Wearable camera, and method for encoding video captured by the wearable camera
By determining the center of rotation and adjusting compression levels based on radial distance, wearable cameras optimize bitrate and storage needs, addressing power and transmission constraints while maintaining image quality.
Patent Information
- Application Number
- JP2024172587
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-08-17
- Filing Date
- 2024-10-01
- Publication Date
- 2025-07-10
- Estimated Expiration
- 2041-08-11
AI Technical Summary
Wearable cameras face limitations in power and bitrate due to battery constraints and wireless transmission capabilities, leading to challenges in efficiently encoding video streams.
A method and device for wearable cameras that determine the center of rotation and adjust compression levels based on radial distance from the center, increasing compression with distance to reduce bitrate and maintain image quality.
Reduces the required bandwidth and storage space for video streams by optimizing compression levels, particularly in areas less affected by motion blur and of lower interest, while maintaining high quality in central regions.
Smart Images

Figure 0007705997000001 
Figure 0007705997000002
Abstract
Description
Technical Field
[0001] The present invention relates to video encoding, and more particularly to encoding video captured by a wearable camera.
Background Art
[0002] Wearable cameras are used by police officers, for example, to capture video streams during patrols and events. Such cameras can also be referred to as body worn cameras (BWCs). Wearable cameras are typically battery-powered. Therefore, there are limitations on the power available for wearable cameras. Furthermore, wearable cameras can transmit the captured video stream via a wireless connection. Therefore, the bitrate available for such transmission is also a limiting factor in encoding the video stream.
Summary of the Invention
[0003] The object of the present invention is to facilitate bitrate savings and / or battery savings for video streams generated by wearable cameras.
[0004] According to a first aspect, a method for encoding video captured by a wearable camera is provided. The method includes determining a center of rotation for an image frame to be encoded. The center of rotation is related to the rotation of the wearable camera when capturing the video. The image frame includes a plurality of groups of pixels. The method further includes setting a compression level for the plurality of groups of pixels of the image frame, wherein the compression level for the plurality of groups of pixels of the image frame is such that the compression level increases with the radial distance from the center of rotation. The method further includes encoding the image frame using the compression level.
[0005] The inventors have recognized that the rotational movement of the wearable camera occurs during use and that such rotational movement leads to movement of the captured images. In such a rotational movement, the movement of the group of pixels between consecutive image frames will increase with the radial distance from the center of rotation within the image frame corresponding to the center of rotation of the image sensor of the wearable camera. This will result in an increased risk of failure to identify the motion vectors associated with the radial distance from the center of rotation, which will lead to problems related to high bitrates, thereby increasing the risk that pixel blocks have to be encoded using intra-coding instead of generally less bit-consuming inter-coding. This risk will be more prominent for motion estimation algorithms optimized for translational motion identification. Therefore, a specific compression principle is introduced where the compression level increases with the radial distance from the center of rotation.
[0006] By increasing the compression level with the radial distance from the center of rotation, the total bitrate required to encode a video including the image frames is reduced compared to the case where the compression level would be equally the same as that at the center of rotation.
[0007] Reducing the bitrate is beneficial in that, for example, it reduces the bandwidth required to wirelessly transmit the resulting video stream and reduces the storage space required to store the resulting video stream.
[0008] The increased risk of failure to identify the motion vectors associated with the radial distance from the center of rotation also generally causes the bitrate cost for the group of pixels to increase with the radial distance of the group of pixels from the center of rotation. Therefore, increasing the compression level with the radial distance from the center of rotation will result in a higher compression level for groups of pixels for which the bitrate cost would otherwise be higher.
[0009] Furthermore, since the effect of motion blur generally increases with the radial distance from the center of rotation, it is advantageous to enable higher image quality (resolution) with lower compression closer to the center of rotation where the motion blur is generally minimal.
[0010] Parts of the image frame closer to the center of rotation may also be parts of higher interest compared to more peripheral parts further away from the center of rotation, so it is advantageous to enable higher image quality (resolution) with lower compression closer to the center of rotation.
[0011] Furthermore, since the effect of motion blur generally increases with the radial distance from the center of rotation, the loss of image quality further away from the center of rotation becomes less important. This is because the effect of motion blur further away from the center of rotation is considered to have affected the image quality anyway.
[0012] A group of pixels can be referred to as, for example, a block, a macroblock, or a coding tree unit.
[0013] The compression level for a plurality of groups of pixels in an image frame can be, for example, a compression value such as the value of a quantization parameter for the plurality of groups of pixels in the image frame.
[0014] The act of determining the center of rotation of the method can further include determining the center of rotation using data from one or more motion sensors within the wearable camera.
[0015] The method can further include determining motion vectors for the image frame. The act of determining the center of rotation can then further include determining the center of rotation using the motion vectors.
[0016] The method can further include determining a pre-rotation center for an image frame and setting a motion vector search area that includes the pre-rotation center and that is a sub-area of the image frame. The act of determining the motion vector can further include determining the motion vector within the motion vector search area. The reliability of the determined motion vector generally decreases with the radial distance from the center of rotation. Thus, selecting a motion vector search area that is a sub-area of the image and that includes the pre-rotation center will exclude the area of the image frame that includes the least reliable motion vectors. As a result, a more reliable identification of the center of rotation becomes possible.
[0017] The method can further include obtaining a previous rotation center for the immediately previous image frame. The act of determining the pre-rotation center can then include determining the pre-rotation center for the image frame so as to correspond to the previous rotation center. The center of rotation tends to move between consecutive image frames but not within a very wide range. Thus, the previous rotation center, i.e., the center of rotation of the immediately previous image frame, is a good approximation to use as the pre-rotation center. Preferably, the previous rotation center is the center of rotation of the image frame immediately preceding the immediate future.
[0018] The method can further include determining the angular velocity of the wearable camera when capturing an image frame. In the act of setting the compression level, the rate of increase of the compression level associated with the radial distance from the center of rotation can then be based on the angular velocity. The incentive to increase the compression level increases with the amount of motion in the image frame. In the case of rotation, the motion in the image frame depends on the radial distance from the center of rotation but also on the angular velocity. For a larger angular velocity, the motion will be larger at the same radial distance from the center of rotation. Therefore, it is beneficial to increase not only the compression level but also the rate of increase based on the angular velocity along with the radial distance from the center of rotation. The loss of image quality for a larger angular velocity will not be significant because the effect of motion blur at a larger angular velocity is likely to affect the image quality anyway.
[0019] The method can further include determining a motion vector for the image frame. In the act of setting the compression level, the rate of increase of the compression level associated with the radial distance from the center of rotation can then be based on the motion vector. For example, the rate of increase of the compression level can be based on the rate of increase of the length of the motion vector associated with the radial distance from the center of rotation.
[0020] According to a second aspect, a non-transitory computer-readable medium is provided. The non-transitory computer-readable medium stores instructions, perhaps in the form of computer-readable program code, which when executed on a device having processing capabilities are configured to implement the method of the first aspect. The device having processing capabilities can be a wearable camera, for example, a body-worn camera.
[0021] The features described above of the method according to the first aspect are also applicable to the second aspect, where applicable. References are made above to avoid undue repetition.
[0022] According to a third aspect, a wearable camera is provided. The wearable camera includes an image sensor, a circuit, and an encoder. The image sensor is configured to capture image data. The circuit is configured to execute a rotation center determination function configured to determine a rotation center for an encoded image frame. The rotation center is related to the rotation of the wearable camera when capturing video. The image frame includes a plurality of groups of pixels based on the image data. The circuit is further configured to execute a compression level setting function configured to set a compression level for the plurality of groups of pixels of the image frame, and the compression level for the plurality of groups of pixels of the image frame is such that the compression level increases with the radial distance from the rotation center. The encoder is configured to encode the image frame into a video stream using the compression level set by the compression level setting function.
[0023] The wearable camera can further include a motion sensor for determining motion data about the wearable camera. The rotation center determination function can then be further configured to use the motion data from the motion sensor to determine the rotation center.
[0024] The encoder of the wearable camera can be further configured to determine a motion vector for the image frame. The rotation center determination function can then be further configured to use the motion vector to determine the rotation center.
[0025] The circuit of the wearable camera can be further configured to execute a pre-rotation center determination function configured to determine a pre-rotation center for the image frame, and a motion vector search area setting function configured to set a motion vector search area that includes the pre-rotation center and is a sub-area of the image frame. The encoder can then be configured to determine a motion vector within the motion vector search area.
[0026] The circuit of the wearable camera can be further configured to execute a function for obtaining the center of rotation immediately before for the immediately preceding image frame. The function for determining the center of rotation in advance can then be configured to determine the center of rotation in advance for the image frame so as to correspond to the center of rotation immediately before.
[0027] The circuit of the wearable camera can be further configured to execute a function for determining the angular velocity of the wearable camera when capturing an image frame. In the compression level setting function, the rate of increase of the compression level associated with the radial distance from the center of rotation can then be based on the angular velocity.
[0028] The encoder of the wearable camera can be further configured to determine a motion vector for the image frame. In the compression level setting function, the rate of increase of the compression level associated with the radial distance from the center of rotation can then be based on the motion vector.
[0029] The further scope of applicability of the present disclosure will become apparent from the detailed description shown below. However, since various changes and modifications within the scope of the present invention will become apparent to those skilled in the art from this detailed description, it should be understood that the detailed description and specific examples are shown by way of illustration only while showing the preferred embodiments of the present invention.
[0030] Accordingly, it is to be understood that the present invention is not limited to the specific component parts of the devices described or the specific acts of the methods described, because such devices and methods may vary. It is also to be understood that the terms used herein are for the purpose of describing particular embodiments only and are not intended to be limiting. It should be noted that when used in this specification and the appended claims, the articles “a,” “an,” “the,” and “said” are intended to mean that one or more of the elements are present unless the context clearly dictates otherwise. Thus, for example, references to “a unit” or “the unit” can include several devices and the like. Further, the terms “comprising,” “including,” “containing,” and the like do not exclude other elements or steps.
Brief Description of the Drawings
[0031] The above and other aspects of the present invention will now be described in more detail with reference to the accompanying drawings. The drawings are not to be considered as limiting, but rather are used for explanation and understanding. Like reference numerals refer to like elements throughout.
Figure 1
Figure 2
Modes for Carrying Out the Invention
[0032] The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which presently preferred embodiments of the invention are shown. The invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
[0033] FIG. 1 shows a wearable camera 100. The wearable camera 100 can be a body-worn camera BWC. The wearable camera 100 can be a separate unit or can be integrated into another unit, such as in a helmet, glasses, etc. The wearable camera 100 can be used by a police officer, for example, to capture video and possibly other data during patrols and events. The captured data may then be needed as evidence, for example, when investigating a crime and prosecuting a suspect. To store the captured data, an external data management system, such as a video management system or an evidence management system, can be used with the wearable camera 100. Such data management systems generally provide storage of the captured data and also viewing of the captured data, either in real time or as playback of recorded data. Typically, the wearable camera 100 is battery-powered and has a limited bitrate. The limited bitrate can be due to limitations in local data storage for the data management system or bandwidth for a wireless connection to a central location where live video is viewed. Further, the bandwidth limitation for the wireless connection can vary over time, thereby causing the bitrate to be even more limited at times. The wearable camera 100 includes an image sensor 110, a circuit 130, and an encoder 120.
[0034] The image sensor 110 is configured to capture image data. The image data can be, for example, data of an image frame. The capture of the image sensor and the image data is well known to those skilled in the art and will not be discussed in more detail in the present disclosure.
[0035] Encoder 120 is configured to encode the image data captured by image sensor 110 into a video stream. Sometimes, the video stream provided by encoder 120 is called an encoded video stream. Typically, video encoder 120 is configured to encode some of the image frames of the video stream as intra-frames or key-frames, and some of the image frames of the video stream as inter-frames or delta-frames. An intra-frame is an encoded video frame that does not require information from other encoded video frames to be decoded. Thus, an intra-frame is encoded based on information from the image frame of the video data for which the intra-frame is set to correspond. Typically, the similarity within the image frame is used to encode the image frame as an intra-frame. In video encoding, an intra-frame is often called an I-frame. The image frames of the video stream between two intra-frames are encoded as inter-frames. Typically, an inter-frame only contains the changes that occur from one frame to the next. Thus, an inter-frame typically contains less data than an intra-frame. In video encoding, an inter-frame is often called a P-frame or a B-frame. A P-frame points to the immediately preceding frame for data reference. Thus, the content of the immediately preceding frame must be known in order to decode the P-frame. A B-frame can point to both the immediately preceding frame and the forward frame for data reference. Thus, the content of both the immediately preceding frame and the forward frame must be known in order to decode the B-frame. When encoding an inter-frame, the image frame is divided into multiple groups of pixels. The groups of pixels can be called, for example, blocks, macroblocks, or coding tree units. The image frame is compared with a reference frame. For example, the reference frame for encoding a P-frame is the immediately preceding image frame.A matching algorithm is used to identify groups of matching pixels between the image frame to be encoded and the reference frame. If a match is found for a group of pixels, that group of pixels can be encoded as a motion vector that specifies how much the group of pixels has moved within the image frame since the reference frame. Determining the motion vector is sometimes called motion estimation. When there is significant motion due to fast movement of the camera or objects within the scene being captured, motion estimation may fail to estimate the motion vector. The fewer motion vectors identified for the image frame to be encoded, the larger the resulting encoded inter-frame will be in bit size, and thus a larger bandwidth will be required to transmit the encoded inter-frame.
[0036] Circuit 130 is configured to perform functions of the wearable camera 100. Circuit 130 can include a processor 132 such as a central processing unit (CPU), a microcontroller, or a microprocessor. Processor 132 is configured to execute program code. The program code can be configured, for example, to perform functions of the wearable camera 100.
[0037] The wearable camera 100 can further include a motion sensor 140 for determining motion data for the wearable camera. The motion sensor 140 is configured to measure motion data of the wearable device. The motion sensor 140 can include a gyroscope and / or an accelerometer. The gyroscope is configured to measure motion data in the form of the orientation and / or angular velocity of the wearable camera 100. The accelerometer is configured to measure motion data in the form of the acceleration (or rate of change of velocity) of the wearable camera 100 in its own instantaneous rest frame. The motion sensor 140 is configured to sample the motion data as a function of time.
[0038] The wearable camera 100 can further include a local data storage 150. The local data storage 150 can be configured to store video streams. The local data storage typically has a limited data storage capacity. The local data storage 150 can be any type of local data storage suitable for storing video streams. For example, the local data storage 150 can be in the form of an SD card reader and an SD card. Another example of the local data storage 150 can be in the form of flash memory, such as NAND flash.
[0039] The wearable camera 100 can further include a transmitter 160. The transmitter 160 can be configured to wirelessly transmit a video stream to a video management system. The transmitter 160 can be configured to continuously transmit the captured video stream to a data management system. Wireless transmission is typically limited by the bandwidth available for wireless transmission.
[0040] The wearable camera 100 can further include a memory 170. The memory 170 can be one or more of a buffer, flash memory, hard drive, removable media, volatile memory, non-volatile memory, random access memory (RAM), or another suitable device. In a typical arrangement, the memory 170 can include non-volatile memory for long-term data storage and volatile memory that functions as system memory for the circuit 130. The memory 170 can exchange data with the circuit 130 through a data bus. There can also be associated control lines and an address bus between the memory 170 and the circuit 130.
[0041] The functions of the wearable camera 100 can be embodied in the form of executable logic routines (e.g., lines of code, software programs, etc.), which are stored on a non-transitory computer-readable medium (e.g., memory 170) of the wearable camera 100 and executed by the circuit 130 (e.g., using the processor 152). Further, the functions of the wearable camera 100 can be a stand-alone software application or form part of a software application that performs additional tasks related to the wearable camera 100. The functions to be described can be considered as a way in which the processing unit of the circuit 150, e.g., the processor 132, is configured to execute them. Similarly, the functions to be described can be implemented in software, but such functions can also be implemented by dedicated hardware or firmware, or by some combination of hardware, firmware, and / or software.
[0042] The circuit 130 is configured to execute a rotation center determination function 181. The rotation center determination function 181 is configured to determine a rotation center for an encoded image frame. The rotation center is related to the rotation of the wearable camera when capturing video. In particular, the rotation center for an image frame corresponds to the image sensor rotation center about which the wearable camera 100 rotates when capturing the image data on which the image frame is based. The image frame is divided into a plurality of groups of pixels used in the encoding of the image frame.
[0043] Circuit 130 is configured to execute a compression level setting function 182. The compression level setting function 182 is configured to set compression levels for a plurality of groups of pixels of an image frame. The compression levels for the plurality of groups of pixels of the image frame are set such that the compression level increases with the radial distance from the center of rotation. Generally, if a first group of pixels is at a greater radial distance from the center of rotation than a second group of pixels, the first group of pixels will have a higher compression level than the second group of pixels. The compression level can increase continuously with the radial distance from the center of rotation. However, the compression level can also increase stepwise, such that, for example, a first group of pixels within a first radial distance range from the center of rotation has the same first compression level, and a second group of pixels within a second radial distance range from the center of rotation that includes a radial distance greater than the first radial distance range has the same second compression level, and so on.
[0044] The increase in the compression level can be further made non-uniform, such that, for example, the compression level increases at a higher rate with the distance from the center of rotation in the horizontal direction than with the vertical distance. In such a case of non-uniform increase, the compression level at a given distance from the center of rotation along a horizontal line from the center of rotation will be higher than the compression level at the same given distance from the center of rotation along a line at an angle greater than 0 degrees from the center of rotation. Generally, the compression level can be based not only on the need for compression due to a high degree of movement further away from the center of rotation, but also on the expected level of interest in different parts of the image frame. It can be assumed, for example, that the peripheral part of the image frame in the horizontal direction is less interesting than the peripheral part of the image frame in the vertical direction.
[0045] The compression level can be related to different measures and characteristics indicating the compression level. For example, the compression level can be related to a compression value such as the value of a quantization parameter, whereby the value of the quantization parameter is set to increase with the radial distance from the center of rotation. In such a case, if a first group of pixels is at a greater radial distance from the center of rotation compared to a second group of pixels, the first group of pixels will have a greater value of the quantization parameter compared to the second group of pixels. This can be achieved, for example, by a gradient quantization parameter map (QMAP).
[0046] The compression level can further describe or indicate the bit rate required for encoding. Thus, as long as the required bit rate is lower, the compression level is considered to be higher, regardless of the means or process used to achieve the lower required bit rate. For example, the compression level can be increased by using different encoding schemes for blocks depending on the radial distance from the center of rotation. Examples of such encoding are the use of skip blocks or DC coding (i.e., single color) I blocks for groups of images, which will result in a higher compression level compared to encoding groups of images as normal P or B blocks. A skip block has no image information or prediction information at all, and the decoder interprets the skip block as if the corresponding block (group of pixels) in the immediately previous frame and the block (group of pixels) are identical. A skip block can also have a motion vector that it inherits from its neighborhood. However, no information is encoded. To increase the compression level, the selection of the block type (I, P, skip) can be biased such that the probability of the block type corresponding to a higher compression level increases with the distance from the center of rotation.
[0047] Encoder 120 can then be configured to encode the image frames into a video stream using the compression level set by the compression level setting function 182.
[0048] The increase in the compression level with the radial distance from the center of rotation will result in a decrease in the bitrate of the video stream of such encoded frames, which will be lower than it would be if the video stream included image frames having the same compression level at the center of rotation but without the increase in the compression level with the radial distance from the center of rotation. Further, increasing the compression level along with the radial distance from the center of rotation will maintain high quality (low compression) for groups of pixels near the center of rotation. This is beneficial because these groups of pixels depict what is straight in front of the wearable camera, are less affected by motion blur, require a lower bitrate for encoding, are easier to find corresponding motion vectors, and are often associated with areas of high interest. Further, most of the bitrate reduction is achieved by a high compression level in the peripheral part compared to the center of rotation. A higher compression level for such groups of pixels is beneficial because these groups of pixels depict what is in the peripheral part compared to the center of rotation, are more affected by motion blur, require a higher bitrate for encoding, are more difficult to find corresponding motion vectors, and are often associated with areas of low interest. Such a method for bitrate reduction will not achieve the benefits identified above compared to uniformly increasing the compression in the image frames and reducing the frame rate.
[0049] The center of rotation determination function 181 can be configured to determine the center of rotation using, for example, motion data from the motion sensor 140 comprising a gyroscope and / or an accelerometer.
[0050] Encoder 120 can be configured to determine motion vectors for image frames. The rotation center determination function 181 is further configured to determine the rotation center using the motion vectors. In the case of rotational motion, the motion vectors generate a circular pattern around the rotation center, and the length increases along with the radial distance from the rotation center. Therefore, the identified motion vectors can be analyzed using an appropriate algorithm for determining the rotation center.
[0051] The circuit can be configured to execute a pre-rotation center determination function 183 and a motion vector search area setting function 184. The pre-rotation center determination function 183 is configured to determine a pre-rotation center for an image frame. The motion vector search area setting function 184 is configured to set a motion vector search area that includes the pre-rotation center and is a sub-area of the image frame. The encoder 120 can then be configured to determine a motion vector within the motion vector search area. The rotation center determination function 181 can then be configured to determine a rotation center using the motion vectors within the motion vector search area. The basis for this is that the radial distance by which a certain group of pixels moves between two consecutive image frames increases along with the radial distance from the rotation center, so the reliability of the determined motion vectors generally decreases along with the radial distance from the rotation center, and this in turn makes it increasingly difficult to determine the motion vectors. Therefore, using only the motion vectors within the sub-area will exclude the area of the image frame that includes the least reliable motion vectors. Thus, a highly reliable identification of the rotation center becomes possible. The motion vector search area can be set, for example, to include a group of pixels for which a motion vector can be determined with a desired reliability. The encoder 120 can then be configured to determine a motion vector of the image frame outside the motion vector search area. However, such a motion vector will only then be used by the encoder 120 to encode the image frame and will not be used by the rotation center determination function 181 that determines the rotation center.
[0052] Encoder 120 can alternatively be configured to determine the motion vectors of the image frames only within the motion vector search area. In such a case, the motion vector search area can be set to exclude a group of pixels of which the encoder 120 has a probability of failing to determine the motion vectors over a certain threshold. The probability of failure will increase with the increase in motion. Therefore, the group of pixels to be excluded can be selected as the group of pixels farther than the threshold radial distance from the previous rotation center. The threshold radial distance can be based on, for example, the angular velocity. In the case of the group of pixels for which the motion vectors are not determined, i.e., the group of pixels of which the motion vectors are not determined, the compression level can be further increased to compensate for the additional bitrate required for encoding in the absence of the identified motion vectors. The increase in the compression level can be achieved by means of adapting the quantization parameter value for the group of pixels of which the motion vectors are not determined. In an alternative method, the group of pixels of which the motion vectors are not determined can be encoded as skip blocks or DC-coded (i.e., single color) I blocks.
[0053] Circuit 130 can be configured to execute the previous rotation center acquisition function 185. The previous rotation center acquisition function 185 is configured to acquire the previous rotation center for the previous image frame. The previous rotation center determination function 183 can then be configured to determine the previous rotation center for the image frame so as to correspond to the previous rotation center. Since the rotation center generally does not move very far between consecutive image frames, using the previous rotation center for the previous image frame will generally be a good approximation for use as the previous rotation center.
[0054] The circuit can be further configured to execute an angular velocity determination function 186. The angular velocity determination function 186 is configured to determine the angular velocity of the wearable camera when capturing an image frame. The compression level setting function 182 can then be configured such that the rate of increase of the compression level with respect to the radial distance from the center of rotation is based on the angular velocity. Generally, the compression level should increase with an increase in the amount of motion within the image frame. In the case of rotation, the amount of motion within the image frame depends on the radial distance from the center of rotation but also on the angular velocity. For a larger angular velocity, the amount of motion will be greater at the same radial distance from the center of rotation than it would be for a smaller angular velocity. Thus, the compression level can increase at a higher rate with the radial distance from the center of rotation for a larger angular velocity, such that the compression level will be higher at the same radial distance from the center of rotation than it would be for a smaller angular velocity. The angular velocity can be determined by data from the motion sensor 140, for example, by a gyroscope.
[0055] Alternatively or additionally, based on the motion vectors determined by the encoder 120, the compression level setting function 182 can be configured such that the rate of increase of the compression level with respect to the radial distance from the center of rotation is based on the motion vectors. For example, for a larger angular velocity, the determined motion vectors will increase in length at a higher rate with the radial distance from the center of rotation than they would be for a smaller angular velocity. Thus, the rate of increase of the compression level can be set to be proportional to the rate of increase of the length of the motion vectors with respect to the radial distance from the center of rotation. Similarly, the compression levels for groups of pixels can be set to be proportional to the lengths of their respective motion vectors.
[0056] With reference to FIG. 2, a method 200 for encoding video captured by a wearable camera is discussed. Method 200 is based on the insight obtained by the inventor that, for example, for groups of pixels that were thought to hold less relevant information and / or that were thought to have lower quality even without a higher compression level, a higher compression level will be applied, so that by increasing the compression level along with the radial distance from the center of rotation, the bitrate required for encoding can be reduced without a correspondingly large loss of quality.
[0057] Some of all the steps of method 200 can be performed by functions of the wearable camera 100 described above. The method includes the following steps. Unless a step is particularly dependent on the result of another step, the steps can be performed in any suitable order.
[0058] The method includes determining S220 a center of rotation for the image frame to be encoded, the center of rotation being related to the rotation of the wearable camera when capturing the video, and the image frame including a plurality of groups of pixels. Method 200 further includes setting S222 a compression level for the plurality of groups of pixels of the image frame, where the compression level for the plurality of groups of pixels of the image frame is such that the compression level increases along with the radial distance from the center of rotation. Method 200 further includes encoding S224 the image frame using the compression level.
[0059] Determining S220 the center of rotation can further include determining the center of rotation using data from one or more motion sensors within the wearable camera.
[0060] Method 200 can further include determining S210 a motion vector for the image frame. Determining S220 the center of rotation can further include determining the center of rotation using the motion vector.
[0061] Method 200 may further include determining a pre-rotation center for an image frame at S206, and setting a motion vector search area that includes the pre-rotation center and is a sub-area of the image frame at S208. Determining the motion vector at S208 may then include determining the motion vector within the motion vector search area.
[0062] Method 200 may further include obtaining a previous rotation center for the previous image frame at S204. Determining the pre-rotation center at S206 may then include determining the pre-rotation center for the image frame so as to correspond to the previous rotation center.
[0063] Method 200 may further include determining the angular velocity of the wearable camera when capturing the image frame at S202. In setting the compression level at S222, the rate of increase of the compression level associated with the radial distance from the rotation center is based on the angular velocity.
[0064] The method may further include storing a video stream in the wearable camera. The method may further include wirelessly transmitting the video stream from the wearable camera. The method may further include corresponding features for the features disclosed for the wearable camera described with respect to FIG. 1.
[0065] Those skilled in the art will recognize that the present invention is not limited to the embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims. Such modifications and variations can be understood and implemented by those skilled in the art when practicing the claimed invention by examining the drawings, the present disclosure, and the appended claims.
Claims
1. A method for encoding video captured by a wearable camera, comprising: determining a center of rotation in an image frame to be encoded, wherein the center of rotation corresponds to the center of rotation of an image sensor of the wearable camera in rotational motion when capturing the image data underlying the image frame, and the image frame includes a plurality of groups of pixels; setting a compression level for the plurality of groups of pixels of the image frame, wherein the compression level for the plurality of groups of pixels of the image frame is such that the compression level increases with the radial distance from the center of rotation; and encoding the image frame using the compression level. A method comprising the above steps.
2. The act of determining the center of rotation further includes: determining the center of rotation using data from one or more motion sensors within the wearable camera. The method according to claim 1.
3. The method further includes determining a motion vector for the image frame, and the act of determining the center of rotation further includes: determining the center of rotation using the motion vector. The method according to claim 1.
4. The method further includes determining an angular velocity of the wearable camera when capturing the image data underlying the image frame, and in the act of setting the compression level, the higher the angular velocity, the higher the rate of increase of the compression level with the radial distance from the center of rotation. The method according to claim 1.
5. The method further includes determining a motion vector for the image frame, and in the act of setting the compression level, the higher the rate of increase of the length of the motion vector with the radial distance from the center of rotation, the higher the rate of increase of the compression level with the radial distance from the center of rotation. The method according to claim 1.
6. A non-transitory computer-readable storage medium storing instructions for encoding video captured by a wearable camera, wherein the instructions are instructions for implementing the method according to any one of claims 1 to 5.
7. A wearable camera, An image sensor configured to capture image data, a circuit, configured to execute a rotation center determination function configured to determine a rotation center in an encoded image frame, the rotation center corresponding to the rotation center of the image sensor of the wearable camera that rotates when capturing the image data serving as the basis of the image frame, the image frame including a plurality of groups of pixels, a circuit configured to execute a compression level setting function configured to set a compression level for the plurality of groups of pixels of the image frame, the compression level for the plurality of groups of pixels of the image frame being such that the compression level increases with the radial distance from the rotation center, an encoder configured to encode the image frame into a video stream using the compression level set by the compression level setting function A wearable camera comprising: **Claim 8** Further comprising a motion sensor for determining motion data for the wearable camera, The wearable camera according to claim 7, wherein the rotation center determination function is further configured to determine the rotation center using the motion data from the motion sensor. **Claim 9** The wearable camera according to claim 7, wherein the encoder is further configured to determine a motion vector for the image frame, and the rotation center determination function is further configured to determine the rotation center using the motion vector. **Claim 10** The circuit is further configured to execute an angular velocity determination function configured to determine the angular velocity of the wearable camera when capturing the image data serving as the basis of the image frame, and in the compression level setting function, the higher the angular velocity, the higher the rate of increase of the compression level with the radial distance from the rotation center. The wearable camera according to claim 7. **Claim 11** The wearable camera according to claim 7, wherein the encoder is further configured to determine a motion vector for the image frame, and in the compression level setting function, the higher the rate of increase of the length of the motion vector with the radial distance from the rotation center, the higher the rate of increase of the compression level with the radial distance from the rotation center.
Citation Information
Patent Citations
Image processor, image processing method and image processing program
JP2005020478A
Imaging apparatus
JP2013017165A
Imaging device
WO2012026122A1