Method for identifying stationary regions in frames of a video sequence

Abstract

A method (10, 30) for identifying stationary regions in frames of a video sequence, comprising: receiving (S1010) an encoded version (20) of a video sequence, wherein the encoded version (20) of the video sequence includes an intra-coded frame (200) followed by a plurality of inter-coded frames (201, 202, 203, 204); reading (S1020) coding mode information in the inter-coded frames of the encoded version of the video sequence, wherein the coding mode information indicates that a block of pixels in the inter-coded frames is skip-coded; using the read coding mode information, discovering (S1030) one or more blocks of pixels (210, 212, 406) that are each skip-coded in a respective plurality of consecutive frames in the encoded version (20) of the video sequence; and designating (S1040) each discovered block of pixels (210, 212, 406) as a stationary region in the respective plurality of consecutive frames.

Description

Technical Field

[0001] This invention relates to a method and apparatus for identifying stationary regions in frames of a video sequence. It also relates to a non-transitory computer-readable medium storing computer code instructions suitable for performing the method. Background Technology

[0002] Today, surveillance cameras that generate large amounts of video data are widely used. Typically, it's necessary to determine whether an image region is static and for how long. Traditionally, this analysis is performed using image analysis algorithms that work on decoded video frames. However, analyzing large volumes of data can require considerable processing power, which may be unavailable or expensive. Therefore, improvements are needed in this context. Summary of the Invention

[0003] In view of the above, one object of the present invention is to solve or at least reduce one or more of the disadvantages discussed above.

[0004] According to a first aspect of the present invention, a method for identifying stationary regions in frames of a video sequence is provided, the method comprising: receiving an encoded version of the video sequence, wherein the encoded version of the video sequence includes intra-coded frames, the intra-coded frames being followed by a plurality of inter-coded frames; reading encoding mode information in the inter-coded frames of the encoded version of the video sequence, wherein the encoding mode information indicates that pixel blocks in the inter-coded frames are skipped in encoding; using the read encoding mode information, identifying one or more pixel blocks that are all skipped in encoding in a corresponding plurality of consecutive frames in the encoded version of the video sequence; and designating each identified pixel block as a stationary region in the corresponding plurality of consecutive frames.

[0005] "Encoded version of a video sequence" should be understood as is known in the field of video coding, which may require, for example, image frames of a video sequence to be represented in a bitstream that may be compressed.

[0006] "Encoding mode information" should be understood as any information provided in the encoded version of the video stream. Therefore, "encoding mode information" and "encoded information" can be considered synonyms.

[0007] A “pixel block” should be understood as a set of pixels in an image frame that can be encoded in an encoded version of the video stream. For example, a pixel block may include a set of luminance samples and / or a set of chrominance samples corresponding to the pixels of the pixel block. For example, a pixel block may correspond to a “macroblock” as is well known in the field of video coding.

[0008] "Skip coding" and "skipped coding" should be understood as they are well-known in the field of video coding and video coding standards. For example, "skip coding" may include an indication in the encoded version of the video stream that one or more pixel blocks (such as one or more macroblocks) should be decoded as "skipped". A skipped-coded block may also be referred to as an "empty" block. A skipped-coded block may be an "empty P-block".

[0009] "Intra-coded frame" should be understood as a frame that is encoded only with reference to other parts of the same frame, as is well known in the field of video coding. For example, an intra-coded frame can be an "I" frame.

[0010] An "inter-coded frame" should be understood as a frame that can be encoded with reference to one or more different frames, as is well known in the field of video coding. For example, an inter-coded frame can be a "P" frame or a "B" frame.

[0011] "Still areas" should be understood as regions in the video sequence that have not changed directly since the previous frame.

[0012] The encoded version of a video sequence, including subsequent intra-coded frames and multiple inter-coded frames, can be, for example, a group of pictures (GOP) as known in the field of video coding.

[0013] This invention stems from the understanding that pixel blocks that have been skipped from encoding in multiple consecutive frames likely correspond to stationary regions in a video sequence. By specifying stationary regions based on encoding mode information indicating that pixel blocks in inter-coded frames have been skipped from encoding, these regions can be designated as stationary without requiring full decoding or image analysis of the corresponding inter-coded frames of the encoded version of the video sequence, as will be explained further below.

[0014] Furthermore, the complete decoding of an encoded video bitstream typically begins with the so-called "unpacking" of the corresponding bitstream, where the contents of the bitstream can be deserialized; that is, the individual items of the bitstream are parsed and can be read, for example, into variables in volatile memory. In one or more further decoding steps, the thus deserialized data can be processed, for example, involving the interpretation of motion vectors and other methods used to encode pixel blocks, ultimately producing a decoded image frame containing actual pixel blocks in the form of, for example, chroma and luminance samples. Typically, these one or more further steps require significantly more processor time than unpacking.

[0015] Typically, coding mode information indicating which pixel blocks in an inter-coded frame are skipped from coding is directly available after unpacking without further processing, and therefore, a region can be designated as stationary without further decoding and / or processing. This saves processing power. This is particularly beneficial for long intervals between intra-coded frames (e.g., "I" frames, as is common in surveillance camera applications), because no image analysis is then needed during the long sequence of inter-coded frames (e.g., long GOPs) until the next intra-coded frame, while it can still be inferred that no motion occurred during the sequence of inter-coded frames.

[0016] The method may further include analyzing video data in frames of a video sequence to detect motion and / or identify foreground objects, wherein static regions of the frames are excluded from the analysis.

[0017] Therefore, by excluding regions designated as static from the analysis, available processing power expenditure can be targeted at areas that are likely to have changed. For example, this could allow for decoding only intra-coded frames and / or only inter-coded frames of a video sequence that do not correspond to regions designated as static. This allows for better use of available resources and / or savings in processing power costs.

[0018] The method may further include one or more timers, each timer corresponding to a specified static region, the timers indicating how many consecutive frames of the pixel block are skipped for encoding before and including the current inter-frame encoded frame.

[0019] This provides a way to keep track of how long a tracking region has remained stationary, while still requiring only moderate processing power.

[0020] Maintaining one or more timers may include incrementing the timer corresponding to a specified static region if a corresponding pixel block that was skipped from encoding in the previous inter-coding frame is found to be skipped from encoding in the current inter-coding frame.

[0021] This is a particularly simple way to implement a timer.

[0022] Maintaining one or more timers may include resetting the timer corresponding to the specified still region if the corresponding pixel block that was skipped in the previous inter-coded frame is not found to be skipped in the current inter-coded frame.

[0023] This is a particularly simple way to implement a timer.

[0024] The method may further include defining a region of intra-coded frames including the object of interest relative to one or more frames of a video sequence; discovering one or more designated still regions, each designated still region being at least partially located within the region including the object of interest; and designating the object of interest as a still object.

[0025] Optionally, the definition of the region including the object of interest can be based on object detection in the decoded frames of the video sequence.

[0026] As is commonly referred to in the art, an "object of interest" can be any object located within a frame of a video sequence. It can be, for example, but is not limited to, objects that have particular significance to the user and / or viewer, operator, or image analysis process of the video sequence. The region including the object of interest can be defined, for example, based on an automatic object detection process or algorithm. In another example, the region including the object of interest can be manually defined by an operator viewing one or more frames of the video sequence as part of a live stream or a recording.

[0027] Based on the above, regions of a frame can be designated as stationary based on encoding mode information. By discovering that one or more designated stationary regions are at least partially located within a region containing the object of interest, this information can be used to infer with a certain degree of certainty that the object of interest has not moved, and the object of interest can be identified as a stationary object.

[0028] This allows for the identification of stationary objects without requiring image analysis of the inter-coded frames, or even complete decoding of them, because information from a specified stationary region can be used to infer that the object in question is likely stationary. This allows for better use of available resources and / or savings in processing power costs.

[0029] Where it is important not to omit any actual stationary objects, as described above, one may only need at least one specified stationary region to be at least partially located within the region containing the object of interest. Alternatively, if a high degree of certainty that the object is stationary is required, and it is acceptable to omit some actual stationary objects, then one may need the entire region containing the object of interest to be covered by one or more specified stationary objects. Methods in between are also possible, such as requiring a specific portion of the region containing the object of interest to be covered by one or more specified stationary regions. This portion can be adjusted according to the degree of certainty required that the object of interest is actually stationary. Alternatively, one may need a predetermined number of specified stationary regions to be at least partially located within the region containing the object of interest.

[0030] Optionally, the object of interest can be designated as a stationary object if the corresponding timer for each specified stationary region exceeds a predetermined threshold. A "stationary object" can then be understood as an object that does not move substantially during the duration of the timer.

[0031] The method may further include decoding frames of an encoded version of a video sequence, wherein the definition of a region including the object of interest is performed relative to the decoded frame, optionally based on object detection in that frame, and multiple consecutive frames corresponding to the specified static region directly follow the decoded frame.

[0032] The decoded frame can be, for example, an intra-coded frame. However, it can also be an inter-coded frame among multiple inter-coded frames.

[0033] Therefore, according to the present invention, the object of interest defined relative to the decoded frame can be specified as static without fully decoding subsequent frames.

[0034] The method may further include modifying the video data of frames of a video sequence so that the video data in the identified still areas is less salient than the video data in other areas, wherein the salientity level of a pixel block in a still area of ​​a frame of a video sequence is set according to a timer maintained for that pixel block.

[0035] By modifying video frames to make static areas less noticeable, operators can perform tasks such as keeping track of and monitoring video feeds, allowing them to focus their attention on areas that may actually have changed.

[0036] The significance level can decrease as the value of the timer increases.

[0037] Video data in static areas can be made less noticeable by reducing brightness, chroma, and / or opacity.

[0038] The reading of encoding mode information, the discovery of one or more pixel blocks that have been skipped from encoding, and the designation of each discovered pixel block as a stationary region can be done sequentially for each of the multiple inter-coded frames.

[0039] This should be understood as performing reads, discoveries, and assignments in parallel or sequentially, but completing an inter-coded frame completely before moving to the next inter-coded frame.

[0040] Designating a discovered pixel block as a stationary region can include designating the discovered pixel block as a spatial region of a frame in which the video data has not changed directly from the previous frame in the video sequence.

[0041] A skipped-encoding pixel block in a frame of a video sequence can be a pixel block that is encoded as a copy of a spatially overlapping pixel block in the previous frame of the video sequence.

[0042] Encoding mode information can be read from pixel blocks and / or inter-coded frame headers in the coded version of the video sequence.

[0043] According to a second aspect, an apparatus is provided for identifying stationary regions in frames of a video sequence. The apparatus includes circuitry configured to: receive an encoded version of the video sequence, wherein the encoded version of the video sequence includes intra-coded frames, followed by a plurality of inter-coded frames; read encoding mode information in the inter-coded frames of the encoded version of the video sequence, wherein the encoding mode information indicates that pixel blocks in the inter-coded frames are skipped in encoding; use the read encoding mode information to identify one or more pixel blocks that are all skipped in encoding in corresponding plurality of consecutive frames of the encoded version of the video sequence; and designate each identified pixel block as a stationary region in the corresponding plurality of consecutive frames.

[0044] The second aspect can generally have the same features and advantages as the first aspect.

[0045] According to a third aspect, a non-transitory computer-readable medium having computer code instructions stored thereon is provided, the computer code instructions being suitable for performing the method of the first aspect when executed by a processing-capable device.

[0046] The third aspect can typically have the same characteristics and advantages as the first aspect. Attached Figure Description

[0047] The above and additional objects, features and advantages of the invention will be better understood from the following illustrative and non-limiting detailed description of embodiments of the invention, with reference to the accompanying drawings, wherein the same reference numerals will be used for similar elements.

[0048] Figure 1 A flowchart illustrating an example of a method for identifying stationary regions in frames of a video sequence is shown.

[0049] Figure 2 The video sequence is shown.

[0050] Figure 3 A flowchart illustrating a second method example for identifying stationary regions in frames of a video sequence is shown.

[0051] Figure 4 The video frame is shown.

[0052] Figure 5 The video processing system and its components are shown. Detailed Implementation

[0053] The invention will now be described more fully below with reference to the accompanying drawings, in which presently preferred embodiments of the invention are illustrated. However, the invention may be practiced in many different forms within the scope of the claims and should not be construed as limited to the embodiments set forth herein.

[0054] Figure 1 A flowchart of a method example 10 for identifying stationary regions in frames of a video sequence is shown.

[0055] Figure 5 A video processing system and its components are shown. A surveillance camera 502 can be aimed at scene 514 and can be connected thereto to implement method example 10 (see [link]). Figure 1 The video processing unit 504 can be connected to the display 506. Alternatively, the display 506 can be connected to a video client unit (not shown), which in turn is connected to the video processing unit. An additional sensor 508, such as an IR sensor, can also be aimed at the scene 514. Furthermore, optionally, the camera 502 can be a so-called fusion camera, which has, for example, a first sensor 510 and a second sensor 512 that can operate at different wavelength intervals.

[0056] Typically, video processing unit 504 can be connected to several surveillance cameras; for simplicity, only one surveillance camera 502 is shown. Surveillance camera 502 and / or video client unit can be connected directly to video processing unit 504, for example, or logically connected to video processing unit 504 via a packet-switched network such as an IP network.

[0057] Reference Figure 1 At S1010, version 20 of the encoded video sequence is received. Specifically, version 20 of the encoded video sequence can be received from the monitoring camera 502 at the video processing unit 504 and can depict scene 514. The video sequence may include frames captured at the sensor 510 of the camera 502.

[0058] Encoding can be performed, for example, at the monitoring camera 502 or in a separate unit (not shown). The encoding has been performed into the bitstream according to video encoding formats and / or standards such as, for example, H.261, H.262, H.263, H.264 / AVC, H.265 / HEVC, EVC, H.266 / VCC, VP8, VP9, ​​or AV1.

[0059] Figure 2Example encoding version 20 of a video sequence is schematically illustrated. As shown, example encoding version 20 of the video sequence includes an intra-frame coded frame 200 (labeled "I"), followed by multiple inter-frame coded frames (labeled "P"), which are shown here as four inter-frame coded frames 201, 202, 203, and 204 for illustrative purposes only. A first pixel block 210 and a second pixel block 212, which will be detailed below, are schematically illustrated.

[0060] Refer again Figure 1 At S1020, for example in video processing unit 504 (see...) Figure 5 At this point, encoding mode information is read from the inter-coded frames of the encoded version of the video sequence. For example, encoding mode information can be provided in the bitstream format of the relevant video coding standard.

[0061] In particular, encoding mode information can indicate that pixel blocks in inter-coded frames are skipped from encoding.

[0062] For example, coding mode information can be read from pixel blocks in an inter-coded frame. Therefore, as defined in the relevant video coding standards, such skipped coding can take the form of skipped blocks in the corresponding frame of version 20 of the video sequence's coding. Alternatively, skipped coding can be indicated in the frame header, where, for example, the entire frame (including all pixel blocks containing the coding) can be indicated as skipped coding.

[0063] Reading encoding mode information can include, for example, unpacking a bitstream including inter-frame coded frames by loading the contents of the bitstream (including the portion of the bitstream corresponding to the macroblock) into non-volatile memory. Typically, such skipping encoding of a block can be indicated as a field in the macroblock that is encoded and therefore available after unpacking, but does not require any further decoding.

[0064] The skipped-encoding pixel blocks in the frames of the video sequence can be interpreted as pixel blocks encoded as copies of spatially overlapping pixel blocks in the previous frame of the video sequence at corresponding positions in each frame of pixel blocks 210 and 212, as in Figure 2 As shown in the image.

[0065] exist Figure 2 In the example, the pixel blocks that were skipped from encoding are represented by "S".

[0066] Refer again Figure 1 At S1030, for example in video processing unit 504 (see...) Figure 5 At point ), using the read encoding mode information, it can be found that one or more pixel blocks that were skipped from encoding in the corresponding multiple consecutive frames of the video sequence encoding version 20.

[0067] exist Figure 2 In the example, pixel block 210 was found to be skipped in encoding in two frames (i.e., in frames 202 and 203). Similarly, pixel block 212 was found to be skipped in encoding in three frames (i.e., in frames 201, 202, and 203). Pixel blocks 210 and 212 (both found at S1030) were therefore skipped in encoding in the corresponding multiple frames.

[0068] At S1040, for example in video processing unit 504 (see...) Figure 5 At point S1030, each discovered pixel block 210, 212 is designated as a corresponding static region in the corresponding plurality of consecutive frames 202, 203; 201, 202, 203. Therefore, in Figure 2 In the example, pixel block 210 is designated as a stationary region relative to frames 202 and 203, and pixel block 212 is designated as a stationary region relative to frames 201, 202, and 203. Designating each found pixel block 210, 212 as a stationary region may include designating the respective found pixel block 210, 212 as a corresponding spatial region of a frame in the video sequence where the video data has not changed directly since the previous frame.

[0069] The reading of encoding mode information S1020, the discovery of one or more pixel blocks that have been skipped from encoding S1030, and the designation of each discovered pixel block as a stationary region S1040 can be performed sequentially for each inter-frame coded frame 201, 202, 203, 204 of the multiple inter-frame coded frames of the encoding version 20 of the video sequence, or alternatively, simultaneously.

[0070] Optionally, with S1030 and S1040 ( Figure 1 Parallel processing can be implemented, for example, in the video processing unit 504 (see...). Figure 5 One or more timers are maintained at a specified rest area, each timer corresponding to a designated rest region. One or more timers can be stored, for example for later analysis, and / or displayed directly on display 506 (see [reference]). Figure 5 On the monitor 506, or through color coding, darkening, desaturation, or some other type of visualization (see [reference]). Figure 5 )superior.

[0071] Each timer can be initialized, for example, for each pixel block encoded in version 20 of the encoding of the video sequence. In another example, a timer can be initialized for each pixel block found to be skipped in at least one frame. In yet another example, a timer can be initialized when a pixel block is found to be skipped in multiple frames.

[0072] Therefore, in Figure 2 In the example, the timer 222 corresponding to pixel block 212 can be initialized at frame 201, when pixel block 212 is first detected as skipped encoding.

[0073] Similarly, the timer 220 corresponding to pixel block 210 can be initialized at the frame 202 where pixel block 210 is first detected as skipped encoding.

[0074] Each timer 220, 222 can therefore be used as an indication of how many consecutive frames of the pixel block have been skipped from being encoded before and including the current inter-frame encoded frame. Thus, each timer 220, 222 can, for example, count a time unit or the number of frames during which the pixel block has been skipped from being encoded. In the case where the timers count in time units, the counting time can be derived, for example, from the number of frames counted, and from the frames per second value derived from the video stream, or an equivalent value in other units.

[0075] Each timer 220, 222 can be incremented when the specified static region corresponding to each timer 220, 222 is found to be skipped in the current inter-coded frame, in addition to being skipped in the previous inter-coded frame.

[0076] Therefore, in Figure 2 In the example, at frame 202, timer 222 increments because pixel block 212, which was skipped from encoding in the previous frame 201, is also skipped from encoding in the current frame 202.

[0077] Similarly, at frame 202, timer 222 increments because pixel block 212, which was skipped from encoding in the previous frame 201, is also skipped from encoding in the current frame 202. Further, at frame 203, timer 222 increments again because pixel block 212, which was skipped from encoding in the previous frame 202, is also skipped from encoding in the current frame 203.

[0078] If the corresponding pixel block that was skipped in the previous inter-coded frame is not found to be skipped in the current inter-coded frame, the timer corresponding to the specified still area can be reset.

[0079] Therefore, in Figure 2 In the example, at frame 204, pixel blocks 210 and 212 were not found to be skipped encoding, therefore, each of timers 220 and 222 was reset.

[0080] Refer again Figure 1 At S1050, optionally, for example, in video processing unit 504 (see...) Figure 5At this point, methods known per se can be used to analyze video data in frames of video sequence 20 to detect motion and / or identify foreground objects. For example, motion detection and / or foreground object identification can be performed on decoded video frames and can involve comparing data from several intra-coded frames and / or inter-coded frames.

[0081] Thus, static regions specified based on coding pattern information indicating skipped coding can be excluded from the analysis to detect motion and / or identify foreground objects.

[0082] Analysis S1050 can be performed on the previously decoded intra-coded frame 200, such that analysis S1050 can be performed at least partially on the decoded intra-coded frame 200. Furthermore, analysis S1050 can be performed on previously decoded intra-coded frames. Therefore, analysis can involve comparing the decoded intra-coded frame 200 with one or more previously decoded intra-coded frames.

[0083] Furthermore, after intra-coded frame 200 and multiple inter-coded frames 201, 202, 203, and 204, when a new intra-coded frame is encountered in the video sequence, the new intra-coded frame can be decoded, and the analysis S1050 can be updated based on the decoded new intra-coded frame. Subsequently, the reading of coding mode information S1020, the discovery of skipped-coded blocks S1030, and the designation of static regions S1040 can be repeated on multiple new inter-coded frames following the new intra-coded frame.

[0084] Figure 3 It shows the relationship with Figure 1 Method Example 10, compared to the second method example 30 which includes further optional steps, or the identification of stationary regions in frames of a video sequence. Just as with method example 10, the second method example 30 can be implemented in video processing unit 504 (see...). Figure 5 Implemented in ).

[0085] The receiving of the encoded version of the video sequence (S1010) and the reading of the encoding mode information (S1020) can be combined as described above. Figure 1 Perform as described in Method Example 10.

[0086] Alternatively, at S3022, it can be, for example, at video processing unit 504 (see...) Figure 5 The decoded video sequence at point ) is the frame of encoding version 20 (see) Figure 2 ). Figure 4 An example of this decoded frame 208 is shown. As a concrete example, Figure 4 The figure 402 and the ball 404 are depicted schematically. Decoded frame 208 can be used with intra-coded frame 200 (see [link]). Figure 2Corresponding to ). Alternatively, frame 208 can be associated with inter-frame coded frames 201, 202, 203, and 204 (see Figure 2 Corresponding to.

[0087] At S3024, optionally, object detection can be performed within frames of the video sequence. Object detection can optionally be performed relative to intra-coded frame 200 and / or inter-coded frame as decoded at S3022, and optionally, it can also be performed on previously decoded intra-coded frames. Thus, object detection can involve comparing such decoded intra-coded frame 200 with one or more previously decoded intra-coded frames.

[0088] For example, in video processing unit 504 (see...) Figure 5 ) Perform object detection at the location.

[0089] Object detection can be based on foreground detection algorithms. Foreground detection algorithms, as known in the art itself, can analyze a sequence of image frames to identify the movement of objects between image frames. For example, object detection may involve selecting an existing background model to see which parts of frame 208 remain unchanged (involving, for example, the use of a threshold), or filtering out known changes due to time of day or changes in image configuration (such as exposure or gain).

[0090] The foreground detection algorithm can therefore, for example, compare the decoded intra-coded frame 200 with one or more previous intra-coded frames. One or more objects 402, 404 that are found to have moved can be designated as foreground objects by the foreground detection algorithm. Conversely, objects that have not moved can be designated as background objects.

[0091] Alternatively, object detection can be based on a classification algorithm as known in the art itself, which can analyze the decoded frame 208 to detect objects such as people 402 (see [link to relevant documentation]). Figure 4 ), Ball 404 (see Figure 4 ) or specific types of objects to be tested, such as suitcases.

[0092] Typically, any other method or algorithm known in the art can be used to perform object detection.

[0093] As in Figure 4 As shown, object detection can produce one or more identified objects, such as a person 402 or a ball 404. Figure 4 In the example, the found static region 406 corresponds to a pixel block partially located within region 405, which includes the object of interest 404 (in this example, a soccer ball 404).

[0094] At S3026, at least one region 403, 405 of the frame of the video sequence is defined, and each defined region includes an object of interest 402, 404. The frame of the video sequence may be a decoded frame 208.

[0095] The definition of at least one region can be performed, for example, at the video processing unit 504.

[0096] At least one region can be defined based on optional object detection at S3024. Figure 4 Regions 403 and 405, schematically shown, can typically have any geometry. Object detection can generally return bounding boxes, as is known in the art, forming rectangles that include the object of interest. The bounding boxes then constitute the region including the object of interest.

[0097] Alternatively, for example, object detection can return a heatmap corresponding to the probability of regions corresponding to the same object of interest. Regions including the object of interest can then be generated from the contours of the heatmap, or bounding boxes can then be created based on the heatmap.

[0098] Alternatively, the definition of at least one region can be based on some other data relating to the content of frames in the video sequence. In one example, the definition can be based on external sensor 508, such as an IR detector (see [link to relevant documentation]). Figure 5 In this case, data from external sensor 508 can provide information such as that from scene 514 (see [reference]). Figure 5 The location information of the object of interest to a person in the video sequence can be used to define the region of interest that includes the object of interest in the frame of the video sequence.

[0099] As another example, the definition of at least one region 403, 405 can be based on an image sensor 512 from the same camera, which is a different sensor than the one on which the frames of the video sequence are based (see [link]). Figure 5 (Data).

[0100] The discovery of one or more skipped-encoding pixel blocks S1030 and the designation of one or more static regions S1040 can be combined as described above. Figure 1 Perform as described in method example 10.

[0101] At S3050, the region 406 designated as stationary at S1040 is evaluated relative to regions 403 and 405 that include objects of interest 402 and 404. Specifically, it is checked whether each designated stationary region 406 is at least partially located within regions 405 and 405 that include objects of interest 402 and 404. If so, at S3060, each such object of interest is designated as a stationary region. Optionally, this designation can only be performed if multiple consecutive frames corresponding to the designated stationary region 406 directly follow the decoded frame 208.

[0102] Optionally, if each corresponding timer 408 corresponding to each identified designated static region exceeds a predetermined threshold, then only the designated S3060 for the static object can be executed.

[0103] Optionally, it may be necessary that the entire region including the object of interest must be covered by one or more designated static regions 406. Alternatively, it may be necessary that at least one designated static region 406 is at least partially located within the region including the objects of interest 402, 404. Again, alternatively, it may be necessary that a specific portion of the region including the objects of interest 403, 405 is covered by one or more designated static regions 406, or that a predetermined number of designated static regions 406 are at least partially located within the region including the objects of interest 403, 405.

[0104] exist Figure 4 In the example, the found static region 406 corresponds to a pixel block partially located within region 405, which includes the object of interest 404 (in this example, a soccer ball 404). Therefore, it is reasonable to infer that the soccer ball 404 is a static object, as it is highly likely that no change occurred in the region corresponding to pixel block 406. Therefore, in Figure 4 In the example, the object of interest 404 is specified as a static object.

[0105] Optionally, an object of interest can be designated as stationary if the associated timer 408 exceeds a predetermined threshold, which ensures, for example, that the designated stationary region has remained stationary for at least a minimum number of time units or the number of inter-coded frames.

[0106] Optionally, at S3070, the video data of the frames in the video sequence can be modified so that the video data in the identified still areas is less prominent than the video data in other areas. Alternatively or additionally, this data can be color-coded.

[0107] Therefore, the modified video data can be displayed, for example, on monitor 506 (see...). Figure 5 )superior.

[0108] The salience level of a pixel block in a still region within a frame of a video sequence can be set according to a timer maintained for that pixel block. For example, the salience level can decrease as the timer value increases. Furthermore, for example, video data in still regions can be made less salient by reducing brightness, reducing chroma, and / or reducing opacity.

[0109] The above teachings can be implemented by a computer. For this purpose, a non-transitory computer-readable storage medium can be provided. The non-transitory computer-readable storage medium includes instructions stored thereon that, when executed on a processing device, are used to implement the method examples.

[0110] The non-transitory computer-readable storage medium and / or processing-capable device may be part of the video processing unit 504 or may be communicatively connected to the video processing unit 504, for example, via electrical and / or wireless connections. The non-transitory computer-readable storage medium and / or processing-capable device may be physically connected to and / or integrated with the video processing unit 504.

[0111] Furthermore, when practicing the claimed invention, a person skilled in the art can understand and implement variations of the disclosed embodiments by studying the drawings, the disclosure, and the appended claims. For example, in addition to the examples already given, method steps presented sequentially for simplicity can also be performed in parallel, and method steps presented as being performed in parallel with other steps can also be performed sequentially with other steps, while still falling within the scope of the appended claims.

Claims

1. A method for identifying stationary regions in frames of a video sequence, the method comprising: Receive an encoded version of the video sequence, wherein the encoded version of the video sequence includes intra-coded frames, which are followed by a plurality of inter-coded frames; Read the encoding mode information in the inter-coded frame of the encoded version of the video sequence, wherein the encoding mode information indicates that a pixel block in the inter-coded frame is skipped from encoding, and wherein the skipped pixel block is a pixel block that is encoded as a copy of a spatially overlapping pixel block in the previous frame of the video sequence. Using the read encoding mode information, one or more pixel blocks were found to have been skipped from encoding in the corresponding multiple consecutive frames of the encoded version of the video sequence. Maintain one or more timers, each timer corresponding to a pixel block and indicating how many consecutive frames, including the current inter-frame encoded frame, have the pixel block skipped encoding; Each discovered pixel block is identified as a stationary region in the corresponding plurality of consecutive frames; and The video data of the frames in the video sequence that include the static region is modified to make the video data in the identified static region less significant than the video data in other regions. This is achieved by reducing the brightness, chroma, and / or opacity of the identified static region to make the video data in the identified static region less significant. The salience level of a pixel block in a static region of a frame in the video sequence is set according to a timer maintained for the pixel block, wherein the salience level decreases as the value of the timer increases.

2. The method according to claim 1, further comprising: Video data in the frames of the video sequence is analyzed to detect motion and / or identify foreground objects, wherein the static regions of the frames are excluded from the analysis.

3. The method according to claim 1, wherein, The maintenance includes incrementing the timer corresponding to the pixel block if the corresponding pixel block that was skipped in the previous inter-coded frame is found to be skipped in the current inter-coded frame.

4. The method according to claim 1, wherein, The maintenance includes: if the corresponding pixel block that was skipped in the previous inter-coding frame is not found to be skipped in the current inter-coding frame, then resetting the timer corresponding to the pixel block.

5. The method of claim 1, further comprising: One or more frames relative to the video sequence define a region including the object of interest; Discover one or more identified static regions, each identified static region being at least partially located within the region including the object of interest; as well as The object of interest is identified as a static object.

6. The method of claim 5, further comprising: Decode the frames of the encoded version of the video sequence. in: The definition of the region including the object of interest is performed relative to the decoded frame, and The plurality of consecutive frames corresponding to the identified static region directly follow the decoded frame.

7. The method according to claim 1, wherein: The reading of the encoding mode information, the discovery of the one or more pixel blocks that have all been skipped from encoding, and the identification of each discovered pixel block as a stationary region are performed sequentially for each of the plurality of inter-coded frames.

8. The method according to claim 1, wherein, Identifying the discovered pixel blocks as stationary regions includes identifying the discovered pixel blocks as spatial regions of the frame in which the video data has not changed directly since the previous frame of the video sequence.

9. The method according to claim 1, wherein, The encoding mode information is read from pixel blocks and / or inter-frame encoded frame headers in the inter-frame encoded version of the video sequence.

10. An apparatus for identifying stationary regions in frames of a video sequence, the apparatus comprising circuitry configured to: Receive the encoded version of the video sequence, wherein, The encoded version of the video sequence includes intra-coded frames, which are followed by a plurality of inter-coded frames; Read the encoding mode information in the inter-coded frame of the encoded version of the video sequence, wherein the encoding mode information indicates that a pixel block in the inter-coded frame is skipped from encoding, and wherein the skipped pixel block is a pixel block that is encoded as a copy of a spatially overlapping pixel block in the previous frame of the video sequence. Using the read encoding mode information, one or more pixel blocks were found to be skipped in encoding in corresponding consecutive frames of the encoded version of the video sequence. Maintain one or more timers, each timer corresponding to a pixel block and indicating how many consecutive frames, including the current inter-frame encoded frame, have the pixel block skipped encoding; Each discovered pixel block is identified as a stationary region in the corresponding plurality of consecutive frames; and The video data of the frames in the video sequence that include the static region is modified to make the video data in the identified static region less significant than the video data in other regions. This is achieved by reducing the brightness, chroma, and / or opacity of the identified static region to make the video data in the identified static region less significant. The salience level of a pixel block in a static region of a frame in the video sequence is set according to a timer maintained for the pixel block, wherein the salience level decreases as the value of the timer increases.

11. A non-transitory computer-readable medium having stored thereon computer code instructions, which, when executed by a processing-capable device, are adapted to perform the method according to claim 1.

Images