Still frame detection in video data

Abstract

Still frame detection for single-pass video data, comprising: determining that an average quantization parameter of a frame of the video data falls below a quantization parameter threshold; determining whether an amount of skipped macroblocks in the frame satisfies a skipped macroblock threshold; and identifying the frame as a still frame in response to the amount of skipped macroblocks exceeding the skipped macroblock threshold.

Description

[Background technology]

[0001] When two or more consecutive frames have identical or very similar content, such frames are referred to as still frames, still images, or static scenes. When still frames are encoded by referencing other frames, the still frames require fewer bits to encode and generally do not use all the bits allocated for their encoding. The accumulated bits result in reduced quantization parameters for encoding, which can increase the size of the encoded still frames over time without resulting in any visually perceptible improvement in quality. [Brief explanation of the drawing]

[0002] [Figure 1] This is an exemplary computer block diagram for still frame detection in video data, according to several embodiments. [Figure 2] This is a flowchart illustrating an exemplary method for detecting still frames in video data, according to several embodiments. [Figure 3] This is a flowchart illustrating an exemplary method for detecting still frames in video data, according to several embodiments. [Figure 4] This is a flowchart illustrating an exemplary method for detecting still frames in video data, according to several embodiments. [Figure 5] This is a flowchart illustrating an exemplary method for detecting still frames in video data, according to several embodiments. [Figure 6] This is a flowchart illustrating an exemplary method for detecting still frames in video data, according to several embodiments. [Figure 7] This is a flowchart illustrating an exemplary method for detecting still frames in video data, according to several embodiments. [Modes for carrying out the invention]

[0003] In some embodiments, a method for detecting still frames in video data includes determining whether the average quantization parameter of a frame in the video data is below a quantization parameter threshold; determining whether the amount of skipped macroblocks in the frame satisfies a skipped macroblock threshold; and identifying the frame as a still frame depending on whether the amount of skipped macroblocks exceeds the skipped macroblock threshold.

[0004] In some embodiments, the method further includes determining that the amount of interframe encoded macroblocks in a frame satisfies an interframe macroblock threshold, in that the amount of skipped macroblocks is below a skipped macroblock threshold; determining that the rate distortion cost ratio of the skipped macroblocks in a frame to the unskipped macroblocks in a frame is below a rate distortion cost ratio threshold; and identifying the frame as a static frame, in that the rate distortion cost ratio is below a rate distortion cost ratio threshold. In some embodiments, the method further includes modifying one or more encoding attributes for the next frame in that the frame has been identified as a static frame; and encoding the next frame according to one or more modified encoding attributes. In some embodiments, the method further includes generating data indicating the frame as a static frame. In some embodiments, the method further includes skipping one or more identified static frames during the analysis of the video data based on the data. In some embodiments, the video data includes single-pass video data. In some embodiments, the average quantization parameter includes the average of each quantization parameter used to encode each macroblock in a frame.

[0005] In some embodiments, an apparatus for detecting still frames in video data performs the steps of determining whether the average quantization parameter of a frame in the video data is below a quantization parameter threshold; determining whether the amount of skipped macroblocks in the frame satisfies a skipped macroblock threshold; and identifying the frame as a still frame depending on whether the amount of skipped macroblocks exceeds the skipped macroblock threshold.

[0006] In some embodiments, the step further includes determining that the amount of interframe encoded macroblocks in a frame satisfies an interframe macroblock threshold, in that the amount of skipped macroblocks is below a skipped macroblock threshold; determining that the rate distortion cost ratio of the skipped macroblocks in a frame to the unskipped macroblocks in a frame is below a rate distortion cost ratio threshold; and identifying the frame as a static frame, in that the rate distortion cost ratio is below a rate distortion cost ratio threshold. In some embodiments, the step further includes modifying one or more encoding attributes for the next frame in that the frame has been identified as a static frame; and encoding the next frame according to one or more modified encoding attributes. In some embodiments, the step further includes generating data indicating the frame as a static frame. In some embodiments, the step further includes skipping one or more identified static frames during the analysis of the video data based on the data. In some embodiments, the video data includes single-pass video data. In some embodiments, the average quantization parameter includes the average of each quantization parameter used to encode each macroblock in a frame.

[0007] In some embodiments, a computer program product located on a non-temporary computer-readable storage medium includes a computer program instruction for detecting still frames in video data, which, when executed, causes a computer system to perform the steps of: determining that the average quantization parameter of a frame in the video data is below a quantization parameter threshold; determining whether the amount of skipped macroblocks in the frame satisfies a skipped macroblock threshold; and identifying the frame as a still frame depending on whether the amount of skipped macroblocks exceeds the skipped macroblock threshold.

[0008] In some embodiments, the step further includes determining that the amount of interframe encoded macroblocks in a frame satisfies an interframe macroblock threshold, in that the amount of skipped macroblocks is below a skipped macroblock threshold; determining that the rate distortion cost ratio of the skipped macroblocks in a frame to the unskipped macroblocks in a frame is below a rate distortion cost ratio threshold; and identifying the frame as a static frame, in that the rate distortion cost ratio is below a rate distortion cost ratio threshold. In some embodiments, the step further includes modifying one or more encoding attributes for the next frame in that the frame has been identified as a static frame; and encoding the next frame according to one or more modified encoding attributes. In some embodiments, the step further includes generating data indicating the frame as a static frame. In some embodiments, the step further includes skipping one or more identified static frames during the analysis of the video data based on the data. In some embodiments, the video data includes single-pass video data.

[0009] In video coding, particularly single-pass video coding, a rate control algorithm allocates the amount of bits to encode each frame. For example, a rate control algorithm allocates bits to frames based on the target bitrate of the video data as a whole. The rate control algorithm then determines the quantization parameter (QP) for encoding the frame or for each block of the frame, based on the allocated bits. The quantization parameter defines the degree of quantization applied when encoding a frame (e.g., per frame, per block). Therefore, a higher quantization parameter indicates that a higher degree of quantization is applied, resulting in increased compression, reduced quality, and smaller file size.

[0010] When two or more consecutive frames have identical or very similar content, such frames are referred to as still frames, still images, or static scenes. For example, a video of a scene without human activity will have identical content for most of its parts. When several consecutive still frames are encoded within video data, the still frames can be encoded as interframes referencing other frames. Therefore, still frames require fewer bits to encode and generally do not use all of the bits allocated for their encoding. In existing solutions, especially for single-pass video encoding, this reduces the QP by a rate-controlled algorithm so that more of the allocated bits are used. As a result, encoded still frames will have a high Peak Signal-to-Noise Ratio (PSNR). When the period of a still frame ends and frames with more movement are encoded, the encoder does not know that the period of the still frame has ended, so the rate-controlled algorithm encodes these frames with a smaller QP. By using a low QP for frames with motion, the encoder quickly exhausts its allocated bits and triggers an increased QP for subsequent encoded macroblocks within the frame to keep bit usage close to the target. This can result in various perceptible quality differences, such as frames with a high-quality upper portion (due to the macroblocks being encoded with a low QP) and a low-quality lower portion (due to the latter macroblocks being encoded with a high QP).

[0011] In particular, for still frames, there is no visual quality benefit from increased bit allocation and high PSNRs above a certain point (e.g., 40 dB for PSNR). In other words, once a still frame reaches a certain PSNR, allocating additional bits to further increase the PSNR does not result in a noticeable quality improvement in the still frame. Conversely, for motion-captured frames, since the PSNR of motion frames is typically lower than that of still frames, the PSNR gain from increased bit allocation results in a perceptible quality improvement. Therefore, it is advantageous to identify still frames during video encoding, especially in single-pass video encoding applications, in order to control the allocation of bits to subsequent encoded frames. Identifying still frames in video data also provides additional benefits. For example, if some analysis needs to be performed on the video data (e.g., security or surveillance footage analysis), removing or skipping identified still frames improves the efficiency of the analysis by reducing the number of frames to be analyzed.

[0012] The static frame detection in video data according to this application is generally implemented using a computer, i.e., an automated computing machine. Therefore, for further explanation, Figure 1 shows a block diagram of a computing machine including an exemplary computer 100 configured for static frame detection in video data according to a particular embodiment. The exemplary computer 100 and the method described herein can be used in a variety of devices or scenarios, including personal computers, security devices, video cameras, mobile devices, game devices, set-top boxes, etc. The computer 100 in Figure 1 includes at least one computer processor 102 or "CPU", and random access memory 104 ("RAM") connected to the processor 102 and other components of the computer 100 via a high-speed memory bus 106 and a bus adapter 108.

[0013] The RAM 104 stores the operating system 110. Operating systems useful in a computer configured for still frame detection in video data according to a particular embodiment include UNIX®, Linux®, Microsoft Windows®, and others conceivable by those skilled in the art. In the example in Figure 1, the operating system 110 is shown in RAM 104, but many components of such software are typically also stored in non-volatile memory such as data storage 112, such as a disk drive. The RAM also stores an encoding module 114a, which is a module for still frame detection in video data according to a particular embodiment.

[0014] The computer 100 in Figure 1 includes a disk drive adapter 116 coupled to a processor 102 and other components of the computer 100 via an expansion bus 118 and a bus adapter 108. The disk drive adapter 116 connects non-volatile data storage to the computer 100 in the form of data storage 112. Disk drive adapters useful in a computer configured for still frame detection in video data according to a particular embodiment include Integrated Drive Electronics ("IDE") adapters, Small Computer System Interface ("SCSI") adapters, and others conceivable by those skilled in the art. In some embodiments, non-volatile computer memory is implemented as an optical disk drive, electrically erasable programmable read-only memory (so-called "EEPROM" or "flash" memory), RAM drive, etc., as conceivable by those skilled in the art.

[0015] The exemplary computer 100 in Figure 1 includes one or more input / output ("I / O") adapters 120. The I / O adapters implement user-oriented input / output through software drivers and computer hardware to control, for example, output to a display device such as a computer display screen, and user input from a user input device 122 such as a keyboard and mouse. The exemplary computer 100 in Figure 1 includes a video adapter 124, which is an example of an I / O adapter specifically designed for graphics output to a display device 126 such as a display screen or computer monitor. The video adapter 124 is connected to the processor 102 via a high-speed video bus 128, a bus adapter 108, and a front-side bus 130, which is also a high-speed bus.

[0016] The exemplary computer 100 in Figure 1 includes a communication adapter 132 for data communication with other computers and with data communication networks. Such data communication is performed serially, via RS-232 connections, via external buses such as the Universal Serial Bus ("USB"), via data communication networks such as IP data communication networks, and / or by other means conceivable by those skilled in the art. The communication adapter implements hardware-level data communication, enabling one computer to transmit data communication directly to another computer or via a data communication network. Examples of communication adapters useful in a computer configured for still frame detection in video data according to a particular embodiment include modems for wired dial-up communication, Ethernet® (IEEE 802.3) adapters for wired data communication, and 802.11 adapters for wireless data communication.

[0017] To perform static frame detection in video data, computer 100 (e.g., encoding module 114) encodes frames of video data. Frames of video data are encoded according to several previously determined bit assignment and quantization parameters. In some embodiments, frames are encoded as interframes referencing one or more previously encoded frames. To begin determining whether a frame is a static frame, encoding module 114 determines whether the average QP of the frame is lower than a QP threshold. For example, suppose a frame is encoded as a plurality of macroblocks, each macroblock containing a contiguous portion of pixels in the frame. As described herein, a macroblock is a processing unit or functional unit of a frame and is not limited to or restricted by any particular video standard. Therefore, steps or processes described herein as being performed with respect to a macroblock may also be performed on frames or other functional units or processing units of video coding. Each macroblock is encoded according to the determined QP. Therefore, the average QP of the frame (QP avg ) is the average of all QPs used to encode each macroblock in the frame. QP threshold (QP thr ) is a predefined threshold. For example, in some embodiments, the QP threshold is selected as a threshold that produces a desired video quality (e.g., subjectively "good" video quality). For example, the QP threshold may be selected as 25 or as another value selected according to engineering or design considerations as can be understood.

[0018] For example, QP avg >QP thr If that is the case, the frame will not be identified as a static frame. Instead, QP avg ≤QP thr If so, the encoding module 114 proceeds to the next step in determining whether the encoded frame is a stationary frame. Next, the encoding module 114 determines the amount of skipped macroblocks (MB) in the frame. skip) determines whether it meets a predefined threshold value (e.g., the skipped macroblock threshold (MB skip_thr )). As described above, when a frame is encoded as an inter-frame that references a previously encoded frame, one or more macroblocks within the frame are encoded as references to one or more corresponding macroblocks in the previous frame. A skipped macroblock (e.g., a SKIP macroblock) is a specific type of inter-frame macroblock encoding that references a previously encoded macroblock. For example, a skipped macroblock is encoded by referencing a previously encoded macroblock without coding any residual error or motion vector. The decoder of the video data then estimates the motion vector of the skipped macroblock based on the motion vectors of other decoded macroblocks. The encoder usually encodes a macroblock as a skipped macroblock due to either of two scenarios: the macroblock to be encoded is the same as its reference, or the QP for encoding the macroblock is high and it does not have enough bits for encoding. MB skip ≥MB skip_thr The evaluation of whether is performed when QP avg ≤QP thr is, thus reducing or eliminating the possibility of the latter scenario.

[0019] In some embodiments, the amount MB of skipped macroblocks within a frame skip is defined as a percentage or ratio of the skipped macroblocks to other macroblocks within the frame. Thus, MB skip_thr is defined as a percentage or ratio threshold. Those skilled in the art will understand that in some embodiments, MB skip is expressed according to another method and MB skip_thr is correspondingly defined. When MB skip meets MB skip_thr (e.g., MB skip ≥MB skip_thrThe encoding module 114 identifies the frame as a static frame.

[0020] MB skip MB skip_thr If the condition is not met (for example, MB skip <MB skip_thr ), however, there is still a possibility that the frame is a static frame. For example, in some embodiments, frames are encoded primarily using interframe macroblock coding other than skipped macroblocks. For example, if frames are encoded according to a high bitrate, even if the frame is very similar to the previous frame, the encoder will encode some macroblocks as unskipped macroblocks because of the large amount of bits available. Thus MB skip <MB skip_thr In this case, the encoding module 114 encodes the amount of interframe encoding macroblocks of the frame in MB. inter The inter-frame macroblock threshold is MB inter_thr Determine whether the condition is met. In some embodiments, the amount of interframe coding macroblocks of the frame is MB. inter MB is defined as the percentage or ratio of an interframe coded macroblock to other macroblocks within a frame. Therefore, MB inter_thr This is defined as a percentage or ratio threshold. Those skilled in the art will know that in some embodiments, MB inter This is expressed according to a different method, MB inter_thr They will understand that it is defined appropriately.

[0021] MB inter MB inter_thr If it falls below (for example, MB) inter <MB inter_thr ), the encoding module 114 identifies the frame as a non-static frame. Rather, MB inter ≥MB inter_thrIf so, the encoding module 114 compares the rate distortion cost (RDCost) of the unskipped macroblock with the RDCost of the skipped macroblock, and then calculates the RDRatio. non-skip Determine the total RDCost(RDCostSkip) of skipped macroblocks in the frame. For example, the encoding module 114 determines the total RDCost(RDCostSkip) of skipped macroblocks in the frame. frame ) and the total RDCost(RDCostInter) of all interframe macroblocks within a frame (including skipped and non-skipped interframe macroblocks). frame The encoding module 114 calculates the average RDCost(RDCostSkip) of the skipped macroblocks. MB ) to RDCostSkip frame The calculation is performed as / Num_Skip_MB, where Num_Skip_MB is the number of skipped macroblocks in the frame. The encoding module 114 then calculates the average RDCost(RDCostNonSkip) of non-skipped interframe macroblocks. MB ) to (RDCostInter frame -RDCostSkip frame The calculation is performed as ) / (Picture_size_in_MBs-Num_Skip_MB), where Picture_size_in_MBs is the total number of macroblocks in the frame. Then, the ratio RDRatio is calculated. non-skip (RDCostNonSkip MB -RDCostSkip MB ) / (RDCostNonSkip MB The encoding module 114 is then calculated as the ratio RDRatio. non-skip threshold RDRatio non-skip_thr Compare with RDRatio. non-skip If it falls below the threshold (for example, RDRatio non-skip <RDRatio non-skip_thr ), the frame is identified as a static frame.

[0022] In some embodiments, the encoding module 114 takes one or more actions depending on whether it has identified a frame as a static frame. For example, in some embodiments such as single-pass video encoding, the encoding module 114 modifies one or more encoding attributes of the next frame to be encoded (e.g., frames that occur consecutively after a frame identified as a static frame). As an example, the QP for encoding the next frame or at least a portion of the next frame (e.g., one or more macroblocks of the next frame) is changed. In some embodiments, changing the QP includes increasing the QP or preventing a decrease in the QP, as determined by the rate control algorithm. Thus, fewer bits are used for the next frame. If a series of frames is identified as static frames, reducing the bit allocation for this series of frames allows additional bits to be allocated to motion frames that are later encoded, thereby improving the quality of motion frames while maintaining the target bitrate.

[0023] As another example of actions taken by the encoding module 114 in response to the identification of a still frame, in some embodiments, the encoding module 114 generates data indicating the still frame. For example, tags or other metadata in the video data indicate that a frame is a still frame. The data indicating the frame and potentially other frames as still frames facilitates various actions by the encoding module 114 or another service, so as can be understood. For example, in some embodiments, one or more identified still frames are skipped in the video data. Therefore, when performing manual or computer-aided analysis of the video data, the total amount of video to be analyzed is reduced. For example, assuming the video data contains security footage, bypassing still frames helps in reviewing the security footage, as still frames are unlikely to contain important information.

[0024] According to the method described above, as a result of various comparisons, a frame will be identified as a static frame. Therefore, as a result of such comparisons, if the comparison produces the opposite result, the frame will also be identified as a non-static frame (e.g., a motion frame). For example, QP avg >QP thr In that case, the frame is not identified as a static frame. Another example is QP avg ≤QP thr However, MB skip <MB skip_thr and MB inter <MB inter_thr In that case, the frame is not identified as a static frame. As a further example, QP avg ≤QP thr and MB skip <MB skip_thr , and MB inter ≥MB inter_thr However, RDRatio non-skip ≥RDRatio non-skip_thr If this is the case, the frame is not identified as a static frame. If the frame is not identified as a static frame, in some embodiments, the coding module 114 allows the rate control algorithm to change the QP of the next frame as needed. Those skilled in the art will understand that certain changes to the QP or other coding attributes vary according to the specific rate control algorithm being implemented.

[0025] For further explanation, Figure 2 shows a flowchart illustrating an exemplary method for detecting still frames in video data, which includes determining (e.g., by encoding module 114) that the average quantization parameter of a frame in the video data is below a quantization parameter threshold.202 In some embodiments, a frame is encoded as a frame-to-frame relationship that references one or more previously encoded frames. Assume that a frame is encoded as a plurality of macroblocks, each macroblock containing a contiguous portion of pixels in the frame. Each macroblock is encoded according to the determined QP. Thus, the average QP of the frame is (QP avg) is the average of all QPs used to encode each macroblock in the frame. QP threshold (QP thr ) is a predefined threshold. For example, in some embodiments, the QP threshold is selected as a threshold that produces a desired video quality (e.g., subjectively "good" video quality). For example, the QP threshold may be selected as 25, or as another value selected according to engineering or design considerations as can be understood.

[0026] Furthermore, the method in Figure 2 shows the amount of skipped macroblocks (MB) within a frame. skip ) is the skipped macroblock threshold (MB skip_thr This includes determining (for example, by encoding module 204) whether the following condition is met. As described above, if a frame is encoded as an interframe that references a previously encoded frame, one or more macroblocks within the frame are encoded as references to one or more corresponding macroblocks in the previous frame. A skipped macroblock (e.g., a SKIP macroblock) is a particular type of interframe macroblock encoding that references a previously encoded macroblock. For example, a skipped macroblock is encoded by referencing a previously encoded macroblock without coding any residual errors or motion vectors. In some embodiments, the amount of skipped macroblocks in a frame is MB skip MB is defined as the percentage or ratio of skipped macroblocks to other macroblocks within the frame. Therefore, MB skip_thr This is defined as a percentage or ratio threshold. Those skilled in the art will know that in some embodiments, MB skip This is expressed according to a different method, MB skip_thr You will understand that this is appropriately defined. If the amount of skipped macroblocks in a frame satisfies the skipped macroblock threshold, the method in Figure 2 proceeds to identify the frame as a static frame (e.g., by encoding module 114) 206. In other words, MBskip MB skip_thr If the conditions are met (for example, MB skip ≥MB skip_thr The encoding module 114 identifies the frame as a static frame.

[0027] For further explanation, Figure 3 shows a flowchart illustrating an exemplary method for detecting still frames in video data, which includes determining (e.g., by coding module 114) that the average quantization parameter of a frame in the video data is below a quantization parameter threshold 202; determining (e.g., by coding module 204) whether the amount of skipped macroblocks in the frame satisfies a skipped macroblock threshold 204; and identifying the frame as a still frame (e.g., by coding module 114, depending on whether the amount of skipped macroblocks in the frame satisfies the skipped macroblock threshold) 206.

[0028] The method in Figure 3 determines that if the mount of a skipped macroblock in the frame falls below the skipped macroblock threshold, then 204 (for example, MB) skip <MB skip_thr In this case, the method in Figure 3 differs from Figure 2 in that it proceeds to determining 302 that the amount of interframe coded macroblocks in a frame satisfies the interframe macroblock threshold. In some embodiments, the amount of interframe coded macroblocks in a frame MB inter MB is defined as the percentage or ratio of an interframe coded macroblock to other macroblocks within a frame. Therefore, MB inter_thr This is defined as a percentage or ratio threshold. Those skilled in the art will know that in some embodiments, MB inter This is expressed according to a different method, MB inter_thr They will understand that it is defined appropriately.

[0029] Also, the method in FIG. 3 includes determining 304 that the rate distortion cost ratio of the frame-skipped macroblock to the non-frame-skipped macroblock of the frame is below the rate distortion cost ratio threshold (e.g., according to MB inter ≧MB inter_thr , by the encoding module 114). For example, the encoding module 114 calculates the total RDCost (RDCostSkip frame ) of the skipped macroblocks in the frame and the total RDCost (RDCostInter frame ) of all the inter-frame macroblocks (including skipped and non-skipped inter-frame macroblocks) in the frame. The encoding module 114 calculates the average RDCost (RDCostSkip MB ) of the skipped macroblocks as RDCostSkip frame / Num_Skip_MB, where Num_Skip_MB is the number of skipped macroblocks in the frame. Also, the encoding module 114 calculates the average RDCost (RDCostNonSkip MB ) of the non-skipped inter-frame macroblocks as (RDCostInter frame - RDCostSkip frame ) / (Picture_size_in_MBs - Num_Skip_MB), where Picture_size_in_MBs is the total number of macroblocks in the frame. Then, the ratio RDRatio non-skip is calculated as (RDCostNonSkip MB - RDCostSkip MB ) / RDCostNonSkip MB . The encoding module 114 then compares the ratio RDRatio non-skip with the threshold RDRatio non-skip_thr . Also, the method in FIG. 3 is based on the fact that the rate distortion cost ratio is below the rate distortion cost threshold (e.g., RDRatio non-skip <RDRatio non-skip_thr), including identifying the frame as a static frame (for example, by encoding module 114) 306.

[0030] For further explanation, Figure 4 shows a flowchart illustrating an exemplary method for detecting still frames in video data, which includes determining (e.g., by coding module 114) that the average quantization parameter of a frame in the video data is below a quantization parameter threshold 202, determining (e.g., by coding module 204) whether the amount of skipped macroblocks in the frame satisfies a skipped macroblock threshold 204, and identifying the frame as a still frame (e.g., by coding module 114, depending on whether the amount of skipped macroblocks in the frame satisfies the skipped macroblock threshold) 206.

[0031] The method in Figure 4 differs from that in Figure 2 in that it includes changing (modifying) one or more coding attributes of the next frame 402. For example, the QP for encoding the next frame or at least a portion of the next frame (e.g., one or more macroblocks of the next frame) is changed. In some embodiments, changing the QP includes increasing the QP or preventing a decrease in the QP, as determined by the rate control algorithm. Thus, fewer bits are used for the next frame. If a series of frames is identified as static frames, reducing the bit allocation for this series of frames allows additional bits to be allocated to motion frames that are later encoded, thereby improving the quality of motion frames while maintaining the target bitrate. The method in Figure 4 also includes encoding the next frame according to one or more modified coding attributes 404.

[0032] For further explanation, Figure 5 shows a flowchart illustrating an exemplary method for detecting still frames in video data, which includes determining (e.g., by encoding module 114) that the average quantization parameter of a frame in the video data is below a quantization parameter threshold 202; determining (e.g., by encoding module 204) whether the amount of skipped macroblocks in the frame satisfies a skipped macroblock threshold 204; and identifying the frame as a still frame (e.g., by encoding module 114, depending on whether the amount of skipped macroblocks in the frame satisfies the skipped macroblock threshold) 206.

[0033] The method in Figure 5 differs from that in Figure 2 in that it includes generating data 502 that indicates a frame as a still frame. For example, the encoding module 114 generates video data tags or other metadata that indicate a frame is a still frame. As additional frames are encoded and potentially identified as still frames, the encoding module 114 generates video data that indicates one or more identified still frames.

[0034] For further explanation, Figure 6 shows a flowchart illustrating an exemplary method for detecting still frames in video data, which includes determining (e.g., by coding module 114) that the average quantization parameter of a frame in the video data is below a quantization parameter threshold 202; determining (e.g., by coding module 204) whether the amount of skipped macroblocks in the frame satisfies a skipped macroblock threshold 204; identifying the frame as a still frame (e.g., by coding module 114, depending on whether the amount of skipped macroblocks in the frame satisfies the skipped macroblock threshold) 206; and generating data indicating the frame as a still frame 502.

[0035] The method in Figure 6 differs from Figure 5 in that it involves skipping one or more identified still frames from the video data.602 For example, the encoding module 114 or another module accesses generated data that indicates the still frames in the video data, as can be understood. The identified still frames are then skipped (e.g., bypassed) from the video data. Thus, the total amount of video data to be analyzed is reduced, as still frames are less likely to contain information relevant to manual or machine-assisted analysis of the data.

[0036] For further explanation, Figure 7 shows a flowchart illustrating an exemplary method for detecting still frames in video data. Figure 7 is similar to Figure 3 in that it includes determining (for example, by the coding module 114) that the average quantization parameter of a frame in the video data is below a quantization parameter threshold 202; determining (for example, by the coding module 204) whether the amount of skipped macroblocks in the frame satisfies a skipped macroblock threshold 204; identifying the frame as a static frame (for example, by the coding module 114, depending on whether the amount of skipped macroblocks in the frame satisfies a skipped macroblock threshold) 206; determining 302 that the amount of interframe coded macroblocks in the frame satisfies an interframe macroblock threshold, depending on whether the amount of skipped macroblocks in the frame is below a skipped macroblock threshold; determining 304 that the rate distortion cost ratio of skipped macroblocks in the frame to unskipped macroblocks in the frame is below a rate distortion cost ratio threshold; and identifying the frame in the video data as a static frame, depending on whether the rate distortion cost ratio is below a rate distortion cost ratio threshold 306.

[0037] The method in Figure 7 differs from that in Figure 3 in that it includes changing (modifying) one or more coding attributes of the next frame 702. For example, the QP for encoding the next frame or at least a portion of the next frame (e.g., one or more macroblocks of the next frame) is changed. In some embodiments, changing the QP includes increasing the QP or preventing a decrease in the QP, as determined by a rate control algorithm. Thus, fewer bits are used for the next frame. If a series of frames is identified as static frames, reducing the bit allocation for this series of frames allows additional bits to be allocated to motion frames that are later encoded, thereby improving the quality of motion frames while maintaining the target bitrate. The method in Figure 7 also includes encoding the next frame according to one or more modified coding attributes 704.

[0038] Considering the above explanation, readers will recognize that still frame detection in video data includes the following advantages: ● Improving the performance of computing systems by changing the encoding of frames based on identified still frames, thereby maintaining or improving overall video quality by increasing the amount of bits allocated to non-static moving frames. ● Improved computing system performance by increasing computational efficiency when analyzing video data by skipping identified still frames.

[0039] The exemplary embodiments of this disclosure are primarily described in the context of a fully functional computer system for still frame detection in video data. However, readers skilled in the art will recognize that this disclosure can be embodied in a computer program product placed on a computer-readable storage medium for use with any suitable data processing system. Such a computer-readable storage medium may be any storage medium for machine-readable information, including magnetic media, optical media, or other suitable media. Examples of such media include magnetic disks in hard drives or floppy disks, compact disks for optical drives, magnetic tapes, and others conceived by those skilled in the art. Those skilled in the art will immediately recognize that any computer system with suitable programming means can perform steps of the methods of this disclosure embodied in a computer program product. Furthermore, while some of the exemplary embodiments described herein are directed towards software installed and run on computer hardware, there are still plenty of alternative embodiments within the scope of this disclosure that can be implemented as firmware or in hardware.

[0040] This disclosure may be a system, method, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions for causing a processor to execute aspects of this disclosure.

[0041] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium may, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any preferred combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically coded devices such as punch cards or grooved raised structures having instructions recorded thereon, and any preferred combination thereof. As used herein, a computer-readable storage medium should not be interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0042] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives computer-readable program instructions from the network and transmits them for storage in a computer-readable storage medium within each computing / processing device.

[0043] Computer-readable program instructions for performing the operations of the Disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk and C, and conventional procedural programming languages ​​such as the C programming language or similar programming languages. Computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer, partially on a remote computer, or fully on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it may be connected to an external computer (for example, via the Internet using an Internet Service Provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer-readable program instructions by individualizing the electronic circuit using state information of computer-readable program instructions in order to carry out an aspect of the present disclosure.

[0044] Aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present disclosure. It will be understood that each block in a flowchart and / or block diagram, and combinations of blocks in a flowchart and / or block diagram, can be implemented by computer-readable program instructions.

[0045] These computer-readable program instructions can be provided to the processor of a general-purpose computer, a dedicated computer, or another programmable data processing device so that instructions executed via the processor of the computer or other programmable data processing device generate a machine that creates means for performing functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions can be stored on a computer-readable storage medium on which the instructions stored thereon can be used to instruct a computer, a programmable data processing device, and / or other device to function in a particular way so that the storage medium contains a product containing instructions that implements a mode of function / action specified in one or more blocks of a flowchart and / or block diagram.

[0046] Furthermore, computer-readable program instructions can be loaded into a computer, another programmable data processing device, or other device to cause a series of operational steps to be performed on the computer, another programmable device, or other device in order to generate a computer implementation process in which instructions executed on the computer, another programmable device, or other device implement the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0047] The flow diagrams and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flow diagram or block diagram may represent a module, segment, or portion of instructions containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions described in a block may occur in a different order than shown in the figure. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or blocks may be executed in reverse order depending on the functions involved. It should also be noted that each block in a block diagram and / or flow diagram, and combinations of blocks in a block diagram and / or flow diagram, may be implemented by a dedicated hardware-based system that performs a specified function or action, or combines dedicated hardware with computer instructions.

[0048] It will be understood from the above description that modifications and changes can be made to various embodiments of this disclosure. The statements herein are for illustrative purposes only and should not be construed as restrictive. The scope of this disclosure is limited only by the following claims.

Claims

1. A method for detecting still frames in video data, This involves determining whether the average quantization parameter of the video data frames falls below the quantization parameter threshold, In response to the average quantization parameter falling below the quantization parameter threshold, it is determined whether the amount of skipped macroblocks in the frame satisfies the skipped macroblock threshold. The process includes identifying the frame as a static frame in accordance with the amount of skipped macroblocks exceeding the skipped macroblock threshold, method.

2. In accordance with the fact that the amount of skipped macroblocks falls below the skipped macroblock threshold, it is determined that the amount of interframe encoded macroblocks in the frame satisfies the interframe macroblock threshold. It is determined that the rate distortion cost ratio of the skipped macroblocks in the frame to the rate distortion cost ratio threshold is below the rate distortion cost ratio threshold. The process includes identifying the frame as the static frame in response to the rate distortion cost ratio falling below the rate distortion cost ratio threshold, The method according to claim 1.

3. Depending on whether the frame is identified as the static frame, one or more encoding attributes for the next frame are changed, Encoding the next frame according to one or more of the modified encoding attributes, The method according to claim 1.

4. This includes generating data that represents the frame as the static frame, The method according to claim 1.

5. Based on the aforementioned data, the analysis of the video data includes skipping one or more identified still frames. The method according to claim 4.

6. The aforementioned video problem includes single-pass video data. The method according to claim 1.

7. The average quantization parameter includes the average of each quantization parameter used to encode each macroblock of the frame, The method according to claim 1.

8. A device for detecting still frames in video data, Computer processors and The computer memory connected to the computer processor, The aforementioned computer memory is This involves determining whether the average quantization parameter of the video data frames falls below the quantization parameter threshold, In response to the average quantization parameter falling below the quantization parameter threshold, it is determined whether the amount of skipped macroblocks in the frame satisfies the skipped macroblock threshold. The frame is identified as a static frame when the amount of skipped macroblocks exceeds the skipped macroblock threshold, It stores executable computer program instructions. Device.

9. Computer program instructions are, In accordance with the fact that the amount of skipped macroblocks falls below the skipped macroblock threshold, it is determined that the amount of interframe encoded macroblocks in the frame satisfies the interframe macroblock threshold. It is determined that the rate distortion cost ratio of the skipped macroblocks in the frame to the rate distortion cost ratio threshold is below the rate distortion cost ratio threshold. In response to the rate distortion cost ratio falling below the rate distortion cost ratio threshold, the frame is identified as the static frame. It is possible to do so. The apparatus according to claim 8.

10. Computer program instructions are, Depending on whether the frame is identified as the static frame, one or more encoding attributes for the next frame are changed, Encoding the next frame according to one or more of the modified encoding attributes: It is possible to do so. The apparatus according to claim 8.

11. Computer program instructions are, It is possible to generate data that represents the aforementioned frame as the static frame. The apparatus according to claim 8.

12. Computer program instructions are, Based on the aforementioned data, it is possible to skip one or more identified still frames during the analysis of the video data. The apparatus according to claim 11.

13. The aforementioned video problem includes single-pass video data. The apparatus according to claim 8.

14. The average quantization parameter includes the average of each quantization parameter used to encode each macroblock of the frame, The apparatus according to claim 8.

Images