Static frame detection method and device

Abstract

The invention discloses a static frame detection method and device. The static frame detection method comprises the steps of obtaining an Nth image group GN; decoding to obtain a key frame in the image group GN; determining the similarity between the key frame in the image group GN and the key frame of the (N-1) th image group GN-1; and determining whether a static frame exists or not based on the similarity and the starting moment of whether the static frame exists or not currently, the scheme is realized based on a frame skip detection strategy of the key frame, that is, the key frame in the video signal is subjected to static frame detection preferentially, and if the static frame condition is not found, detection of other frame types in the current image group is directly skipped; according to the implementation, not only is the cost of processing non-key frames by a decoder reduced, but also the frame data volume required by static frame detection is greatly reduced, so that the detection speed is greatly improved; and the efficient detection mode significantly reduces the occupation of server resources, and reduces the hardware cost and energy consumption.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to a still frame detection method and device. Background Art

[0002] With the rapid development of video technology, video quality is becoming increasingly important. However, in real-time signal source scenarios, video playback often experiences static frames, also known as freeze frames, due to issues such as monitoring equipment or camera failures, line interruptions, or human error. This phenomenon occurs when the image remains static for multiple consecutive frames or even for extended periods of time. If the static frames are caused by video playback failures (such as prolonged freezes caused by stuttering), they can severely impact video quality, similar to the effects of bad frames.

[0003] Traditional still frame detection methods often face challenges when processing high-frame-rate, high-resolution live streams, such as those used for video surveillance and television broadcasts. The continuous improvement of video clarity, resolution, and the number of concurrent live video sources has placed even higher demands on the real-time, accurate, and efficient nature of still frame detection. Summary of the Invention

[0004] In view of this, this application provides the following technical solutions:

[0005] The first aspect of the present application provides a static frame detection method, comprising:

[0006] Get the Nth image group G N , where N is a positive integer;

[0007] Decode to obtain image group G N The key frames in

[0008] Determine the image group G N The key frame and the N-1th image group G N-1 The similarity of the key frames;

[0009] Based on the similarity and whether there is a still frame start time, it is determined whether there is a still frame. The still frame start time is the time when the image group G M The key frame and image group G M-1 The data is set when the similarity of the key frames meets the set conditions, and M is less than N.

[0010] In a possible implementation, determining whether a still frame exists based on the similarity and whether a still frame start time currently exists includes:

[0011] If the similarity is greater than or equal to the noise threshold, further determining whether there is a still frame start moment;

[0012] If a still frame start time exists, continue to perform still frame detection processing on subsequent image groups; if a still frame start time does not exist, determine the still frame start time based on image frame data of cached historical image groups;

[0013] If the similarity is less than the noise threshold, further determining whether there is a still frame start moment;

[0014] If there is a still frame start time, based on the image group G N-1 Determine the still frame end time; if there is no still frame start time, continue with the still frame detection process for subsequent image groups;

[0015] It is determined whether a still frame exists based on the still frame start time and the still frame end time.

[0016] In a possible implementation, the image group G is determined N The key frame and the N-1th image group G N-1 The similarity of the key frames includes:

[0017] Determine the image group G N The key frames and image group G in N-1 The mean absolute difference of all components in the image coding format of the key frames.

[0018] In a possible implementation, during the still frame detection process, as the video signal is received, the image groups obtained are sequentially cached. The maximum number of cached image groups is three. When there are three cached image groups, the cached image groups include the currently processed image group G. N , image group G N-1 and image group G N-2 ;

[0019] The determining of the still frame start time based on the image frame data of the cached historical image group includes:

[0020] If there are only two image groups currently cached, set image group G N-1 The moment corresponding to the key frame is determined as the start moment of the still frame;

[0021] If the current cache has three image groups, extract image group G N-2 All frame data are fully decoded, and all decoded image frames are compared with the image group G frame by frame. N The key frames are compared with the image group G N The moment corresponding to the image frame whose similarity to the key frame is greater than or equal to the noise threshold is determined as the still frame start moment;

[0022] If the current cache has three image groups, and image group G N-2All image frames and image groups G obtained by decoding N The similarity of the key frames is less than the noise threshold, and the image group G N-1 The moment corresponding to the key frame is determined as the still frame start moment.

[0023] In a possible implementation, the image group G N-1 Determine the end time of the still frame, including:

[0024] Extract image group G N-1 All frame data are fully decoded, and all decoded image frames are compared with the image group G frame by frame. N-1 The key frames are compared with the image group G N-1 The moment corresponding to the image frame whose similarity to the key frame is greater than or equal to the noise threshold is determined as the still frame end moment.

[0025] In a possible implementation, determining whether a still frame exists based on the still frame start time and the still frame end time includes:

[0026] Determine a time difference between the start time of the still frame and the end time of the still frame;

[0027] If the time difference is greater than the time threshold, it is determined that a still frame exists;

[0028] If the time difference is less than or equal to the time threshold, it is determined that no still frame exists, and the still frame start time is set to blank.

[0029] In one possible implementation, the following is also included:

[0030] The noise threshold and the time threshold are preconfigured.

[0031] In one possible implementation, the following is also included:

[0032] If there is a still frame, the still frame data including the still frame time will be saved or uploaded to the cloud.

[0033] In a possible implementation, the still frame detection method is implemented by reusing a first multimedia processing framework.

[0034] A second aspect of the present application provides a still frame detection device, comprising:

[0035] Image group acquisition module, used to obtain the Nth image group G N , where N is a positive integer;

[0036] Decoding processing module, used to decode and obtain image group G N The key frames in

[0037] Similarity determination module, used to determine the image group G N The key frame and the N-1th image group G N-1 The similarity of the key frames;

[0038] A still frame judging module is configured to determine whether a still frame exists based on the similarity and whether a still frame start time exists currently. The still frame start time is the time when the image group G M The key frame and image group G M-1 The data is set when the similarity of the key frames meets the set conditions, and M is less than N.

[0039] Through the above technical solutions, it can be seen that the embodiment of the present application discloses a still frame detection method and device, including: obtaining the Nth image group G N ; Decode to obtain image group G N The key frames in the image group G N The key frame and the N-1th image group G N-1 Determine whether there is a still frame based on the similarity and whether there is a still frame start time, the still frame start time is in the image group G M The key frame and image group G M-1 The scheme is based on the key frame skipping detection strategy, that is, the key frames in the video signal are given priority for still frame detection. If no still frame is found, the detection of other frame types in the current image group is skipped directly. This implementation not only reduces the decoder's overhead in processing non-key frames, but also greatly reduces the amount of frame data required for still frame detection, greatly improving the detection speed. This efficient detection method significantly reduces the occupation of server resources, reduces hardware costs and energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0041] Figure 1 This is a flowchart of a still frame detection method disclosed in an embodiment of the present application;

[0042] Figure 2 This is a first flow chart for determining whether a still frame exists according to an embodiment of the present application;

[0043] Figure 3 A second flow chart for determining whether a still frame exists according to an embodiment of the present application is provided;

[0044] Figure 4 A flowchart of a tool-based implementation of the still frame detection solution disclosed in an embodiment of the present application;

[0045] Figure 5 This is a schematic diagram of the overall implementation process of the solution disclosed in the embodiment of this application;

[0046] Figure 6 This is a structural diagram of a still frame detection device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] Figure 1 This is a flow chart of a still frame detection method disclosed in an embodiment of the present application. Figure 1 As shown, the still frame detection method may include:

[0049] Step 101: Obtain the Nth image group G N , where N is a positive integer.

[0050] During the transmission process, video streams are continuously transmitted to devices requiring video output in the form of groups of pictures (GOPs), which are transmitted sequentially in chronological order. A Group of Pictures (GOP) is a set of consecutive image frames, typically starting with an I-frame, followed by P-frames and B-frames. I-frames are keyframes and contain complete image information, while P-frames (predictively encoded using the previous I-frame or P-frame and containing only the differences from the reference frame) and B-frames (bidirectionally predicted using the previous and next I-frames or P-frames, typically used to improve compression efficiency) are encoded using the differences from the previous I-frame or frames.

[0051] Step 102: Decode to obtain image group G N The keyframes in .

[0052] In the traditional scheme, when performing still frame detection, all the frames in the image group are usually decoded and then compared frame by frame. Such an implementation not only consumes a lot of resources but also has very low efficiency. In the embodiment of the present application, the image group G is obtained. NAfter that, only the key frames in the image group, i.e., I frames, are decoded. The subsequent comparison of frame images is also the comparison of key frames in different image groups. Only when specific conditions are met will other image frames outside the key frames be decoded.

[0053] In other words, a key frame is a frame type that can be fully decoded without relying on other frames. For each image group, the key frame can be decoded first, and other types of frames are decoded only when needed. Therefore, compared with traditional solutions, the solution of this application can significantly reduce the amount of decoded data, thereby saving related resources and significantly improving the efficiency of static frame detection.

[0054] Step 103: Determine the image group G N The key frame and the N-1th image group G N-1 The similarity of the key frames.

[0055] Image Group G N It is the latest image group. Each time the latest image group is obtained, the key frame of the image group is decoded and obtained, and then the key frame of the current processed image group is compared with the key frame of the previous image group to determine the similarity between the two.

[0056] Step 104: Determine whether there is a still frame based on the similarity and whether there is a still frame start time. The still frame start time is the time when the image group G M The key frame and image group G M-1 The data is set when the similarity of the key frames meets the set conditions, and M is less than N.

[0057] Determining a still frame requires two conditions: whether the previous and next images are identical (determined based on similarity) and the duration of the image remaining unchanged. Therefore, determining whether a still frame exists requires determining the presence of a still frame based on the similarity and the presence of a still frame start time. The specific implementation will be detailed in the following embodiments and will not be further elaborated here.

[0058] The still frame start time is a parameter that is set every time a new image group is received, and indicates the start time of a possible still frame state. The specific determination method will be described in detail in the embodiments below.

[0059] The still frame detection method described in this embodiment is implemented based on a key frame skipping detection strategy, that is, still frame detection is performed preferentially on key frames in the video signal. If no still frame is found, the detection of other frame types in the current image group is directly skipped. This implementation not only reduces the decoder's overhead in processing non-key frames, but also greatly reduces the amount of frame data required for still frame detection, thereby greatly improving the detection speed. Moreover, this efficient detection method significantly reduces the occupancy of server resources, reduces hardware costs and energy consumption.

[0060] Figure 2 This is the first flow chart of determining whether there is a still frame disclosed in the embodiment of this application. Figure 2 As shown, the determining whether there is a still frame based on the similarity and whether there is a still frame start moment at present may include:

[0061] Step 201: If the similarity is greater than or equal to the noise threshold and there is a still frame start moment, continue still frame detection processing for subsequent image groups;

[0062] If the similarity is greater than or equal to the noise threshold, it indicates that the key frame of the current image group is the same as or similar to the key frame of the previous image group, and a still frame may exist. If the start time of the still frame has been determined before, the detection continues until the end time of the still frame is detected.

[0063] Step 202 : If the similarity is greater than or equal to the noise threshold and there is no still frame start time, determine the still frame start time based on the image frame data of the cached historical image group, and proceed to step 205 .

[0064] In a specific implementation, during the still frame detection process, as the video signal is received, the image groups obtained are sequentially cached. The maximum number of cached image groups is three. When there are three cached image groups, the cached image groups include the currently processed image group G. N , image group G N-1 and image group G N-2 .

[0065] Determining the still frame start time based on the image frame data of the cached historical image group may include:

[0066] If there are only two image groups currently cached, set image group G N-1 The moment corresponding to the key frame is determined as the start time of the still frame. If there are only two image groups cached, and the key frames of the two image groups are the same or similar, it means that the still frame state may have been in place from the beginning. The first image group cached (image group G N-2 ), that is, the moment of the key frame of the first image group, is recorded as the still frame start time.

[0067] If the current cache has three image groups, extract image group GN-2 All frame data are fully decoded, and all decoded image frames are compared with the image group G frame by frame. N The key frames are compared with the image group G N The time corresponding to the image frame whose key frame similarity is greater than or equal to the noise threshold is determined as the static frame start time. If there are three image groups in the cache, and the second image group in the cache (image group G N-1 ) and the third cached (image group G N ), it means that the start time of the still frame starts at least from the key frame time of the second picture group, and may also be at the non-key frame time of the first picture group. Therefore, it is necessary to decode all the frame data of the first picture group, that is, the picture group G. N-2 All frame data of the third image group, that is, G N The moment corresponding to the image frame having the same or highly similar key frame is determined as the still frame start moment.

[0068] If the current cache has three image groups, and image group G N-2 All image frames and image groups G obtained by decoding N The similarity of the key frames is less than the noise threshold, and the image group G N-1 The moment corresponding to the key frame is determined as the start time of the still frame. If there are three image groups in the cache, and the first image group G N-2 All image frames in the third image group G N The similarity of the key frames of G is low, then the second image group G can be directly N-1 The moment corresponding to the key frame is determined as the still frame start moment.

[0069] Step 203: If the similarity is less than the noise threshold and there is a still frame start time, based on the image group G N-1 Determine the still frame end time and proceed to step 205 .

[0070] Specifically, the image group G N-1 Determining the still frame end time may include: extracting the image group G N-1 All frame data are fully decoded, and all decoded image frames are compared with the image group G frame by frame. N-1 The key frames are compared with the image group G N-1 The moment corresponding to the image frame whose similarity to the key frame is greater than or equal to the noise threshold is determined as the still frame end moment.

[0071] If the third image group G N The key frame and the second image group G N-1The similarity of the key frames does not meet the requirements, that is, the similarity is low. Since the first frame of each image group is a key frame, it means that the end time of the static frame is within the time period corresponding to the second image group, so it is necessary to extract the second image group G N-1 All frame data are fully decoded to determine which ones are related to the second image group G N-1 The image frame whose similarity to the key frame is lower than the noise threshold is selected, and the moment corresponding to the image frame is determined as the still frame end moment.

[0072] Step 204: If the similarity is less than the noise threshold and there is no still frame start time, continue with the still frame detection process for subsequent image groups.

[0073] Step 205: Determine whether there is a still frame based on the still frame start time and the still frame end time.

[0074] Figure 3 The second flow chart for determining whether a still frame exists disclosed in the embodiment of the present application is combined with Figure 3 As shown, the determining whether there is a still frame based on the still frame start time and the still frame end time may include:

[0075] Step 301: Determine the time difference between the start time of the still frame and the end time of the still frame.

[0076] Step 302: If the time difference is greater than the time threshold, it is determined that a still frame exists.

[0077] Step 303: If the time difference is less than or equal to the time threshold, it is determined that no still frame exists, and the still frame start time is set to blank.

[0078] For example, if the time threshold is 3 seconds, if the time difference is 2.5 seconds, it is determined that there is no still frame; if the time difference is 3.5 seconds, it is determined that there is a still frame.

[0079] It is understandable that if the time difference between the start time of the still frame and the end time of the still frame is lower than or equal to the time threshold, there may be a reasonable range of stillness in the video content, such as the timed display of a certain object to create a tense atmosphere, but this does not belong to a still frame situation. At this time, it is necessary to set the start time of the still frame to blank and re-determine whether there is a still frame situation for subsequent image groups.

[0080] The implementation of the still frame detection scheme described in this embodiment incorporates a cache mechanism to store the data of the three most recently processed image groups. When two adjacent key frames are detected as suspected still frames, full-frame decoding of the cached image groups is performed, depending on the specific situation, to determine the start and end times of the still frame. This retroactive decoding mechanism effectively avoids errors that may be caused by frame skipping, ensuring the accuracy of still frame detection.

[0081] In implementation, the noise threshold and the time threshold may be pre-configured so that the noise threshold and the time threshold can be directly called to perform corresponding judgment processing when performing still frame detection.

[0082] If a still frame is detected, the still frame data including the still frame time can be saved or uploaded to the cloud, so as to timely lock the abnormal state of the video and perform corresponding processing.

[0083] In the above embodiment, the image group G is determined N The key frame and the N-1th image group G N-1 The similarity of the key frames of the image group G may include: determining N The key frames and image group G in N-1 The mean absolute difference of all components in the image coding format of the key frames.

[0084] The image encoding format of the key frame is not fixed, such as RGB format or YUV format. Each encoding format contains multiple components. The image group G N The key frames and image group G in N-1 The mean absolute difference of all components in the image coding format of the key frame, that is, to determine the image group G N The key frames and image group G in N-1 The larger the mean absolute difference of all components, the smaller the similarity between the two key frames. The smaller the mean absolute difference of all components, the greater the similarity between the two key frames.

[0085] Subsequently, whether one of the conditions for the still frame state is met, ie, whether the same or similar image content exists in different image groups, can be determined by comparing the mean absolute difference of all components with the noise threshold.

[0086] The static frame detection solution described in this application not only significantly improves static frame detection efficiency but also ensures the accuracy of detection results, providing an efficient and reliable static frame detection solution for high-concurrency, high-resolution live video signal sources. For enterprises that need to process large-scale video signals (such as video surveillance service providers, TV stations, and broadcast centers), this solution can significantly reduce operating costs while improving overall system throughput and response speed. It can also be easily integrated into existing video surveillance and live broadcast systems, making it suitable for a variety of application scenarios.

[0087] In practical applications, the freeze frame detection method can be implemented by reusing a first multimedia processing framework, such as the FFmpeg framework. The freeze detect filter in the FFmpeg framework can be used to detect video freeze frames. The freeze detect filter calculates the mean absolute difference (MAD) of all components of key frames in two adjacent decoded image groups and compares it to a preset noise threshold. When the MAD is less than the noise threshold (corresponding to a similarity greater than or equal to the noise threshold) and exceeds a preset freeze frame duration, the filter records a freeze frame message.

[0088] The present application solution can make full use of the powerful audio and video processing capabilities of the FFmpeg framework to achieve efficient engineering tooling. The AVFormat library of the FFmpeg framework supports common public live transmission protocols and transcapacitor protocols on the market, including the RTSP protocol widely used in the field of video surveillance, the UDP multicast protocol in the field of digital television, the HTTP-HLS and RTMP protocols with higher latency, and low-latency transmission protocols such as WebRTC and SRT that have been open sourced in recent years. The present invention directly reuses the AVFormat library to realize the transmission protocol reception and encapsulation format parsing of the video signal source. In addition, the AVCodec library of the FFmpeg framework integrates the functions of most decoders on the market. The present application implements on-demand decoding logic in a specific decoder, and only decodes key frames, further optimizing resource utilization. At the same time, the caching logic of multiple GOP groups is implemented in the FFtools tool layer, and retroactive decoding control logic and still frame detection logic are added to ensure the accuracy of the detection results.

[0089] The still frame detection solution of the present application is based on the engineering tool implementation of the FFmpeg framework, which can quickly process a variety of video format sources by calling the FFmpeg command line, greatly enhancing the simplicity and ease of use of the still frame detection algorithm. By reusing the powerful audio and video processing capabilities of the FFmpeg framework, especially its decoding library, transmission protocol library and transpackaging format library, the present invention avoids the need for repeated development of related functional components, significantly saving development and maintenance costs. For enterprises, this not only reduces the manpower investment of the technical team, but also shortens the cycle of function launch, thereby realizing commercial applications more quickly. In addition, as an open source framework, FFmpeg's extensive support and community resources further reduce technical risks and maintenance costs.

[0090] Figure 4 This is a flowchart of the tool-based implementation of the still frame detection solution disclosed in the embodiment of this application. Figure 5 This is a schematic diagram of the overall implementation process of the solution disclosed in the embodiment of this application. Figure 4 and Figure 5 As shown in the figure, the still frame detection scheme includes the following:

[0091] S101, threshold parameter configuration, according to the specific application scenario, determine the threshold parameters required for still frame detection. It mainly includes two key parameters: one parameter is the noise threshold (noise), which is used to judge the similarity between two key frames. When the mean absolute difference (MAFD) of the two frames is less than or equal to the threshold (the corresponding similarity is greater than or equal to the threshold), the two frames are considered similar and a still frame may exist; the other parameter is the still frame duration threshold (duration), which is used to judge whether the duration of the still frame reaches the preset minimum value. Only when the still frame duration exceeds the threshold, it is determined to be a valid still frame scene;

[0092] S102, video signal source reception and parsing, reuses the AVFormat library of the FFmpeg framework to support a variety of common live broadcast transmission protocols (such as RTSP, UDP, HTTP-HLS, RTMP, WebRTC, SRT, etc.). Through the AVFormat library, the system can receive live video signal sources, parse their encapsulation formats, and extract video data for subsequent processing. This step ensures that the system is compatible with multiple video transmission protocols and adapts to the needs of different live broadcast scenarios.

[0093] S103, GOP group cache management, in the FFmpeg framework tool layer, the system performs cache management on the parsed three adjacent GOP (Group of Pictures) groups. Specifically, it includes: the currently processed GOP group (G n ), the GOP group being decoded and detected; the previous GOP group (G n-1 ), used to compare frames with the current GOP group to determine whether there is a still frame, and the end moment of the still frame scene may be in this GOP group; the first two GOP groups (G n-2 ), used to backtrack detection when needed to accurately determine the start of a still frame. By caching multiple GOP groups, the system can backtrack historical frame data when a suspected still frame is detected, ensuring the accuracy of the detection result.

[0094] S104, key frame decoding and caching, reuse the AVCodec library of the FFmpeg framework, call the corresponding decoder to decode the current GOP group (G n ) in the IDR frame (I n ) is decoded and cached. n The system temporarily caches other types of frames in the group (such as P frames and B frames) without decoding them to reduce computing resource consumption. When the system first caches a GOP group, it records the first frame time (t0) of the video signal and initializes the still frame start time (T0) to empty. This step provides an initial time reference for subsequent still frame detection;

[0095] S105, key frame similarity calculation, when the system caches at least two GOP groups, the current GOP group (G n ) of the IDR frame (I n ) and the previous GOP group (G n-1 ) of the IDR frame (I n-1 ) to perform similarity detection. The specific steps are as follows: Step 1, calculate the mean absolute difference (MAFD) of all components of the two key frames. Step 2, if MAFD ≤ noise, it is determined to be a suspected still frame scene. At this time: if T0 ≠ empty, it means that there is a suspected still frame scene in the previous period. The system only needs to continue to detect the IDR frames of the subsequent GOP groups, that is, continue to execute S105, and finally determine the end time of the still frame; if T0 = empty, it is necessary to determine the start time of the still frame. If there are only two GOP groups at present, the still frame start time T0 is the first frame (corresponding to the key frame / I frame) time t0, otherwise, the system needs to extract G n-2 All frame data of the group are fully decoded and compared with I frame by frame. n The frames are compared to accurately determine the start time of the static frame. In the third step, if mafd > noise, it means that the similarity between the two key frames is not high, and the system needs to perform further processing based on whether T0 is empty (see S106);

[0096] S106, non-still frame scene processing, that is, when the mafd of two adjacent key frames is greater than noise, it means that the two frames are not similar, and the system needs to process them in different cases: when T0≠empty, it means that there is a suspected still frame scene before, and the system needs to n-1 ) to determine the end time (T1) of the still frame. The specific operation is: n-1 All frames of the group are fully decoded and compared with I frame by frame. n-1 The frames are compared to determine the end time of the still frame. If T1-T0 ≥ duration, it is determined to be a valid still frame scene. Otherwise, it is only a suspected still picture with insufficient duration. The non-empty T0 value still needs to be reassigned to empty and enter the next still frame scene detection; when T0 = empty, it means that the three adjacent cached GOP groups do not belong to still frame scenes, and the system continues to perform key frame still frame detection on subsequent live video signals.

[0097] S107: Continuous Detection and Results Storage: The system continues to perform still-frame detection on the live signal source according to the above steps until the live signal ends. During the detection process, the system saves the detected still-frame scenes as files locally or reports them to a cloud database for storage and easy subsequent review and processing. By saving the detection results in real time, the system can provide detailed still-frame information to operation and maintenance personnel, facilitating rapid problem location and resolution.

[0098] During the entire still frame detection process, dynamic frame detection control is performed on the keyframe detection results according to the situation. For non-keyframes, the system only decodes and detects them when necessary. For example, when a still frame is detected in a keyframe, decoding of non-keyframes can be dynamically enabled to further confirm the duration of the still frame. This dynamic frame detection control strategy further reduces computing resource consumption while ensuring detection accuracy. This solution not only significantly reduces computing resource consumption, but also maintains high detection speed and accuracy in high-concurrency, high-resolution live broadcast scenarios.

[0099] In actual application scenarios, the keyframe-based frame skipping detection mechanism involved in this application can be integrated into the transcoding FFmpeg tool, and an ffmpeg task process can be enabled for each live TV video signal source to perform separate detection. This can greatly reduce the computational complexity while ensuring detection accuracy, thereby achieving efficient real-time processing of live stream static frame detection, and can realize quality monitoring and detection processing of multiple high-concurrency, high-resolution live video signals based on the original hardware monitoring resources, greatly improving the system processing throughput.

[0100] An example of the ffmpeg detection command line used for each live TV signal source is: . / ffmpeg -i udp: / / 233.233.233.233:1234 -work_threads_num 4 -keyframe_decode_enable 1 -video_freeze_noise_threshold 0.001 -video_freeze_duration_seconds 5.0 -detect_file_output . / info.txt -f null / dev / null. This task command line can detect still frames on the multicast live video source udp: / / 233.233.233.233:1234. work_threads_num=4 enables four threads for parallel decoding of the source signal, accelerating speed. keyframe_decode_enable=1 enables keyframe decoding, passing in specific decoder control to decode only keyframes. video_freeze_noise_threshold=0.001 indicates that two frames are considered similar when the mean absolute difference of all components is less than 0.001. video_freeze_duration_seconds=5.0 indicates that a still frame scene is considered when consecutive similar frames last for more than 5.0 seconds. detect_file_output=. / info.txt saves the still frame detection results to a file. -f null dev / null indicates that subsequent encoding processing and file storage are not required for the decoded data frames. This method achieves a speed improvement of several dozen times compared to performing full-frame decoding detection on live video signal sources.

[0101] The static frame detection scheme proposed in this application prioritizes decoding and static frame detection for key frames (I frames) in the video stream, skipping the decoding process of a large number of non-key frames (P frames and B frames). This strategy has three technical advantages. First, it significantly reduces computing resource consumption: By reducing the decoding of non-key frames, the system can significantly reduce computing resource consumption, especially in high-resolution, high-frame-rate video streams, where the resource optimization effect is more obvious. Second, it improves detection speed. The frame skipping detection strategy reduces the number of frames to be processed, significantly improving detection speed and meeting real-time requirements. Third, it ensures detection accuracy: By caching multiple GOP groups and combining a retrospective detection mechanism, the system can accurately determine the start and end times of the static frame when a suspected static frame is detected, avoiding misjudgment.

[0102] Furthermore, the solution leverages the FFmpeg framework for engineering tooling, fully leveraging its powerful audio and video processing capabilities. This approach offers three advantages: First, it reuses FFmpeg's powerful functionality: By reusing FFmpeg's AVFormat and AVCodec libraries, the system supports a variety of common live broadcast transmission protocols (such as RTSP, UDP, HTTP-HLS, RTMP, WebRTC, SRT, etc.), and achieves efficient video decoding and encapsulation format parsing. Second, it reduces development and maintenance costs: Reusing the FFmpeg framework avoids the need for repeated development of video processing modules, significantly reducing development and maintenance costs. Third, it improves system compatibility: The wide support of the FFmpeg framework makes this technology compatible with a variety of video formats and transmission protocols, making it suitable for different application scenarios.

[0103] For the sake of simplicity, the aforementioned method embodiments are described as a series of action combinations. However, those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0104] The method is described in detail in the embodiments disclosed in the above application. The method of the application can be implemented using various forms of devices. Therefore, the application also discloses a device, and a specific embodiment is given below for detailed description.

[0105] Figure 6 This is a schematic diagram of the structure of a static frame detection device disclosed in an embodiment of the present application. Figure 6 As shown, the still frame detection device 60 may include:

[0106] Image group acquisition module 601, used to obtain the Nth image group G N , where N is a positive integer.

[0107] Decoding processing module 602, used for decoding to obtain image group G N The keyframes in .

[0108] Similarity determination module 603, used to determine the image group G N The key frame and the N-1th image group G N-1 The similarity of the key frames.

[0109] The still frame judging module 604 is configured to determine whether a still frame exists based on the similarity and whether a still frame start time exists currently. The still frame start time is the time when the image group G M The key frame and image group G M-1The data is set when the similarity of the key frames meets the set conditions, and M is less than N.

[0110] The still frame detection device described in this embodiment is implemented based on a key frame skipping detection strategy, that is, still frame detection is performed on key frames in the video signal first. If no still frame is found, the detection of other frame types in the current image group is directly skipped. This implementation not only reduces the decoder's overhead in processing non-key frames, but also greatly reduces the amount of frame data required for still frame detection, thereby greatly improving the detection speed. Moreover, this efficient detection method significantly reduces the occupancy of server resources, reduces hardware costs and energy consumption.

[0111] The specific implementation of the still frame detection device and the modules it contains, as well as other possible implementations, can be found in the corresponding parts of the method embodiment, and will not be repeated here.

[0112] Any one of the still frame detection devices described in the above embodiments includes a processor and a memory. The image group acquisition module, decoding processing module, decoding processing module, still frame judgment module, etc. in the above embodiments are all stored in the memory as program modules, and the processor executes the above program modules stored in the memory to realize corresponding functions.

[0113] The processor contains a kernel, which retrieves the corresponding program module from the memory. There can be one or more kernels, and the kernel parameters can be adjusted to process the returned data.

[0114] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0115] In an exemplary embodiment, a computer-readable storage medium is also provided, which can be directly loaded into the internal memory of a computer and contains software code. After being loaded and executed by a computer, the computer program can implement the steps shown in any embodiment of the above-mentioned still frame detection method.

[0116] In an exemplary embodiment, a computer program product is also provided, which can be directly loaded into the internal memory of a computer and contains software code. After being loaded and executed by a computer, the computer program can implement the steps shown in any embodiment of the still frame detection method described above.

[0117] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0118] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0119] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0120] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A still frame detection method, characterized in that: include: Get the Nth image group G N , where N is a positive integer; Decode to obtain image group G N The key frames in Determine the image group G N The key frame and the N-1th image group G N-1 The similarity of the key frames; Based on the similarity and whether there is a still frame start time, it is determined whether there is a still frame. The still frame start time is the time when the image group G M The key frame and image group G M-1 The data is set when the similarity of the key frames meets the set conditions, and M is less than N.

2. The still frame detection method according to claim 1, wherein: The determining whether there is a still frame based on the similarity and whether there is a still frame start moment currently includes: If the similarity is greater than or equal to the noise threshold, further determining whether there is a still frame start moment; If a still frame start time exists, continue to perform still frame detection processing on subsequent image groups; if a still frame start time does not exist, determine the still frame start time based on image frame data of cached historical image groups; If the similarity is less than the noise threshold, further determining whether there is a still frame start moment; If there is a still frame start time, based on the image group G N-1 Determine the still frame end time; if there is no still frame start time, continue with the still frame detection process for subsequent image groups; It is determined whether a still frame exists based on the still frame start time and the still frame end time.

3. The still frame detection method according to claim 1, wherein: The determined image group G N The key frame and the N-1th image group G N-1 The similarity of the key frames includes: Determine the image group G N The key frames and image group G in N-1 The mean absolute difference of all components in the image coding format of the key frames.

4. The still frame detection method according to claim 2, wherein: During the still frame detection process, as the video signal is received, the image groups obtained are sequentially cached. The maximum number of cached image groups is three. When there are three cached image groups, the cached image groups include the currently processed image group G N , image group G N-1 and image group G N-2 ; The determining of the still frame start time based on the image frame data of the cached historical image group includes: If there are only two image groups currently cached, set image group G N-1 The moment corresponding to the key frame is determined as the start moment of the still frame; If the current cache has three image groups, extract image group G N-2 All frame data are fully decoded, and all decoded image frames are compared with the image group G frame by frame. N The key frames are compared with the image group G N The moment corresponding to the image frame whose similarity to the key frame is greater than or equal to the noise threshold is determined as the still frame start moment; If the current cache has three image groups, and image group G N-2 All image frames and image groups G obtained by decoding N The similarity of the key frames is less than the noise threshold, and the image group G N-1 The moment corresponding to the key frame is determined as the still frame start moment.

5. The still frame detection method according to claim 2, wherein: The image group G N-1 Determine the end time of the still frame, including: Extract image group G N-1 All frame data are fully decoded, and all decoded image frames are compared with the image group G frame by frame. N-1 The key frames are compared with the image group G N-1 The moment corresponding to the image frame whose similarity to the key frame is greater than or equal to the noise threshold is determined as the still frame end moment.

6. The still frame detection method according to claim 2, wherein: The determining whether a still frame exists based on the still frame start time and the still frame end time includes: Determine a time difference between the start time of the still frame and the end time of the still frame; If the time difference is greater than the time threshold, it is determined that a still frame exists; If the time difference is less than or equal to the time threshold, it is determined that no still frame exists, and the still frame start time is set to blank.

7. The still frame detection method according to claim 6, wherein: Also includes: The noise threshold and the time threshold are preconfigured.

8. The still frame detection method according to claim 1, wherein: Also includes: If there is a still frame, the still frame data including the still frame time will be saved or uploaded to the cloud.

9. The static frame detection method according to any one of claims 1 to 8, characterized in that: The still frame detection method is implemented by reusing the first multimedia processing framework.

10. A still frame detection device, characterized in that: include: Image group acquisition module, used to obtain the Nth image group G N , where N is a positive integer; Decoding processing module, used to decode and obtain image group G N The key frames in Similarity determination module, used to determine the image group G N The key frame and the N-1th image group G N-1 The similarity of the key frames; A still frame judging module is configured to determine whether a still frame exists based on the similarity and whether a still frame start time exists currently. The still frame start time is the time when the image group G M The key frame and image group G M-1 The data is set when the similarity of the key frames meets the set conditions, and M is less than N.

Images