A method, device and equipment for detecting image freeze frame and storage medium

Abstract

Embodiments of the present application disclose a kind of image freeze frame detection method, device, equipment and storage medium, relevant embodiments can be applied to artificial intelligence, map, intelligent transportation and various scenes, for improving the accuracy of detecting image freeze frame.The method of the embodiment of the present application comprises: obtaining target image pair, wherein the target image pair is the gray image corresponding to two adjacent frames, performing saliency processing on each gray image in the target image pair to retain high-contrast area, obtaining first saliency image and second saliency image, performing gray difference processing on the first saliency image and the second saliency image, obtaining gray difference image, performing contour processing on the gray difference image, obtaining the target contour with the largest difference area value, and if the difference area value of the target contour is less than the area threshold, it is determined that the target image pair has image freeze frame.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device, and storage medium for detecting frozen frames in images. Background Technology

[0002] With the rapid development of information technology, people have increasingly higher demands for audio and video viewing experiences, and the smoothness of live streaming or video playback is an important indicator for measuring viewing experience. One of the most commonly used metrics is frames per second (FPS). However, FPS is often insufficient to detect static scenes. For example, during a live stream, if a static scene is pushed, from the viewer's perspective, the live stream might appear to be frozen, but from an FPS perspective, there are no abnormalities between the image frames. Therefore, accurately obtaining the relationship between image frames to improve the smoothness of live streaming or video playback has become an urgent problem to solve.

[0003] Currently, the main techniques for detecting inter-frame relationships in images are based on the proportion of pixel differences between images or image similarity. However, these techniques generally require averaging the overall detection results and calculating percentages, which can easily overlook or weaken local differences between image frames, leading to misjudgments of these differences and a decrease in the accuracy of obtaining inter-frame relationships. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for detecting frozen frames in images. It uses saliency processing to obtain images of areas with high contrast and significantly high grayscale values ​​that better match human visual perception of differences. Furthermore, based on saliency processing, it gradually emphasizes differences between images that better match human visual characteristics, avoiding the neglect or weakening of local differences between images. It can quantify these differences using difference area values, thereby determining whether a target image pair has frozen frames based on the maximum difference area value, thus improving the accuracy of obtaining inter-frame relationships.

[0005] One embodiment of this application provides a method for detecting frozen frames in an image, including:

[0006] Obtain target image pairs, where a target image pair is a grayscale image corresponding to two adjacent frames;

[0007] For each grayscale image in the target image pair, perform saliency processing to preserve the high-contrast regions, resulting in a first saliency image and a second saliency image;

[0008] Gray-level difference processing is performed on the first saliency image and the second saliency image to obtain the gray-level difference image;

[0009] Contour processing is performed on grayscale difference images to obtain the target contour with the largest difference area value;

[0010] If the difference area value of the target contour is less than the area threshold, then the target image pair is determined to have image freeze.

[0011] Another aspect of this application provides an image freeze detection device, comprising:

[0012] The acquisition unit is used to acquire target image pairs, wherein the target image pairs are grayscale images corresponding to two adjacent frames;

[0013] The processing unit is used to perform saliency processing on each grayscale image in the target image pair to preserve the high-contrast region, so as to obtain a first saliency image and a second saliency image;

[0014] The processing unit is also used to perform grayscale difference processing on the first saliency image and the second saliency image to obtain a grayscale difference image;

[0015] The processing unit is also used to perform contour processing on grayscale difference images to obtain the target contour with the largest difference area value;

[0016] The determination unit is used to determine that if the difference area value of the target contour is less than the area threshold, the target image pair has an image freeze frame.

[0017] In one possible design, in another implementation of the embodiments of this application, the processing unit may specifically be used for:

[0018] Gaussian blur filtering is applied to each grayscale image in the target image pair to obtain the first Gaussian blurred image and the second Gaussian blurred image.

[0019] The first Gaussian blurred image and the second Gaussian blurred image are subjected to saliency processing to preserve the high-contrast regions, respectively, to obtain the first saliency image and the second saliency image.

[0020] In one possible design, in another implementation of the embodiments of this application, the processing unit may specifically be used for:

[0021] Median filtering is applied to the grayscale difference image to obtain the median-filtered difference image;

[0022] Contour processing is performed on the median-filtered difference image to obtain the target contour with the largest difference area value.

[0023] In one possible design, in another implementation of the embodiments of this application, the acquisition unit may specifically be used for:

[0024] Obtain the first color histogram and the second color histogram corresponding to the original image pair, wherein the original image pair is the color image corresponding to two adjacent frames;

[0025] The similarity between the first color histogram and the second color histogram is calculated to obtain the color histogram similarity.

[0026] If the color histogram similarity is greater than the first similarity threshold, then the original image pair is processed into grayscale to obtain the target image pair.

[0027] In one possible design, in another implementation of the embodiments of this application, the acquisition unit may specifically be used for:

[0028] If the color histogram similarity is greater than the first similarity threshold, then the original image pair is processed into grayscale to obtain the basic grayscale image pair.

[0029] Structural similarity is calculated for basic grayscale image pairs to obtain structural similarity.

[0030] If the structural similarity is greater than the second similarity threshold, then the basic grayscale image pair is taken as the target image pair.

[0031] In one possible design, in another implementation of the embodiments of this application, the acquisition unit may specifically be used for:

[0032] The average gray level of the basic gray level image pair is calculated to obtain the basic brightness contrast value of the basic gray level image pair.

[0033] The grayscale standard deviation is calculated for the basic grayscale image pairs to obtain the basic contrast measurement value of the basic grayscale image pairs;

[0034] The structural contrast value of the basic grayscale image pair is calculated based on the basic brightness contrast value and the basic contrast measurement value.

[0035] Structural similarity is calculated based on the basic brightness contrast value, the basic contrast measurement value, and the structural contrast value.

[0036] In one possible design, in another implementation of the embodiments of this application, the acquisition unit may specifically be used for:

[0037] The first color histogram is normalized to obtain the first color distribution interval of the first color histogram;

[0038] Based on the number of intervals, the mean of the first color distribution interval is calculated to obtain the mean of the first color distribution.

[0039] The second color histogram is normalized to obtain the second color distribution range of the second color histogram;

[0040] Based on the number of intervals, the mean of the second color distribution intervals is calculated to obtain the mean of the second color distribution.

[0041] The similarity is calculated based on the first color distribution interval, the mean of the first color distribution, the second color distribution interval, and the mean of the second color distribution to obtain the color histogram similarity.

[0042] In one possible design, in another implementation of the embodiments of this application, the acquisition unit may specifically be used for:

[0043] Based on the decoding frame rate, the target object under test is decoded to obtain the set of image frames corresponding to the target object under test;

[0044] Take any pair of adjacent image frames in the image frame set as the original image pair;

[0045] The color pixel values ​​in the original image pair are counted to obtain the number of pixel values ​​corresponding to each color interval, so as to obtain the first color histogram and the second color histogram.

[0046] In one possible design, in another implementation of the embodiments of this application, the processing unit may specifically be used for:

[0047] The difference between the first spectrum corresponding to the first Gaussian blurred image and the cepstral of the mean of the first image is calculated to obtain the first difference;

[0048] The first difference is used as the first saliency weight corresponding to the first Gaussian blurred image;

[0049] The first saliency weight is mapped to the first Gaussian blurred image to obtain the first saliency image;

[0050] The difference between the second spectrum corresponding to the second Gaussian blurred image and the cepstral of the mean of the second image is calculated to obtain the second difference.

[0051] The second difference is used as the second saliency weight corresponding to the second Gaussian blurred image;

[0052] The second saliency weights are mapped to the second Gaussian blurred image to obtain the second saliency image.

[0053] In one possible design, in another implementation of the embodiments of this application, the processing unit may specifically be used for:

[0054] Using a Gaussian filter convolution kernel of the same size as the Gaussian filter, each pixel in each grayscale image is scanned sequentially to obtain the first neighboring pixels of each pixel.

[0055] Based on the filtering weights, the gray values ​​corresponding to the pixels in the first neighborhood are weighted and averaged to obtain the weighted average gray value corresponding to each pixel in the first neighborhood.

[0056] Replace the gray value of the center pixel with the weighted average gray value corresponding to each pixel in the first neighborhood to obtain the first Gaussian blurred image and the second Gaussian blurred image.

[0057] In one possible design, in another implementation of the embodiments of this application, the processing unit may specifically be used for:

[0058] Using a median filter convolution kernel of the same size as the median filter, each pixel in the grayscale difference image is scanned sequentially to obtain the pixels in the second neighborhood corresponding to each pixel.

[0059] The median of the gray values ​​corresponding to the pixels in the second neighborhood is calculated to obtain the median of each pixel in the second neighborhood.

[0060] The median value of the center pixel is replaced by the median value of each neighboring pixel to obtain a median-filtered difference image.

[0061] In one possible design, in another implementation of the embodiments of this application, the processing unit may specifically be used for:

[0062] Contour feature extraction is performed on the median-filtered difference image to obtain multiple difference contours containing the target object;

[0063] Calculate the area of ​​each differential contour to obtain the differential area value for each differential contour;

[0064] Based on the difference area value of each difference contour, pairwise comparisons are performed to obtain the target contour with the largest difference area value.

[0065] This application also provides a computer device, including: a memory, a processor, and a bus system;

[0066] The memory is used to store programs;

[0067] The processor implements the methods described above when executing a program in memory;

[0068] Bus systems are used to connect memory and processor to enable communication between them.

[0069] Another aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described above.

[0070] As can be seen from the above technical solutions, the embodiments of this application have the following beneficial effects:

[0071] By acquiring a target image pair consisting of grayscale images corresponding to two adjacent frames, saliency processing is performed on each grayscale image in the target image pair to retain the high-contrast region, thereby obtaining a first saliency image and a second saliency image. Then, grayscale difference processing is performed on the first saliency image and the second saliency image to obtain a grayscale difference image. Then, contour processing can be performed on the grayscale difference image to obtain the target contour with the largest difference area value. When the difference area value of the target contour is less than the area threshold, it is determined that there is image freeze frame in the target image pair. By employing the above method, saliency processing can be performed on the grayscale images corresponding to two adjacent frames in a target image pair to retain high-contrast regions. This results in images with high contrast regions that exhibit significantly high grayscale values, which are more consistent with human visual perception. Furthermore, based on the saliency processing, grayscale difference processing is applied to the first and second saliency images to further emphasize the differences between the images that are more consistent with human visual characteristics, avoiding the neglect or weakening of local differences between images. Then, based on the grayscale difference processing and combined with contour processing, the differences between the images that are more consistent with human visual characteristics can be quantified through the difference area value. Thus, the presence of frozen frames in the target image pair can be determined based on the maximum difference area value, thereby improving the accuracy of obtaining the inter-frame relationship of images. Attached Figure Description

[0072] Figure 1 This is a schematic diagram of the architecture of the image data control system in an embodiment of this application;

[0073] Figure 2 This is a flowchart of one embodiment of the image frozen frame detection method in this application;

[0074] Figure 3 This is a flowchart of another embodiment of the image freeze detection method in this application;

[0075] Figure 4 This is a flowchart of another embodiment of the image freeze detection method in this application;

[0076] Figure 5 This is a flowchart of another embodiment of the image freeze detection method in this application;

[0077] Figure 6This is a flowchart of another embodiment of the image freeze detection method in this application;

[0078] Figure 7 This is a flowchart of another embodiment of the image freeze detection method in this application;

[0079] Figure 8 This is a flowchart of another embodiment of the image freeze detection method in this application;

[0080] Figure 9 This is a flowchart of another embodiment of the image freeze detection method in this application;

[0081] Figure 10 This is a flowchart of another embodiment of the image freeze detection method in this application;

[0082] Figure 11 This is a flowchart of another embodiment of the image freeze detection method in this application;

[0083] Figure 12 This is a flowchart of another embodiment of the image freeze detection method in this application;

[0084] Figure 13 This is a flowchart of another embodiment of the image freeze detection method in this application;

[0085] Figure 14 This is a schematic diagram illustrating the principle of the image frozen frame detection method in the embodiments of this application;

[0086] Figure 15 This is another schematic diagram of the principle of the image frozen frame detection method in the embodiments of this application;

[0087] Figure 16 This is a grayscale image schematic diagram of the image freeze detection method in the embodiments of this application;

[0088] Figure 17 This is a Gaussian blurred image schematic diagram of the image freeze detection method in the embodiments of this application;

[0089] Figure 18 This is a saliency image diagram of the image freezing detection method in the embodiments of this application;

[0090] Figure 19 This is a grayscale difference image schematic diagram of the image freeze detection method in the embodiments of this application;

[0091] Figure 20 This is a median-filtered difference image schematic diagram of the image freeze frame detection method in the embodiments of this application;

[0092] Figure 21 This is a schematic diagram of an original image pair of the image freeze detection method in the embodiments of this application;

[0093] Figure 22 This is a color histogram diagram of the image freeze detection method in the embodiments of this application;

[0094] Figure 23 This is a schematic diagram comparing the frozen frame detection effect of the image frozen frame detection method in the embodiments of this application;

[0095] Figure 24 This is a schematic diagram comparing the average calculation time for frozen frame detection in the image frozen frame detection method in the embodiments of this application;

[0096] Figure 25 This is a schematic diagram of one embodiment of the image freeze detection device in this application;

[0097] Figure 26 This is a schematic diagram of one embodiment of the computer device described in this application. Detailed Implementation

[0098] This application provides a method, apparatus, device, and storage medium for detecting frozen frames in images. It uses saliency processing to obtain images of areas with high contrast and significantly high grayscale values ​​that better match human visual perception of differences. Furthermore, based on saliency processing, it gradually emphasizes differences between images that better match human visual characteristics, avoiding the neglect or weakening of local differences between images. It can also quantify these differences using difference area values, thereby determining whether a target image pair has frozen frames based on the maximum difference area value, thus improving the accuracy of obtaining inter-frame relationships.

[0099] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0100] To facilitate understanding, some terms or concepts involved in the embodiments of this application will be explained first.

[0101] 1. Frame freeze detection

[0102] Frame freeze detection refers to detecting whether a target video or live stream has a frame freeze (stuttering) phenomenon.

[0103] 2. The data metric is the number of frames transmitted per second (FPS).

[0104] The number of frames transmitted per second (fps) refers to the number of frames in an animation or video.

[0105] 3. Image similarity

[0106] Image similarity refers to the degree of similarity between images quantified through image algorithms.

[0107] It is understood that in the specific embodiments of this application, data related to target images and the like are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0108] It is understood that the image freeze detection method disclosed in this application specifically involves Intelligent Vehicle Infrastructure Cooperative Systems (IVICS), which will be further introduced below. Intelligent Vehicle Infrastructure Cooperative Systems, or simply Vehicle-Road Cooperative Systems, are a development direction of Intelligent Transportation Systems (ITS). IVICS utilizes advanced wireless communication and next-generation Internet technologies to implement comprehensive, real-time dynamic information interaction between vehicles and roads. Based on the collection and fusion of dynamic traffic information across all times and spaces, it conducts active safety control of vehicles and cooperative road management, fully realizing effective coordination between people, vehicles, and roads, ensuring traffic safety, improving traffic efficiency, and thus forming a safe, efficient, and environmentally friendly road traffic system.

[0109] Understandably, the image freeze detection method disclosed in this application also involves Artificial Intelligence (AI) technology, which will be further introduced below. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, enabling them to have the functions of perception, reasoning, and decision-making.

[0110] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0111] Secondly, Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close connection with linguistic research. NLP technologies typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0112] Secondly, Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0113] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0114] It should be understood that the image freeze detection method provided in this application can be applied to various scenarios, including but not limited to artificial intelligence, maps, intelligent transportation, cloud technology, etc., to optimize the smoothness of live broadcasts or video screens by performing freeze detection on image pairs corresponding to adjacent frames, so as to be applied to scenarios such as live broadcast playback, intelligent audio-visual interaction, intelligent map recognition or intelligent traffic navigation.

[0115] To address the aforementioned problems, this application proposes a method for detecting image frozen frames, which is applied to... Figure 1 Please refer to the text data control system shown. Figure 1 , Figure 1 This is a schematic diagram of the architecture of a text data control system in an embodiment of this application, such as... Figure 1 As shown, the server acquires a target image pair consisting of grayscale images corresponding to two adjacent frames provided by the terminal device. It performs saliency processing on each grayscale image in the target image pair to retain the high-contrast region, thereby obtaining a first saliency image and a second saliency image. Then, it performs grayscale difference processing on the first saliency image and the second saliency image to obtain a grayscale difference image. Then, it can perform contour processing on the grayscale difference image to obtain the target contour with the largest difference area value. When the difference area value of the target contour is less than the area threshold, it is determined that there is image freezing in the target image pair. By employing the above method, saliency processing can be performed on the grayscale images corresponding to two adjacent frames in a target image pair to retain high-contrast regions. This results in images with high contrast regions that exhibit significantly high grayscale values, which are more consistent with human visual perception. Furthermore, based on the saliency processing, grayscale difference processing is applied to the first and second saliency images to further emphasize the differences between the images that are more consistent with human visual characteristics, avoiding the neglect or weakening of local differences between images. Then, based on the grayscale difference processing and combined with contour processing, the differences between the images that are more consistent with human visual characteristics can be quantified through the difference area value. Thus, the presence of frozen frames in the target image pair can be determined based on the maximum difference area value, thereby improving the accuracy of obtaining the inter-frame relationship of images.

[0116] Understandable, Figure 1Only one type of terminal device is shown in the diagram. In real-world scenarios, many more types of terminal devices can participate in the data processing. These include, but are not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, and in-vehicle terminals. The specific number and types depend on the actual scenario and are not limited here. Furthermore, Figure 1 The diagram shows one server, but in real-world scenarios, multiple servers can be involved, especially in scenarios involving multi-model training and interaction. The number of servers depends on the specific scenario and is not limited here.

[0117] It should be noted that in this embodiment, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal devices and servers can be directly or indirectly connected via wired or wireless communication, and terminal devices and servers can be connected to form a blockchain network; this application does not impose any limitations on this.

[0118] Based on the above introduction, the image freeze detection method in this application will be described below. Please refer to [link / reference]. Figure 2 One embodiment of the image frozen frame detection method in this application includes:

[0119] In step S101, a target image pair is obtained, wherein the target image pair is a grayscale image corresponding to two adjacent frames;

[0120] In this embodiment, when a user needs to detect frame freezes (stuttering) in audio and video services such as live streams or videos, the decoding frame rate can be controlled to obtain target image pairs.

[0121] The target image pair refers to the grayscale image corresponding to two adjacent frames. Specifically, it can be represented as an image pair consisting of two adjacent frames downloaded from a live stream or video, and the image obtained after grayscale processing of the image pair.

[0122] Specifically, such as Figure 14 As shown, when a user needs to perform frame freeze (stuttering) detection on audio and video services such as live streams or videos, the target object to be tested (e.g., [example object]) can be determined based on the decoding frame rate. Figure 14 Decode the live stream or video (as shown) to obtain the target object to be tested (e.g., Figure 14 The set of image frames corresponding to the indicated live stream or video (e.g., Figure 14The images shown are 001, 002, 003 to n, etc. Then, the image frame set (e.g., ...) is... Figure 14 By combining adjacent image frames from images 001, 002, 003 to n (as shown), the original image pair can be obtained (e.g., ...). Figure 14 The original image pairs are then subjected to grayscale processing to obtain the target image pairs, which are shown as image pairs 001, 002 to n-1, etc.

[0123] In step S102, each grayscale image in the target image pair is subjected to saliency processing to preserve the high-contrast region, resulting in a first saliency image and a second saliency image.

[0124] In this embodiment, after obtaining the target image pair, the high-contrast region of each grayscale image in the target image pair can be preserved through saliency processing, and the high-contrast region can be displayed with a significantly higher grayscale value to obtain the first saliency image and the second saliency image, thereby achieving pre-noise reduction of the grayscale image and reducing the interference of unnecessary noise in the image.

[0125] Specifically, such as Figure 15 As shown, after obtaining the target image pair, some contrast details will be lost when the original image pair is converted from a colored RGB image to a grayscale image. However, since the human eye often notices the area with higher contrast first when observing the difference, this embodiment can perform saliency processing on each grayscale image in the target image pair to retain the high contrast area, so as to obtain the first saliency image and the second saliency image.

[0126] For each grayscale image in the target image pair (e.g.) Figure 16 The grayscale image shown is subjected to saliency processing to preserve high-contrast regions. This can be done using a saliency algorithm, which can specifically be a visual saliency algorithm, an image saliency algorithm, or other saliency algorithms. No specific limitation is made here. Preferably, in this embodiment, it can be done by statistically analyzing the first saliency spectrum and the cepstral of the first image mean for each grayscale image in the target image pair, and the second saliency spectrum and the cepstral of the second image mean. Further, the difference between the first saliency spectrum and the cepstral of the first image mean can be calculated to obtain a first difference, which is used as a first saliency weight. This first saliency weight is then mapped to the grayscale image to obtain the first saliency image (e.g., a grayscale image). Figure 18 The saliency image is shown. Similarly, the difference between the second spectrum and the cepstral of the second image mean can be calculated to obtain the second difference, which is used as the second saliency weight. The second saliency weight is then mapped to the grayscale image to obtain the second saliency image.

[0127] In step S103, grayscale difference processing is performed on the first saliency image and the second saliency image to obtain a grayscale difference image;

[0128] In this embodiment, after obtaining the first saliency image and the second saliency image, in order to more accurately capture or display the differences between the first saliency image and the second saliency image, grayscale difference processing can be performed on the first saliency image and the second saliency image to obtain a grayscale difference image.

[0129] Specifically, such as Figure 15 As shown, after obtaining the first saliency image and the second saliency image, the first saliency image (such as...) can be... Figure 18 The grayscale difference between the first salient image (as shown) and the second salient image is processed. Specifically, a grayscale difference calculation can be performed to obtain the first salient image (e.g., the image with the grayscale difference). Figure 18 The grayscale difference between the saliency image (shown as intended) and the second saliency image can be expressed as follows (1):

[0130] Sub_img=abs(S_img_0-S_img_1) (1);

[0131] Where Sub_img represents the grayscale difference image, and abs(S_img_0-S_img_1) represents the first saliency image (e.g., ... Figure 18 The absolute value of the difference between the first saliency image (shown as intended) and the second saliency image.

[0132] In step S104, contour processing is performed on the grayscale difference image to obtain the target contour with the largest difference area value;

[0133] In this embodiment, after obtaining the grayscale difference image, since the sum of all area differences existing in the entire grayscale difference image is statistically calculated, the judgment of image freezing may be difficult to interpret due to the insignificant area interference. Therefore, this embodiment can perform contour processing on the grayscale difference image to obtain the target contour with the largest difference area value.

[0134] Specifically, such as Figure 15 As shown, after acquiring the grayscale difference image, contour processing can be performed on the grayscale difference image to obtain the target contour with the largest difference area value. Specifically, this can be achieved by extracting contour features from the grayscale difference image to obtain multiple contours containing the target object (such as...). Figure 19The white area shown is used to define the difference contours, and the area of ​​each difference contour is calculated to obtain the difference area value of each difference contour. Then, the difference area values ​​of each difference contour can be compared pairwise to obtain the target contour with the largest difference area value.

[0135] In step S105, if the difference area value of the target contour is less than the area threshold, it is determined that the target image pair has an image freeze frame.

[0136] Specifically, such as Figure 15 As shown, after obtaining the target contour with the largest difference area value, the difference area value of the target contour can be compared with the area threshold. When the difference area value of the target contour is less than the area threshold, it can be understood that there is almost no difference between the two grayscale images in the target image pair, that is, the grayscale images corresponding to the two adjacent frames are similar or the same, and then it can be determined that there is image freeze in the target image pair.

[0137] Similarly, such as Figure 14 As shown, after obtaining the target object to be tested (such as...) Figure 14 The indicated live stream or video) corresponds to multiple target image pairs (e.g. Figure 14 The target contour with the largest difference area value for the image combination (001, 002 to n-1, etc.) is given. For example, assuming the maximum difference area value of the target contour corresponding to image combination 001 is 120.07, the maximum difference area value of the target contour corresponding to image combination 002 is 280.12, the maximum difference area value of the target contour corresponding to image combination 003 is 30.21, the maximum difference area value of the target contour corresponding to image combination 004 is -1, and the maximum difference area value of the target contour corresponding to image combination 005 is -1, and assuming the area threshold is 35, then each maximum difference area value is compared with the area threshold, and the image freeze can be represented as "false, false, true, false and false", that is, image combination 003 has an image freeze.

[0138] Understandably, experiments have shown that, for example... Figure 23 The diagram illustrates the comparison of frozen frame detection effects. Compared to traditional frozen frame detection techniques, this embodiment, based on human visual characteristics and using the maximum difference area to determine whether an image is frozen, shows a significant improvement in accuracy.

[0139] It is understood that this embodiment can be applied to business scenarios such as discovering stuttering issues in live streaming platforms or video playback applications based on comment systems, automated regression testing of live streaming platforms or video playback applications, detection of live short video issues, and verification of automated test results of the live streaming toolkit SDK for live streaming platforms or video playback applications. It can also be applied to other business scenarios, and no specific limitations are made here.

[0140] In this embodiment, a method for detecting frozen frames in images is provided. Through the above method, the grayscale images corresponding to two adjacent frames in a target image pair are subjected to saliency processing to preserve high-contrast regions. This yields images of regions with significantly higher grayscale values ​​and higher contrast, which better match the differences observed by the human eye. Furthermore, based on the saliency processing, grayscale difference processing is performed on the first and second saliency images. This further emphasizes the differences between the images that better match human visual characteristics, avoiding the neglect or weakening of local differences between images. Then, based on the grayscale difference processing, contour processing is combined to quantify the differences between the images that better match human visual characteristics through the difference area value. Therefore, the presence of frozen frames in the target image pair can be determined based on the maximum difference area value, thereby improving the accuracy of obtaining the relationship between image frames.

[0141] Optionally, in the above Figure 2 Based on the corresponding embodiments, in another optional embodiment of the image frozen frame detection method provided in this application, such as... Figure 3 As shown, step S102 performs saliency processing on each grayscale image in the target image pair to preserve the high-contrast regions, obtaining a first saliency image and a second saliency image, including:

[0142] In step S301, Gaussian blur filtering is performed on each grayscale image in the target image pair to obtain a first Gaussian blurred image and a second Gaussian blurred image.

[0143] In step S302, the first Gaussian blurred image and the second Gaussian blurred image are subjected to saliency processing to preserve the high contrast region, respectively, to obtain the first saliency image and the second saliency image.

[0144] In this embodiment, after obtaining the target image pair, Gaussian blur filtering can be performed on each grayscale image in the target image pair to obtain a first Gaussian blurred image and a second Gaussian blurred image, thereby achieving pre-denoising of the grayscale images and reducing noise interference in the grayscale images. Then, the first Gaussian blurred image and the second Gaussian blurred image can be subjected to saliency processing to retain the high contrast region to obtain a first saliency image and a second saliency image.

[0145] Specifically, such as Figure 15 As shown, after obtaining the target image pair, you can first process each grayscale image in the target image pair (e.g., ... Figure 16 The grayscale image shown is subjected to Gaussian blur filtering. Specifically, this can be achieved by sequentially scanning each pixel in the grayscale image using a Gaussian filter convolution kernel corresponding to the size of the Gaussian filter to obtain the pixels in the first neighborhood of each pixel. Then, based on the filter weights, a weighted average of the grayscale values ​​corresponding to the pixels in the first neighborhood is calculated to obtain the weighted average grayscale value corresponding to each pixel in the first neighborhood. This weighted average grayscale value is then used to replace the grayscale value of the center pixel to obtain the first Gaussian blurred image (e.g., ...). Figure 17 (The Gaussian blurred image shown) and the second Gaussian blurred image.

[0146] Furthermore, such as Figure 15 As shown, after obtaining the first Gaussian blurred image and the second Gaussian blurred image, saliency processing to preserve high-contrast regions can be performed on both images. Specifically, this can be achieved by statistically analyzing the first saliency spectrum corresponding to the first Gaussian blurred image and the cepstrum of the first image's mean, and calculating the difference between the first saliency spectrum and the cepstrum of the first image's mean to obtain the first difference. This first difference can then be used as the first saliency weight corresponding to the first Gaussian blurred image, and mapping this first saliency weight back to the first Gaussian blurred image to obtain the first saliency image (e.g., ...). Figure 17 The Gaussian blurred image is shown in the figure. Similarly, the second spectrum and the cepstral of the mean of the second image corresponding to the second Gaussian blurred image can be statistically analyzed, and the difference between the second spectrum and the cepstral of the mean of the second image can be calculated to obtain the second difference. Then, the second difference can be used as the second saliency weight corresponding to the second Gaussian blurred image, and the second saliency weight can be mapped to the second Gaussian blurred image to obtain the second saliency image.

[0147] Optionally, in the above Figure 2 Based on the corresponding embodiments, in another optional embodiment of the image frozen frame detection method provided in this application, such as... Figure 4 As shown, step S104 performs contour processing on the grayscale difference image to obtain the target contour with the largest difference area value, including:

[0148] In step S401, the grayscale difference image is subjected to median filtering to obtain a median-filtered difference image;

[0149] In step S402, contour processing is performed on the median-filtered difference image to obtain the target contour with the largest difference area value.

[0150] In this embodiment, after obtaining the grayscale difference image, median filtering can be performed on the grayscale difference image to obtain a median-filtered difference image, thereby achieving post-denoising of the grayscale image and reducing isolated noise points in the grayscale image. Then, contour processing can be performed on the median-filtered difference image to better obtain the target contour with the largest difference area value.

[0151] Specifically, such as Figure 15 As shown, after obtaining the grayscale difference image (such as...) Figure 19 After obtaining the grayscale difference image (as shown), median filtering can be applied to the grayscale difference image to obtain the median-filtered difference image (e.g., ...). Figure 20 The median-filtered difference image shown can be obtained by sequentially scanning each pixel in the grayscale difference image using a median filter convolution kernel corresponding to the median filter size, to obtain the pixels in the second neighborhood corresponding to each pixel, and then calculating the median of the grayscale values ​​of the pixels in the second neighborhood to obtain the median of each pixel in the second neighborhood. Finally, the median of each pixel in the neighborhood is used to replace the grayscale value of the center pixel to obtain the median-filtered difference image (e.g., Figure 20 (The median-filtered difference image is shown).

[0152] Optionally, in the above Figure 2 Based on the corresponding embodiments, in another optional embodiment of the image frozen frame detection method provided in this application, such as... Figure 5 As shown, step S101, acquiring the target image pair, includes:

[0153] In step S501, the first color histogram and the second color histogram corresponding to the original image pair are obtained, wherein the original image pair is the color image corresponding to two adjacent frames;

[0154] In step S502, the similarity between the first color histogram and the second color histogram is calculated to obtain the color histogram similarity.

[0155] In step S503, if the color histogram similarity is greater than the first similarity threshold, the original image pair is processed to obtain the target image pair.

[0156] In this embodiment, when acquiring the target object to be tested (e.g. Figure 14 After obtaining the live stream or video (as shown), the original image pairs can be obtained from the target object to be tested. Then, the first color histogram and the second color histogram corresponding to the original image pairs can be obtained. The similarity between the first color histogram and the second color histogram can be calculated to obtain the color histogram similarity. Then, when the color histogram similarity is greater than the first similarity threshold, the original image pairs can be processed into grayscale to obtain the target image pairs.

[0157] The original image pair consists of the color images corresponding to two adjacent frames (e.g., ...). Figure 21 The original image pair shown.

[0158] Specifically, in acquiring the target object to be tested (such as...) Figure 14 After the live stream or video (as shown), the target object to be tested is decoded according to the decoding frame rate to obtain the set of image frames corresponding to the target object. Then, any pair of adjacent image frames in the set can be used as the original image pair. Then, the color pixel values ​​of the color images in the original image pair are counted to obtain the interval distribution list composed of the number of pixel values ​​corresponding to each color interval, that is, the first color histogram (e.g., ...). Figure 22 The diagram shows a color histogram containing 16 color ranges and a second color histogram.

[0159] Furthermore, the similarity between the first color histogram and the second color histogram can be calculated to obtain the color histogram similarity. Specifically, the first color histogram can be normalized to obtain the first color distribution interval of the first color histogram, and the mean of the first color distribution interval can be calculated based on the number of intervals to obtain the first color distribution mean. Similarly, the second color histogram can be normalized to obtain the second color distribution interval of the second color histogram, and the mean of the second color distribution interval can be calculated based on the number of intervals to obtain the second color distribution mean. Then, the similarity between the first color distribution interval, the first color distribution mean, the second color distribution interval, and the second color distribution mean can be calculated to obtain the color histogram similarity.

[0160] Furthermore, the color histogram similarity is compared with a first similarity threshold, which is set according to actual application requirements and is not specifically limited here. When the color histogram similarity is greater than the first similarity threshold, it can be understood that the two color images of the original image pair are similar or identical, that is, the original image pair is very likely to have image freeze-frames, and the original image pair can be processed into grayscale to obtain the target image pair. Conversely, when the color histogram similarity is less than or equal to the first similarity threshold, it can be understood that the two color images of the original image pair are dissimilar and have differences (e.g., ...). Figure 21 If the original image pair (circled as shown) has no image freeze-frames, then the original image pair can be filtered.

[0161] For example, Figure 14As shown, assuming the color histogram similarity of image combination 001 is 0.993, the color histogram similarity of image combination 002 is 0.991, the color histogram similarity of image combination 003 is 0.998, the color histogram similarity of image combination 004 is 0.931, and the color histogram similarity of image combination 005 is 0.93, and assuming the first similarity threshold is 0.9, then these image combinations are all similar or identical. Therefore, grayscale processing can be performed on these image combinations separately to obtain the target image pairs.

[0162] Optionally, in the above Figure 5 Based on the corresponding embodiments, in another optional embodiment of the image frozen frame detection method provided in this application, such as... Figure 6 As shown, in step S503, if the color histogram similarity is greater than the first similarity threshold, then the original image pair is subjected to grayscale processing to obtain the target image pair, including:

[0163] In step S601, if the color histogram similarity is greater than the first similarity threshold, the original image pair is subjected to grayscale processing to obtain a basic grayscale image pair.

[0164] In step S602, structural similarity is calculated for the basic grayscale image pairs to obtain the structural similarity.

[0165] In step S603, if the structural similarity is greater than the second similarity threshold, the basic grayscale image pair is taken as the target image pair.

[0166] In this embodiment, when the color histogram similarity is greater than the first similarity threshold, the original image pair can be processed into grayscale to obtain a basic grayscale image pair. In order to more accurately identify image pairs with large differences in the original image pair and achieve accurate filtering of non-image frozen frames, this embodiment can perform structural similarity calculation on the basic grayscale image pair. Then, when the structural similarity is greater than the second similarity threshold, the basic grayscale image pair is used as the target image pair.

[0167] Specifically, when the color histogram similarity is greater than the first similarity threshold, the original image pair can be processed into grayscale to obtain a basic grayscale image pair. Then, the structural similarity algorithm SSIM can be used to filter the basic grayscale image pair. Specifically, the average grayscale of the basic grayscale image pair can be calculated to obtain the basic brightness contrast value of the basic grayscale image pair. At the same time, the grayscale standard deviation of the basic grayscale image pair can be calculated to obtain the basic contrast measurement value of the basic grayscale image pair. Then, the structural contrast value of the basic grayscale image pair can be calculated based on the basic brightness contrast value and the basic contrast measurement value. Finally, the structural similarity can be calculated based on the basic brightness contrast value, the basic contrast measurement value, and the structural contrast value to obtain the structural similarity.

[0168] Furthermore, the obtained structural similarity can be compared with a second similarity threshold, which is set according to actual application requirements and is not specifically limited here. When the structural similarity is greater than the second similarity threshold, it can be understood that the basic grayscale image pair is very likely to be similar or identical images, that is, there may be image freeze frames. In this case, the basic grayscale image pair is taken as the target image pair. Conversely, when the structural similarity is less than or equal to the second similarity threshold, it can be understood that the two grayscale images of the basic grayscale image pair are not similar, that is, there are no image freeze frames in the basic grayscale image pair. In this case, the basic grayscale image pair can be filtered.

[0169] For example, Figure 14 As shown, assuming the structural similarity of image combination 001 is 0.987, the structural similarity of image combination 002 is 0.972, the structural similarity of image combination 003 is 0.991, the structural similarity of image combination 004 is -1, and the structural similarity of image combination 005 is 0.932, and assuming the first similarity threshold is 0.95, then only if image combinations 001 to 003 are similar or identical can image combinations 001 to 003 be considered as target image pairs.

[0170] Optionally, in the above Figure 6 Based on the corresponding embodiments, in another optional embodiment of the image frozen frame detection method provided in this application, such as... Figure 7 As shown, step S602 calculates the structural similarity of the basic grayscale image pairs to obtain the structural similarity, including:

[0171] In step S701, the average gray level of the basic gray level image pair is calculated to obtain the basic brightness contrast value of the basic gray level image pair.

[0172] In step S702, the grayscale standard deviation is calculated for the basic grayscale image pair to obtain the basic contrast measurement value of the basic grayscale image pair;

[0173] In step S703, the structural contrast value of the basic grayscale image pair is calculated based on the basic brightness contrast value and the basic contrast measurement value;

[0174] In step S704, structural similarity is calculated based on the basic brightness contrast value, the basic contrast measurement value, and the structural contrast value to obtain the structural similarity.

[0175] Specifically, after obtaining the basic grayscale image pair, the average grayscale of the basic grayscale image pair can be calculated using the following formula (2) to obtain the basic brightness contrast value of the basic grayscale image pair:

[0176]

[0177] Where, x i This represents the grayscale value corresponding to each pixel in each grayscale image of the basic grayscale image pair. N is the number of pixels, C1 is a constant parameter value that can be set according to actual application requirements, and no specific restrictions are imposed here. I(x, y) represents the basic brightness contrast value of the basic grayscale image pair, where x and y are used to represent a grayscale image of the basic grayscale image pair, and μ x and μ y These represent the average gray level of one of the gray-level images in the basic gray-level image pair.

[0178] Meanwhile, after obtaining the basic grayscale image pair, the grayscale standard deviation of the basic grayscale image pair can be calculated using the following formula (3) to obtain the basic contrast measurement value of the basic grayscale image pair:

[0179]

[0180] Where, x i It represents the gray value corresponding to each pixel in each grayscale image of the basic grayscale image pair, μ. x σ represents the average gray level of one of the grayscale images in the basic grayscale image pair, N is the number of pixels, C2 is a constant parameter value that can be set according to actual application requirements, and no specific restrictions are imposed here. x and σ y The values ​​are respectively used to represent the grayscale standard deviation of one grayscale image in the basic grayscale image pair, and c(x, y) represents the basic contrast measurement value of the basic grayscale image pair, where x and y are used to represent one grayscale image in the basic grayscale image pair.

[0181] Furthermore, after obtaining the basic brightness contrast value and the basic contrast measurement value, the structural contrast value of the basic grayscale image pair can be calculated based on the basic brightness contrast value and the basic contrast measurement value using the following formula (4):

[0182]

[0183] Where, σ x and σ y The values ​​represent the grayscale standard deviation of one grayscale image in the basic grayscale image pair, respectively. C3 is a constant parameter value that can be set according to actual application requirements; no specific restrictions are imposed here. x and y are used to represent one grayscale image in the basic grayscale image pair, respectively, and μ. x y represents the average gray level of one of the gray-level images in the basic gray-level image pair, and s(x,y) represents the structural contrast value of the basic gray-level image pair.

[0184] Furthermore, combining the above three dimensions, the following formula (5) can be used to calculate the structural similarity based on the basic luminance contrast value, the basic contrast measurement value, and the structural contrast value, in order to obtain the structural similarity:

[0185]

[0186] Here, S(x, y) represents the structural similarity, which can be specifically represented by the ssim value.

[0187] Furthermore, such as Figure 15 As shown, the SSIM value is calculated based on two grayscale images from the obtained base grayscale image pair and compared with a preset second similarity threshold, such as SSIM_THRESHOLD = 0.95. The second similarity threshold, SSIM_THRESHOLD = 0.95, was selected based on actual experimental results and can be set to other values; no specific restrictions are imposed here. Therefore, the second layer of filtering (such as...) Figure 15 The SSIM filter layer shown can be expressed as follows (6):

[0188] SSIMCheck=CalSSIM(img_0,img_1)>SSIM_THRESHOLD (6);

[0189] Here, SSIMCheck is used to represent the filtering result of the SSIM algorithm, CalSSIM method is used to represent the SSIM calculation, and img_0 and img_1 are used to represent one grayscale image in the basic grayscale image pair.

[0190] Optionally, in the above Figure 5 Based on the corresponding embodiments, in another optional embodiment of the image frozen frame detection method provided in this application, such as... Figure 8 As shown, step S502 calculates the similarity between the first color histogram and the second color histogram to obtain the color histogram similarity, including:

[0191] In step S801, the first color histogram is normalized to obtain the first color distribution range of the first color histogram.

[0192] In step S802, the mean of the first color distribution interval is calculated based on the number of intervals to obtain the mean of the first color distribution;

[0193] In step S803, the second color histogram is normalized to obtain the second color distribution range of the second color histogram;

[0194] In step S804, the mean of the second color distribution interval is calculated based on the number of intervals to obtain the mean of the second color distribution;

[0195] In step S805, similarity is calculated based on the first color distribution interval, the first color distribution mean, the second color distribution interval, and the second color distribution mean to obtain the color histogram similarity.

[0196] Specifically, after obtaining the first color histogram (e.g.) Figure 22 After obtaining the color histogram (shown as a representation of a color range containing 16 color intervals), the first color histogram can be normalized. Specifically, this can be done by normalizing the statistically obtained first color histogram (e.g., ...). Figure 22 The distribution interval values ​​of the color histogram (shown as an example, containing 16 color intervals) are mapped to [0,1] to obtain the first color distribution interval for convenient subsequent processing. Furthermore, the mean of the first color distribution interval can be calculated based on the number of intervals using the following formula (7) to obtain the mean of the first color distribution:

[0197]

[0198] in, Different k values ​​represent the mean of the first color distribution (e.g.) ) or the mean of the second color distribution (e.g. M is used to represent the number of intervals, and H is used to represent the number of intervals. k (I) The first color distribution interval (e.g., H1(I)) or the second color distribution interval (e.g., H2(I)) is represented by different k values.

[0199] Similarly, after obtaining the second color histogram, the second color histogram can be normalized. Specifically, the distribution interval values ​​of the statistically obtained second color histogram can be mapped to [0,1] to obtain the second color distribution interval that is convenient for subsequent processing. Furthermore, the above formula (7) can be used to calculate the mean of the second color distribution interval based on the number of intervals to obtain the mean of the second color distribution.

[0200] Furthermore, after obtaining the first color distribution interval, the first color distribution mean, the second color distribution interval, and the second color distribution mean, the following formula (8) can be used to calculate the similarity based on the first color distribution interval, the first color distribution mean, the second color distribution interval, and the second color distribution mean, so as to obtain the color histogram similarity:

[0201]

[0202] Where H1(I) represents the first color distribution interval, and H2(I) represents the second color distribution interval. This represents the mean of the first color distribution. d represents the mean of the second color distribution, and d(H1,H2) represents the color histogram similarity.

[0203] Furthermore, such as Figure 15 As shown, specifically, after calculating the color histogram similarity based on the obtained first and second color histograms, it can be compared with a preset first similarity threshold, such as HIST_THRESHOLD = 0.99, to determine whether to proceed to the next layer of filtering. Therefore, the first layer of filtering (such as...) Figure 15 The color histogram filter layer shown can be represented by the following expression (9):

[0204] HistCheck=CalHist(img_0,Img_1)>HIST_THRESHOLD (9);

[0205] Here, HistCheck represents the color histogram filtering result, CalHist method represents the color histogram calculation method, and img_0 and Img_1 are used to represent the first color histogram and the second color histogram, respectively.

[0206] Understandably, experiments have shown that, for example... Figure 24 The diagram illustrates the comparison of average time taken for frozen frame detection calculation. This embodiment adds a pre-filtering fast layer (such as...). Figure 15 The color histogram filter layer and SSIM filter layer shown can be used to improve detection speed, so as to achieve high availability standards in real-world scenarios.

[0207] Optionally, in the above Figure 5 Based on the corresponding embodiments, in another optional embodiment of the image frozen frame detection method provided in this application, such as... Figure 9 As shown, step S501 obtains the first color histogram and the second color histogram corresponding to the original image pair, including:

[0208] In step S901, the target object to be tested is decoded according to the decoding frame rate to obtain the set of image frames corresponding to the target object to be tested;

[0209] In step S902, any pair of adjacent image frames in the image frame set is taken as the original image pair;

[0210] In step S903, the color pixel values ​​of the color images in the original image pair are counted to obtain the number of pixel values ​​corresponding to each color interval, so as to obtain the first color histogram and the second color histogram.

[0211] In this embodiment, after obtaining the target object to be tested (e.g., Figure 14 When dealing with a live stream or video (as shown), the target object to be tested can be decoded according to the decoding frame rate to obtain the set of image frames corresponding to the target object. Then, any pair of adjacent image frames in the set of image frames can be used as the original image pair, and the color pixel values ​​of the color images in the original image pair can be counted to obtain the number of pixel values ​​corresponding to each color interval, so as to obtain the corresponding interval distribution list, i.e., the first color histogram or the second color histogram.

[0212] Specifically, the target object to be tested can be obtained from a local video address (e.g., / data / test.mp4), a URL formatted video address, or a live stream address (e.g., http: / / www.test.com / test.mp4). Other methods can also be used to obtain the target object (e.g., ...). Figure 14 (The live stream or video shown is not specifically limited here. For example, the target object to be tested can be obtained through the entry point and input of the detection program (such as python freeze_detect.py data / test.mp4).

[0213] Furthermore, based on the decoding frame rate, the target object under test is decoded to obtain the set of image frames corresponding to the target object under test. Specifically, the tool FFMPEG can be used to decode the target object under test according to the required frame rate, i.e., the decoding frame rate. For example, assuming the decoding frame rate is 1 image per second, then a 24-second video will yield 24 images.

[0214] Furthermore, any pair of adjacent image frames in the image frame set can be used as the original image pair. For example, they can be paired up one after another to form the image group to be tested, i.e., the original image pair. For example, if there are 24 images, they can be combined into 23 image groups.

[0215] Furthermore, the color pixel values ​​in the original image pair can be counted to obtain the number of pixel values ​​corresponding to each color interval, forming a corresponding interval distribution list, namely the first color histogram and the second color histogram.

[0216] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment of the image frozen frame detection method provided in this application, such as... Figure 10 As shown, step S302 performs saliency processing on the first Gaussian blurred image and the second Gaussian blurred image respectively, preserving the high-contrast regions, to obtain the first saliency image and the second saliency image, including:

[0217] In step S1001, the difference between the first spectrum corresponding to the first Gaussian blurred image and the cepstral of the mean of the first image is calculated to obtain the first difference.

[0218] In step S1002, the first difference is used as the first saliency weight corresponding to the first Gaussian blurred image;

[0219] In step S1003, the first saliency weight is mapped to the first Gaussian blurred image to obtain the first saliency image;

[0220] Specifically, the first Gaussian blurred image is processed using a saliency processing algorithm to preserve high-contrast regions for subsequent comparison. Since the cepstrum of the images has similarity, this embodiment can obtain the spectral residual by subtracting the image's cepstrum from the image's mean cepstrum using the following formula (10), thereby analyzing the salient regions:

[0221] R(f)=L(f)-A(f) (10);

[0222] Where A(f) represents the mean cepstral obtained from general image statistics, i.e. the mean cepstral of the first image, L(f) represents the first cepstral of the first Gaussian blurred image, and R(f) represents the first difference between the two, i.e. the statistical singularity.

[0223] Furthermore, the first difference can be used as the first saliency weight corresponding to the first Gaussian blurred image. Then, the following expression (11) can be used to map back to the grayscale image, i.e., the first Gaussian blurred image, according to the first saliency weight, thereby obtaining the grayscale image after saliency processing (e.g., Figure 18 The first salient image (shown as a salient image) has high-contrast regions with significantly high grayscale values, where the maximum grayscale value is MAX_GRAY = 256.

[0224] S_img_n=Saliency(G_img_n)*MAX_GRAY (11);

[0225] Where S_img_n represents the image after saliency processing, such as the first saliency image, Saliency(G_img_n) represents the saliency processing method, and n∈[0,1] can be used to represent the first Gaussian blurred image and the second Gaussian blurred image.

[0226] In step S1004, the difference between the second spectrum corresponding to the second Gaussian blurred image and the cepstral of the mean of the second image is calculated to obtain the second difference.

[0227] In step S1005, the second difference is used as the second saliency weight corresponding to the second Gaussian blurred image;

[0228] In step S1006, the second saliency weight is mapped to the second Gaussian blurred image to obtain the second saliency image.

[0229] Specifically, the second Gaussian blurred image is processed using a saliency processing algorithm similar to that of the first Gaussian blurred image, so as to retain the high contrast area for subsequent comparison. That is, the difference between the spectrum of the image and the cepstral of the image mean can be obtained by the above formula (10) to obtain the spectrum residual, i.e. the second difference, and used as the second saliency weight corresponding to the second Gaussian blurred image. Then, the above expression (11) can be used to map back to the grayscale image, i.e. the second Gaussian blurred image, according to the second saliency weight, so as to obtain the grayscale image after saliency processing, whose high contrast area has obvious high grayscale value, i.e. the second saliency image.

[0230] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment of the image frozen frame detection method provided in this application, such as... Figure 11 As shown, step S301 performs Gaussian blur filtering on each grayscale image in the target image pair to obtain a first Gaussian blurred image and a second Gaussian blurred image, including:

[0231] In step S1101, the Gaussian filter convolution kernel corresponding to the size of the Gaussian filter is used to scan each pixel in each grayscale image in turn to obtain the first neighboring pixels of each pixel.

[0232] In step S1102, based on the filtering weights, the gray values ​​corresponding to the pixels in the first neighborhood are weighted and averaged to obtain the weighted average gray value corresponding to each pixel in the first neighborhood.

[0233] In step S1103, the gray value of the center pixel is replaced by the weighted average gray value corresponding to each pixel in the first neighborhood to obtain the first Gaussian blurred image and the second Gaussian blurred image.

[0234] Specifically, Gaussian blur filtering is performed on each grayscale image in the target image pair. Since the essence of Gaussian blur is a process of filtering high-frequency signals and retaining low-frequency signals, a convolution filtering algorithm based on Gaussian kernel can be used. The Gaussian filter convolution kernel corresponding to the size of the Gaussian filter is used to scan each pixel in each grayscale image in turn to obtain the pixels in the first neighborhood corresponding to each pixel. The size of the Gaussian filter can be expressed as (2n*1) or other sizes. No specific restrictions are made here. Then, the grayscale values ​​corresponding to the pixels in the first neighborhood can be weighted and averaged based on the filter weight to obtain the weighted average grayscale corresponding to each pixel in the first neighborhood. Then, the weighted average grayscale corresponding to each pixel in the first neighborhood replaces the grayscale value of the center pixel to obtain the first Gaussian blurred image and the second Gaussian blurred image. The Gaussian kernel convolution filtering can be specifically performed using the following formula (12):

[0235]

[0236] Here, G(u, v) can be used to represent the value of each pixel coordinate in each grayscale image after Gaussian filtering.

[0237] Furthermore, Gaussian blur filtering can be applied to each grayscale image in the target image pair using the expression: G_img_n = GaussBlur(img_n).

[0238] Where G_img_n represents the image after Gaussian blurring, such as the first Gaussian blurred image and the second Gaussian blurred image, GaussBlur(img_n) is used to represent the Gaussian blur method, and n∈[0,1] can be used to represent the grayscale image in the target image pair.

[0239] Optionally, in the above Figure 4 Based on the corresponding embodiments, in another optional embodiment of the image frozen frame detection method provided in this application, such as... Figure 12 As shown, step S401 performs median filtering on the grayscale difference image to obtain a median-filtered difference image, including:

[0240] In step S1201, the median filter convolution kernel corresponding to the median filter size is used to scan each pixel in the grayscale difference image in turn to obtain the second neighboring pixels corresponding to each pixel.

[0241] In step S1202, the median value of the gray values ​​corresponding to the pixels in the second neighborhood is calculated to obtain the median value corresponding to each pixel in the second neighborhood.

[0242] In step S1203, the gray value of the center pixel is replaced by the median value of each neighboring pixel to obtain a median-filtered difference image.

[0243] Specifically, when acquiring grayscale difference images (such as...) Figure 19 After the grayscale difference image shown, it can be seen that there are still a lot of noise in the grayscale difference image. However, these noises are not the part that needs special attention in the difference area calculation. Therefore, the grayscale difference image can be processed by median filtering. The median filtering convolution kernel corresponding to the median filtering size is used to scan each pixel in the grayscale difference image in turn to obtain the second neighboring pixels of each pixel. The median filtering size can be expressed as (2n*1) or other sizes. There is no specific restriction here. Then, the median value of the grayscale value corresponding to the second neighboring pixel can be calculated to obtain the median value corresponding to each second neighboring pixel. The median value corresponding to each neighboring pixel is used to replace the grayscale value of the center pixel to obtain the median filtered difference image. That is, it can be understood as replacing the value of a point in the image with the median value of each point in a neighborhood of that point, so that the value of the surrounding pixels is close to the real value, thereby eliminating isolated noise points. It can be expressed as the following expression (13):

[0244] g(x,y)=med{f(xk,yl),(k,l∈W)} (13);

[0245] Where f(x,y) is used to represent the original image, i.e., the grayscale difference image (e.g., ...). Figure 19 The image shown is a grayscale difference image. g(x,y) is used to represent the processed image, i.e., the median-filtered difference image (e.g., the image with grayscale difference). Figure 20 The median-filtered difference image is shown in the figure. W is a two-dimensional template, which can be understood as the median filter convolution kernel corresponding to the median filter size (e.g., 2n*1).

[0246] Among them, the median filtering method for eliminating snowflake noise in an image while retaining the area of ​​the larger difference can also be expressed as the following expression (14):

[0247] M_img = med(Sub_img) (14);

[0248] Where M_img represents the image after median filtering, i.e., the median-filtered difference image (e.g.) Figure 20 The image shown is a median-filtered difference image, and med(Sub_img) represents the median filtering method.

[0249] Optionally, in the above Figure 4 Based on the corresponding embodiments, in another optional embodiment of the image frozen frame detection method provided in this application, such as... Figure 12As shown, step S402 performs contour processing on the median-filtered difference image to obtain the target contour with the largest difference area value, including:

[0250] In step S1301, contour features are extracted from the median-filtered difference image to obtain multiple difference contours containing the target object;

[0251] In step S1302, the area of ​​each differential contour is calculated to obtain the differential area value of each differential contour;

[0252] In step S1303, pairwise comparisons are performed based on the difference area value of each difference contour to obtain the target contour with the largest difference area value.

[0253] Specifically, after obtaining the median-filtered difference image, the contours in the image can be obtained through a contour extraction algorithm, that is, the contour feature extraction can be performed on the median-filtered difference image as shown in (15) to obtain multiple difference contours containing the target object:

[0254] Contours=FindContours(M_img) (15);

[0255] Where M_img represents the image after median filtering, i.e., the median-filtered difference image (e.g.) Figure 20 The image shown is a median-filtered difference image. Contours represents the extracted contours within the image, i.e., the difference contours. FindContours(M_img) is the contour extraction algorithm.

[0256] Furthermore, through experimental testing, it was found that summing all area differences in the entire image would lead to difficulties in interpreting the calculation results and defining threshold standards due to insignificant area interference. Therefore, this embodiment uses the following expression (16) to take the largest difference area, which is more in line with human visual observation standards, as the total calculation result:

[0257] Areas=ContourArea(Contours) (16);

[0258] Here, Areas represents the list of contour areas, which is the set of difference area values ​​for each different contour, and ContourArea(Contours) represents the contour area calculation method.

[0259] Furthermore, pairwise comparisons are performed based on the difference area values ​​of each difference contour, and the target contour with the largest difference area value is obtained through the following expression (17):

[0260] Max_Area = max(Areas) (17);

[0261] Here, Max_Area represents the maximum area, and max(Areas) represents obtaining the maximum area value.

[0262] Furthermore, after obtaining the target contour with the largest difference area value, the following formula (18) can be used to compare the difference area value of the target contour with the area threshold such as FREEZE_AREA_THRESHOLD=100 to obtain the frozen frame determination result. That is, when the difference area value of the target contour is less than the area threshold, it can be understood that the target image pair has an image frozen frame:

[0263] Freeze = MAX_Area <FREEZE_AREA_THRESHOLD (18);

[0264] Here, Freeze represents the final frozen frame determination result of the target image, and FREEZE_AREA_THRESHOLD represents the area threshold used for frozen frame determination. FREEZE_AREA_THRESHOLD is set according to the actual use scenario and in combination with interpretability. There is no specific restriction here. Usually, the value of FREEZE_AREA_THRESHOLD can be set to 100, which is equivalent to the area value of a 10px*10px (i.e., 10*10 pixels) region in the image.

[0265] The image freeze detection device of this application is described in detail below. Please refer to [link / reference]. Figure 25 , Figure 25 This is a schematic diagram of one embodiment of the image frozen frame detection device in this application. The image frozen frame detection device 20 includes:

[0266] The acquisition unit 201 is used to acquire a target image pair, wherein the target image pair is a grayscale image corresponding to two adjacent frames;

[0267] The processing unit 202 is used to perform saliency processing on each grayscale image in the target image pair to preserve the high-contrast region, so as to obtain a first saliency image and a second saliency image.

[0268] The processing unit 202 is also used to perform grayscale difference processing on the first saliency image and the second saliency image to obtain a grayscale difference image;

[0269] The processing unit 202 is also used to perform contour processing on the grayscale difference image to obtain the target contour with the largest difference area value;

[0270] The determining unit 203 is used to determine that the target image pair has an image freeze frame if the difference area value of the target contour is less than the area threshold.

[0271] Optionally, in the above Figure 25 Based on the corresponding embodiments, in another embodiment of the image frozen frame detection device provided in this application, the processing unit 202 may specifically be used for:

[0272] Gaussian blur filtering is applied to each grayscale image in the target image pair to obtain the first Gaussian blurred image and the second Gaussian blurred image.

[0273] The first Gaussian blurred image and the second Gaussian blurred image are subjected to saliency processing to preserve the high-contrast regions, respectively, to obtain the first saliency image and the second saliency image.

[0274] Optionally, in the above Figure 25 Based on the corresponding embodiments, in another embodiment of the image frozen frame detection device provided in this application, the processing unit 202 may specifically be used for:

[0275] Median filtering is applied to the grayscale difference image to obtain the median-filtered difference image;

[0276] Contour processing is performed on the median-filtered difference image to obtain the target contour with the largest difference area value.

[0277] Optionally, in the above Figure 25 Based on the corresponding embodiments, in another embodiment of the image frozen frame detection device provided in this application, the acquisition unit 201 can specifically be used for:

[0278] Obtain the first color histogram and the second color histogram corresponding to the original image pair, wherein the original image pair is the color image corresponding to two adjacent frames;

[0279] The similarity between the first color histogram and the second color histogram is calculated to obtain the color histogram similarity.

[0280] If the color histogram similarity is greater than the first similarity threshold, then the original image pair is processed into grayscale to obtain the target image pair.

[0281] Optionally, in the above Figure 25 Based on the corresponding embodiments, in another embodiment of the image frozen frame detection device provided in this application, the acquisition unit 201 can specifically be used for:

[0282] If the color histogram similarity is greater than the first similarity threshold, then the original image pair is processed into grayscale to obtain the basic grayscale image pair.

[0283] Structural similarity is calculated for basic grayscale image pairs to obtain structural similarity.

[0284] If the structural similarity is greater than the second similarity threshold, then the basic grayscale image pair is taken as the target image pair.

[0285] Optionally, in the above Figure 25 Based on the corresponding embodiments, in another embodiment of the image frozen frame detection device provided in this application, the acquisition unit 201 can specifically be used for:

[0286] The average gray level of the basic gray level image pair is calculated to obtain the basic brightness contrast value of the basic gray level image pair.

[0287] The grayscale standard deviation is calculated for the basic grayscale image pairs to obtain the basic contrast measurement value of the basic grayscale image pairs;

[0288] The structural contrast value of the basic grayscale image pair is calculated based on the basic brightness contrast value and the basic contrast measurement value.

[0289] Structural similarity is calculated based on the basic brightness contrast value, the basic contrast measurement value, and the structural contrast value.

[0290] Optionally, in the above Figure 25 Based on the corresponding embodiments, in another embodiment of the image frozen frame detection device provided in this application, the acquisition unit 201 can specifically be used for:

[0291] The first color histogram is normalized to obtain the first color distribution interval of the first color histogram;

[0292] Based on the number of intervals, the mean of the first color distribution interval is calculated to obtain the mean of the first color distribution.

[0293] The second color histogram is normalized to obtain the second color distribution range of the second color histogram;

[0294] Based on the number of intervals, the mean of the second color distribution intervals is calculated to obtain the mean of the second color distribution.

[0295] The similarity is calculated based on the first color distribution interval, the mean of the first color distribution, the second color distribution interval, and the mean of the second color distribution to obtain the color histogram similarity.

[0296] Optionally, in the above Figure 25 Based on the corresponding embodiments, in another embodiment of the image frozen frame detection device provided in this application, the acquisition unit 201 can specifically be used for:

[0297] Based on the decoding frame rate, the target object under test is decoded to obtain the set of image frames corresponding to the target object under test;

[0298] Take any pair of adjacent image frames in the image frame set as the original image pair;

[0299] The color pixel values ​​in the original image pair are counted to obtain the number of pixel values ​​corresponding to each color interval, so as to obtain the first color histogram and the second color histogram.

[0300] Optionally, in the above Figure 25 Based on the corresponding embodiments, in another embodiment of the image frozen frame detection device provided in this application, the processing unit 202 may specifically be used for:

[0301] The difference between the first spectrum corresponding to the first Gaussian blurred image and the cepstral of the mean of the first image is calculated to obtain the first difference;

[0302] The first difference is used as the first saliency weight corresponding to the first Gaussian blurred image;

[0303] The first saliency weight is mapped to the first Gaussian blurred image to obtain the first saliency image;

[0304] The difference between the second spectrum corresponding to the second Gaussian blurred image and the cepstral of the mean of the second image is calculated to obtain the second difference.

[0305] The second difference is used as the second saliency weight corresponding to the second Gaussian blurred image;

[0306] The second saliency weights are mapped to the second Gaussian blurred image to obtain the second saliency image.

[0307] Optionally, in the above Figure 25 Based on the corresponding embodiments, in another embodiment of the image frozen frame detection device provided in this application, the processing unit 202 may specifically be used for:

[0308] Using a Gaussian filter convolution kernel of the same size as the Gaussian filter, each pixel in each grayscale image is scanned sequentially to obtain the first neighboring pixels of each pixel.

[0309] Based on the filtering weights, the gray values ​​corresponding to the pixels in the first neighborhood are weighted and averaged to obtain the weighted average gray value corresponding to each pixel in the first neighborhood.

[0310] Replace the gray value of the center pixel with the weighted average gray value corresponding to each pixel in the first neighborhood to obtain the first Gaussian blurred image and the second Gaussian blurred image.

[0311] Optionally, in the above Figure 25 Based on the corresponding embodiments, in another embodiment of the image frozen frame detection device provided in this application, the processing unit 202 may specifically be used for:

[0312] Using a median filter convolution kernel of the same size as the median filter, each pixel in the grayscale difference image is scanned sequentially to obtain the pixels in the second neighborhood corresponding to each pixel.

[0313] The median of the gray values ​​corresponding to the pixels in the second neighborhood is calculated to obtain the median of each pixel in the second neighborhood.

[0314] The median value of the center pixel is replaced by the median value of each neighboring pixel to obtain a median-filtered difference image.

[0315] Optionally, in the above Figure 25 Based on the corresponding embodiments, in another embodiment of the image frozen frame detection device provided in this application, the processing unit 202 may specifically be used for:

[0316] Contour feature extraction is performed on the median-filtered difference image to obtain multiple difference contours containing the target object;

[0317] Calculate the area of ​​each differential contour to obtain the differential area value for each differential contour;

[0318] Based on the difference area value of each difference contour, pairwise comparisons are performed to obtain the target contour with the largest difference area value.

[0319] This application also provides a schematic diagram of another computer device, such as... Figure 26 As shown, Figure 26 This is a schematic diagram of a computer device structure provided in an embodiment of this application. The computer device 300 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 331 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the computer device 300. Furthermore, the CPU 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the computer device 300.

[0320] Computer device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 333, such as Windows Server. TMMac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0321] The aforementioned computer device 300 is also used to perform, for example Figures 2 to 13 The steps in the corresponding embodiments.

[0322] Another aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements... Figures 2 to 13 The steps in the method described in the illustrated embodiment.

[0323] Another aspect of this application provides a computer program product comprising a computer program, which, when executed by a processor, implements as follows: Figures 2 to 13 The steps in the method described in the illustrated embodiment.

[0324] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0325] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0326] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0327] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0328] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for detecting frozen frames in an image, characterized in that, include: Obtain target image pairs, wherein the target image pairs are grayscale images corresponding to two adjacent frames; Gaussian blur filtering is applied to each grayscale image in the target image pair to obtain a first Gaussian blurred image and a second Gaussian blurred image; The first Gaussian blurred image and the second Gaussian blurred image are subjected to saliency processing to preserve high-contrast regions, resulting in a first saliency image and a second saliency image. This includes: calculating the difference between the first cepstral spectrum corresponding to the first Gaussian blurred image and the cepstral spectrum of the first image mean, obtaining a first difference; using the first difference as a first saliency weight corresponding to the first Gaussian blurred image; mapping the first saliency weight to the first Gaussian blurred image, obtaining the first saliency image; calculating the difference between the second cepstral spectrum corresponding to the second Gaussian blurred image and the cepstral spectrum of the second image mean, obtaining a second difference; using the second difference as a second saliency weight corresponding to the second Gaussian blurred image; mapping the second saliency weight to the second Gaussian blurred image, obtaining the second saliency image. The first saliency image and the second saliency image are subjected to grayscale difference processing to obtain a grayscale difference image; Perform contour processing on the grayscale difference image to obtain the target contour with the largest difference area value; If the difference area value of the target contour is less than the area threshold, then it is determined that the target image pair has an image freeze.

2. The method according to claim 1, characterized in that, The step of performing contour processing on the grayscale difference image to obtain the target contour with the largest difference area value includes: The grayscale difference image is subjected to median filtering to obtain a median-filtered difference image; The median-filtered difference image is subjected to contour processing to obtain the target contour with the largest difference area value.

3. The method according to claim 1, characterized in that, The acquisition of the target image pair includes: Obtain the first color histogram and the second color histogram corresponding to the original image pair, wherein the original image pair is the color image corresponding to two adjacent frames; The similarity between the first color histogram and the second color histogram is calculated to obtain the color histogram similarity. If the color histogram similarity is greater than the first similarity threshold, then the original image pair is processed into grayscale to obtain the target image pair.

4. The method according to claim 3, characterized in that, If the color histogram similarity is greater than a first similarity threshold, then the original image pair is subjected to grayscale processing to obtain the target image pair, including: If the color histogram similarity is greater than the first similarity threshold, then the original image pair is subjected to grayscale processing to obtain a basic grayscale image pair; Structural similarity is calculated for the basic grayscale image pairs to obtain the structural similarity. If the structural similarity is greater than the second similarity threshold, then the basic grayscale image pair is taken as the target image pair.

5. The method according to claim 4, characterized in that, The step of calculating the structural similarity of the base grayscale image pairs to obtain the structural similarity includes: The average gray level of the base gray level image pair is calculated to obtain the base brightness contrast value of the base gray level image pair. The grayscale standard deviation is calculated for the basic grayscale image pair to obtain the basic contrast measurement value of the basic grayscale image pair; The structural contrast value of the basic grayscale image pair is calculated based on the basic brightness contrast value and the basic contrast measurement value. The structural similarity is calculated based on the basic brightness contrast value, the basic contrast measurement value, and the structural contrast value.

6. The method according to claim 3, characterized in that, The step of calculating the similarity between the first color histogram and the second color histogram to obtain the color histogram similarity includes: The first color histogram is normalized to obtain the first color distribution range of the first color histogram; Based on the number of intervals, the mean of the first color distribution interval is calculated to obtain the mean of the first color distribution. The normalization process is applied to the second color histogram to obtain the second color distribution range of the second color histogram; Based on the number of intervals, the mean of the second color distribution intervals is calculated to obtain the mean of the second color distribution; The similarity is calculated based on the first color distribution interval, the first color distribution mean, the second color distribution interval, and the second color distribution mean to obtain the color histogram similarity.

7. The method according to claim 3, characterized in that, The step of obtaining the first color histogram and the second color histogram corresponding to the original image pair includes: Based on the decoding frame rate, the target object under test is decoded to obtain the set of image frames corresponding to the target object under test; Take any pair of adjacent image frames in the image frame set as the original image pair; The color pixel values ​​in the original image pair are counted to obtain the number of pixel values ​​corresponding to each color interval, thereby obtaining the first color histogram and the second color histogram.

8. The method according to claim 1, characterized in that, The step of performing Gaussian blur filtering on each grayscale image in the target image pair to obtain a first Gaussian blurred image and a second Gaussian blurred image includes: Using a Gaussian filter convolution kernel of the same size as the Gaussian filter, each pixel in each grayscale image is scanned sequentially to obtain the first neighboring pixel of each pixel; Based on the filtering weights, the gray values ​​corresponding to the pixels in the first neighborhood are weighted and averaged to obtain the weighted average gray value corresponding to each pixel in the first neighborhood. Replace the gray value of the center pixel with the weighted average gray value corresponding to each pixel in the first neighborhood to obtain the first Gaussian blurred image and the second Gaussian blurred image.

9. The method according to claim 2, characterized in that, The step of performing median filtering on the grayscale difference image to obtain a median-filtered difference image includes: The median filter convolution kernel corresponding to the median filter size is used to scan each pixel in the grayscale difference image sequentially to obtain the second neighboring pixels corresponding to each pixel. The median of the gray values ​​corresponding to the pixels in the second neighborhood is calculated to obtain the median of each pixel in the second neighborhood. The median value of the center pixel is replaced by the median value of each pixel in the neighborhood to obtain the median filtered difference image.

10. The method according to claim 2, characterized in that, The step of performing contour processing on the median-filtered difference image to obtain the target contour with the largest difference area value includes: Contour feature extraction is performed on the median-filtered difference image to obtain multiple difference contours containing the target object; Calculate the area of ​​each difference contour to obtain the difference area value of each difference contour. Based on the difference area value of each of the difference contours, a pairwise comparison is performed to obtain the target contour with the largest difference area value.

11. A device for detecting frozen frames in images, characterized in that, include: An acquisition unit is used to acquire a pair of target images, wherein the pair of target images are grayscale images corresponding to two adjacent frames; A processing unit is configured to perform Gaussian blur filtering on each grayscale image in the target image pair to obtain a first Gaussian blurred image and a second Gaussian blurred image; and to perform saliency processing on the first Gaussian blurred image and the second Gaussian blurred image respectively, preserving high-contrast regions, to obtain a first saliency image and a second saliency image, including: calculating the difference between the first cepstral of the first Gaussian blurred image and the cepstral of the first image mean to obtain a first difference; using the first difference as a first saliency weight for the first Gaussian blurred image; mapping the first saliency weight to the first Gaussian blurred image to obtain the first saliency image; calculating the difference between the second cepstral of the second Gaussian blurred image and the cepstral of the second image mean to obtain a second difference; using the second difference as a second saliency weight for the second Gaussian blurred image; and mapping the second saliency weight to the second Gaussian blurred image to obtain the second saliency image; The processing unit is further configured to perform grayscale difference processing on the first saliency image and the second saliency image to obtain a grayscale difference image; The processing unit is also used to perform contour processing on the grayscale difference image to obtain the target contour with the largest difference area value. The determining unit is configured to determine that the target image pair has an image freeze if the difference area value of the target contour is less than the area threshold.

12. The apparatus according to claim 11, characterized in that, The processing unit is specifically used for: The grayscale difference image is subjected to median filtering to obtain a median-filtered difference image; The median-filtered difference image is subjected to contour processing to obtain the target contour with the largest difference area value.

13. The apparatus according to claim 11, characterized in that, The acquisition unit is specifically used for: Obtain the first color histogram and the second color histogram corresponding to the original image pair, wherein the original image pair is the color image corresponding to two adjacent frames; The similarity between the first color histogram and the second color histogram is calculated to obtain the color histogram similarity. If the color histogram similarity is greater than the first similarity threshold, then the original image pair is processed into grayscale to obtain the target image pair.

14. The apparatus according to claim 13, characterized in that, The acquisition unit is specifically used for: If the color histogram similarity is greater than the first similarity threshold, then the original image pair is subjected to grayscale processing to obtain a basic grayscale image pair; Structural similarity is calculated for the basic grayscale image pairs to obtain the structural similarity. If the structural similarity is greater than the second similarity threshold, then the basic grayscale image pair is taken as the target image pair.

15. A computer device comprising a memory, a processor, and a bus system, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 10; The bus system is used to connect the memory and the processor to enable communication between the memory and the processor.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images