Image freezing detection method and device and computer storage medium
By selecting multiple detection points in the image screen, verifying the pixel value frame by frame and determining the same continuous frames, the problem of low freezing detection accuracy and misjudgment in the prior art is solved, and freezing detection with high sensitivity and strong real-time performance is achieved.
Patent Information
- Application Number
- CN202410936882.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-08
AI Technical Summary
The existing image freezing detection methods are not very accurate under the condition of limited hardware resources, weak anti-interference ability, and large computing resources, which cannot effectively distinguish between video content and screen freezing caused by interruption of signal transmission.
By selecting multiple points at the preset position in the image screen as detection points, the pixel values of each frame of the image are checked one by one by one by one, and the screen is frozen when the verification values of multiple consecutive frames are the same, adapting to different application scenarios, including the full area and some areas with fixed backgrounds to effectively display the scene.
It improves the freezing detection sensitivity of micro-motion scenes, reduces hardware resource consumption, has real-time and universal applicability, can distinguish between image content static and signal interruption, and reduces the rate of error judgment.
Smart Images

Figure CN118918063A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of image and video processing, and in particular relates to an image freezing detection method, device and computer storage medium. Background Art
[0002] Image freeze detection technology is mainly used in monitoring, streaming media transmission, video conferencing, digital instruments and other scenarios to ensure the continuity and smoothness of the image. This technology mainly determines whether there is a picture freeze phenomenon by analyzing the changes in the image frame content in real time. Some common existing image freeze detection methods are as follows:
[0003] 1. Based on the frame difference method: This is the most basic method. The principle is to calculate the pixel difference between two or several consecutive frames. If the difference between multiple consecutive frames is very small or there is no change, it can be judged that the image is frozen. 2. Pixel histogram statistics method: Perform histogram statistics on the pixel points of a frame of the video, and track and match them. If the statistical value does not change between consecutive frames, it is judged that the picture is frozen. 3. Motion vector analysis: In modern video coding standards such as H.264, each frame has corresponding motion vector information. The change of motion vector can be analyzed to detect whether the video is frozen. When the motion vector is close to zero for a continuous period of time, it can be judged that the picture is frozen. 4. Deep learning method: Use deep learning models such as convolutional neural networks to learn and analyze video sequences. The model can automatically learn the difference between normal video playback and picture freezing from a large number of training samples, thereby realizing freeze detection.
[0004] The existing freeze detection method has the following disadvantages:
[0005] Disadvantages of the frame difference method: 1. Limited accuracy: For scenes with small movements or large background noise, frame difference may lead to misjudgment, especially small movements in low light conditions or static backgrounds. 2. Weak anti-interference ability: It cannot effectively distinguish between the stillness of the video content itself and the freezing of the picture caused by the real signal transmission interruption.
[0006] Disadvantages of pixel histogram statistics: Limited accuracy: may lead to misjudgment of small movements. Inflexible: also not very adaptable to complex environments.
[0007] Disadvantages of motion vector analysis: 1. Dependence on encoding format: It is only applicable to encoded videos with motion vector information, and cannot be directly applied to original unencoded videos. 2. Parsing cost: Obtaining and parsing motion vector information may increase computing resource consumption.
[0008] Disadvantages of deep learning methods: 1. High training requirements: a large amount of labeled data is required for training, and the training process is often time-consuming. 2. High consumption of computing resources: high hardware support is required to run deep learning models.
[0009] How to achieve simple and efficient image freeze detection under the condition of limited hardware resources is a problem that needs to be solved urgently. Summary of the invention
[0010] In view of this, the present invention provides an image freezing detection method, device and computer storage medium to solve the deficiencies of the prior art.
[0011] In order to achieve the above object, the present invention adopts the following technical scheme:
[0012] An embodiment of the present invention provides an image freezing detection method, comprising:
[0013] Selecting multiple points at preset positions in the image as detection points;
[0014] Selecting a preset rule to check and calculate the pixel values of the detection points in each frame of the image one by one to obtain a final check value;
[0015] When the corresponding check values of multiple consecutive frames are the same, it is determined that the picture is frozen.
[0016] Furthermore, before selecting a plurality of points at preset positions in the image as detection points, the method further includes: acquiring an application scenario of the video image;
[0017] Selecting a detection point at a preset position in the image according to the application scenario;
[0018] When the application scenario of the video image is a full-area effective display scenario, only multiple pixel points starting from any coordinate position of the video image data source are selected as detection points for a verification calculation;
[0019] When the application scenario of the video image is a partial area effective display scene with a fixed background, multiple pixel points starting from any coordinate position of the non-background area in the video image data source are selected as detection points for a verification calculation.
[0020] Furthermore, when the application scenario of the video image is an effective display scene of a partial area with a fixed background, while selecting multiple pixel points starting from any coordinate position of the non-background area in the video image data source as detection points, all points in a preset rectangular area within the fixed background area are also selected as detection points and a verification calculation is performed.
[0021] Furthermore, the effective display scene of the partial area with a fixed background includes: a digital instrument; the effective display scene of the full area includes monitoring, streaming media transmission, and video conferencing.
[0022] Furthermore, there are pre-set pixel points for background keep-alive detection in a preset rectangular area within the fixed background area; and the pixel points for background keep-alive detection are changing frame by frame.
[0023] Furthermore, the preset rectangular area is located at an edge of the video image, and a change in pixel values of pixels used for background keep-alive detection within the preset rectangular area is within a preset range.
[0024] Furthermore, the preset rule is to calculate the pixel value of the first detection point and the pixel value of the second detection point, and then calculate the result obtained by calculating the pixel value of the third detection point, and then calculate the result obtained by calculating the pixel value of the fourth detection point, and so on, until the final verification value is obtained by calculating the pixel value of the last detection point.
[0025] Furthermore, the operation is a preset binary logic operation.
[0026] Further, when the check values of the consecutive multiple frames are the same, the number of frames N with the same check values is configurable, which specifically includes: at the end of each frame, the check point check value Pn of the current frame is compared with the check point check value P(n-1) of the previous frame; if Pn is the same as P(n-1), the count value M is automatically increased by 1; if they are not the same, the count value M is reset to zero; then the count value M is compared with N; if M is not less than N, it means that the check values of N consecutive frames are the same, and it is determined that the image is frozen;
[0027] If each frame has multiple check values, the check value of the current frame is considered to be equal to the check value of the previous frame only when all the check values are equal to the corresponding check value of the previous frame, and the count value M is increased by 1.
[0028] The embodiment of the present invention further provides an image freezing detection device, comprising:
[0029] Selection module: used to select multiple points at preset positions in the image as detection points;
[0030] Calculation module: used to select preset rules to check and calculate the pixel values of the detection points in each frame of the image one by one to obtain the final verification value;
[0031] Determination module: used to determine that the picture is frozen when the calculation module calculates that the corresponding check values of multiple consecutive frames are the same.
[0032] An embodiment of the present invention further provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned image freezing detection method is implemented.
[0033] The image freezing detection method and device provided by the present invention verifies and calculates the pixels of the selected rectangular window area point by point. The calculated verification value will change with the change of any pixel value, thereby improving the freezing detection sensitivity of the scene with slight motion. The pixel point by point verification calculation of the selected detection point can distinguish whether it is the stillness of the image content itself or the real signal transmission interruption, solving the problem that the image content does not change for a long time in the slow motion scene may lead to misjudgment. The freezing detection threshold N (the number of frames with the same continuous verification value) can also be adjusted to increase the fault tolerance rate and eliminate misjudgment.
[0034] On the other hand, since the freezing of the picture is judged only by checking the pixel values of several detection points in the picture, the freezing detection method and device of the present invention are not affected by the image content and are only controlled by the video image data source, so they are also well adaptable to complex environment scenes.
[0035] On the other hand, the freeze detection method and device of the present invention are applicable to all image data formats, and therefore are not dependent on the format of the video image data, do not require format parsing, and are generally applicable.
[0036] In addition, freeze detection can perform calculations and detections on selected pixels in real time. It only needs to save the detection values, and there is no need to cache large amounts of image data for complex algorithm calculations, which greatly saves hardware resources. The hardware implementation is simple, efficient, and has strong real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0038] Figure 1 A schematic diagram of a flow chart of an image freezing detection method provided by an embodiment of the present invention;
[0039] Figure 2 A flow chart of screen freeze detection for effectively displaying a scene in a partial area with a fixed background;
[0040] Figure 3 A flowchart for detecting freezing of the screen for effectively displaying the scene in the entire area;
[0041] Figure 4A schematic structural diagram of an image freezing detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0043] Figure 1 An embodiment of the present invention provides an image freezing detection method, comprising:
[0044] S101, selecting a plurality of points at preset positions in an image as detection points.
[0045] The number of detection points is configurable, and users can set it according to their own needs. The more settings, the more accurate the judgment, but the more data involved in the verification calculation will increase the amount of calculation; the fewer settings, the more likely it is that the detection coverage will not be comprehensive, and the active picture with a small change range will be mistakenly judged as a frozen picture. Therefore, in the actual work process, different values can be set to debug to get the best effect.
[0046] The positions of the detection points of the image can be evenly distributed, that is, detection points are set at certain row and column intervals. The setting of the detection points can also be adjusted and modified during the image display process. This allows users to set a more reasonable number of detection points and specific positions for specific display images. For example: In a video surveillance image facing a door, since the door position is the place where the entire image changes the most, more detection points can be set at the door position to improve the detection efficiency of image freezing.
[0047] Before selecting a plurality of points at preset positions in the image as detection points, the method further includes: acquiring an application scenario of the video image; and selecting detection points at preset positions in the image according to the application scenario.
[0048] In this specific implementation manner, the application scenarios of the image include a full-area effective display scenario and a partial-area effective display scenario with a fixed background.
[0049] The detection points at preset positions in the image screen are selected according to the application scenario, including: when the application scenario of the video image is a full-area effective display scenario, only multiple pixel points starting from any coordinate position of the video image data source are selected as detection points for a verification calculation; relatively speaking, the time interval between changes in the image content of this type of scene is short. For example, monitoring, streaming media transmission, and video conferencing all belong to this type of scene. The image content of this scene can be fast or slow.
[0050] When the application scenario of the video image is a partial area effective display scene with a fixed background, multiple pixel points starting from any coordinate position of the non-background area in the video image data source are selected as detection points for a verification calculation. At the same time, all detection points in a rectangular area within the fixed background area are also selected for a verification calculation. Relatively speaking, the time interval between changes in the image content of this type of scene is relatively long. For example, like: digital instrument. The instrument background remains basically unchanged, and the dial pointer, numbers, etc. change slowly. When the application scenario of the video image is a partial area effective display scene with a fixed background; there are pre-set pixel points for background keep alive detection in a preset rectangular area within the fixed background area, and the number of pixel points for background keep alive detection can be 1 or more; the pixel points are located in the selected rectangular area and the pixel values are changing frame by frame. The pixel value of the background image is fixed and unchanged. In order to perform background keep alive detection, the pixel values of one or more pixel points pre-set in the background image are constantly changing; when setting the preset rectangular area, it is necessary to ensure that the pixel points used for background keep alive detection can be covered. Setting background keep alive detection can effectively solve the problem that the car is driving at a constant speed, the instrument panel pointer does not change, and it is impossible to distinguish whether the picture is normal or frozen. The preset rectangular area is located at the edge of the video image, and the pixel value of the pixel point of the background keep-alive detection changes within a preset range. The pixel point of the background keep-alive detection is located at the edge of the video image and the pixel value changes within a preset range, which can ensure that the keep-alive pixel point will not have a significant impact on the user's viewing.
[0051] The pixel points in the selected rectangular window area are checked and calculated point by point. Any change in the pixel value will change the calculated check value, which improves the freeze detection sensitivity of the scene with slight motion. The pixel points of the selected detection points are checked and calculated point by point, which can distinguish whether it is the stillness of the image content itself or the real signal transmission interruption, solving the problem that the image content does not change for a long time in the slow motion scene, which may lead to misjudgment.
[0052] S102, using a preset rule to check and calculate the pixel values of the detection points in each frame of the image one by one to obtain a final check value.
[0053] The preset rule is to calculate the pixel value of the first detection point and the pixel value of the second detection point, and then calculate the result obtained by calculating the pixel value of the third detection point, and then calculate the result obtained by calculating the pixel value of the fourth detection point, and so on, until the final verification value is obtained by calculating the pixel value of the last detection point. The operation is a preset binary logic operation. Binary logic operation is more conducive to chip design to implement related operations. Each time a one-by-one operation is completed, a final verification value is obtained for storage. Multiple pixel value data can be effectively simplified into one data volume for storage, without the need to cache a large amount of image data for complex algorithm operations, saving hardware resources, simple hardware implementation and strong real-time performance. Binary logic operations can be AND operations, OR operations, etc., and each pixel value is converted into a binary value before the operation to perform related binary logic operations.
[0054] When the application scenario of the video image is a full-area effective display scenario, a verification calculation is performed once for all detection points in each frame to obtain a verification value to be saved.
[0055] When the application scenario of the video image is a partial area effective display scenario with a fixed background, a check calculation is performed once for the detection points in the non-background area in each frame to obtain a check value to be saved. At the same time, a check calculation is performed once for all points in the rectangular area in the background area to obtain another check value to be saved. In other embodiments, the detection points in the non-background area and all points in the rectangular area in the background area can also be checked together when performing the check calculation, and only one check value to be saved is obtained.
[0056] S103: When the corresponding check values of multiple consecutive frames are the same, it is determined that the picture is frozen.
[0057] When the check values of multiple consecutive frames are the same, the determination of image freezing specifically includes: at the end of each frame, the check point check value Pn of the current frame is compared with the check point check value P(n-1) of the previous frame; if Pn is the same as P(n-1), the count value M is automatically increased by 1; if they are not the same, the count value M is cleared; the freeze detection threshold is set to N, and at the end of each frame, N is compared with the count value M; if M is not less than N, it means that the check values of N consecutive frames are the same, that is, Pn is the same as Pn+1, Pn+2...Pn+N, then it is determined that the image is frozen. The number of check values calculated according to each frame is 1 or 2. If it is 2 check values, the count values corresponding to the 2 check values must be no less than the set freeze detection threshold to determine that the image is frozen.
[0058] The freeze detection threshold N can be adjusted to increase the error tolerance and eliminate misjudgment. In addition, the pixel value of the detection point is not affected by the image content, but only controlled by the video image data source, so it is also well adaptable to complex environmental scenes. Freeze detection does not depend on the format of video image data, does not require format parsing, and has universal applicability. When the application scenario of the video image is to effectively display a scene with a fixed background in a partial area, the pre-set background keep-alive detection point must be included when selecting the detection point.
[0059] Figure 2 This is a flow chart of screen freeze detection for effectively displaying scenes in a partial area with a fixed background; there are background keep-alive detection points in the rectangular area, and the verification calculation is completed twice, and two verification values are obtained respectively. Only when the two verification values are the same for multiple consecutive frames, the screen is determined to be frozen. The details are as follows:
[0060] One or more pixel points starting from any coordinate position (X, Y) of the effective display area of the video image data source are selected as detection points, so that their pixel values change every fixed M frames and are not affected by the image content.
[0061] The pixel points of the detection points in the above steps are verified and calculated one by one. Specifically, the (X, Y) pixel point is verified and calculated with the (X+1, Y) pixel point, and the result is verified and calculated with the (X+2, Y) pixel point, and so on until the verification calculation is performed with the (X+A, Y+B) pixel point to obtain the final verification value of the detection point of the current frame, which is denoted as Xn (X represents the verification value, and n represents the nth frame).
[0062] At the end of each frame, the checkpoint value Pn of the current frame is compared with the checkpoint value P(n-1) of the previous frame. If Pn is the same as P(n-1), the count value M+1 is added; if not, the count value M is reset to zero.
[0063] Then, starting from any coordinate (x, y) position in each frame of the video stream except the detection point, set a window (background area) of any size (a*b), which is a rectangular area from (x, y) to (x+a, y+b). This rectangular area contains the background keep-alive detection point.
[0064] The pixel points contained in the selected window are checked and calculated one by one. Specifically, the (x, y) pixel point is checked and calculated with the (x+1, y) pixel point, and the result is then checked and calculated with the (x+2, y) pixel point, and so on until the (x+a, y+b) pixel point is checked and calculated, and the final check value of the window of the current frame is obtained, which is denoted as Qn (Q represents the check value, and n represents the nth frame).
[0065] At the end of each frame, the check value Qn of the window of the current frame is compared with the check value Q(n-1) of the window of the previous frame. If Qn is the same as Q(n-1), the count value is m+1; if not, the count value is reset to zero.
[0066] Set a freeze detection threshold N, and compare N with M and m at the end of each frame. If M is not less than N, and m is not less than N, it means that the check values of N consecutive frames are the same, that is, Pn is the same as Pn+1, Pn+2...Pn+N-1, and Qn is the same as Qn+1, Qn+2...Qn+N-1, then it is determined that the image is frozen. In other embodiments, the two check calculations can be combined to obtain one check value.
[0067] Figure 3 This is a flowchart for detecting freeze of the screen with full-area effective display scenes. Figure 2 Compared with the above method, the only difference is the calculation and judgment of the check value of the background keep-alive detection point in the rectangular area.
[0068] like Figure 4 As shown, the embodiment of the present invention further provides an image freezing detection device, comprising:
[0069] Selection module: used to select multiple points at preset positions in the image as detection points;
[0070] Calculation module: used to select preset rules to check and calculate the detection points in each frame of the image one by one to obtain the final verification value;
[0071] Judgment module: used to determine that the picture is frozen when the calculation module calculates that the check values of multiple consecutive frames are the same.
[0072] The image freezing detection device also includes a scene acquisition module for acquiring the application scene of the video image;
[0073] The selection module is used to select different verification and detection methods according to different scenarios;
[0074] When the application scenario of the video image is a full-area effective display scenario, only multiple pixel points starting from any coordinate position of the video image data source are selected as detection points for a verification calculation;
[0075] When the application scenario of the video image is an effective display scene of a partial area with a fixed background, multiple pixel points starting from any coordinate position in the non-background area of the video image data source are selected as detection points for a verification calculation. At the same time, all detection points in a rectangular area within the fixed background area are also selected for a verification calculation.
[0076] The effective display scenes of the partial area with a fixed background include: digital instruments; the effective display scenes of the full area include monitoring, streaming media transmission, and video conferencing.
[0077] The preset rule for verification calculation is to calculate the pixel value of the first detection point and the pixel value of the second detection point, and then calculate the result with the pixel value of the third detection point, and then calculate the result with the pixel value of the fourth detection point, and so on, until the final verification value is obtained by calculating with the pixel value of the last detection point.
[0078] It should be noted that: the image freeze detection device provided in the above embodiment only uses the division of the above program modules as an example to illustrate when performing chromatic noise reduction. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the image freeze detection device provided in the above embodiment and the image freeze detection method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment. The beneficial effects are the same as those in the method embodiment, and will not be repeated here.
[0079] The embodiment of the present invention further provides a computer storage medium, which is a computer-readable storage medium, on which a computer program is stored, and the computer program can be executed by a processor of an image freezing detection device to complete the steps of the aforementioned image freezing detection method. The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM).
[0080] In the several embodiments provided by the present invention, it should be understood that the disclosed methods and intelligent devices can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0081] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. An image freezing detection method, characterized in that: include: Selecting multiple points at preset positions in the image as detection points; Selecting a preset rule to check and calculate the pixel values of the detection points in each frame of the image one by one to obtain a final check value; When the corresponding check values of multiple consecutive frames are the same, it is determined that the picture is frozen.
2. The method according to claim 1, characterized in that Before selecting a plurality of points at preset positions in the image as detection points, the method further includes: obtaining an application scenario of the video image; Selecting a detection point at a preset position in the image according to the application scenario; When the application scenario of the video image is a full-area effective display scenario, only multiple pixel points starting from any coordinate position of the video image data source are selected as detection points; When the application scenario of the video image is a partial area effective display scenario with a fixed background, multiple pixel points starting from any coordinate position of the non-background area in the video image data source are selected as detection points.
3. The method according to claim 2, characterized in that When the application scenario of the video image is an effective display scenario of a partial area with a fixed background, multiple pixel points starting from any coordinate position of the non-background area in the video image data source are selected as detection points, and all points in a preset rectangular area within the fixed background area are also selected as detection points.
4. The method according to claim 3, characterized in that There are pre-set pixel points for background keep-alive detection in a preset rectangular area within the fixed background area, and the pixel points for background keep-alive detection are changing frame by frame.
5. The method according to claim 4, characterized in that The preset rectangular area is located at an edge position of the video image, and pixel values of pixels used for background keep-alive detection within the preset rectangular area vary within a preset range.
6. The method according to any one of claims 1 to 5, characterized in that: The preset rule is to calculate the pixel value of the first detection point and the pixel value of the second detection point, and then calculate the result obtained by calculating the pixel value of the third detection point, and then calculate the result obtained by calculating the pixel value of the fourth detection point, and so on, until the final verification value is obtained by calculating the pixel value of the last detection point.
7. The method according to claim 6, characterized in that The operation is a preset binary logic operation.
8. The method according to any one of claims 1 to 7, characterized in that: When the check values of multiple consecutive frames are the same, it is determined that the image is frozen, and the number of frames N with the same check value is configurable, specifically including: at the end of each frame, the check point check value Pn of the current frame is compared with the check point check value P(n-1) of the previous frame; if Pn is the same as P(n-1), the count value M is automatically increased by 1; if they are not the same, the count value M is reset to zero; then the count value M is compared with N; if M is not less than N, it means that the check values of N consecutive frames are the same, and it is determined that the image is frozen; If each frame has multiple check values, the check value of the current frame is considered to be equal to the check value of the previous frame only when all the check values are equal to the corresponding check value of the previous frame, and the count value M is increased by 1.
9. An image freezing detection device, characterized in that: include: Selection module: used to select multiple points at preset positions in the image as detection points; Calculation module: used to select preset rules to check and calculate the pixel values of the detection points in each frame of the image one by one to obtain the final verification value; Determination module: used to determine that the picture is frozen when the calculation module calculates that the corresponding check values of multiple consecutive frames are the same.
10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image freezing detection method according to any one of claims 1 to 6 is implemented.
