Frozen screen detection method, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a frozen screen detection method, an electronic device, a storage medium and a program product, for at least two frame images continuously output to a display screen, according to the least significant bit of each frame image under a preset color channel, a to-be-detected image block of the frame image is determined, and according to the to-be-detected image block of the frame image and a preset watermark block, the to-be-detected image block of the frame image is detected. The watermark blocks carried by the frame images are determined, and then whether the display screen is frozen or not is determined according to the change conditions of the watermark blocks carried by at least two frame images. The to-be-detected image block is inserted into the least significant bit of the preset color channel, so that the influence on the frame image is small, and the image quality of the frame image is ensured while the frozen screen detection is realized.

Description

Technical Field

[0001] This application relates to the field of display technology, and more particularly to a method for detecting frozen screens, electronic devices, storage media, and program products. Background Technology

[0002] Screen freezing is a type of display device malfunction that occurs when video frames are repeatedly transmitted within the display device for a certain period of time, which is inconsistent with the input video stream, resulting in the screen freezing.

[0003] Taking in-vehicle use as an example, existing surround-view cameras transmit real-time video streams of the vehicle's surroundings to the in-vehicle display system or processing unit. If there is a screen freezing issue, it will affect functional safety and user experience.

[0004] Therefore, screen freeze detection is particularly important. Summary of the Invention

[0005] This application provides a method for detecting frozen screens, an electronic device, a storage medium, and a program product for implementing frozen screen detection.

[0006] In a first aspect, embodiments of this application provide a method for detecting frozen screens, including:

[0007] For at least two consecutive frame images output to the display screen, the image block to be tested in the frame image is determined based on the least significant bit in the preset color channel of each frame image.

[0008] The watermark block carried by the frame image is determined based on the image block to be tested and the preset watermark block of the frame image;

[0009] Based on the changes in the watermark blocks carried by the at least two frame images, it is determined whether the display screen has frozen.

[0010] In one possible implementation, determining the image block to be tested in the frame image based on the least significant bit in a preset color channel of each frame image includes:

[0011] The image to be tested is determined based on the least significant bit of the preset color channel of the preset pixels in each frame image;

[0012] The image to be tested is divided into multiple image blocks to be tested in the frame image.

[0013] In one possible implementation, the image to be tested is divided to determine multiple image blocks of the frame image, including:

[0014] Based on the size of the preset watermark block, the image to be tested is divided to determine multiple image blocks to be tested in the frame image.

[0015] In one possible implementation, determining the watermark block carried by the frame image based on the image block to be tested and the preset watermark block includes:

[0016] Determine the structural similarity between the image block to be tested and the preset watermark block;

[0017] Based on the structural similarity, it is determined whether the frame image carries the preset watermark block.

[0018] In one possible implementation, the number of the image block to be tested and the preset watermark block is multiple;

[0019] The step of determining whether the frame image carries the preset watermark block based on the structural similarity includes:

[0020] Determine a first number of pre-defined watermark blocks whose structural similarity is greater than a first threshold.

[0021] When the first number corresponding to any of the preset watermark blocks is greater than or equal to the second threshold, it is determined that the frame image carries the preset watermark block;

[0022] When the first number corresponding to each preset watermark block is less than the second threshold, it is determined that the frame image does not carry the preset watermark block.

[0023] In one possible implementation, the number of the image block to be tested and the preset watermark block is multiple; the step of determining whether the frame image carries the preset watermark block based on the structural similarity includes:

[0024] Determine the average structural similarity of each preset watermark block;

[0025] When the average value corresponding to any of the preset watermark blocks is greater than or equal to the third threshold, it is determined that the frame image carries the preset watermark block;

[0026] When the average value corresponding to each preset watermark block is less than the third threshold, it is determined that the frame image does not carry the preset watermark block.

[0027] In one possible implementation, determining whether the display screen has frozen based on the changes in the watermark blocks carried by the at least two frame images includes:

[0028] If the preset watermark block is carried in a portion of the frame images of at least two frame images, it is determined that the display screen has not frozen.

[0029] When the preset watermark block is carried in each of the at least two frame images, and at least some frame images carry different preset watermark blocks, it is determined that the display screen has not frozen.

[0030] When the display screen freezes, it is determined that the same preset watermark block is carried or the preset watermark block is not carried in each of the at least two frame images.

[0031] In one possible implementation, the method further includes:

[0032] A timer begins when the display screen is determined to have frozen.

[0033] If the timeout period does not reach the preset duration, a watermark error signal is sent to the display screen; if the timeout period reaches the preset duration, a screen freeze fault signal is sent to the display screen.

[0034] In one possible implementation, the method further includes:

[0035] If it is determined that the latter of two adjacent frame images carries the preset watermark block, the least significant bit of the preset color channel of the latter is replaced according to the least significant bit of the preset color channel of the former.

[0036] In one possible implementation, the method further includes:

[0037] The preset watermark block is inserted into the least significant bit of the preset color channel of at least a portion of the frame images of the at least two frame images.

[0038] In one possible implementation, inserting the preset watermark block into the least significant bit of a preset color channel of at least a portion of the at least two frame images includes:

[0039] In the least significant bit of the preset color channel of at least a portion of the frame images of the at least two frame images, a plurality of the preset watermark blocks are inserted in a preset order.

[0040] Secondly, this application provides a frozen screen detection device, comprising:

[0041] The first determining module is used to determine the image block to be tested of at least two consecutively output frame images to the display screen based on the least significant bit of the preset color channel of each frame image.

[0042] The second determining module is used to determine the watermark block carried by the frame image based on the image block to be tested and the preset watermark block of the frame image;

[0043] The third determining module is used to determine whether the display screen has frozen based on the changes in the watermark blocks carried by the at least two frame images.

[0044] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0045] The memory stores computer-executed instructions;

[0046] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0047] Fourthly, this application provides a signal transmission system, including: a first device, a second device, and a third device, wherein the second device is communicatively connected to the first device and the third device;

[0048] The second device is used to determine the image block to be tested of the frame image based on the least significant bit of the preset color channel of each frame image, for at least two frame images continuously output by the first device to the third device.

[0049] The watermark block carried by the frame image is determined based on the image block to be tested and the preset watermark block of the frame image;

[0050] Based on the changes in the watermark blocks carried by the at least two frame images, it is determined whether the third device has experienced screen freezing.

[0051] In one possible implementation, it further includes: a fourth device, which is communicatively connected to the first device and the second device;

[0052] The fourth device is used to insert the preset watermark block into the least significant bit of a preset color channel of at least a portion of the frame images of at least two consecutively output frame images by the first device.

[0053] In one possible implementation, it further includes a fifth device, which includes a serializer, an image compression module, and a deserializer; the fifth device is communicatively connected to the fourth device and the second device.

[0054] The serializer is used to convert the parallel data of the frame image output by the fourth device into serial data.

[0055] The image compression module is used to compress the serial data;

[0056] The deserializer is used to decompress the compressed serial data, convert it into parallel data, and send it to the second device.

[0057] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0058] In a sixth aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0059] The screen freeze detection method, electronic device, storage medium, and program product provided in this application, for at least two consecutively output frame images to a display screen, determine the image block to be tested in each frame image based on the least significant bit of a preset color channel. Based on the image block to be tested and a preset watermark block, determine the watermark block carried by the frame image. Then, determine whether the display screen has frozen based on the changes in the watermark blocks carried by the at least two frame images. By inserting the image block to be tested into the least significant bit of the preset color channel, the impact on the frame image is minimal, thereby ensuring the image quality of the frame image while achieving screen freeze detection. Attached Figure Description

[0060] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0061] Figure 1 Flowchart of the screen freeze detection method provided in this application Figure 1 ;

[0062] Figure 2 Flowchart of the screen freeze detection method provided in this application Figure 2 ;

[0063] Figure 3 A schematic diagram illustrating the insertion method of the watermark block provided in this application;

[0064] Figure 4 A schematic diagram of the frozen screen detection device provided in this application;

[0065] Figure 5 A schematic diagram of the structure of the electronic device provided in this application;

[0066] Figure 6 A schematic diagram of the signal transmission system provided in this application.

[0067] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0068] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0069] Screen freezing is a type of display device malfunction that occurs when video frames are repeatedly transmitted within the display device for a certain period of time, which is inconsistent with the input video stream, resulting in the screen freezing.

[0070] Taking in-vehicle systems as an example, existing surround-view cameras transmit real-time video streams of the vehicle's surroundings to the in-vehicle display system or processing unit. If screen freezing issues occur, it will affect functional safety and user experience. Therefore, screen freezing detection is particularly important.

[0071] For example, a watermark image can be added to a video frame to determine if a freeze screen is present. However, adding a watermark image to a video frame will affect the image quality of the video frame, impacting the user experience.

[0072] To address this issue, this application proposes a screen freeze detection method. Based on the least significant bit of a preset color channel in each frame image, the method determines the image block to be tested within the frame image. Then, based on the image block to be tested and a preset watermark block, it determines whether screen freeze has occurred. By inserting the image block to be tested into the least significant bit of the preset color channel, the impact on the frame image is minimal, thus ensuring image quality of the frame image while achieving screen freeze detection.

[0073] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0074] Figure 1 Flowchart of the screen freeze detection method provided in this application Figure 1 ,like Figure 1 As shown, the method includes:

[0075] S101. For at least two consecutive frame images output to the display screen, determine the image block to be tested of the frame image based on the least significant bit of the preset color channel of each frame image.

[0076] In this embodiment, for at least two consecutively output frame images to the display screen, the least significant bit of each frame image represents the binary information of the image block to be tested, thereby determining the image block to be tested in the frame image. By representing the image block to be tested with the least significant bit of the preset color channel, the visual effect of the frame image is minimized, ensuring the image quality of the frame image.

[0077] For example, the image block to be tested includes multiple pixels, each of which is represented by a binary value, meaning each pixel can only be 0 (black) or 1 (white). Accordingly, multiple pixels in the frame image correspond one-to-one with multiple pixels in the image block to be tested, and the least significant bit of the binary information of the pixel in the frame image under the preset color channel can represent the binary information of the corresponding pixel in the image block to be tested.

[0078] For example, if the image type of the frame image is RGB888 format, then each pixel in the frame image consists of three channels: red (R), green (G), and blue (B). Each channel has an 8-bit binary number to represent the brightness. That is, the brightness value in each channel can be represented by 8 bits of binary. For example, when the brightness value of the red channel is 138, it can be represented by 10001010.

[0079] For example, the image type of the frame image can also be RGB161616 format. In this case, each channel in the frame image has a 16-bit binary number to represent the brightness. That is, the brightness value in each channel can be represented by 16 bits of binary. For example, when the brightness value of the red channel is 285, it can be represented by 0000000100011101.

[0080] Accordingly, the least significant bit of the preset color channel is the rightmost bit of the binary representation.

[0081] In one possible implementation, the preset color channel may include a red channel, a green channel, or a blue channel.

[0082] During high-speed transmission, the data transmission rate is very high, and any transmission error or delay may cause video frames to be displayed repeatedly, leading to screen freezing. For example, in order to reduce bandwidth requirements and storage space during high-speed transmission, video streams are usually compressed and decompressed. The compression and decompression processes may mask or change certain features of video frames, causing partial image damage and affecting the accuracy of screen freezing detection. For example, DSC (Display Stream Compression) can partially damage the least significant bits of image data in areas of uneven color.

[0083] In some optional implementations, there are multiple image blocks to be tested, thereby enabling frozen screen detection based on multiple image blocks and reducing the impact of local damage caused by image compression. In this case, the image block to be tested corresponds to a region of the frame image, and the number of pixels in the image block is less than the number of pixels in the frame image.

[0084] In some embodiments, the test image of a frame image is determined based on the least significant bit of the preset color channel of the preset pixel of each frame image. The preset pixel can represent the position of the frame image corresponding to the test image. The preset pixel corresponds one-to-one with the pixel of the test image. Thus, the binary information of the pixel of the frame image can be determined based on the binary information of the least significant bit of the preset color channel of the preset pixel, thereby determining the test image of the frame image.

[0085] Then, the image to be tested can be divided into multiple test image blocks of the frame image. By using multiple test image blocks, it can be determined whether the display screen has frozen, reducing the impact of local damage caused by image compression. For example, if a test image block has pixel damage due to image compression, it can be determined whether the display screen has frozen by using other test image blocks, thus improving the accuracy of frozen screen detection.

[0086] For example, the size of the image to be tested can be less than or equal to the size of the frame image, that is, the number of pixels in the image to be tested can be less than or equal to the number of pixels in the frame image.

[0087] In one possible implementation, the image to be tested is divided according to the size of a preset watermark block, determining multiple image blocks to be tested within the frame image. The size of each image block to be tested is consistent with the size of the preset watermark block, facilitating subsequent comparison between the image blocks to be tested and the preset watermark block to determine whether the frame image carries the preset watermark block. Here, a watermark refers to information data embedded in an image in some way; in this case, it often refers to information data that is imperceptible to the human eye and requires extraction and detection using specific methods.

[0088] S102. Determine the watermark block carried by the frame image based on the image block to be tested and the preset watermark block of the frame image.

[0089] In this embodiment, the watermark block carried by the frame image is determined to be a preset watermark block based on the image block to be tested and the preset watermark block of the frame image, so as to determine whether at least two frame images continuously output to the display screen are consistent.

[0090] In some optional implementations, the structural similarity between the image block to be tested and the preset watermark block is determined. Based on the structural similarity, it is determined whether the watermark block carried by the frame image is the preset watermark block, so as to facilitate the determination of whether at least two frame images continuously output to the display screen are consistent.

[0091] For example, the structural similarity between the image block to be tested and the preset watermark block is determined based on the pixels of the image block to be tested and the pixels of the preset watermark block.

[0092] Based on the pixels of the image block to be tested and the preset watermark block, the brightness similarity between the two is determined; based on the pixels of the image block to be tested and the preset watermark block, the contrast similarity between the two is determined; based on the pixels of the image block to be tested and the preset watermark block, the structural similarity between the two is determined. Then, based on the brightness similarity, contrast similarity, and structural similarity, the structural similarity between the image block to be tested and the preset watermark block is determined.

[0093] For example, if the image patch to be tested has N pixels, x i Multiply the pixel value of each pixel by 255 to achieve the inverse transformation of normalized data, that is, to restore the normalized pixel value (range 0 to 1) back to the standard 8-bit image format (range 0 to 255).

[0094] Then the average brightness u of the image block under test x for:

[0095]

[0096] The average brightness u of the preset watermark block y The same calculation method is used. The function representation of brightness similarity is l(x,y):

[0097]

[0098] Where C1 is a coefficient, which is a constant of 6.5025.

[0099] Contrast ratio reflects the degree of drastic change in the brightness of an image, which is the standard deviation of pixel values. The standard deviation σ of the image patch under test is... x The formula is:

[0100]

[0101] The standard deviation σ of the preset watermark block y The same calculation method is used. The functional representation of contrast similarity is c(x,y):

[0102]

[0103] C2 is a coefficient, taken as a constant of 58.5225.

[0104] The formula for the covariance between the image patch x to be tested and the preset watermark patch y is:

[0105]

[0106] The structural similarity function is represented as s(x,y):

[0107]

[0108] C3 is a coefficient, taken as a constant of 29.26125.

[0109] Combining brightness, contrast, and structural similarity, we obtain structural similarity using the following formula:

[0110]

[0111] Based on this, the structural similarity between the image patch to be tested and the preset watermark patch can be determined. The larger the value of SSIM(x,y), the greater the similarity between the image patch to be tested and the preset watermark patch.

[0112] Then, based on structural similarity, it can be determined whether the frame image carries a preset watermark block.

[0113] As one implementation method, if the structural similarity between the image block to be tested and the preset watermark block is greater than or equal to the first threshold, it is determined that the frame image carries the preset watermark block; if the structural similarity between the image block to be tested and the preset watermark block is less than the first threshold, it is determined that the frame image does not carry the preset watermark block.

[0114] In some embodiments, the number of image blocks to be tested is multiple. For scenarios requiring high detection speed, multiple different preset watermark blocks can be used. Then, the structural similarity between each image block to be tested and each preset watermark block can be determined.

[0115] For ease of description, we take a single image frame as an example. We determine the structural similarity between each image block to be tested and each preset watermark block in the image frame. The structural similarity of each preset watermark block includes the structural similarity between each preset watermark block and each image block to be tested in the image frame.

[0116] In one possible implementation, a first number of predefined watermark blocks with structural similarity greater than a first threshold can be determined, referred to as the first number corresponding to each predefined watermark block. When the first number corresponding to any predefined watermark block is greater than or equal to a second threshold, it indicates that the number of test image blocks similar to that predefined watermark block is relatively large, and the frame image is determined to carry a predefined watermark block. When the first number corresponding to each predefined watermark block is less than the second threshold, it indicates that the number of test image blocks similar to that predefined watermark block is relatively small, and the frame image is determined not to carry a predefined watermark block. Based on this, it is possible to determine more accurately whether a frame image carries a predefined watermark block.

[0117] For example, for each preset watermark block, the structural similarity between each image block to be tested and the preset watermark block can be calculated, resulting in multiple structural similarities. Each structural similarity is then compared sequentially with a first threshold. When the structural similarity is greater than or equal to the first threshold, the similarity counter is incremented; when the structural similarity is less than the first threshold, the similarity counter remains unchanged. Then, the similarity counter results corresponding to each preset watermark block are counted, and the preset watermark block with the largest similarity counter result greater than a second threshold is taken as the preset watermark block carried by the frame image. If the similarity counters corresponding to each preset watermark block are all less than the second threshold, it is determined that the frame image does not carry a preset watermark block.

[0118] For example, there are four preset watermark blocks, denoted as the first preset watermark block, the second preset watermark block, the third preset watermark block, and the fourth preset watermark block, respectively. The first structural similarity between the first preset watermark block and each image block to be tested in the frame image is calculated to obtain multiple first structural similarities corresponding to the first preset watermark block. The second structural similarity between the second preset watermark block and each image block to be tested is calculated to obtain multiple second structural similarities corresponding to the second preset watermark block. The third structural similarity between the third preset watermark block and each image block to be tested is calculated to obtain multiple third structural similarities corresponding to the third preset watermark block. The fourth structural similarity between the fourth preset watermark block and each image block to be tested is calculated to obtain multiple fourth structural similarities corresponding to the fourth preset watermark block.

[0119] If, among the first, second, third, and fourth structural similarities, the number of structural similarities greater than or equal to the first threshold is greater than or equal to the second threshold, then the preset watermark block corresponding to the target structural similarity is taken as the preset watermark block carried by the frame image. The target structural similarity is defined as the largest number of structural similarities greater than or equal to the first threshold among the first, second, third, and fourth structural similarities. If the number of structural similarities less than the first threshold is less than the second threshold, the number of structural similarities less than the first threshold is less than the second threshold, the number of structural similarities less than the first threshold is less than the second threshold, the number of structural similarities less than the first threshold is less than the second threshold, and the number of structural similarities less than the first threshold is less than the second threshold, then the frame image is determined not to carry a preset watermark block.

[0120] In one possible implementation, the average structural similarity corresponding to each preset watermark block can be determined. If the average value corresponding to any preset watermark block is greater than or equal to a third threshold, it indicates that the frame image is relatively similar to the preset watermark block, and the frame image is determined to carry the preset watermark block. If the average value corresponding to each preset watermark block is less than the third threshold, it indicates that the frame image is less similar to the preset watermark block, and the frame image is determined not to carry the preset watermark block. Based on this, it is possible to determine more accurately whether a frame image carries a preset watermark block.

[0121] For example, for each preset watermark block, the structural similarity between each image block to be tested and the preset watermark block can be calculated to obtain multiple structural similarities. The average value of these multiple structural similarities is calculated. If there is a preset watermark block with an average value greater than a third threshold, the structural similarity with the largest average value is determined to be the preset watermark block carried by the frame image. If there is no preset watermark block with an average value greater than the third threshold, it is determined that the frame image does not carry a preset watermark block.

[0122] For example, there are four preset watermark blocks, denoted as the first preset watermark block, the second preset watermark block, the third preset watermark block, and the fourth preset watermark block. The first structural similarity between the first preset watermark block and each image block to be tested in the frame image is calculated, resulting in multiple first structural similarities corresponding to the first preset watermark block, and a first average value is calculated for these multiple first structural similarities. The second structural similarity between the second preset watermark block and each image block to be tested is calculated, resulting in multiple second structural similarities corresponding to the second preset watermark block, and a second average value is calculated for these multiple second structural similarities. The third structural similarity between the third preset watermark block and each image block to be tested is calculated, resulting in multiple third structural similarities corresponding to the third preset watermark block, and a third average value is calculated for these multiple third structural similarities. Finally, the fourth structural similarity between the fourth preset watermark block and each image block to be tested is calculated, resulting in multiple fourth structural similarities corresponding to the fourth preset watermark block, and a fourth average value is calculated for these multiple fourth structural similarities.

[0123] Among the first average value, the second average value, the third average value, and the fourth average value, the preset watermark block corresponding to the largest average value that is greater than the third threshold is taken as the preset watermark block carried by the frame image; if the first average value, the second average value, the third average value, and the fourth average value are all less than the third threshold, it is determined that the frame image does not carry a preset watermark block.

[0124] It should be noted that, based on the above two methods for determining whether a frame image carries a preset watermark block, watermark images can also be accurately detected even under processing such as image rotation (processing from clockwise rotation of 5 degrees to counterclockwise rotation of 5 degrees), image area occlusion (approximately 25% of the image area is occluded), image horizontal flipping, image vertical flipping, and DSC compression.

[0125] S103. Determine whether the display screen has frozen based on the changes in the watermark blocks carried by at least two frame images.

[0126] In this embodiment of the application, if the watermark blocks carried by at least two frames of images change, it is determined that the display screen has not frozen; if the watermark blocks carried by at least two frames of images do not change, it is determined that the display screen has frozen.

[0127] In some alternative implementations, if a preset watermark block is carried in a portion of at least two frame images, then the watermark block carried in at least two frame images changes (e.g., a change occurs where the preset watermark block is carried and where it is not carried), and it is determined that the display screen has not frozen.

[0128] If a preset watermark block is carried in each of at least two frames, and at least some frames carry different preset watermark blocks, then the watermark blocks carried in at least two frames change (e.g., the preset watermark blocks change), and it is determined that the display screen has not frozen.

[0129] If the same preset watermark block is carried in each of at least two frames of images, or if no preset watermark block is carried, then the watermark block carried in at least two frames of images has not changed, and it is determined that the display screen is frozen.

[0130] In one possible implementation, timing begins when at least two frames of images carry the same preset watermark block or do not carry a watermark block. If the timing duration has not reached the preset duration, a watermark error flag is sent to the display screen. If the timing duration has reached the preset duration, a screen freeze fault signal is sent to the display screen to prevent false screen freeze detection and improve the accuracy of screen freeze detection.

[0131] For example, a screen freeze fault signal (e.g., a fault image) can be output to the display screen to replace the original video stream until the screen freeze fault is resolved, at which point the output of the screen freeze fault signal stops.

[0132] In one possible implementation, if it is determined that the latter of two adjacent frame images carries a preset watermark block, the least significant bit of the preset color channel of the latter is replaced according to the least significant bit of the preset color channel of the former, which helps to restore image quality.

[0133] For example, when inserting a preset watermark block, the watermark block is inserted in frames 2, 4, 6, and 8 out of eight consecutive frames. In this case, the preset watermark block is inserted in four frames and not inserted in the other four frames. After the preset watermark block is identified, the least significant bit of the preset color channel in the current frame can be replaced with the least significant bit of the preset color channel in the previous frame.

[0134] The screen freeze detection method provided in this application determines the image block to be tested in each frame image based on the least significant bit of a preset color channel. Then, based on the image block to be tested and a preset watermark block, it determines whether the display screen has frozen. By inserting the image block to be tested into the least significant bit of the preset color channel, the impact on the frame image is minimal, thus ensuring image quality while achieving screen freeze detection. Furthermore, the screen freeze detection process provided in this application can encompass most of the video stream transmission process, enabling screen freeze detection over longer video links. The entire detection process does not require fixed pixels as a reference, and the detection method allows for image corruption and noise, making it suitable for screen freeze detection in high-speed transmission scenarios.

[0135] Figure 2 Flowchart of the screen freeze detection method provided in this application Figure 2 ,like Figure 2 As shown, in this embodiment... Figure 2 Based on the embodiments, the method includes:

[0136] S201. Insert a preset watermark block into the least significant bit of the preset color channel of at least a portion of the frame images of at least two frame images.

[0137] In this embodiment, a preset watermark block is inserted into the least significant bit of the preset color channel of at least a portion of at least two frame images. Compared to inserting the watermark into the entire image, this avoids noise caused by local image processing or image compression affecting the subsequent watermark detection accuracy. Furthermore, the local storage requirement is reduced, requiring only one preset watermark block to be stored. Since the preset watermark block is inserted into the least significant bit, the visual impact on the frame image is minimal.

[0138] For example, the same preset watermark block can be inserted in some frames of at least two frame images, different preset watermark blocks can be inserted in some frames of at least two frame images, or different preset watermark blocks can be inserted in all frames of at least two frame images.

[0139] For example, for eight frames of images continuously output to the display screen, the same preset watermark block can be inserted in two of the frames, the same preset watermark block can be inserted in four of the frames, the same preset watermark block can be inserted in six of the frames, or different preset watermark blocks can be inserted in two of the frames, different preset watermark blocks can be inserted in four of the frames, different preset watermark blocks can be inserted in six of the frames, different preset watermark blocks can be inserted in eight of the frames, etc.

[0140] For example, the preset watermark block can have central symmetry features and a clear binary boundary, such as a circle, star, or polygon shape, to facilitate determining whether a frame image carries the preset watermark block. When there are multiple preset watermark blocks, the similarity between the multiple preset watermarks can be low to ensure uniqueness and facilitate differentiation and detection.

[0141] In one possible implementation, multiple preset watermark blocks are inserted in a preset order into the least significant bit of a preset color channel of at least a portion of at least two frame images. Accordingly, the image to be tested and the image blocks to be tested within the image to be tested are determined based on the positions of the inserted preset watermark blocks. This allows for the determination of whether a frame image carries a preset watermark based on the multiple image blocks to be tested and the preset watermark blocks, reducing the impact of localized damage caused by image compression.

[0142] For example, taking a 512*512 frame image as an example, the preset number of watermark blocks is compared to inserting a watermark of the same size. During insertion, watermark blocks of 32*32, 64*64, or 128*128 sizes are selected according to the precision requirements. Figure 3 The repeated insertion method avoids the impact of noise caused by local image processing or image compression on the accuracy of subsequent watermark detection compared to full image insertion, and the insertion method is simple.

[0143] S202. For at least two consecutive frame images output to the display screen, determine the image block to be tested of the frame image based on the least significant bit of the preset color channel of each frame image.

[0144] S203. Determine the watermark block carried by the frame image based on the image block to be tested and the preset watermark block of the frame image.

[0145] S204. Determine whether the display screen has frozen based on the changes in the watermark blocks carried by at least two frame images.

[0146] The implementation method of step S202 is as follows: Figure 1 Similar to S101, the implementation method of step S203 is the same as... Figure 1 Similar to S102, the implementation of step S204 is the same as... Figure 1 Similar to S103, it will not be described again here.

[0147] The frozen screen detection method provided in this application inserts a preset watermark block into the least significant bit of the preset color channel of the frame image. Compared with inserting the entire image, segmentation can avoid noise caused by local image processing or image compression affecting the subsequent watermark detection accuracy. It can reduce the storage space requirement corresponding to the preset watermark block. Furthermore, the detection part only performs similarity detection on a fixed small block, without needing to store the entire image calculation process. It only uses a calculation circuit that matches the block size, without needing to perform a complete calculation on the entire frame. The required storage space is reduced, the circuit complexity is lower, and the characteristics of the adaptable circuit are improved.

[0148] Figure 4 This is a schematic diagram of the frozen screen detection device provided in this application, as shown below. Figure 4 As shown, the frozen screen detection device 10 provided in this embodiment includes:

[0149] The first determining module 11 is used to determine the image block to be tested of the frame image based on the least significant bit of the preset color channel of each frame image for at least two frame images that are continuously output to the display screen.

[0150] The second determining module 12 is used to determine the watermark block carried by the frame image based on the image block to be tested and the preset watermark block of the frame image.

[0151] The third determining module 13 is used to determine whether the display screen has frozen based on the changes in the watermark blocks carried by at least two frame images.

[0152] In one possible implementation, the first determining module 11 is specifically used to determine the image to be tested of the frame image based on the least significant bit of the preset color channel of the preset pixel of each frame image; and to divide the image to be tested to determine multiple image blocks to be tested of the frame image.

[0153] In one possible implementation, the first determining module 11 is specifically used to divide the image to be tested according to the size of the preset watermark block, and determine multiple image blocks to be tested of the frame image.

[0154] In one possible implementation, the second determining module 12 is specifically used to determine the structural similarity between the image block to be tested and the preset watermark block; based on the structural similarity, it determines whether the frame image carries the preset watermark block.

[0155] In one possible implementation, there are multiple image blocks to be tested and preset watermark blocks; the second determining module 12 is specifically used to determine a first number of preset watermark blocks whose structural similarity is greater than a first threshold; when the first number of any preset watermark block is greater than or equal to a second threshold, the frame image is determined to carry a preset watermark block; when the first number of each preset watermark block is less than the second threshold, the frame image is determined not to carry a preset watermark block.

[0156] In one possible implementation, there are multiple image blocks to be tested and preset watermark blocks; the second determining module 12 is specifically used to determine the average value of the structural similarity corresponding to each preset watermark block; when the average value corresponding to any preset watermark block is greater than or equal to a third threshold, it is determined that the frame image carries a preset watermark block; when the average value corresponding to each preset watermark block is less than the third threshold, it is determined that the frame image does not carry a preset watermark block.

[0157] In one possible implementation, the third determining module 13 is specifically used to determine that the display screen has not frozen when a preset watermark block is carried in some frames of at least two frame images; to determine that the display screen has not frozen when a preset watermark block is carried in each frame of at least two frame images and at least some frames carry different preset watermark blocks; and to determine that the display screen has frozen when the same preset watermark block is carried or no preset watermark block is carried in each frame of at least two frame images.

[0158] In one possible implementation, the third determining module 13 is further configured to start timing when it is determined that the display screen has frozen; send a watermark error signal to the display screen when the timing duration has not reached the preset duration; and send a frozen screen fault signal to the display screen when the timing duration has reached the preset duration.

[0159] In one possible implementation, the second determining module 13 is further configured to, when determining that the latter of two adjacent frame images carries a preset watermark block, replace the least significant bit of the preset color channel of the latter with the least significant bit of the preset color channel of the former.

[0160] In one possible implementation, an insertion module 14 is also included for inserting a preset watermark block in the least significant bit of a preset color channel of at least a portion of the frame images of at least two frame images.

[0161] In one possible implementation, the insertion module 14 is specifically used to insert a plurality of preset watermark blocks in a preset order at least a portion of the preset color channels of at least two frame images.

[0162] The frozen screen detection device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0163] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the electronic device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus.

[0164] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0165] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0166] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0167] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0168] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0169] This application also provides a signal transmission system, such as Figure 6 As shown, the signal transmission system includes a first device 601, a second device 602, and a third device 603, wherein the second device 602 is communicatively connected to the first device 601 and the third device 603.

[0170] The second device 602 is used to determine the image block to be tested of the frame image based on the least significant bit of the preset color channel of each frame image, based on the image block to be tested and the preset watermark block of the frame image, based on the image block to be tested and the preset watermark block of the frame image, and to determine whether the third device 603 has frozen based on the changes of the watermark blocks carried by the at least two frame images.

[0171] In one possible implementation, such as Figure 6 As shown, the signal transmission system further includes: a fourth device 604, which is communicatively connected to the first device 601 and the second device 602; the fourth device 604 is used to insert a preset watermark block into the least significant bit of a preset color channel of at least a portion of the frame images of at least two consecutively output frame images by the first device 601.

[0172] In one possible implementation, such as Figure 6 As shown, the signal transmission system further includes a fifth device 605, which includes a serializer 6051, an image compression module 6052, and a deserializer 6053. The fifth device 605 is communicatively connected to the fourth device 604 and the second device 602. The serializer 6051 is used to convert the parallel data of the frame image output by the fourth device 604 into serial data; the image compression module 6052 is used to compress the serial data; and the deserializer 6053 is used to decompress the compressed serial data, convert it back into parallel data, and send it to the second device 602.

[0173] For example, the first device can be the host computer, and the third device can be the display device.

[0174] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0175] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0176] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0177] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0178] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0179] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0180] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0181] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0182] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0183] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for detecting frozen screen, characterized in that, include: For at least two consecutive frame images output to the display screen, the image block to be tested in the frame image is determined based on the least significant bit in the preset color channel of each frame image. The watermark block carried by the frame image is determined based on the image block to be tested and the preset watermark block of the frame image; Based on the changes in the watermark blocks carried by the at least two frame images, it is determined whether the display screen has frozen.

2. The method according to claim 1, characterized in that, The step of determining the image block to be tested in a frame image based on the least significant bit in a preset color channel of each frame image includes: The image to be tested is determined based on the least significant bit of the preset color channel of the preset pixels in each frame image; The image to be tested is divided into multiple image blocks to be tested in the frame image.

3. The method according to claim 2, characterized in that, The image to be tested is divided into multiple image blocks to be tested within the frame image, including: Based on the size of the preset watermark block, the image to be tested is divided to determine multiple image blocks to be tested in the frame image.

4. The method according to claim 1, characterized in that, The step of determining the watermark block carried by the frame image based on the image block to be tested and the preset watermark block includes: Determine the structural similarity between the image block to be tested and the preset watermark block; Based on the structural similarity, it is determined whether the frame image carries the preset watermark block.

5. The method according to claim 4, characterized in that, The number of the image block to be tested and the preset watermark block are multiple; The step of determining whether the frame image carries the preset watermark block based on the structural similarity includes: Determine a first number of pre-defined watermark blocks whose structural similarity is greater than a first threshold. When the first number corresponding to any of the preset watermark blocks is greater than or equal to the second threshold, it is determined that the frame image carries the preset watermark block; When the first number corresponding to each preset watermark block is less than the second threshold, it is determined that the frame image does not carry the preset watermark block.

6. The method according to claim 5, characterized in that, The number of the image block to be tested and the preset watermark block is multiple; the step of determining whether the frame image carries the preset watermark block based on the structural similarity includes: Determine the average structural similarity of each preset watermark block; When the average value corresponding to any of the preset watermark blocks is greater than or equal to the third threshold, it is determined that the frame image carries the preset watermark block; When the average value corresponding to each preset watermark block is less than the third threshold, it is determined that the frame image does not carry the preset watermark block.

7. The method according to claim 2, characterized in that, The step of determining whether the display screen has frozen based on the changes in the watermark blocks carried by the at least two frame images includes: If the preset watermark block is carried in a portion of the frame images of at least two frame images, it is determined that the display screen has not frozen. When the preset watermark block is carried in each of the at least two frame images, and at least some frame images carry different preset watermark blocks, it is determined that the display screen has not frozen. When the display screen freezes, it is determined that the same preset watermark block is carried or the preset watermark block is not carried in each of the at least two frame images.

8. The method according to claim 7, characterized in that, The method further includes: A timer begins when the display screen is determined to have frozen. If the timeout period does not reach the preset duration, a watermark error signal is sent to the display screen; if the timeout period reaches the preset duration, a screen freeze fault signal is sent to the display screen.

9. The method according to claim 2, characterized in that, The method further includes: If it is determined that the latter of two adjacent frame images carries the preset watermark block, the least significant bit of the preset color channel of the latter is replaced according to the least significant bit of the preset color channel of the former.

10. The method according to any one of claims 1-9, characterized in that, The method further includes: The preset watermark block is inserted into the least significant bit of the preset color channel of at least a portion of the frame images of the at least two frame images.

11. The method according to claim 10, characterized in that, The insertion of the preset watermark block into the least significant bit of the preset color channel of at least a portion of the frames in at least two frames includes: In the least significant bit of the preset color channel of at least a portion of the frame images of the at least two frame images, a plurality of the preset watermark blocks are inserted in a preset order.

12. A frozen screen detection device, characterized in that, include: The first determining module is used to determine the image block to be tested of at least two consecutively output frame images to the display screen based on the least significant bit of the preset color channel of each frame image. The second determining module is used to determine the watermark block carried by the frame image based on the image block to be tested and the preset watermark block of the frame image; The third determining module is used to determine whether the display screen has frozen based on the changes in the watermark blocks carried by the at least two frame images.

13. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-11.

14. A signal transmission system, characterized in that, include: A first device, a second device, and a third device, wherein the second device is communicatively connected to the first device and the third device; The second device is used to determine the image block to be tested of the frame image based on the least significant bit of the preset color channel of each frame image, for at least two frame images continuously output by the first device to the third device. The watermark block carried by the frame image is determined based on the image block to be tested and the preset watermark block of the frame image; Based on the changes in the watermark blocks carried by the at least two frame images, it is determined whether the third device has experienced screen freezing.

15. The system according to claim 14, characterized in that, Also includes: A fourth device, which is communicatively connected to the first device and the second device; The fourth device is used to insert the preset watermark block into the least significant bit of a preset color channel of at least a portion of the frame images of at least two consecutively output frame images by the first device.

16. The system according to claim 15, characterized in that, Also includes: The fifth device includes: a serializer, an image compression module, and a deserializer; the fifth device is communicatively connected to the fourth device and the second device. The serializer is used to convert the parallel data of the frame image output by the fourth device into serial data. The image compression module is used to compress the serial data; The deserializer is used to decompress the compressed serial data, convert it into parallel data, and send it to the second device.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-11.

18. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-11.

Images