A method and apparatus for monitoring a video freeze based on a SoC

Abstract

The application discloses a kind of based on SoC's video picture freeze monitoring method and device, the method includes: the monitoring signal in the system level chip inside application equipment is collected;Monitoring signal is handled, and the image frame sequence corresponding to monitoring signal is obtained;Determine whether image frame sequence satisfies the target video picture freeze condition set in advance, when judging that image frame sequence satisfies target video picture freeze condition, determine that application equipment exists video picture freeze situation.Visibly, the present application can carry out video picture freeze monitoring before original video signal is displayed in display equipment, improves video picture freeze monitoring efficiency;In addition, the monitoring of video picture freeze is realized by image verification algorithm, can reduce video picture freeze monitoring algorithm complexity while guaranteeing monitoring accuracy, to reduce hardware resource occupancy rate, reduce development cost, especially in the scene of automobile instrument video picture freeze monitoring, can greatly improve monitoring reliability and safety.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a video frame freeze monitoring method and apparatus based on SoC. Background Technology

[0002] With the development of technology, more and more car instrument displays are shifting from mechanical displays to full LCD video displays. In addition to displaying traditional information such as vehicle speed, mileage, RPM, fuel consumption, and battery level, full LCD video displays also include a variety of emerging video information such as perceived vehicle environment, real-time road conditions, and entertainment settings. This not only meets the driver's basic driving safety requirements but also further enhances the driver's driving experience.

[0003] However, in practical applications, due to software and hardware malfunctions in automotive electronics, the video feed on the dashboard frequently freezes. When this happens, drivers generally cannot detect it in time and take appropriate action, posing a significant safety hazard. Currently, existing technologies typically use two methods to detect video feed freezes on automotive dashboards: the first is to use a region histogram of image frames to measure the image similarity between adjacent frames; the second is to calculate the pixel difference values ​​at corresponding positions in adjacent frames and measure the image similarity between adjacent frames by the proportion of differing pixels to the total number of pixels. However, practice has shown that the aforementioned two methods, as well as other existing methods, are mostly based on computer vision principles. They either require complex image processing or the deployment of complex neural network models. These methods are coupled or dependent on hardware modules such as GPUs and memory. If related hardware resources malfunction (e.g., fail), the detection of video feed freezes cannot be achieved, reducing the effectiveness and stability of video feed freeze detection.

[0004] Therefore, it is particularly important to propose a technical solution to improve the stability and effectiveness of video image freeze monitoring. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a video freeze monitoring method and device based on a SoC (System-on-a-Chip). The monitoring signal for video freeze monitoring originates from within the system-on-a-chip, enabling video freeze monitoring to be performed in front of the original video signal displayed on the display device. This improves the efficiency of video freeze monitoring and ensures its timeliness. Furthermore, video freeze monitoring can be implemented using image verification algorithms or lightweight monitoring models that can be deployed within the system-on-a-chip. This reduces the complexity of the video freeze monitoring algorithm while ensuring monitoring accuracy, thereby reducing hardware resource consumption and development costs. Especially in automotive instrument panel video freeze monitoring scenarios, this significantly improves monitoring reliability and security.

[0006] To address the aforementioned technical problems, the first aspect of this invention discloses a video freeze monitoring method based on a SoC, applied in a monitoring device, wherein the monitoring device is used to implement video freeze monitoring of an application device with video output functionality, and the method includes: Collect monitoring signals from the system-on-a-chip in the application device; The monitoring signal is processed to obtain the image frame sequence corresponding to the monitoring signal; Determine whether the image frame sequence meets the preset target video frame freeze condition. When it is determined that the image frame sequence meets the target video frame freeze condition, it is determined that the application device has a video frame freeze.

[0007] As an optional implementation, in the first aspect of the present invention, determining whether the image frame sequence meets the preset video frame freeze conditions includes: Each image frame in the image frame sequence is verified according to a preset verification algorithm to obtain a verification value for each image frame; Based on the checksum of each image frame, a target value is calculated, and it is determined whether the target value meets a pre-set first video frame freeze condition. When it is determined that the target value meets the first video frame freeze condition, the image frame sequence is determined to meet the target video frame freeze condition; the target value is used to represent the cumulative number of consecutively frozen frames; or, The image frame sequence is input into a pre-trained video frame freeze detection model to obtain the output result of the video frame freeze detection model. It is then determined whether the output result meets the second video frame freeze condition. When it is determined that the output result meets the second video frame freeze condition, the image frame sequence is determined to meet the target video frame freeze condition.

[0008] As an optional implementation, in the first aspect of the present invention, the step of calculating a target value based on the check value of each image frame, determining whether the target value satisfies a preset first video frame freeze condition, and determining that the image frame sequence satisfies the target video frame freeze condition when it is determined that the target value satisfies the first video frame freeze condition includes: Determine whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame; When a match is found, the current image frame is determined to be a frozen frame, and the cumulative value of the frozen frame is updated. It is then determined whether the cumulative value of the frozen frame is greater than or equal to a preset threshold. When the cumulative value of the frozen frame is greater than or equal to the preset threshold, the image frame sequence is determined to meet the target video frame freezing condition. When the cumulative value of the frozen frame is not greater than or equal to the preset threshold, it is determined whether there is a subsequent adjacent image frame to the current image frame. When there is a subsequent adjacent image frame to the current image frame, the subsequent adjacent image frame to the current image frame is determined as a new current image frame to be judged, and the operation of determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated. When a mismatch is detected, the current image frame is determined not to be a frozen frame, and the cumulative value of the frozen frame is reset. It is then determined whether there is a subsequent adjacent image frame to the current image frame. If a preceding adjacent image frame to the current image frame is detected, the subsequent adjacent image frame to the current image frame is determined as a new current image frame to be judged, and the operation of determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated.

[0009] As an optional implementation, in the first aspect of the present invention, the step of calculating a target value based on the check value of each image frame, determining whether the target value satisfies a preset first video frame freeze condition, and determining that the image frame sequence satisfies the target video frame freeze condition when it is determined that the target value satisfies the first video frame freeze condition includes: Determine whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame; When a match is found, the current image frame is determined to be a frozen frame, and the cumulative value of the frozen frame is updated. It is then determined whether there is a subsequent adjacent image frame to the current image frame. If a subsequent adjacent image frame is found, the subsequent adjacent image frame is determined as a new current image frame to be judged, and the operation of judging whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated. When a mismatch is detected, the current image frame is determined not to be a frozen frame, and the cumulative value of frozen frames is determined to be greater than or equal to a preset threshold. When the cumulative value of frozen frames is determined to be greater than or equal to the preset threshold, the image frame sequence is determined to meet the target video frame freezing condition. When the cumulative value of frozen frames is determined not to be greater than or equal to the preset threshold, the cumulative value of frozen frames is reset, and it is determined whether there is a subsequent adjacent image frame to the current image frame. When it is determined that there is a subsequent adjacent image frame to the current image frame, the subsequent adjacent image frame to the current image frame is determined as a new current image frame to be judged, and the operation of determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated.

[0010] As an optional implementation, in the first aspect of the present invention, before determining whether the check value of the current image frame to be determined matches the check value of the preceding adjacent image frame of the current image frame, the method further includes: After determining the current image frame to be judged, it is determined whether there is a preceding adjacent image frame in the image frame sequence. If it is determined that there is a preceding adjacent image frame, the operation of determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame is triggered. If it is determined that there is no preceding adjacent image frame, the following adjacent image frame of the current image frame is determined as the new current image frame to be judged, and the operation of determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame is triggered; or, After determining the current image frame to be judged, it is determined whether a checksum judgment operation for adjacent image frames has been performed before the current time. If it is determined that a checksum judgment operation for adjacent image frames has been performed, the operation of judging whether the checksum of the current image frame to be judged matches the checksum of the preceding adjacent image frame of the current image frame is triggered. If it is determined that no checksum judgment operation for adjacent image frames has been performed, the following adjacent image frame of the current image frame is determined as the new current image frame to be judged, and the operation of judging whether the checksum of the current image frame to be judged matches the checksum of the preceding adjacent image frame of the current image frame is triggered.

[0011] As an optional implementation, in the first aspect of the present invention, the monitoring signal includes: a first signal output by the display engine inside the system-on-a-chip; or a second signal output by the timing control circuit inside the system-on-a-chip; or a third signal output by a functional component disposed after the timing control circuit inside the system-on-a-chip and before the display interface of the application device. The first signal is obtained by the display engine acquiring a standard video signal and performing a preset first conversion operation on the standard video signal; The second signal is obtained by the timing control circuit performing a preset second conversion operation on the first output signal output by the display engine; The third signal is obtained by the functional component performing a preset third conversion operation on the second output signal output by the timing control circuit.

[0012] As an optional implementation, in the first aspect of the present invention, after processing the monitoring signal to obtain the image frame sequence corresponding to the monitoring signal, the method further includes: For each original image frame in the image frame sequence, the region of interest of the original image frame is determined, and the region of interest of the original image frame is cropped out from the original image frame as the target image frame corresponding to the original image frame. The target image frame includes the region of interest of the original image frame. The image frame sequence is updated based on the target image frames corresponding to all the original image frames.

[0013] As an optional implementation, in the first aspect of the present invention, the method further includes: When it is determined that the video screen of the application device is frozen, an interrupt signal is generated, and a matching exception handling operation is performed according to the interrupt signal; The step of performing a matching exception handling operation based on the interrupt signal includes: The interrupt signal is transmitted to the video transmission control device of the application device, so that the video transmission control device stops transmitting the video signal; and / or, An alarm message is generated based on the interrupt signal, and an alarm operation is performed based on the alarm message; and / or, The interrupt signal is transmitted to the video signal processing device of the application device to reset the video signal processing device.

[0014] A second aspect of the present invention discloses a video frame freeze monitoring device based on a SoC, the device comprising: The acquisition module is used to acquire monitoring signals from inside the system-on-a-chip in the application device. The processing module is used to process the monitoring signal to obtain the image frame sequence corresponding to the monitoring signal; The judgment module is used to determine whether the image frame sequence meets the preset target video frame freezing conditions; The first determining module is used to determine that the application device has a video frame freeze when the judging module determines that the image frame sequence meets the target video frame freeze condition.

[0015] As an optional implementation, in the second aspect of the present invention, the specific method by which the determining module performs the operation of determining whether the image frame sequence meets the preset target video frame freezing conditions is as follows: Each image frame in the image frame sequence is verified according to a preset verification algorithm to obtain a verification value for each image frame; Based on the checksum of each image frame, a target value is calculated, and it is determined whether the target value meets a pre-set first video frame freeze condition. When it is determined that the target value meets the first video frame freeze condition, the image frame sequence is determined to meet the target video frame freeze condition; the target value is used to represent the cumulative number of consecutively frozen frames; or, The image frame sequence is input into a pre-trained video frame freeze detection model to obtain the output result of the video frame freeze detection model. It is then determined whether the output result meets the second video frame freeze condition. When it is determined that the output result meets the second video frame freeze condition, the image frame sequence is determined to meet the target video frame freeze condition.

[0016] As an optional implementation, in the second aspect of the present invention, the specific method by which the judging module performs the operation of calculating a target value based on the check value of each image frame and judging whether the target value satisfies a preset first video frame freezing condition, and determining that the image frame sequence satisfies the target video frame freezing condition when it is determined that the target value satisfies the first video frame freezing condition, is as follows: Determine whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame; When a match is found, the current image frame is determined to be a frozen frame, and the cumulative value of the frozen frame is updated. It is then determined whether the cumulative value of the frozen frame is greater than or equal to a preset threshold. When the cumulative value of the frozen frame is greater than or equal to the preset threshold, the image frame sequence is determined to meet the target video frame freezing condition. When the cumulative value of the frozen frame is not greater than or equal to the preset threshold, it is determined whether there is a subsequent adjacent image frame to the current image frame. When there is a subsequent adjacent image frame to the current image frame, the subsequent adjacent image frame to the current image frame is determined as a new current image frame to be judged, and the operation of determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated. When a mismatch is detected, the current image frame is determined not to be a frozen frame, and the cumulative value of the frozen frame is reset. It is then determined whether there is a subsequent adjacent image frame to the current image frame. If a preceding adjacent image frame to the current image frame is detected, the subsequent adjacent image frame to the current image frame is determined as a new current image frame to be judged, and the operation of determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated.

[0017] As an optional implementation, in the second aspect of the present invention, the specific method by which the judging module performs the operation of calculating a target value based on the check value of each image frame and judging whether the target value satisfies a preset first video frame freezing condition, and determining that the image frame sequence satisfies the target video frame freezing condition when it is determined that the target value satisfies the first video frame freezing condition, is as follows: Determine whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame; When a match is found, the current image frame is determined to be a frozen frame, and the cumulative value of the frozen frame is updated. It is then determined whether there is a subsequent adjacent image frame to the current image frame. If a subsequent adjacent image frame is found, the subsequent adjacent image frame is determined as a new current image frame to be judged, and the operation of judging whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated. When a mismatch is detected, the current image frame is determined not to be a frozen frame, and the cumulative value of frozen frames is determined to be greater than or equal to a preset threshold. When the cumulative value of frozen frames is determined to be greater than or equal to the preset threshold, the image frame sequence is determined to meet the target video frame freezing condition. When the cumulative value of frozen frames is determined not to be greater than or equal to the preset threshold, the cumulative value of frozen frames is reset, and it is determined whether there is a subsequent adjacent image frame to the current image frame. When it is determined that there is a subsequent adjacent image frame to the current image frame, the subsequent adjacent image frame to the current image frame is determined as a new current image frame to be judged, and the operation of determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated.

[0018] As an optional implementation, in a second aspect of the invention, the determining module is further configured to: After determining the current image frame to be judged, it is determined whether there is a preceding adjacent image frame in the image frame sequence. If it is determined that there is a preceding adjacent image frame, the operation of determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame is triggered. If it is determined that there is no preceding adjacent image frame, the following adjacent image frame of the current image frame is determined as the new current image frame to be judged, and the operation of determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame is triggered; or, After determining the current image frame to be judged, it is determined whether a checksum judgment operation for adjacent image frames has been performed before the current time. If it is determined that a checksum judgment operation for adjacent image frames has been performed, the operation of judging whether the checksum of the current image frame to be judged matches the checksum of the preceding adjacent image frame of the current image frame is triggered. If it is determined that no checksum judgment operation for adjacent image frames has been performed, the following adjacent image frame of the current image frame is determined as the new current image frame to be judged, and the operation of judging whether the checksum of the current image frame to be judged matches the checksum of the preceding adjacent image frame of the current image frame is triggered.

[0019] As an optional implementation, in a second aspect of the present invention, the monitoring signal includes: a first signal output by the display engine inside the system-on-a-chip; or a second signal output by the timing control circuit inside the system-on-a-chip; or a third signal output by a functional component disposed after the timing control circuit inside the system-on-a-chip and before the display interface of the application device. The first signal is obtained by the display engine acquiring a standard video signal and performing a preset first conversion operation on the standard video signal; The second signal is obtained by the timing control circuit performing a preset second conversion operation on the first output signal output by the display engine; The third signal is obtained by the functional component performing a preset third conversion operation on the second output signal output by the timing control circuit.

[0020] As an optional implementation, in a second aspect of the invention, the apparatus further includes: The second determining module is used to determine the region of interest for each original image frame in the image frame sequence after the processing module performs the operation of processing the monitoring signal to obtain the image frame sequence corresponding to the monitoring signal. The cropping module is used to crop the region of interest of the original image frame as the target image frame corresponding to the original image frame, and the target image frame includes the region of interest of the original image frame; An update module is used to update the image frame sequence based on the target image frames corresponding to all the original image frames.

[0021] As an optional implementation, in a second aspect of the invention, the apparatus further includes: The generation module is used to generate an interrupt signal when it is determined that the application device has a video frame freeze. The processing module is also configured to perform a matching exception handling operation based on the interrupt signal; The specific method by which the processing module performs the matching exception handling operation based on the interrupt signal is as follows: The interrupt signal is transmitted to the video transmission control device of the application device, so that the video transmission control device stops transmitting the video signal; and / or, An alarm message is generated based on the interrupt signal, and an alarm operation is performed based on the alarm message; and / or, The interrupt signal is transmitted to the video signal processing device of the application device to reset the video signal processing device.

[0022] A third aspect of the present invention discloses another video frame freeze monitoring device based on a SoC, the device comprising: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the SoC-based video frame freeze monitoring method disclosed in the first aspect of the present invention.

[0023] The fourth aspect of the present invention discloses a computer-storable medium storing computer instructions, which, when invoked, are used to execute the SoC-based video frame freeze monitoring method disclosed in the first aspect of the present invention.

[0024] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: In this embodiment of the invention, monitoring signals from within the system-on-a-chip (SoC) of the application device are collected; the monitoring signals are processed to obtain the corresponding image frame sequence; it is determined whether the image frame sequence meets the preset target video freeze conditions. When it is determined that the image frame sequence meets the target video freeze conditions, it is determined that the application device has a video freeze situation. It can be seen that the monitoring signals used for video freeze monitoring in this invention originate from within the SoC, enabling video freeze monitoring to be performed in front of the original video signal displayed on the display device, thus improving the efficiency of video freeze monitoring and ensuring the timeliness of monitoring. Furthermore, video freeze monitoring can be implemented through image verification algorithms or lightweight monitoring models that can be deployed within the SoC. This reduces the complexity of the video freeze monitoring algorithm while ensuring monitoring accuracy, thereby reducing hardware resource consumption and development costs. Especially in the scenario of video freeze monitoring in automotive instrument panels, it can greatly improve monitoring reliability and security. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of the internal structure of the SoC to which the SoC-based video frame freeze monitoring method disclosed in this embodiment of the invention is applicable; Figure 2 This is a flowchart illustrating a video frame freeze monitoring method based on SoC disclosed in an embodiment of the present invention; Figure 3 This is a flowchart illustrating another video frame freeze monitoring method based on SoC disclosed in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a video frame freeze monitoring device based on SoC disclosed in an embodiment of the present invention; Figure 5 This is a schematic diagram of another SoC-based video frame freeze monitoring device disclosed in an embodiment of the present invention; Figure 6 This is a schematic diagram of another SoC-based video frame freeze monitoring device disclosed in an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or end that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or ends.

[0029] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0030] This invention discloses a video freeze monitoring method and apparatus based on a SoC (System-on-a-Chip). It collects monitoring signals from within the system-on-a-chip (SoC) of an application device; processes the monitoring signals to obtain image frame sequences corresponding to the monitoring signals; and determines whether the image frame sequences meet pre-set target video freeze conditions. When the image frame sequences meet the target video freeze conditions, it is determined that the application device is experiencing video freeze. As can be seen, the monitoring signals used in this invention for video freeze monitoring originate from within the SoC, enabling video freeze monitoring to be performed in front of the original video signal displayed on the display device, thus improving the efficiency of video freeze monitoring and ensuring its timeliness. Furthermore, video freeze monitoring can be implemented using image verification algorithms or lightweight monitoring models that can be deployed within the SoC. This reduces the complexity of the video freeze monitoring algorithm while ensuring monitoring accuracy, thereby reducing hardware resource consumption and development costs. Especially in automotive instrument panel video freeze monitoring scenarios, it can greatly improve monitoring reliability and security. Detailed explanations follow.

[0031] To better understand the SoC-based video freeze detection method and apparatus described in this invention, the internal structure of the system-on-a-chip (SoC) in the application device requiring video freeze detection is first described. Figure 1 This is a schematic diagram of a system-on-a-chip (SoC) disclosed in an embodiment of the present invention. Figure 1 As shown, the internal structure of a SoC may include a display engine (DE), a timing controller (TCON), and various display interfaces (such as...). Figure 1 The structural units shown include LVDS interface, DSI interface, RGB interface, etc. Specifically, when monitoring video frame freeze in an application device, the monitoring device collects monitoring signals from within the SoC. These monitoring signals can be signals output by the TCON, signals output by the DE, or signals output by functional components connected after the TCON and before the display interface. Figure 1 This explanation uses the "signal output by TCON" as an example.

[0032] Optionally, the monitoring device can be integrated within the SoC or externally connected to the SoC. Figure 1 The example is that the monitoring device is integrated inside the SoC and specifically connected after the TCON.

[0033] In practical applications, if the monitoring device detects video freezing on the application device based on the collected monitoring signals, the monitoring device can intelligently handle the video freezing phenomenon, such as... Figure 1The diagram shows the generation of an interrupt signal and the freezing process based on the generated interrupt signal.

[0034] It should be noted that, Figure 1 The schematic diagram of the SoC internal structure shown is only intended to illustrate the internal structure of an SoC applicable to a SoC-based video freeze detection method. The positions and sizes of structural units such as the display engine, timing control circuit, monitoring device, and various display interfaces are also only schematic representations. Figure 1 The schematic diagram of the SoC's internal structure shown is not intended to limit this. The following is a detailed description of a SoC-based method and apparatus for detecting video frame freezes.

[0035] Example 1 Please see Figure 2 , Figure 2 This is a flowchart illustrating a video frame freeze detection method based on a SoC disclosed in an embodiment of the present invention. Figure 2 The described SoC-based video freeze monitoring method can be applied to a monitoring device used to monitor whether video freeze occurs in an application device with video output functionality. Optionally, the monitoring device can be integrated into the application device, its in-vehicle device, or a smart server or smart platform; this embodiment of the invention does not impose any limitations. Figure 2 As shown, the SoC-based video frame freeze detection method may include the following operations: 101. Collect monitoring signals from the system-on-a-chip in the application device.

[0036] 102. Process the monitoring signal to obtain the image frame sequence corresponding to the monitoring signal.

[0037] 103. Determine whether the image frame sequence meets the preset target video frame freezing conditions. If the image frame sequence meets the target video frame freezing conditions, trigger step 104. If the image frame sequence does not meet the target video frame freezing conditions, the current process can be terminated or step 101 can be triggered.

[0038] 104. Confirm that the video feed on the application device is frozen.

[0039] The monitoring device used to monitor video frame freezing has an independent power supply, clock and reset mechanism, as well as an IC layout.

[0040] As can be seen, the monitoring device for monitoring video frame freeze in this embodiment of the invention is not coupled or dependent on other units involved in image display, such as GPU and memory, which can enhance the stability of video frame freeze monitoring. The monitoring signal for video frame freeze monitoring originates from inside the system-on-a-chip (SoC), and video frame freeze monitoring can be performed in front of the original video signal displayed on the display device, which improves the efficiency of video frame freeze monitoring and ensures the timeliness of monitoring. In addition, video frame freeze monitoring can be achieved directly by judging whether the image frame sequence obtained after processing the monitoring signal from inside the SoC meets the preset video frame freeze conditions. The implementation method is simple and conducive to improving monitoring efficiency.

[0041] In this embodiment of the invention, optionally, the monitoring signal may include: a first signal output by the display engine inside the system-on-a-chip; or a second signal output by the timing control circuit inside the system-on-a-chip; or a third signal output by a functional component located after the timing control circuit inside the system-on-a-chip and before the display interface of the application device.

[0042] The first signal is obtained by the display engine acquiring a standard video signal and performing a preset first conversion operation on the standard video signal. It is evident that the monitoring signal used for video freeze detection is obtained after conversion by the display engine. This simplifies the difficulty of embedding the monitoring device into the system-on-a-chip, and the monitoring device's acquisition of the monitoring signal is not affected by the timing control circuit, thereby improving the reliability of video freeze detection.

[0043] The aforementioned second signal is obtained by the timing control circuit performing a preset second conversion operation on the first output signal output by the display engine. It is evident that the monitoring signal used for video freeze detection, obtained after conversion by the timing control circuit, is closer to the display signal output by the display interface, thus improving the accuracy of video freeze detection.

[0044] Furthermore, the aforementioned third signal is obtained by performing a preset third conversion operation on the second output signal output by the timing control circuit by the functional component.

[0045] It is evident that converting the video signal into the driving signal required by the display interface, and using the driving signal required by the display interface as the monitoring signal for video frame freeze monitoring, can simulate the real output scene and reduce the processing difficulty of the monitoring device for the monitoring signal. This is beneficial to improving the processing efficiency of the monitoring device for the monitoring signal, and in turn, improving the processing efficiency of the monitoring device for image frame sequences, and improving the accuracy of video frame freeze monitoring.

[0046] Meanwhile, the display interfaces include, but are not limited to, RGB, LVDS, DSI, HDMI, etc. It can be seen that the video freeze monitoring device proposed in this invention can perform monitoring without changing the input signal format. It supports video freeze monitoring signal formats covering all existing video display formats and has the characteristics of strong monitoring signal compatibility and strong versatility in application scenarios.

[0047] In this embodiment of the invention, optionally, after processing the monitoring signal to obtain the image frame sequence corresponding to the monitoring signal, the process may further include: For each original image frame in the image frame sequence, the region of interest of the original image frame is determined, and the region of interest of the original image frame is cropped out from the original image frame as the target image frame corresponding to the original image frame. The target image frame includes the region of interest of the original image frame. Update the image frame sequence based on the target image frames corresponding to all original image frames.

[0048] As can be seen, the embodiments of the present invention can also identify the region of interest for each original image frame in the image frame sequence and crop the identified region to form a new image frame sequence. This helps to reduce unnecessary information in the image frame sequence, improves the efficiency and accuracy of judging whether the image frame sequence meets the video freeze condition, and further improves the monitoring efficiency and accuracy of video freeze monitoring.

[0049] In an optional embodiment, the above-mentioned determination of whether the image frame sequence meets the preset video frame freeze condition may include: Each image frame in the image frame sequence is verified according to a preset verification algorithm to obtain the verification value of each image frame; Based on the check value of each image frame, the target value is calculated, and it is determined whether the target value meets the preset first video frame freezing condition. When it is determined that the target value meets the first video frame freezing condition, the image frame sequence is determined to meet the target video frame freezing condition. The target value is used to represent the cumulative number of consecutively frozen frames.

[0050] Optionally, the preset verification algorithm includes, but is not limited to, one of the following: MD5 algorithm, CRC algorithm, MD2 algorithm, checksum algorithm, SHA series algorithm, etc.

[0051] As can be seen, this optional embodiment can reduce the complexity of the video freeze monitoring algorithm while ensuring the accuracy of video freeze monitoring, thereby reducing hardware resource consumption and development costs. At the same time, the image verification algorithm that the video freeze monitoring device can use covers a variety of mainstream image verification algorithms, which shows that the monitoring device has the characteristic of high compatibility of built-in algorithms. Especially in the scenario of video freeze monitoring of automotive instrument panel, it can greatly improve the reliability and security of monitoring.

[0052] In this optional embodiment, as an optional implementation, the above-mentioned calculation of the target value based on the check value of each image frame, and determination of whether the target value meets the preset first video frame freeze condition, and determination that the image frame sequence meets the target video frame freeze condition when it is determined that the target value meets the first video frame freeze condition, may include: Determine whether the check value of the current image frame to be judged matches the check value of the preceding image frame; When a match is found, the current image frame is determined to be a frozen frame, and the cumulative value of the frozen frame is updated; it is determined whether the cumulative value of the frozen frame is greater than or equal to a preset threshold; when the cumulative value of the frozen frame is greater than or equal to the preset threshold, the image frame sequence is determined to meet the target video frame freezing condition; when the cumulative value of the frozen frame is not greater than or equal to the preset threshold, it is determined whether there is a subsequent adjacent image frame to the current image frame; when it is determined that there is a subsequent adjacent image frame to the current image frame, the subsequent adjacent image frame to the current image frame is determined as the new current image frame to be judged, and the operation of judging whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated. When a mismatch is detected, the current image frame is determined not to be a frozen frame, and the cumulative value of the frozen frame is reset. It is then determined whether there is a subsequent adjacent image frame to the current image frame. If a preceding adjacent image frame to the current image frame is detected, the subsequent adjacent image frame to the current image frame is determined as the new current image frame to be judged, and the operation of judging whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated.

[0053] As can be seen, this optional embodiment can improve the monitoring accuracy while ensuring the timeliness of monitoring by updating the cumulative value of frozen frames when the current image frame is determined to be a frozen frame and determining whether the cumulative value of frozen frames is greater than or equal to a preset threshold. In this optional embodiment, as another optional implementation, the above-mentioned calculation of the target value based on the check value of each image frame, and determination of whether the target value meets the preset first video frame freezing condition, and determination that the image frame sequence meets the target video frame freezing condition when it is determined that the target value meets the first video frame freezing condition, may also include: Determine whether the check value of the current image frame to be judged matches the check value of the preceding image frame; When a match is found, the current image frame is determined to be a frozen frame, and the cumulative value of the frozen frame is updated. It is then determined whether there is a subsequent adjacent image frame to the current image frame. If a subsequent adjacent image frame is found, the subsequent adjacent image frame is determined as the new current image frame to be judged, and the operation of judging whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated. When a mismatch is detected, the current image frame is determined not to be a frozen frame, and the cumulative value of frozen frames is checked to see if it is greater than or equal to a preset threshold. When the cumulative value of frozen frames is greater than or equal to the preset threshold, the image frame sequence is determined to meet the target video frame freezing condition. When the cumulative value of frozen frames is not greater than or equal to the preset threshold, the cumulative value of frozen frames is reset, and it is checked whether there is a subsequent adjacent image frame to the current image frame. When a subsequent adjacent image frame is found, the subsequent adjacent image frame is determined as the new current image frame to be judged, and the operation of checking whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated.

[0054] As can be seen, this optional embodiment can also reduce the number of times the cumulative value of the frozen frame is judged when the current image frame is determined not to be a frozen frame, by judging whether the cumulative value of the current frozen frame is greater than or equal to a preset threshold.

[0055] In this optional embodiment, further optionally, before determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame, the method may further include the following operations: After determining the current image frame to be judged, it is checked whether there is a preceding adjacent image frame in the image frame sequence. If a preceding adjacent image frame exists, an operation is triggered to determine whether the checksum of the current image frame to be judged matches the checksum of its preceding adjacent image frame. If no preceding adjacent image frame exists, the following adjacent image frame is determined as the new current image frame to be judged, and an operation is triggered to determine whether the checksum of the current image frame to be judged matches the checksum of its preceding adjacent image frame. Or, After determining the current image frame to be judged, it is determined whether the check value judgment operation of adjacent image frames has been performed before the current time. If it is determined that the check value judgment operation of adjacent image frames has been performed, an operation is triggered to determine whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame of the current image frame. If it is determined that the check value judgment operation of adjacent image frames has not been performed, the following adjacent image frame of the current image frame is determined as the new current image frame to be judged, and an operation is triggered to determine whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame of the current image frame.

[0056] As can be seen, this optional embodiment can also determine whether there is a preceding image frame before determining the check value of two adjacent frames, which can reduce unnecessary judgment operations and thus improve the efficiency of video frame freeze detection.

[0057] In another optional embodiment, the above-mentioned determination of whether the image frame sequence meets the preset video frame freeze conditions may also include: The image frame sequence is input into the pre-trained video frame freeze detection model to obtain the output result of the video frame freeze detection model, and it is determined whether the output result meets the second video frame freeze condition. When it is determined that the output result meets the second video frame freeze condition, it is determined that the image frame sequence meets the target video frame freeze condition.

[0058] As can be seen, this optional embodiment can reduce the complexity of video freeze detection algorithms while ensuring the accuracy of video freeze detection, thereby reducing hardware resource consumption and development costs. Especially in the scenario of video freeze detection for automotive dashboards, it can greatly improve the reliability and security of monitoring.

[0059] In another optional embodiment, the above-mentioned processing of the monitoring signal to obtain the image frame sequence corresponding to the monitoring signal may specifically include: The monitoring signal is subjected to grayscale processing / binarization to obtain a second monitoring signal, which is the monitoring signal obtained by grayscale processing / binarization of the original monitoring signal; and the image frame acquisition processing operation is performed on the second monitoring signal to obtain the image frame sequence corresponding to the second monitoring signal.

[0060] Among them, grayscale processing / binarization processing algorithms include, but are not limited to, bimodal histogram thresholding segmentation method, average grayscale method, differential histogram, Ostu method, Abutaleb method, Bernsen method, etc.

[0061] As can be seen, this optional embodiment further reduces unnecessary information in the image frame sequence by performing image grayscale processing / image binarization processing on the monitoring signal and performing image frame acquisition processing on the processed monitoring signal. This helps to improve the efficiency and accuracy of judging whether the image frame sequence meets the video freeze condition, and further improves the monitoring efficiency and accuracy of video freeze monitoring.

[0062] Example 2 Please see Figure 3 , Figure 3 This is a flowchart illustrating a video frame freeze detection method based on a SoC disclosed in an embodiment of the present invention. Figure 3 The described SoC-based video freeze monitoring method can be applied to a monitoring device used to monitor whether video freeze occurs in an application device with video output functionality. Optionally, the monitoring device can be integrated into the application device, its in-vehicle device, or a smart server or smart platform; this embodiment of the invention does not impose any limitations. Figure 3 As shown, the SoC-based video frame freeze detection method may include the following operations: 201. Collect monitoring signals from the system-on-a-chip in the application device.

[0063] 202. Process the monitoring signal to obtain the image frame sequence corresponding to the monitoring signal.

[0064] 203. Determine whether the image frame sequence meets the preset target video frame freezing conditions. If the image frame sequence meets the target video frame freezing conditions, trigger step 204. If the image frame sequence does not meet the target video frame freezing conditions, the process can end or continue to trigger step 201.

[0065] 204. Confirm that the video feed on the application device is frozen.

[0066] In this embodiment of the invention, for other descriptions of steps 201-204, please refer to the detailed description of steps 101-104 in Embodiment 1. These descriptions will not be repeated in this embodiment of the invention.

[0067] In this embodiment of the invention, after step 204 is completed, step 205 is triggered.

[0068] 205. Generate an interrupt signal.

[0069] 206. Execute the corresponding exception handling operation based on the interrupt signal.

[0070] In this embodiment of the invention, optionally, the execution of a matching exception handling operation based on the interrupt signal may include, but is not limited to: The interrupt signal is transmitted to the video transmission control device of the application device to cause the video transmission control device to stop transmitting video signals; and / or, An alarm message is generated based on the interrupt signal, and an alarm operation is performed based on the alarm message; and / or, The interrupt signal is transmitted to the video signal processing unit of the application device to reset the video signal processing unit.

[0071] As can be seen, the monitoring signal used for video freeze detection in this embodiment of the invention originates from within the system-on-a-chip, enabling video freeze detection to be performed in front of the original video signal displayed on the display device, thereby improving the efficiency of video freeze detection and ensuring the timeliness of monitoring. In addition, after determining that a video freeze has occurred, a corresponding interrupt signal can be generated, and a matching exception handling operation can be performed based on the interrupt signal, which further helps to improve the efficiency of video freeze detection.

[0072] Meanwhile, this invention provides a monitoring mechanism for video frame freeze detection. This invention does not limit how to handle video frame freeze, and therefore proposes a variety of video frame freeze processing methods for users to choose from.

[0073] In an optional embodiment, after determining that the application device has a frozen video frame, the method may further include the following operations: A video frame freeze signal is generated, and a video signal retransmission command is sent to the video transmission control device based on the video frame freeze signal, so as to trigger the video transmission control device to retransmit the video signal to be displayed corresponding to the monitoring signal.

[0074] It is evident that the freezing of the video image in the video signal to be displayed corresponding to the monitoring signal may be related to a temporary software or hardware malfunction at the current moment. Therefore, when the video image of the video signal to be displayed is detected to be frozen, the video signal to be displayed can be resent by the video transmission control device for further monitoring, which is beneficial to improving the accuracy of monitoring.

[0075] Example 3 Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a SoC-based video frame freeze monitoring device disclosed in an embodiment of the present invention. Figure 4 The described SoC-based video freeze monitoring device can be used to monitor whether video freeze occurs in application devices with video output capabilities. Optionally, the monitoring device can be integrated into the application device, the in-vehicle device of the application device, or a smart server or smart platform. This invention does not limit the scope of the invention. Figure 4 As shown, the SoC-based video freeze monitoring device may include: The acquisition module 301 is used to acquire monitoring signals from the system-on-a-chip in the application device.

[0076] The processing module 302 is used to process the monitoring signal to obtain the image frame sequence corresponding to the monitoring signal.

[0077] The judgment module 303 is used to determine whether the image frame sequence meets the preset target video screen freezing conditions. When it is determined that the image frame sequence meets the target video screen freezing conditions, the first determination module 304 is triggered to perform the corresponding operation. When it is determined that the image frame sequence does not meet the target video screen freezing conditions, the current process can be terminated, or the acquisition module 301 can be continuously triggered to perform the corresponding operation.

[0078] The first determining module 304 is used to determine that the application device has a video screen freezing situation when the determining module determines that the image frame sequence meets the target video screen freezing condition.

[0079] The monitoring device used to monitor video frame freezing has an independent power supply, clock and reset mechanism, as well as an IC layout.

[0080] As can be seen, the monitoring device for monitoring video frame freeze in this embodiment of the invention is not coupled or dependent on other units involved in image display, such as GPU and memory, which can enhance the stability of video frame freeze monitoring. The monitoring signal for video frame freeze monitoring originates from inside the system-on-a-chip (SoC), and video frame freeze monitoring can be performed in front of the original video signal displayed on the display device, which improves the efficiency of video frame freeze monitoring and ensures the timeliness of monitoring. In addition, video frame freeze monitoring can be achieved directly by judging whether the image frame sequence obtained after processing the monitoring signal from inside the SoC meets the preset video frame freeze conditions. The implementation method is simple and conducive to improving monitoring efficiency.

[0081] In this embodiment of the invention, optionally, the monitoring signal may include: a first signal output by the display engine inside the system-on-a-chip; or a second signal output by the timing control circuit inside the system-on-a-chip; or a third signal output by a functional component located after the timing control circuit inside the system-on-a-chip and before the display interface of the application device.

[0082] The first signal is obtained by the display engine acquiring a standard video signal and performing a preset first conversion operation on the standard video signal. It is evident that the monitoring signal used for video freeze detection is obtained after conversion by the display engine. This simplifies the difficulty of embedding the monitoring device into the system-on-a-chip, and the monitoring device's acquisition of the monitoring signal is not affected by the timing control circuit, thereby improving the reliability of video freeze detection.

[0083] The aforementioned second signal is obtained by the timing control circuit performing a preset second conversion operation on the first output signal output by the display engine. It is evident that the monitoring signal used for video freeze detection, obtained after conversion by the timing control circuit, is closer to the display signal output by the display interface, thus improving the accuracy of video freeze detection.

[0084] Furthermore, the aforementioned third signal is obtained by performing a preset third conversion operation on the second output signal output by the timing control circuit by the functional component.

[0085] It is evident that converting the video signal into the driving signal required by the display interface, and using the driving signal required by the display interface as the monitoring signal for video frame freeze monitoring, can simulate the real output scene and reduce the processing difficulty of the monitoring device for the monitoring signal. This is beneficial to improving the processing efficiency of the monitoring device for the monitoring signal, and in turn, improving the processing efficiency of the monitoring device for image frame sequences, and improving the accuracy of video frame freeze monitoring.

[0086] Meanwhile, the display interfaces include, but are not limited to, RGB, LVDS, DSI, HDMI, etc. It can be seen that the video freeze monitoring device proposed in this invention can perform monitoring without changing the input signal format. It supports video freeze monitoring signal formats covering all existing video display formats and has the characteristics of strong monitoring signal compatibility and strong versatility in application scenarios.

[0087] In this embodiment of the invention, optionally, such as Figure 5 As shown, the device may further include: The second determining module 305 is used to determine the region of interest for each original image frame in the image frame sequence after the processing module 302 processes the monitoring signal to obtain the image frame sequence corresponding to the monitoring signal.

[0088] The cropping module 306 is used to crop the region of interest of the original image frame from the original image frame, and use it as the target image frame corresponding to the original image frame. The target image frame includes the region of interest of the original image frame.

[0089] The update module 307 is used to update the image frame sequence based on the target image frames corresponding to all the original image frames.

[0090] Specifically, after the update module 307 updates the image frame sequence, the image frame sequence judged by the judgment module 303 is the image frame sequence updated by the update module 307.

[0091] As can be seen, the embodiments of the present invention can also identify the region of interest for each original image frame in the image frame sequence and crop the identified region to form a new image frame sequence. This helps to reduce unnecessary information in the image frame sequence, improves the efficiency and accuracy of judging whether the image frame sequence meets the video freeze condition, and further improves the monitoring efficiency and accuracy of video freeze monitoring.

[0092] In an optional embodiment, the specific implementation of the operation by which the judgment module 303 performs the judgment on whether the image frame sequence meets the preset target video frame freezing conditions may include: Each image frame in the image frame sequence is verified according to a preset verification algorithm to obtain the verification value of each image frame; Based on the check value of each image frame, the target value is calculated, and it is determined whether the target value meets the preset first video frame freezing condition. When it is determined that the target value meets the first video frame freezing condition, the image frame sequence is determined to meet the target video frame freezing condition. The target value is used to represent the cumulative number of consecutively frozen frames.

[0093] Optionally, the preset verification algorithm includes, but is not limited to, one of the following: MD5 algorithm, CRC algorithm, MD2 algorithm, checksum algorithm, SHA series algorithm, etc.

[0094] As can be seen, this optional embodiment can reduce the complexity of the video freeze monitoring algorithm while ensuring the accuracy of video freeze monitoring, thereby reducing hardware resource consumption and development costs. At the same time, the image verification algorithm that the video freeze monitoring device can use covers a variety of mainstream image verification algorithms, which shows that the monitoring device has the characteristic of high compatibility of built-in algorithms. Especially in the scenario of video freeze monitoring of automotive instrument panel, it can greatly improve the reliability and security of monitoring.

[0095] In this optional embodiment, as an optional implementation, the aforementioned judgment module 303 calculates the target value based on the check value of each image frame and determines whether the target value meets the preset first video frame freezing condition. When it is determined that the target value meets the first video frame freezing condition, the specific implementation of the operation of determining that the image frame sequence meets the target video frame freezing condition includes: Determine whether the check value of the current image frame to be judged matches the check value of the preceding image frame; When a match is found, the current image frame is determined to be a frozen frame, and the cumulative value of the frozen frame is updated; it is determined whether the cumulative value of the frozen frame is greater than or equal to a preset threshold; when the cumulative value of the frozen frame is greater than or equal to the preset threshold, the image frame sequence is determined to meet the target video frame freezing condition; when the cumulative value of the frozen frame is not greater than or equal to the preset threshold, it is determined whether there is a subsequent adjacent image frame to the current image frame; when it is determined that there is a subsequent adjacent image frame to the current image frame, the subsequent adjacent image frame to the current image frame is determined as the new current image frame to be judged, and the operation of judging whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated. When a mismatch is detected, the current image frame is determined not to be a frozen frame, and the cumulative value of the frozen frame is reset. It is then determined whether there is a subsequent adjacent image frame to the current image frame. If a preceding adjacent image frame to the current image frame is detected, the subsequent adjacent image frame to the current image frame is determined as the new current image frame to be judged, and the operation of judging whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated.

[0096] As can be seen, this optional embodiment can improve the monitoring accuracy while ensuring the timeliness of monitoring by updating the cumulative value of frozen frames when the current image frame is determined to be a frozen frame and determining whether the cumulative value of frozen frames is greater than or equal to a preset threshold.

[0097] In this optional embodiment, as another optional implementation, the judgment module 303 calculates the target value based on the check value of each image frame and determines whether the target value meets the preset first video frame freezing condition. When it is determined that the target value meets the first video frame freezing condition, the specific implementation of the operation of determining that the image frame sequence meets the target video frame freezing condition may further include: Determine whether the check value of the current image frame to be judged matches the check value of the preceding image frame; When a match is found, the current image frame is determined to be a frozen frame, and the cumulative value of the frozen frame is updated. It is then determined whether there is a subsequent adjacent image frame to the current image frame. If a subsequent adjacent image frame is found, the subsequent adjacent image frame is determined as the new current image frame to be judged, and the operation of judging whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated. When a mismatch is detected, the current image frame is determined not to be a frozen frame, and the cumulative value of frozen frames is checked to see if it is greater than or equal to a preset threshold. When the cumulative value of frozen frames is greater than or equal to the preset threshold, the image frame sequence is determined to meet the target video frame freezing condition. When the cumulative value of frozen frames is not greater than or equal to the preset threshold, the cumulative value of frozen frames is reset, and it is checked whether there is a subsequent adjacent image frame to the current image frame. When a subsequent adjacent image frame is found, the subsequent adjacent image frame is determined as the new current image frame to be judged, and the operation of checking whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated.

[0098] As can be seen, this optional embodiment can also reduce the number of times the cumulative value of the frozen frame is judged when the current image frame is determined not to be a frozen frame, by judging whether the cumulative value of the current frozen frame is greater than or equal to a preset threshold.

[0099] In this optional embodiment, the determination module 303 may further include the following operations: After determining the current image frame to be judged, it is checked whether there is a preceding adjacent image frame in the image frame sequence. If a preceding adjacent image frame exists, an operation is triggered to determine whether the checksum of the current image frame to be judged matches the checksum of its preceding adjacent image frame. If no preceding adjacent image frame exists, the following adjacent image frame is determined as the new current image frame to be judged, and an operation is triggered to determine whether the checksum of the current image frame to be judged matches the checksum of its preceding adjacent image frame. Or, After determining the current image frame to be judged, it is determined whether the check value judgment operation of adjacent image frames has been performed before the current time. If it is determined that the check value judgment operation of adjacent image frames has been performed, an operation is triggered to determine whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame of the current image frame. If it is determined that the check value judgment operation of adjacent image frames has not been performed, the following adjacent image frame of the current image frame is determined as the new current image frame to be judged, and an operation is triggered to determine whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame of the current image frame.

[0100] As can be seen, this optional embodiment can also determine whether there is a preceding image frame before determining the check value of two adjacent frames, which can reduce unnecessary judgment operations and thus improve the efficiency of video frame freeze detection.

[0101] In another optional embodiment, the specific implementation of the operation by which the judgment module 303 performs the judgment on whether the image frame sequence meets the preset target video frame freezing conditions may further include: The image frame sequence is input into the pre-trained video frame freeze detection model to obtain the output result of the video frame freeze detection model, and it is determined whether the output result meets the second video frame freeze condition. When it is determined that the output result meets the second video frame freeze condition, it is determined that the image frame sequence meets the target video frame freeze condition.

[0102] As can be seen, this optional embodiment can reduce the complexity of video freeze detection algorithms while ensuring the accuracy of video freeze detection, thereby reducing hardware resource consumption and development costs. Especially in the scenario of video freeze detection for automotive dashboards, it can greatly improve the reliability and security of monitoring.

[0103] In another optional embodiment, the processing module 302 processes the monitoring signal to obtain an image frame sequence corresponding to the monitoring signal, which may specifically include: The monitoring signal is subjected to grayscale processing / binarization to obtain a second monitoring signal, which is the monitoring signal obtained by grayscale processing / binarization of the original monitoring signal; and the image frame acquisition processing operation is performed on the second monitoring signal to obtain the image frame sequence corresponding to the second monitoring signal.

[0104] Among them, grayscale processing / binarization processing algorithms include, but are not limited to, bimodal histogram thresholding segmentation method, average grayscale method, differential histogram, Ostu method, Abutaleb method, Bernsen method, etc.

[0105] As can be seen, this optional embodiment further reduces unnecessary information in the image frame sequence by performing image grayscale processing / image binarization processing on the monitoring signal and performing image frame acquisition processing on the processed monitoring signal. This helps to improve the efficiency and accuracy of judging whether the image frame sequence meets the video freeze condition, and further improves the monitoring efficiency and accuracy of video freeze monitoring.

[0106] In yet another alternative embodiment, such as Figure 5 As shown, the device may further include: The generation module 308 is used to generate an interrupt signal when the first determining module 304 determines that the application device has a video screen freeze.

[0107] The processing module 302 is also used to perform a matching exception handling operation based on the interrupt signal generated by the generation module 308.

[0108] The specific methods by which the processing module 302 performs the matching exception handling operation based on the interrupt signal generated by the generation module 308 include, but are not limited to: The interrupt signal is transmitted to the video transmission control device of the application device to cause the video transmission control device to stop transmitting video signals; and / or, An alarm message is generated based on the interrupt signal, and an alarm operation is performed based on the alarm message; and / or, The interrupt signal is transmitted to the video signal processing unit of the application device to reset the video signal processing unit.

[0109] As can be seen, the monitoring signal used for video freeze detection originates from within the system-on-a-chip (SoC). It can perform video freeze detection in front of the original video signal displayed on the display device, thereby improving the efficiency of video freeze detection and ensuring the timeliness of monitoring. In addition, it can generate a corresponding interrupt signal after determining that a video freeze has occurred, and perform matching abnormal handling operations based on the interrupt signal, which further helps to improve the efficiency of video freeze detection.

[0110] Meanwhile, this invention provides a monitoring mechanism for video frame freeze detection. This invention does not limit how to handle video frame freeze, and therefore proposes a variety of video frame freeze processing methods for users to choose from.

[0111] In an optional embodiment, after the first determining module 304 performs the operation of determining that the application device has a video screen freeze, the device may further include the following operations: The generation module 308 is also used to generate a video frame freeze signal and send a video signal retransmission command to the video transmission control device according to the video frame freeze signal, so as to trigger the video transmission control device to retransmit the video signal to be displayed corresponding to the monitoring signal.

[0112] It is evident that the freezing of the video image in the video signal to be displayed corresponding to the monitoring signal may be related to a temporary software or hardware malfunction at the current moment. Therefore, when the freezing of the video image in the video signal to be displayed is detected, the video signal to be displayed can be resent by the video transmission control device for further monitoring, which is beneficial to improving the accuracy of monitoring.

[0113] Example 4 Please see Figure 6 , Figure 6 This is a schematic diagram of another SoC-based video frame freeze monitoring device disclosed in an embodiment of the present invention. Figure 6 As shown, the SoC-based video freeze monitoring device may include: Memory 401 storing executable program code; Processor 402 coupled to memory 401; The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the SoC-based video frame freeze monitoring method described in Embodiment 1 or Embodiment 2 of the present invention.

[0114] Example 5 This invention discloses a computer-storable medium storing computer instructions. When these computer instructions are invoked, they are used to execute the steps in the SoC-based video frame freeze monitoring method described in Embodiment 1 or Embodiment 2 of this invention.

[0115] Example 6 This invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps in the method for intelligent control of reusable container circulation described in Embodiment 1 or Embodiment 2.

[0116] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0117] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0118] Finally, it should be noted that the video frame freeze monitoring method and apparatus based on SoC disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video frame freeze detection method based on SoC, characterized in that, The method is applied in a monitoring device, and the monitoring device is used to implement video image freeze monitoring of an application device with video image output function. The method includes: Collect monitoring signals from the system-on-a-chip in the application device; The monitoring signal is processed to obtain the image frame sequence corresponding to the monitoring signal; Each image frame in the image frame sequence is verified according to a preset verification algorithm to obtain a verification value for each image frame; Determine whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame; When a match is found, the current image frame is determined to be a frozen frame, and the cumulative value of the frozen frame is updated. It is then determined whether there is a subsequent adjacent image frame to the current image frame. If a subsequent adjacent image frame is found, the subsequent adjacent image frame is determined as a new current image frame to be judged, and the operation of judging whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated. When a mismatch is detected, the current image frame is determined not to be a frozen frame, and the cumulative value of frozen frames is determined to be greater than or equal to a preset threshold. When the cumulative value of frozen frames is determined to be greater than or equal to the preset threshold, the image frame sequence is determined to meet the target video frame freezing condition. When the cumulative value of frozen frames is determined not to be greater than or equal to the preset threshold, the cumulative value of frozen frames is reset, and it is determined whether there is a subsequent adjacent image frame to the current image frame. When it is determined that there is a subsequent adjacent image frame to the current image frame, the subsequent adjacent image frame to the current image frame is determined as a new current image frame to be judged, and the operation of determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated. When it is determined that the image frame sequence meets the target video frame freeze condition, it is determined that the application device has a video frame freeze.

2. The SoC-based video frame freeze monitoring method according to claim 1, characterized in that, Before determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame, the method further includes: After determining the current image frame to be judged, it is determined whether there is a preceding adjacent image frame in the image frame sequence. If it is determined that there is a preceding adjacent image frame, the operation of determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame is triggered. If it is determined that there is no preceding adjacent image frame, the following adjacent image frame of the current image frame is determined as the new current image frame to be judged, and the operation of determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame is triggered; or, After determining the current image frame to be judged, it is determined whether a checksum judgment operation for adjacent image frames has been performed before the current time. If it is determined that a checksum judgment operation for adjacent image frames has been performed, the operation of judging whether the checksum of the current image frame to be judged matches the checksum of the preceding adjacent image frame of the current image frame is triggered. If it is determined that no checksum judgment operation for adjacent image frames has been performed, the following adjacent image frame of the current image frame is determined as the new current image frame to be judged, and the operation of judging whether the checksum of the current image frame to be judged matches the checksum of the preceding adjacent image frame of the current image frame is triggered.

3. The SoC-based video frame freeze monitoring method according to claim 2, characterized in that, The monitoring signals include: a first signal output by the display engine inside the system-on-a-chip; or a second signal output by the timing control circuit inside the system-on-a-chip; or a third signal output by a functional component located after the timing control circuit inside the system-on-a-chip and before the display interface of the application device. The first signal is obtained by the display engine acquiring a standard video signal and performing a preset first conversion operation on the standard video signal; The second signal is obtained by the timing control circuit performing a preset second conversion operation on the first output signal output by the display engine; The third signal is obtained by the functional component performing a preset third conversion operation on the second output signal output by the timing control circuit.

4. The SoC-based video frame freeze monitoring method according to claim 3, characterized in that, After processing the monitoring signal to obtain the image frame sequence corresponding to the monitoring signal, the method further includes: For each original image frame in the image frame sequence, the region of interest of the original image frame is determined, and the region of interest of the original image frame is cropped out from the original image frame as the target image frame corresponding to the original image frame. The target image frame includes the region of interest of the original image frame. The image frame sequence is updated based on the target image frames corresponding to all the original image frames.

5. The SoC-based video frame freeze monitoring method according to claim 3, characterized in that, The application device includes a video transmission control device and a video signal processing device, and the method further includes: When it is determined that the video screen of the application device is frozen, an interrupt signal is generated, and a matching exception handling operation is performed according to the interrupt signal; The step of performing a matching exception handling operation based on the interrupt signal includes: The interrupt signal is transmitted to the video transmission control device of the application device, so that the video transmission control device stops transmitting the video signal; and / or, An alarm message is generated based on the interrupt signal, and an alarm operation is performed based on the alarm message; and / or, The interrupt signal is transmitted to the video signal processing device of the application device to reset the video signal processing device.

6. A video frame freeze monitoring device based on SoC, characterized in that, The apparatus is used to implement video freeze monitoring of application devices with video output function, the apparatus is used to execute the SoC-based video freeze monitoring method as described in any one of claims 1-5, and the apparatus includes: The acquisition module is used to acquire monitoring signals from inside the system-on-a-chip in the application device. The processing module is used to process the monitoring signal to obtain the image frame sequence corresponding to the monitoring signal; The judgment module is used to determine whether the image frame sequence meets the preset target video frame freezing conditions; The first determining module is used to determine that the application device has a video frame freeze situation when the judging module determines that the image frame sequence meets the target video frame freeze condition; Furthermore, the specific methods by which the judgment module determines whether the image frame sequence meets the preset target video frame freezing conditions include: Each image frame in the image frame sequence is verified according to a preset verification algorithm to obtain a verification value for each image frame; Determine whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame; When a match is found, the current image frame is determined to be a frozen frame, and the cumulative value of the frozen frame is updated. It is then determined whether there is a subsequent adjacent image frame to the current image frame. If a subsequent adjacent image frame is found, the subsequent adjacent image frame is determined as a new current image frame to be judged, and the operation of judging whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated. When a mismatch is detected, the current image frame is determined not to be a frozen frame, and the cumulative value of frozen frames is determined to be greater than or equal to a preset threshold. When the cumulative value of frozen frames is determined to be greater than or equal to the preset threshold, the image frame sequence is determined to meet the target video frame freezing condition. When the cumulative value of frozen frames is determined not to be greater than or equal to the preset threshold, the cumulative value of frozen frames is reset, and it is determined whether there is a subsequent adjacent image frame to the current image frame. When it is determined that there is a subsequent adjacent image frame to the current image frame, the subsequent adjacent image frame to the current image frame is determined as a new current image frame to be judged, and the operation of determining whether the check value of the current image frame to be judged matches the check value of the preceding adjacent image frame to the current image frame is repeated.

7. A video frame freeze monitoring device based on SoC, characterized in that, The device includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the SoC-based video frame freeze monitoring method as described in any one of claims 1-5.

Images