A CMS delay detection method and device based on time watermark

By using time watermarking technology in the image transmission channel, the processing time difference between the image processing unit and the display driving unit is detected, and the problem of remote real-time online measurement of video delay in the prior art is solved, thus achieving efficient, safe and stable image transmission , ensuring driving safety.

CN115134587BActive Publication Date: 2024-11-08SHANGHAI YUXING ELECTRONICS TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202210687347.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-16
Publication Date
2024-11-08
Estimated Expiration
2042-06-16

AI Technical Summary

Technical Problem

The existing video delay testing methods cannot achieve remote real-time online measurement, and the specific location of the delay during the video transmission cannot be determined, resulting in the inability to promptly troubleshoot and deal with faults.

Method used

By using the CMS delay detection method based on time watermark, by marking the time watermark in the image transmission channel, using multiple transmission channels and detection units, the processing time difference between the image processing unit and the display driving unit is detected, abnormalities are judged and the channels are switched, and the image is ensured. Transmission quality.

Benefits of technology

Remote real-time online delay detection is realized, which can promptly identify and repair delay problems in image transmission, prevent driving safety risks, and meet ASILB functional safety requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115134587B_ABST
    Figure CN115134587B_ABST
Patent Text Reader

Abstract

The present invention provides a CMS delay detection device and method based on time watermark. The CMS delay detection device is arranged between an image acquisition device and a display device. A first time detection unit detects and records a first time at an output port of the image acquisition device; a second time detection unit detects and records a second time at a data input port of a first virtual switch; a third time detection unit detects and / or records a third time at a data input port of a first display driving screen unit; a fourth time detection unit detects and / or records a fourth time at an output port of a first image processing unit; and each time detection unit performs delay detection on each data transmission channel to determine whether each component has an abnormality, thereby satisfying the CMS requirement of reaching ASILB for the functional safety key point of delay.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of electronic rearview mirrors, and in particular to a CMS delay detection method and device based on time watermark. Background Art

[0002] With the continuous development of image processing technology, more and more devices are equipped with image acquisition devices to acquire images and realize other functions based on the acquired images. For example, in the field of artificial intelligence, smart devices use acquired images for positioning or navigation, electronic rearview mirrors, etc. Time delay is an important issue in the image transmission process.

[0003] The existing video delay test method mainly uses mobile phones, computers, etc. as reference timers, points the camera of the image transmission end at the timer, and calculates the time difference between the output display result and the reference timer by observing the receiving end, so as to estimate the end-to-end video delay. However, this delay test method requires that the sending and receiving ends must be placed together, and it is impossible to perform remote real-time online measurement. It is also impossible to determine where the delay occurs during video transmission and to troubleshoot it in time. Summary of the invention

[0004] The present invention proposes a CMS delay detection method and device based on time watermark, which can solve video transmission failure and determine the location where the failure occurs.

[0005] The present invention aims to provide a CMS delay detection method of a CMS delay device of a time watermark, characterized in that the CMS delay detection method comprises the following steps:

[0006] Step 1, marking the collected image and superimposing the first time and the second time;

[0007] Step 2, determining whether the first image processing unit and the first display driving unit are operating normally;

[0008] Step 3, performing unit delay detection to determine whether the first image processing unit and the first image driving unit have returned to normal;

[0009] Step 4: Select the optimal processing method according to the unit delay detection result;

[0010] The first time is the time of acquisition at the output of the image acquisition device, and the second time is the time of acquisition at the input of the first virtual switch data;

[0011] Specifically, in step 2, the time difference between the second time and the first time is calculated; if the time difference is less than the first preset time, the first counting unit is reset to zero, and the delay detection of the collected data continues; if the time difference is greater than or equal to the first preset time, the first counting unit counts;

[0012] When the first counting unit value reaches the threshold, the image is processed through the first backup transmission channel; when the first counting unit value does not reach the threshold, delay detection continues to be performed on the collected data.

[0013] Furthermore, in step 3, the following steps are also included:

[0014] Step 311, analyzing whether the first image processing unit is abnormal;

[0015] The control unit controls the collected image to be marked and superimposed, the first time detection unit detects and records the current first time, copies the image with the superimposed mark, enters the second image processing unit, and the control unit instructs the third time detection unit to detect the image with the superimposed mark and record the third time;

[0016] The control unit determines whether the difference between the third time and the first time is greater than or equal to a third preset time;

[0017] If the difference between the third time and the first time is greater than or equal to the third preset time, the second counting unit counts, and the third counting unit is reset. When the second counting unit does not reach the threshold, the unit delay detection continues. When the second counting unit reaches the threshold, the second counting unit is reset, the first image processing unit is abnormal, and the first image processing unit is restarted. The fifth counting unit counts to see whether the fifth unit count value reaches the threshold. If not, the unit delay detection continues. If so, the delay detection of the first image processing unit is abnormal.

[0018] If the difference between the third time and the first time is less than the third preset time, the third counting unit counts and the second counting unit is reset. When the third counting unit does not reach the threshold, the unit delay detection continues. When the third counting unit reaches the threshold, the third counting unit is reset and the delay detection of the first image processing unit is normal.

[0019] Furthermore, in step 3, the following steps are also included:

[0020] Step 321, analyzing whether the first display driving unit is abnormal;

[0021] The control unit instructs the third time detection unit to detect the image with the superimposed mark and record the third time, and the control unit instructs the second time detection unit to detect the image with the superimposed mark and record the second time.

[0022] The control unit determines whether the difference between the second time and the third time is greater than or equal to a fourth preset time;

[0023] If the control unit calculates that the difference between the second time and the third time is greater than or equal to the fourth preset time, the first counting unit counts, and the fourth counting unit is reset. When the first counting unit does not reach the threshold, the unit delay detection continues. When the first counting unit reaches the threshold, the first counting unit is reset, the first display driving unit is abnormal, and the first display driving unit is restarted. The sixth counting unit counts to see whether the sixth unit count value reaches the threshold. If not, the unit delay detection continues. If it reaches, the delay detection of the first display driving unit is abnormal.

[0024] If the control unit calculates that the difference between the second time and the third time is less than the fourth preset time, the fourth counting unit starts counting and the first counting unit is reset. When the fourth counting unit does not reach the threshold, the unit delay detection continues. When the fourth counting unit reaches the threshold, the fourth counting unit is reset and the delay detection of the first display driving unit is normal.

[0025] Furthermore, in step 3, the following steps are also included:

[0026] Step 312, another method analyzes whether the first image processing unit is abnormal;

[0027] The control unit controls the OSD to be superimposed on the collected image, the first time detection unit detects and records the current first time, and the control unit controls the fourth time detection unit to detect the image with the superimposed mark and record the fourth time;

[0028] The control unit determines whether the difference between the fourth time and the first time is greater than or equal to the third preset time;

[0029] The control unit calculates that if the difference between the fourth time and the first time is greater than or equal to the third preset time, the second counting unit counts, and the third counting unit is reset. When the second counting unit does not reach the threshold, the unit delay detection continues; when the second counting unit reaches the threshold, the second counting unit is reset, the first image processing unit is abnormal, the first image processing unit is restarted, the fifth counting unit counts, and whether the fifth unit count value reaches the threshold. If not, the unit delay detection continues. If it reaches, the delay detection of the first image processing unit is abnormal.

[0030] If the control unit calculates that the difference between the fourth time and the first time is less than the third preset time, the third counting unit starts counting and the second counting unit is reset. When the third counting unit does not reach the threshold, the unit delay detection continues. When the third counting unit reaches the threshold, the third counting unit is reset and the delay detection of the first image processing unit is normal.

[0031] A CMS delay detection device based on time watermark is also provided, the CMS delay detection device comprising a first image processing unit, a first display driving unit and a detection unit;

[0032] The first image processing unit is connected to the image acquisition device and is used to process the image output by the image acquisition device; the input end of the first display driving unit is connected to the first image processing unit, and the output end is connected to the display device, and is used to process the image in a format to adapt to the display device;

[0033] The detection unit includes a first time detection unit, a second time detection unit, and a third time detection unit;

[0034] The first time detection unit detects and records a first time at the output end of the image acquisition device;

[0035] A second time detection unit detects and records a second time at a data output terminal of the first display driving unit;

[0036] The third time detection unit detects and / or records the third time at the data input terminal of the first display driving screen unit;

[0037] The CMS delay detection device calculates the processing time of the first image processing unit and the first display driving unit for processing a single frame image through the first time, the second time and the third time, and performs CMS delay detection based on the processing time.

[0038] Furthermore, the CMS delay detection device also includes a control unit, a counting unit and a storage unit;

[0039] The control unit is used for time delay detection and unit delay detection;

[0040] The storage unit is used to store a first preset time, a second preset time, a third preset time, a fourth preset time and a threshold;

[0041] The counting unit comprises a first counting unit, a second counting unit, a third counting unit, a fourth counting unit, a fifth counting unit and a sixth counting unit;

[0042] The first counting unit is used to count the abnormalities of the first display driving unit;

[0043] The second counting unit is used to count the abnormalities of the first image processing unit;

[0044] The third counting unit is used to count the first image processing unit normally;

[0045] The fourth counting unit is used to count the first display driving unit normally;

[0046] The fifth counting unit is used to count the number of restarts of the first image processing unit;

[0047] The sixth counting unit is used to count the restarts of the first display driving unit.

[0048] Furthermore, the CMS delay detection device includes a second image processing unit, a second display driving unit and a virtual switch unit;

[0049] The first image processing unit and the first display driving unit are connected to form an original transmission channel; the second image processing unit and the second display driving unit are connected to form a first spare transmission channel; the first image processing unit and the second display driving unit are connected to form a second spare transmission channel; the second image processing unit and the first display driving unit are connected to form a third spare transmission channel;

[0050] The virtual switch unit includes a first virtual switch and a second virtual switch; the first virtual switch is connected to the first display driving unit; the second virtual switch is connected to the second display driving unit;

[0051] The control unit controls the OSD overlay of the collected image, the first time detection unit detects and records the current first time, the control unit controls the second time detection unit to detect the image with the overlay mark and records the second time, after the first time exceeds the first preset time, the second time detection unit does not detect the image with the overlay mark, the first counting unit counts, when the first counting unit value does not reach the threshold, the delay detection of the collected data continues, when the first counting unit value reaches the threshold, the first counting unit is cleared, the data collected by the image acquisition device is processed through the first backup transmission channel, the first virtual switch is turned off, the second virtual switch is turned on, and an early warning is issued.

[0052] Furthermore, the control unit analyzes whether the first image processing unit is abnormal;

[0053] The control unit controls the collected image to be marked and superimposed, the first time detection unit detects and records the current first time, copies the image with the superimposed mark, enters the second image processing unit, and the control unit instructs the third time detection unit to detect the image with the superimposed mark and record the third time;

[0054] The control unit determines whether the difference between the third time and the first time is greater than or equal to a third preset time;

[0055] If the difference between the third time and the first time is greater than or equal to the third preset time, the second counting unit counts, and the third counting unit is reset. When the second counting unit does not reach the threshold, the unit delay detection continues. When the second counting unit reaches the threshold, the second counting unit is reset, the first image processing unit is abnormal, and the first image processing unit is restarted. The fifth counting unit counts to see whether the fifth unit count value reaches the threshold. If not, the unit delay detection continues. If so, the delay detection of the first image processing unit is abnormal.

[0056] If the difference between the third time and the first time is less than the third preset time, the third counting unit counts and the second counting unit is reset. When the third counting unit does not reach the threshold, the unit delay detection continues. When the third counting unit reaches the threshold, the third counting unit is reset and the delay detection of the first image processing unit is normal.

[0057] Furthermore, the control unit analyzes whether the first display driving unit is abnormal;

[0058] The control unit instructs the third time detection unit to detect the image with the superimposed mark and record the third time, and the control unit instructs the second time detection unit to detect the image with the superimposed mark and record the second time.

[0059] The control unit determines whether the difference between the second time and the third time is greater than or equal to a fourth preset time;

[0060] If the control unit calculates that the difference between the second time and the third time is greater than or equal to the fourth preset time, the first counting unit counts, and the fourth counting unit is reset. When the first counting unit does not reach the threshold, the unit delay detection continues. When the first counting unit reaches the threshold, the first counting unit is reset, the first display driving unit is abnormal, and the first display driving unit is restarted. The sixth counting unit counts to see whether the sixth unit count value reaches the threshold. If not, the unit delay detection continues. If it reaches, the delay detection of the first display driving unit is abnormal.

[0061] If the control unit calculates that the difference between the second time and the third time is less than the fourth preset time, the fourth counting unit starts counting and the first counting unit is reset. When the fourth counting unit does not reach the threshold, the unit delay detection continues. When the fourth counting unit reaches the threshold, the fourth counting unit is reset and the delay detection of the first display driving unit is normal.

[0062] Furthermore, the detection unit includes a fourth time detection unit, and the fourth time detection unit detects and / or records a fourth time obtained at an output port of the first image processing unit;

[0063] The control unit controls the OSD overlay on the captured image, the first time detection unit detects and records the current first time, and the control unit controls the fourth time detection unit to detect the image with the overlay mark and record the fourth time;

[0064] The control unit determines whether the difference between the fourth time and the first time is greater than or equal to the third preset time;

[0065] The control unit calculates that if the difference between the fourth time and the first time is greater than or equal to the third preset time, the second counting unit counts, and the third counting unit is reset. When the second counting unit does not reach the threshold, the unit delay detection continues; when the second counting unit reaches the threshold, the second counting unit is reset, the first image processing unit is abnormal, the first image processing unit is restarted, the fifth counting unit counts, and whether the fifth unit count value reaches the threshold. If not, the unit delay detection continues. If it reaches, the delay detection of the first image processing unit is abnormal.

[0066] If the control unit calculates that the difference between the fourth time and the first time is less than the third preset time, the third counting unit starts counting and the second counting unit is reset. When the third counting unit does not reach the threshold, the unit delay detection continues. When the third counting unit reaches the threshold, the third counting unit is reset and the delay detection of the first image processing unit is normal.

[0067] The beneficial effects achieved by the present invention are:

[0068] The present invention proposes a CMS delay detection device based on time watermark, which has multiple channels and detection units, and checks the specific unit that causes the delay and restarts and repairs it and enables a replacement channel, so that even if the data transmission encounters an obstacle, the display can still display normally and remind the user, thereby solving the driving hazards caused by display delays or frozen screens, and protecting the user's driving safety.

[0069] The delay detection method of the present invention is easy to control and can meet the requirements of CMS for achieving ASILB for the functional safety key point of delay.

[0070] In the present invention, the time detection unit is respectively arranged at each node of the original image transmission channel. By comparing the signal transmission time between each node with the preset standard time, it is determined whether there is an abnormality in the original image transmission channel. If an abnormality is found, the channel can be switched immediately to ensure the image transmission quality.

[0071] The present invention includes one original transmission channel and three backup transmission channels for image transmission. The four transmission channels share two sets of main image transmission units. Through the delay detection of each node, it is inferred whether each main image transmission unit can work normally, so as to select the optimal transmission channel. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 It is a structural schematic diagram of a CMS delay detection device based on time watermark provided by an embodiment of the present invention;

[0073] Figure 2 It is a flow chart of a CMS delay detection method based on time watermark provided by an embodiment of the present invention;

[0074] Figure 3 It is a flow chart of the first part of a unit test detection algorithm in a CMS delay detection method based on a time watermark provided by an embodiment of the present invention;

[0075] Figure 4 It is a flow chart of the second part of a unit test detection algorithm in a CMS delay detection method based on a time watermark provided by an embodiment of the present invention;

[0076] Figure 5 It is a flow chart of the third part of a unit test detection algorithm in a CMS delay detection method based on time watermark provided in an embodiment of the present invention.

[0077] Description of the drawings: 100-image acquisition device, 101-first image processing unit, 102-first display driving unit, 103-second image processing unit, 104-second display driving unit, 105-first virtual switch, 106-second virtual switch, 200-display device, 111-first time detection unit, 112-second time detection unit, 113-third time detection unit, 114-fourth time detection unit. DETAILED DESCRIPTION

[0078] The technical solution of the present invention is described in more detail below in conjunction with the accompanying drawings. The present invention includes but is not limited to the following embodiments.

[0079] As attached Figure 1 As shown, the present invention provides a CMS delay detection device based on time watermark, and the CMS delay detection device includes a first image processing unit 101, a first display driving unit 102, a second image processing unit 103, a second display driving unit 104, a detection unit, a superimposed marking unit, a control unit, a virtual switch unit, a control unit, a storage unit, an early warning unit and a counting unit;

[0080] The first image processing unit 101 is a unit for processing the data output by the image acquisition device 100, for example, performing data processing functions such as AEC (automatic exposure control), AGC (automatic gain control), AWB (automatic white balance), black level correction, color correction, defective pixel correction, lens correction, Gamma correction, bad pixel removal, AI, image stitching, distortion correction, and automatic exposure control.

[0081] The second image processing unit 103 includes basic display functions such as color correction.

[0082] The first display driving unit 102 is a unit that performs format processing on the final display effect on the display, such as scaling, brightness, contrast, etc.

[0083] The second display driving unit 104 includes basic functions such as zooming.

[0084] The CMS delay detection device transmits, processes and delays data through four channels; namely: original transmission channel: first image processing unit 101, first display driver unit 102; first backup transmission channel: second image processing unit 103, second display driver unit 104; second backup transmission channel: first image processing unit 101, second display driver unit 104; third backup transmission channel: second image processing unit 103, first display driver unit 102.

[0085] The superimposed marking unit is used to superimpose marks on the image captured by the image capture device 100, and send a signal to notify the control unit to identify the superimposed mark, and the control unit sends a signal to notify the detection unit to detect the image of the superimposed mark and / or record the time.

[0086] The detection unit includes a first time detection unit 111 , a second time detection unit 112 , a third time detection unit 113 and a fourth time detection unit 114 .

[0087] The first time detection unit 111 is located at the output port of the image acquisition device 100 and detects and records the first time.

[0088] The second time detection unit 112 is located at the data input port of the first virtual switch 105, or is set at the data output port of the first display driving unit 102; detects and / or records the second time.

[0089] The third time detection unit 113 is located at the data input port of the first display driving screen unit, and detects and / or records the third time.

[0090] The fourth time detection unit 114 is located at the output port of the first image processing unit 101 and detects and / or records the fourth time.

[0091] The virtual switch unit includes a first virtual switch 105 and a second virtual switch 106 ; the first virtual switch 105 receives a signal transmitted by the first display driving unit 102 ; and the second virtual switch 106 receives a signal transmitted by the second display driving unit 104 .

[0092] The control unit is used for delay detection and unit delay detection.

[0093] The storage unit is used to store a first preset time A, where A can take a value of [100, 150], for example, 100ms, 110ms, 130ms, 150ms; a second preset time t, where t can take a value of [50ms, 100ms], for example, 50ms, 53ms, 70ms, 97ms, 100ms; a third preset time x, where x can take a value of [40ms, 80ms], for example, 41ms, 47ms, 55ms, 70ms, 79ms; a fourth preset time y, where y can take a value of [10ms, 20ms], for example, 11ms, 15ms, 17ms, 19ms; a threshold value B, where B can take a value of [1, 8], for example, 1, 3, 5, 8.

[0094] The warning unit is used to display on the display device 200, sound warning, vibration warning, etc. In addition to warning, it also has the function of prompting.

[0095] The counting units include a first counting unit, a second counting unit, a third counting unit, a fourth counting unit, a fifth counting unit and a sixth counting unit.

[0096] The first counting unit is used to count the abnormalities of the original transmission channel, or count the abnormalities of the first display driving unit 102; the abnormality is for example: the time from T1 exceeds Ams, and the second time detection unit 112 does not detect the image of the superimposed mark; the control unit calculates that the difference ΔT between the second time T2 and the first time T1 is greater than or equal to the preset time t; the control unit calculates that the difference Y between the second time T2 and the third time T3 is greater than or equal to the preset value y;

[0097] A second counting unit: used for counting the abnormalities of the first image processing unit 101 and the second image processing unit 103, for example, the control unit calculates that the difference X between the third time T3 and the first time T1 is greater than or equal to a preset value x;

[0098] The third counting unit: counts the first image processing unit 101 and the second image processing unit 103 normally, for example: the control unit calculates that the difference X between the third time T3 and the first time T1 is less than the preset value x;

[0099] The fourth counting unit: counts the first display driving unit 102 normally, for example: the control unit calculates that the difference Y between the second time T2 and the third time T3 is less than the preset value y;

[0100] A fifth counting unit: counting the number of restarts of the first image processing unit 101;

[0101] The sixth counting unit counts the number of restarts of the first display driving unit 102 .

[0102] The working principle of the CMS delay detection device provided by the present invention is described below through the following embodiments.

[0103] Embodiment 1:

[0104] The data collected by the image acquisition device 100 is processed through the original transmission channel, and is displayed on the display device 200 through the first virtual switch 105. The original transmission channel is: the first image processing unit 101 processes the data output by the image acquisition device 100, for example: performing data processing functions such as AEC (automatic exposure control), AGC (automatic gain control), AWB (automatic white balance), black level correction, color correction, defective pixel correction, lens correction, Gamma correction, bad pixel removal, AI, image stitching, distortion correction, automatic exposure control, etc.; the first display driving unit 102 performs format processing such as scaling, brightness, contrast, etc. on the data output by the first image processing unit 101 to adapt to the display device 200.

[0105] Embodiment 2:

[0106] When the vehicle starts the detection mode, the data collected by the image acquisition device 100 is processed through the original transmission channel, the control unit controls the OSD superposition of the collected image, the first time detection unit 111 detects and records the current first time T1, the control unit instructs the second time detection unit 112 to detect the image with the superimposed mark and record the second time T2, after more than Ams, for example, 150ms from the first time T1, the second time detection unit 112 does not detect the image with the superimposed mark, the first counting unit counts, when the first counting unit value does not reach the threshold value B, for example, 3, the delay detection of the collected data continues, when the first counting unit value reaches the threshold value B, for example, 3, the first counting unit is cleared, the data collected by the image acquisition device 100 is processed through the first backup transmission channel, the first virtual switch 105 is closed, the second virtual switch 106 is opened, the picture is displayed on the display, the unit delay detection is performed, and an early warning is issued, for example: "The original transmission channel is abnormal, the first backup transmission channel has been opened, and the unit delay detection begins."

[0107] Data processing is performed through the first backup transmission channel, that is, the second image processing unit 103 performs functional processing such as color correction on the data output by the image acquisition device 100; the second display driving unit 104 performs functional processing such as scaling on the data output by the second image processing unit 103.

[0108] If the delay detection of the first image processing unit 101 and the delay detection of the first display driving unit 102 are both normal, the data collected by the image acquisition device 100 is processed through the original transmission channel, and the second virtual switch 106 is closed, and the first virtual switch 105 is turned on, and displayed on the display device 200, and a prompt is given, for example: the original transmission channel has been restored, please feel free to use it!

[0109] If the time delay detection of the first image processing unit 101 is normal, and the time delay detection of the first display driving unit 102 is abnormal or has been in detection, the data collected by the image acquisition device 100 is processed through the first backup transmission channel or the second backup transmission channel, and is displayed on the display device 200 through the second virtual switch 106, and an early warning is issued, for example: the first display driving unit 102 is abnormal, the first backup transmission channel has been opened, please go to the relevant agency for maintenance! Or the first display driving unit 102 is abnormal, the second backup transmission channel has been opened, please go to the relevant agency for maintenance!

[0110] If the delay detection of the first display driver unit 102 is normal, and the delay detection of the first image processing unit 101 is abnormal or has been in detection, the data collected by the image acquisition device 100 is processed through the first backup transmission channel, and displayed on the display device 200 through the second virtual switch 106, and an early warning is issued, for example: the first display driver unit 102 is abnormal, the first backup transmission channel has been opened, please go to the relevant agency for maintenance! Or the data is processed through the third backup transmission channel, and displayed on the display device 200 through the first virtual switch 105, and an early warning is issued, for example: the first display driver unit 102 is abnormal, the third backup transmission channel has been opened, please go to the relevant agency for maintenance!

[0111] If the delay detection of the first image processing unit 101 is abnormal or is always in detection, and the delay detection of the first display driving unit 102 is abnormal or is always in detection, the data collected by the image acquisition device 100 is maintained through the first backup transmission channel for data processing, and is displayed on the display device 200 through the second virtual switch 106, and an early warning is issued, for example: the first display driving unit 102 is abnormal, the first backup transmission channel has been opened, please go to the relevant agency for maintenance!

[0112] Embodiment three:

[0113] When the vehicle starts the detection mode, the data collected by the image acquisition device 100 is processed through the original transmission channel, the control unit controls the collected image to be marked and superimposed, such as OSD superimposition, the first time detection unit 111 detects and records the current first time T1, the control unit instructs the second time detection unit 112 to detect the image with the superimposed mark and record the second time T2, the second time detection unit 112 detects the image with the superimposed mark within Ams, such as 150ms from the first time T1, the second time detection unit 112 records the time when the image with the superimposed mark is detected as the second time T2, and the control unit calculates the difference ΔT between the second time T2 and the first time T1,

[0114] If ΔT<t, the first counting unit is reset to zero, and the delay detection of the collected data continues;

[0115] If ΔT≥t, the first counting unit counts. When the value of the first counting unit reaches a threshold value B, such as 3, the first counting unit is reset, and the data collected by the image acquisition device 100 is processed through the first backup transmission channel. The first virtual switch 105 is closed, the second virtual switch 106 is opened, and the picture is displayed on the display. The unit delay detection is performed, and an early warning is issued, for example: "The original transmission channel is abnormal, the first backup transmission channel has been opened, and the unit delay detection begins."

[0116] Data processing is performed through the first backup transmission channel, that is, the second image processing unit 103 performs functional processing such as color correction on the data output by the image acquisition device 100; the second display driving unit 104 performs functional processing such as scaling on the data output by the second image processing unit 103.

[0117] Embodiment 4:

[0118] The unit delay detection mode is turned on, and the delay detection is performed on the first image processing unit 101 and the first display driving unit 102 respectively. The control unit controls the marking and superposition of the collected image, such as OSD superposition. The first time detection unit 111 detects and records the current first time T1, copies the image with the superimposed mark, and enters the first image processing unit 101 and the second image processing unit 103 respectively. The control unit instructs the third time detection unit 113 to detect the image with the superimposed mark and record the third time T3.

[0119] The control unit calculates the difference X between the third time T3 and the first time T1, that is, X=T3-T1, and determines whether X is greater than a preset value x, for example, 80ms;

[0120] If X is greater than or equal to x, the second counting unit counts and the third counting unit is reset;

[0121] When the second counting unit does not reach the threshold value B, for example, 8, the unit delay detection continues.

[0122] When the second counting unit reaches the threshold value B, for example, 8, the second counting unit is cleared, the first image processing unit 101 is abnormal, the first image processing unit 101 is restarted, the fifth counting unit counts, and the fifth unit count value is checked to see whether it reaches the threshold value B, for example, 2. If not, the unit delay detection is continued. If it is reached, the delay detection of the first image processing unit 101 or the second image processing unit 103 is abnormal.

[0123] If X is less than x, the third counting unit counts and the second counting unit is reset.

[0124] When the third counting unit does not reach the threshold value B, for example, 8, the unit delay detection continues.

[0125] When the third counting unit reaches the threshold value B, for example, 8, the third counting unit is reset, and the delay detection of the first image processing unit 101 is normal.

[0126] Embodiment five:

[0127] The control unit instructs the third time detection unit 113 to detect the image with the superimposed mark and record the third time T3, and the control unit instructs the second time detection unit 112 to detect the image with the superimposed mark and record the second time T2.

[0128] The control unit calculates the difference Y between the second time T2 and the third time T3, that is, Y=T2-T3, and determines whether Y is greater than a preset value y, for example, 20ms;

[0129] If Y is greater than or equal to y, the first counting unit counts and the fourth counting unit is reset.

[0130] When the first counting unit does not reach the threshold value B, for example, 8, the unit delay detection continues.

[0131] When the first counting unit reaches the threshold value B, for example, 8, the first counting unit is cleared, the first display driving unit 102 is abnormal, the first display driving unit 102 is restarted, the sixth counting unit counts, and the sixth unit count value is checked to see whether it reaches the threshold value B, for example, 2. If not, the unit delay detection is continued. If it reaches, the delay detection of the first display driving unit 102 is abnormal.

[0132] If Y is less than y, the fourth counting unit starts counting and the first counting unit is reset.

[0133] When the fourth counting unit does not reach the threshold value B, for example, 8, the unit delay detection continues.

[0134] When the fourth counting unit reaches the threshold value B, for example, 8, the fourth counting unit is reset, and the delay detection of the first display driving unit 102 is normal.

[0135] Embodiment six:

[0136] The unit delay detection mode is turned on, and the delay detection is performed on the first image processing unit 101 and the first display driving unit 102 respectively. The control unit controls the marking and superposition of the collected image, such as OSD superposition. The first time detection unit 111 detects and records the current first time T1, copies the image with the superimposed mark, and enters the first image processing unit 101 and the second image processing unit 103 respectively. The control unit instructs the fourth time detection unit 114 to detect the image with the superimposed mark and record the fourth time T4.

[0137] The control unit calculates the difference Z between the fourth time T4 and the first time T1, that is, Z=T4-T1, and determines whether Z is greater than a preset value x, for example, 80ms;

[0138] If Z is greater than or equal to x, the second counting unit counts and the third counting unit is reset;

[0139] When the second counting unit does not reach the threshold value B, for example, 8, the unit delay detection continues.

[0140] When the second counting unit reaches the threshold value B, for example, 8, the second counting unit is cleared, the first image processing unit 101 is abnormal, the first image processing unit 101 is restarted, the fifth counting unit counts, and the fifth unit count value is checked to see whether it reaches the threshold value B, for example, 2. If not, the unit delay detection is continued. If it reaches, the delay detection of the first image processing unit 101 is abnormal.

[0141] If Z is less than x, the third counting unit counts and the second counting unit is reset.

[0142] When the third counting unit does not reach the threshold value B, for example, 8, the unit delay detection continues.

[0143] When the third counting unit reaches the threshold value B, for example, 8, the third counting unit is reset, and the delay detection of the first image processing unit 101 is normal.

[0144] The present invention also provides a CMS delay detection method based on time watermark, which is detected by a CMS delay detection device and includes the following steps:

[0145] Step 1, marking the collected image and superimposing the first time and the second time;

[0146] The control unit controls the superimposed marking unit to mark and superimpose the collected image, the first time detection unit 111 detects and records the current first time T1, and the control unit instructs the second time detection unit 112 to detect the image with the superimposed mark at the data input port of the first virtual switch 105 and record the second time T2.

[0147] Step 2, determining whether the first image processing unit 101 and the first display driving unit 102 are working normally;

[0148] Specifically, when the time from T1 exceeds Ams, the second time detection unit 112 detects the image of the superimposed mark. If the image of the superimposed mark is detected, the second time detection unit 112 records the time when the image of the superimposed mark is detected as the current second time T2; if the image of the superimposed mark is not detected, the first counting unit counts.

[0149] The control unit calculates the difference ΔT between the second time T2 and the first time T1, and compares ΔT with the preset time t; if ΔT<t, the first counting unit is reset to zero and continues to perform delay detection on the collected data; if ΔT≥t, the first counting unit counts.

[0150] When the first counting unit value does not reach the threshold, the delay detection of the collected data continues; when the first counting unit value reaches the threshold, the first counting unit is reset, the data collected by the image acquisition device 100 is processed through the first backup transmission channel, the first virtual switch 105 is closed, the second virtual switch 106 is opened, the picture is displayed on the display, the unit delay detection is performed, and an early warning is issued.

[0151] Step 3, performing unit delay detection to determine whether the first image processing unit 101 and the first image driving unit have returned to normal;

[0152] Specifically, the unit delay detection includes the following steps:

[0153] Step 311, analyzing whether the first image processing unit 101 is abnormal;

[0154] The control unit controls the marking and superposition of the collected image, the first time detection unit 111 detects and records the current first time T1, copies the image with the superimposed mark, and enters the first image processing unit 101 and the second image processing unit 103 respectively, and the control unit instructs the fourth time detection unit 114 to detect the image with the superimposed mark and record the fourth time T4.

[0155] The control unit calculates the difference Z between the fourth time T4 and the first time T1, that is, Z=T4-T1, and determines whether Z is greater than a preset value x;

[0156] If Z is greater than or equal to x, the second counting unit counts and the third counting unit is reset;

[0157] When the second counting unit does not reach the threshold value B, the unit delay detection continues.

[0158] When the second counting unit reaches the threshold value B, the second counting unit is cleared, the first image processing unit 101 is abnormal, the first image processing unit 101 is restarted, the fifth counting unit counts, and the fifth unit count value is checked to see if it reaches the threshold value B. If not, the unit delay detection is continued. If so, the delay detection of the first image processing unit 101 is abnormal.

[0159] If Z is less than x, the third counting unit counts and the second counting unit is reset.

[0160] When the third counting unit does not reach the threshold value B, the unit delay detection continues.

[0161] When the third counting unit reaches the threshold value B, the third counting unit is reset, and the delay detection of the first image processing unit 101 is normal.

[0162] Step 321, analyzing whether the first display driving unit 102 is abnormal;

[0163] The control unit instructs the third time detection unit 113 to detect the image with the superimposed mark and record the third time T3, and the control unit instructs the second time detection unit 112 to detect the image with the superimposed mark and record the second time T2.

[0164] The control unit calculates the difference Y between the second time T2 and the third time T3, that is, Y=T2-T3, and determines whether Y is greater than a preset value y;

[0165] If Y is greater than or equal to y, the first counting unit counts and the fourth counting unit is reset.

[0166] When the first counting unit does not reach the threshold value B, the unit delay detection continues.

[0167] When the first counting unit reaches the threshold value B, the first counting unit is cleared, the first display driving unit 102 is abnormal, the first display driving unit 102 is restarted, the sixth counting unit counts, and the sixth unit count value is checked to see if it reaches the threshold value B. If not, the unit delay detection is continued. If it reaches, the delay detection of the first display driving unit 102 is abnormal.

[0168] If Y is less than y, the fourth counting unit starts counting and the first counting unit is reset.

[0169] When the fourth counting unit does not reach the threshold value B, the unit delay detection continues.

[0170] When the fourth counting unit reaches the threshold value B, the fourth counting unit is reset, and the delay detection of the first display driving unit 102 is normal.

[0171] Step 312, another method analyzes whether the first image processing unit 101 is abnormal;

[0172] In another embodiment, the control unit controls the marking and superposition of the captured image, the first time detection unit 111 detects and records the current first time T1, copies the image with the superimposed mark, and enters the first image processing unit 101 and the second image processing unit 103 respectively, and the control unit instructs the third time detection unit 113 to detect the image with the superimposed mark and record the third time T3.

[0173] The control unit calculates the difference X between the third time T3 and the first time T1, that is, X=T3-T1, and determines whether X is greater than a preset value x, for example, 80ms;

[0174] If X is greater than or equal to x, the second counting unit counts and the third counting unit is reset;

[0175] When the second counting unit does not reach the threshold value B, the unit delay detection continues.

[0176] When the second counting unit reaches the threshold value B, the second counting unit is cleared, the first image processing unit 101 is abnormal, the first image processing unit 101 is restarted, the fifth counting unit counts, and the fifth unit count value is checked to see if it reaches the threshold value B. If not, the unit delay detection is continued. If so, the delay detection of the first image processing unit 101 is abnormal.

[0177] If X is less than x, the third counting unit counts and the second counting unit is reset.

[0178] When the third counting unit does not reach the threshold value B, the unit delay detection continues.

[0179] When the third counting unit reaches the threshold value B, the third counting unit is reset, and the delay detection of the first image processing unit 101 is normal.

[0180] Step 4: Select the optimal processing method based on the unit delay detection result.

[0181] If the first image processing unit 101 and the first image driving unit are restored to normal, the data collected by the image acquisition device 100 is processed through the original transmission channel, and the second virtual switch 106 is closed. The first virtual switch 105 is opened and closed, and displayed on the display device 200.

[0182] If the first image processing unit 101 returns to normal, but the first image driving unit does not return to normal, the data collected by the image acquisition device 100 is processed through the first backup transmission channel or the second backup transmission channel, and is displayed on the display device 200 through the second virtual switch 106, and an early warning is issued.

[0183] If the first image processing unit 101 does not return to normal, the first image driving unit returns to normal, and the data collected by the image acquisition device 100 is processed through the first backup transmission channel, and displayed on the display device 200 and warned through the second virtual switch 106; or processed through the third backup transmission channel, and displayed on the display device 200 and warned through the first virtual switch 105.

[0184] If the first image processing unit 101 and the first image driving unit are not restored to normal, the data collected by the image acquisition device 100 is processed through the first backup transmission channel and displayed on the display device 200 through the second virtual switch 106 and an early warning is issued.

[0185] The present invention is not limited to the above-mentioned specific implementation modes. A person skilled in the art may implement the present invention in various other specific implementation modes according to the embodiments and the disclosure of the drawings. Therefore, any design that adopts the design structure and concept of the present invention and makes some simple transformations or changes shall fall within the scope of protection of the present invention.

Claims

1. A CMS delay detection method for a CMS delay device based on a time watermark, characterized in that: The CMS delay detection method comprises the following steps: Step 1, marking the collected image and superimposing the first time and the second time; Step 2, determining whether the first image processing unit (101) and the first display driving unit (102) are operating normally; Step 3, performing unit delay detection to determine whether the first image processing unit (101) and the first image driving unit have returned to normal; Step 4: Select the optimal processing method according to the unit delay detection result; The first time is the time of acquisition at the output of the image acquisition device (100), and the second time is the time of acquisition at the data input of the first virtual switch (105); Specifically, in step 2, the time difference between the second time and the first time is calculated; if the time difference is less than the first preset time, the first counting unit is reset to zero, and the delay detection of the collected data continues; if the time difference is greater than or equal to the first preset time, the first counting unit counts; When the first counting unit value reaches the threshold, the image is processed through the first backup transmission channel; when the first counting unit value does not reach the threshold, delay detection continues to be performed on the collected data.

2. The CMS delay detection method according to claim 1, characterized in that: In step 3, the following steps are also included: Step 311, analyzing whether the first image processing unit (101) is abnormal; The control unit controls the marking and superimposition of the collected image, the first time detection unit (111) detects and records the current first time, copies the image with the superimposed mark, enters the second image processing unit (103), and the control unit instructs the third time detection unit (113) to detect the image with the superimposed mark and record the third time; The control unit determines whether the difference between the third time and the first time is greater than or equal to a third preset time; If the difference between the third time and the first time is greater than or equal to the third preset time, the second counting unit counts, the third counting unit is reset, when the second counting unit does not reach the threshold, the unit delay detection continues, when the second counting unit reaches the threshold, the second counting unit is reset, the first image processing unit (101) is abnormal, the first image processing unit (101) is restarted, the fifth counting unit counts, whether the fifth unit count value reaches the threshold, if not, the unit delay detection continues, if reached, the first image processing unit (101) delay detection is abnormal; If the difference between the third time and the first time is less than the third preset time, the third counting unit starts counting and the second counting unit is reset. When the third counting unit does not reach the threshold, the unit delay detection continues. When the third counting unit reaches the threshold, the third counting unit is reset and the delay detection of the first image processing unit (101) is normal.

3. The CMS delay detection method according to claim 1, characterized in that: In step 3, the following steps are also included: Step 321, analyzing whether the first display driving unit (102) is abnormal; The control unit instructs the third time detection unit (113) to detect the image with the superimposed mark and record the third time, and the control unit instructs the second time detection unit (112) to detect the image with the superimposed mark and record the second time; The control unit determines whether the difference between the second time and the third time is greater than or equal to a fourth preset time; If the control unit calculates that the difference between the second time and the third time is greater than or equal to a fourth preset time, the first counting unit counts, the fourth counting unit is reset, when the first counting unit does not reach the threshold, the unit delay detection continues, when the first counting unit reaches the threshold, the first counting unit is reset, the first display drive unit (102) is abnormal, the first display drive unit (102) is restarted, the sixth counting unit counts, whether the sixth unit count value reaches the threshold, if not, the unit delay detection continues, if reached, the first display drive unit (102) delay detection is abnormal; If the difference between the second time and the third time calculated by the control unit is less than a fourth preset time, the fourth counting unit starts counting and the first counting unit is reset. When the fourth counting unit does not reach a threshold, the unit delay detection continues. When the fourth counting unit reaches a threshold, the fourth counting unit is reset and the delay detection of the first display driving unit (102) is normal.

4. The CMS delay detection method according to claim 1, characterized in that: In step 3, the following steps are also included: Step 312, analyzing whether the first image processing unit (101) is abnormal; The control unit controls the OSD to be superimposed on the collected image, the first time detection unit (111) detects and records the current first time, and the control unit controls the fourth time detection unit (114) to detect the image with the superimposed mark and record the fourth time; The control unit determines whether the difference between the fourth time and the first time is greater than or equal to the third preset time; The control unit calculates that if the difference between the fourth time and the first time is greater than or equal to the third preset time, the second counting unit counts, the third counting unit is reset, and when the second counting unit does not reach a threshold, the unit delay detection continues; when the second counting unit reaches a threshold, the second counting unit is reset, the first image processing unit (101) is abnormal, the first image processing unit (101) is restarted, the fifth counting unit counts, whether the fifth unit count value reaches the threshold, if not, the unit delay detection continues, if reached, the first image processing unit (101) delay detection is abnormal; If the control unit calculates that the difference between the fourth time and the first time is less than the third preset time, the third counting unit starts counting and the second counting unit is reset. When the third counting unit does not reach the threshold, the unit delay detection continues. When the third counting unit reaches the threshold, the third counting unit is reset and the delay detection of the first image processing unit (101) is normal.

5. A CMS delay detection device based on the CMS delay detection method according to any one of claims 1 to 4, characterized in that: The CMS delay detection device comprises a first image processing unit (101), a first display driving unit (102) and a detection unit; The first image processing unit (101) is connected to the image acquisition device (100) and is used to process the image output by the image acquisition device (100); the first display driving unit (102) has an input end connected to the first image processing unit (101) and an output end connected to the display device (200) and is used to process the image in a format suitable for the display device (200); The detection unit comprises a first time detection unit (111), a second time detection unit (112), and a third time detection unit (113); The first time detection unit (111) detects and records a first time at an output end of the image acquisition device (100); The second time detection unit (112) detects and records a second time at a data output end of the first display driving unit (102); The third time detection unit (113) detects and / or records the third time at the data input terminal of the first display driving unit (102); The CMS delay detection device calculates the processing time of the first image processing unit (101) and the first display driving unit (102) for processing a single frame image respectively through the first time, the second time and the third time, and performs CMS delay detection based on the processing time.

6. The CMS delay detection device according to claim 5, characterized in that: The CMS delay detection device also includes a control unit, a counting unit and a storage unit; The control unit is used for time delay detection and unit delay detection; The storage unit is used to store a first preset time, a second preset time, a third preset time, a fourth preset time and a threshold; The counting unit comprises a first counting unit, a second counting unit, a third counting unit, a fourth counting unit, a fifth counting unit and a sixth counting unit; The first counting unit is used to count the abnormalities of the first display driving unit (102); The second counting unit is used to count the abnormalities of the first image processing unit (101); The third counting unit is used to normally count the first image processing unit (101); The fourth counting unit is used to normally count the first display driving unit (102); The fifth counting unit is used to count the number of restarts of the first image processing unit (101); The sixth counting unit is used to count the number of restarts of the first display driving unit (102).

7. The CMS delay detection device according to claim 6, characterized in that: The CMS delay detection device comprises a second image processing unit (103), a second display driving unit (104) and a virtual switch unit; The first image processing unit (101) and the first display driver unit (102) are connected to form an original transmission channel; the second image processing unit (103) and the second display driver unit (104) are connected to form a first standby transmission channel; the first image processing unit (101) and the second display driver unit (104) are connected to form a second standby transmission channel; and the second image processing unit (103) and the first display driver unit (102) are connected to form a third standby transmission channel; The virtual switch unit comprises a first virtual switch (105) and a second virtual switch (106); the first virtual switch (105) is connected to the first display drive unit (102); the second virtual switch (106) is connected to the second display drive unit (104); The control unit controls the OSD superposition of the collected image, the first time detection unit (111) detects and records the current first time, the control unit controls the second time detection unit (112) to detect the image with the superposition mark and record the second time, after the first time exceeds the first preset time, the second time detection unit (112) does not detect the image with the superposition mark, the first counting unit counts, when the first counting unit value does not reach the threshold value, the time delay detection of the collected data continues, when the first counting unit value reaches the threshold value, the first counting unit is reset, the data collected by the image collection device (100) is processed through the first standby transmission channel, the first virtual switch (105) is closed, the second virtual switch (106) is opened, and an early warning is issued.

8. The CMS delay detection device according to claim 7, characterized in that: The control unit analyzes whether the first image processing unit (101) is abnormal; The control unit controls the marking and superimposing of the collected image, the first time detection unit (111) detects and records the current first time, copies the image with the superimposed mark, enters the second image processing unit (103), and the control unit instructs the third time detection unit (113) to detect the image with the superimposed mark and record the third time; The control unit determines whether the difference between the third time and the first time is greater than or equal to a third preset time; If the difference between the third time and the first time is greater than or equal to the third preset time, the second counting unit counts, the third counting unit is reset, when the second counting unit does not reach the threshold, the unit delay detection continues, when the second counting unit reaches the threshold, the second counting unit is reset, the first image processing unit (101) is abnormal, the first image processing unit (101) is restarted, the fifth counting unit counts, whether the fifth unit count value reaches the threshold, if not, the unit delay detection continues, if reached, the first image processing unit (101) delay detection is abnormal; If the difference between the third time and the first time is less than the third preset time, the third counting unit starts counting and the second counting unit is reset. When the third counting unit does not reach the threshold, the unit delay detection continues. When the third counting unit reaches the threshold, the third counting unit is reset and the delay detection of the first image processing unit (101) is normal.

9. The CMS delay detection device according to claim 7, characterized in that: The control unit analyzes whether the first display driving unit (102) is abnormal; The control unit instructs the third time detection unit (113) to detect the image with the superimposed mark and record the third time, and the control unit instructs the second time detection unit (112) to detect the image with the superimposed mark and record the second time; The control unit determines whether the difference between the second time and the third time is greater than or equal to a fourth preset time; If the control unit calculates that the difference between the second time and the third time is greater than or equal to a fourth preset time, the first counting unit counts, the fourth counting unit is reset, when the first counting unit does not reach the threshold, the unit delay detection continues, when the first counting unit reaches the threshold, the first counting unit is reset, the first display drive unit (102) is abnormal, the first display drive unit (102) is restarted, the sixth counting unit counts, whether the sixth unit count value reaches the threshold, if not, the unit delay detection continues, if reached, the first display drive unit (102) delay detection is abnormal; If the difference between the second time and the third time calculated by the control unit is less than a fourth preset time, the fourth counting unit starts counting and the first counting unit is reset. When the fourth counting unit does not reach a threshold, the unit delay detection continues. When the fourth counting unit reaches a threshold, the fourth counting unit is reset and the delay detection of the first display driving unit (102) is normal.

10. The CMS delay detection device according to claim 7, characterized in that: The detection unit comprises a fourth time detection unit (114), wherein the fourth time detection unit (114) detects and / or records a fourth time acquired at an output port of the first image processing unit (101); The control unit controls the OSD superposition of the collected image, the first time detection unit (111) detects and records the current first time, and the control unit controls the fourth time detection unit (114) to detect the image with the superposition mark and record the fourth time; The control unit determines whether the difference between the fourth time and the first time is greater than or equal to the third preset time; The control unit calculates that if the difference between the fourth time and the first time is greater than or equal to the third preset time, the second counting unit counts, the third counting unit is reset, and when the second counting unit does not reach a threshold, the unit delay detection continues; when the second counting unit reaches a threshold, the second counting unit is reset, the first image processing unit (101) is abnormal, the first image processing unit (101) is restarted, the fifth counting unit counts, whether the fifth unit count value reaches the threshold, if not, the unit delay detection continues, if reached, the first image processing unit (101) delay detection is abnormal; If the control unit calculates that the difference between the fourth time and the first time is less than the third preset time, the third counting unit starts counting and the second counting unit is reset. When the third counting unit does not reach the threshold, the unit delay detection continues. When the third counting unit reaches the threshold, the third counting unit is reset and the delay detection of the first image processing unit (101) is normal.

Citation Information

Patent Citations

  • Fault detection system and method for video shooting equipment and video shooting equipment

    CN109842796A

  • Mouse delay detection method and device

    CN114120195A

  • System and method for the identification of motional media of widely varying picture content

    US20070242880A1