Video stream signal processing method and device

Abstract

The invention provides a video stream signal processing method and device, and particularly relates to the technical field of image signal processing.The method comprises the steps that in the process of receiving a video data stream (which can be understood as a camera serial interface data stream), signal integrity monitoring is conducted on the video data stream, and a monitoring result is obtained; under the condition that the monitoring result shows that the target signal is missing, complementing the target signal based on the compensation signal; wherein the video data stream comprises a plurality of key signals, and the missing of the key signals can cause the subsequent processing of the video data stream to be abnormal; the target signal is one of the plurality of key signals. By guaranteeing the data integrity of the key signal, the transmission integrity of the video data stream can be improved, and then the risk that the video data stream is abnormal in the subsequent application stage is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image signal processing, and particularly to a method and device for signal processing of a video stream. Background Art

[0002] Operations such as switching of video image signal sources may cause loss of some signals (such as frame end, line start, line end, etc.) in the data stream of the camera serial interface, thereby affecting the subsequent image processing process. Seriously, it may even cause the subsequent image processing or display program to go awry. Summary of the Invention

[0003] The purpose of the present invention is to provide a method and device for signal processing of a video stream, which are used to solve the technical problem of poor transmission integrity of the image signal transmission scheme provided by the prior art.

[0004] In a first aspect, an embodiment of the present invention provides a method for signal processing of a video stream, and the method includes:

[0005] During the process of receiving a video data stream, perform signal integrity monitoring on the video data stream to obtain a monitoring result;

[0006] In the case where the monitoring result indicates that a target signal is missing in the video data stream, based on a preset compensation signal, complement the missing target signal;

[0007] Wherein, the video data stream includes multiple key signals, and the absence of the key signals will cause abnormalities in subsequent processing of the video data stream;

[0008] The target signal is one of the multiple key signals.

[0009] In a second aspect, an embodiment of the present invention further provides a device for signal processing of a video stream, and the device includes:

[0010] A state machine, which is used to perform signal integrity monitoring on the video data stream during the process of receiving the video data stream to obtain a monitoring result;

[0011] The state machine is further used for,

[0012] In the case where the monitoring result indicates that a target signal is missing in the video data stream, based on a preset compensation signal, complement the missing target signal;

[0013] Wherein, the video data stream includes multiple key signals, each key signal includes multiple data signals and multiple flag signals, and the absence of the key signals will cause abnormalities in subsequent processing of the video data stream;

[0014] The target key signal is one of the multiple key signals.

[0015] In a third aspect, the present invention provides an electronic device, including a processor, a memory, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the steps of the method described in the first aspect are implemented.

[0016] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0017] In a fifth aspect, the present invention provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0018] In the present invention, for the key signals whose loss can cause anomalies in the subsequent processing of the video data stream, integrity monitoring is performed on them. So that when they are missing, the missing key data is complemented based on a preset compensation signal, thereby ensuring the data integrity of the key signals during the transmission of the video data stream, that is, improving the transmission reliability and transmission integrity of the video data stream, and further reducing the risk of anomalies in the subsequent application stage of the video data stream. Description of the Drawings

[0019] Figure 1 is a flowchart of a signal processing method for a video stream provided by an embodiment of the present invention;

[0020] Figure 2 is a schematic structural diagram of a video stream transmission system provided by an embodiment of the present invention;

[0021] Figure 3 is a schematic structural diagram of a MIPI_CSI signal loss detection and compensation module provided by an embodiment of the present invention;

[0022] Figure 4 is a schematic diagram of waveforms of some key signals of a camera serial interface data stream provided by an embodiment of the present invention;

[0023] Figure 5 is a schematic diagram of a state machine provided by an embodiment of the present invention;

[0024] Figure 6 is a flowchart of loading a sampling signal in a sampling state provided by an embodiment of the present invention;

[0025] Figure 7 is a schematic diagram of a frame end signal detection mechanism in a locked state provided by an embodiment of the present invention;

[0026] Figure 8 is a schematic diagram of a downlink end signal detection mechanism in a locked state provided by an embodiment of the present invention;

[0027] Figure 9 is a schematic diagram of a line start signal detection mechanism for the first line in a locked state provided by an embodiment of the present invention;

[0028] Figure 10 is a schematic diagram of a line start signal detection mechanism for non - first lines in a locked state provided by an embodiment of the present invention;

[0029] Figure 11 is a schematic diagram of a pixel data signal detection mechanism in a locked state provided by an embodiment of the present invention;

[0030] Figure 12 is a schematic diagram of the structure of a signal processing device for a frequency stream provided by an embodiment of the present invention;

[0031] Figure 13 is a schematic diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0032] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0033] The embodiments of the present invention provide a method for processing signals of a video stream. Refer to Figure 1 , Figure 1 is a schematic flowchart of a method for processing signals of a video stream provided by an embodiment of the present invention. As shown in Figure 1 , it includes the following steps:

[0034] Step 101: During the process of receiving a video data stream, perform signal integrity monitoring on the video data stream to obtain a monitoring result.

[0035] Among them, the above - mentioned video data stream is a data stream corresponding to video images of a specific service / scenario / system. For example: monitoring service (the video data stream can be understood as the data stream of the monitoring video transmitted from a monitoring camera to a backend server), vehicle system (the video data stream can be understood as the data stream of the panoramic image transmitted from a vehicle camera to a vehicle processor), robot, artificial intelligence system, etc.

[0036] The transmission protocol used during the transmission of the above video data stream is any transmission protocol / specification that supports video transmission, such as: Camera Serial Interface (CSI) protocol / specification.

[0037] Among them, signal integrity monitoring can be understood as: the process of monitoring whether the signal data of key signals (such as data signals and flag signals) in the video data stream are successfully received within a set time period. The specific definition of the key signals is described below and will not be elaborated here for the time being.

[0038] The above monitoring result is used to indicate whether the signal data of the key signals in the video data stream are successfully received within a set time period.

[0039] The aforementioned set time period can be determined according to the signal appearance interval of the key signal. The signal appearance interval of the key signal is used to represent: in the video data stream, the interval between the expected reception times of two adjacent data signals of the key signal. It should be noted that the signal appearance interval of the key signal can be determined based on the transmission protocol used by the video data stream, or can be obtained by analyzing the signal reception times of multiple data signals of the key signal that have been received.

[0040] Step 102, in the case where the monitoring result indicates that a target signal is missing in the video data stream, based on a preset compensation signal, complete the missing target signal.

[0041] Among them, the video data stream includes multiple key signals, and the absence of the key signals will cause abnormalities in subsequent processing of the video data stream;

[0042] The target signal is one of the multiple key signals.

[0043] Among them, the multiple key signals include one or more of an end-of-frame signal, a line start signal, a line end signal, a pixel data type signal, a pixel data valid signal, and a pixel data signal.

[0044] It should be understood that the end-of-frame signal, the line start signal, the line end signal, the pixel data type signal, the pixel data valid signal, and the pixel data signal are different key signals respectively.

[0045] In some embodiments, the method of the present application can also be applied to the transmission scenario of multiple groups of video streams (such as 2 groups, 3 groups, 4 groups, etc.). In this scenario, multiple groups of video streams are input, and at least one group of video streams among the multiple groups of video streams is output / displayed. Each group of video streams in the multiple groups of video streams corresponds to a virtual channel ID, and the virtual channel IDs corresponding to different groups of video streams are different. The aforementioned key signal can also include a virtual channel ID (vcid) signal indicating the virtual channel ID corresponding to the video stream. The video data stream can be understood as one group of video streams among the multiple groups of video streams. For example, the aforementioned multiple groups of video streams can represent multiple reverse images collected by multiple on-vehicle cameras when the vehicle is reversing. In this embodiment, the multiple key signals can also include a virtual channel ID signal.

[0046] In this embodiment, corresponding enable signals can be set for the frame_end signal, line_start signal, line_end signal, pix_data_type signal, and pix_data_valid signal. When the enable signal is enabled, the loss detection and complement processing of the corresponding key signal are performed. If the enable signal is not enabled, the loss detection and complement processing of the corresponding key signal are not performed.

[0047] It should be noted that the pix_data_type signal and the pix_data_valid signal are controlled by the same enable signal to determine whether to be enabled, while the pix_data signal and the virtual channel ID (vcid) signal are defaulted to perform loss detection and complement processing.

[0048] Among them, the situation where the target signal is missing in the video data stream can be understood as the situation where the target signal is not successfully received within its corresponding time period.

[0049] It should be understood that the above compensation signal can be adaptively selected according to the value range corresponding to the target signal. In this embodiment, when the target signal is a pixel data signal, the compensation signal used to compensate the target signal is set to 0.

[0050] The complement operation in the present application is specifically: using the compensation signal as the missing target signal in the video data stream.

[0051] The method of the present application is applied to a video image processing system. The video image processing system includes a signal loss detection and compensation module. The data stream input to the signal loss detection and compensation module can be understood as the aforementioned video data stream.

[0052] In the present invention, for a key signal whose loss can cause anomalies in the video data stream during processing, integrity monitoring is performed on it so that when it is missing, the missing key signal is complemented based on a preset compensation signal, thereby ensuring the data integrity of the key signal during the transmission of the video data stream, that is, improving the transmission reliability of the video data stream, and further reducing the risk of anomalies in the video data stream during subsequent application phases.

[0053] In this application, it can be set to default to perform the above-mentioned signal integrity monitoring and subsequent missing data complementation operations during the process of receiving the video data stream. It can also be set to determine whether to perform the above-mentioned signal integrity monitoring and subsequent missing data complementation operations based on the user's selection during the process of receiving the video data stream. It can also be set to determine that when certain conditions are met, the above-mentioned signal integrity monitoring and subsequent missing data complementation operations are performed during the process of receiving the video data stream.

[0054] In one embodiment, the signal integrity monitoring of the video data stream to obtain a monitoring result includes:

[0055] When it is determined that the video data stream has a risk of signal loss, signal integrity monitoring is performed on the video data stream to obtain a monitoring result.

[0056] Based on the above settings, the above-mentioned signal integrity monitoring and subsequent missing data complementation operations are selectively performed to reduce the overall energy consumption while ensuring the transmission reliability of the video data stream.

[0057] Furthermore, before the signal integrity monitoring of the video data stream to obtain a monitoring result, the method further includes:

[0058] When the video data stream is switched from a first data source to a second data source, it is determined that the video data stream has a risk of signal loss, where the first data source and the second data source are different data sources.

[0059] Specifically, in a vehicle-mounted system, when a user switches the display content of a vehicle-mounted display from one vehicle-mounted camera to another vehicle-mounted camera, it can be regarded as the situation where the video data stream is switched from a first data source to a second data source. Here, the video data stream corresponds to the display content of the vehicle-mounted display. The certain vehicle-mounted camera can be understood as the first data source, and the other vehicle-mounted camera can be understood as the second data source. Based on the foregoing data signal loss monitoring and complementation measures, problems such as vehicle-mounted system freezing and failures caused by data signal loss during the switching of vehicle-mounted cameras can be avoided, enabling the user to timely and accurately observe the conditions around the vehicle through the vehicle-mounted camera to ensure the user's usage experience and safety during vehicle use.

[0060] In one embodiment,

[0061] The process of receiving the video data stream includes a sampling stage and a locking stage. Monitoring the signal integrity of the video data stream to obtain a monitoring result includes:

[0062] In the sampling stage, based on multiple signal sampling times corresponding to the target signal, generate a signal period of the target signal, where the signal period of the target signal is used to represent: the time interval between two consecutive target signals;

[0063] In the locking stage, determine a safe time period for the target signal based on the signal period of the target signal, and the safe time period for the target signal is a time period for waiting to receive the target signal;

[0064] In the case that the target signal is not successfully received within the safe time period of the target signal, generate a monitoring result indicating that the target signal is missing in the video data stream;

[0065] In the case that the target signal is successfully received within the safe time period of the target signal, generate a monitoring result indicating that the video data stream includes the target signal;

[0066] Wherein, the sampling stage is a time stage for determining the signal period of the target signal, the locking stage is a time stage for detecting whether the target signal is successfully received, and the sampling stage is before the locking stage.

[0067] Wherein, the signal sampling time can be understood as: the time / moment when a target signal is successfully received.

[0068] In this embodiment, based on multiple signal sampling times corresponding to the target signal, to generate a more accurate signal period of the target signal, so that the determined safe time period is more accurate, avoiding the abnormal execution of the missing monitoring of the target signal due to improper setting of the safe time period, and making the missing detection of the target signal more accurate and effective.

[0069] In one embodiment,

[0070] In the sampling stage, based on multiple signal sampling times corresponding to the target signal, generating the signal period of the target signal includes:

[0071] When it is detected that there is a risk of signal loss in the video data stream, enter the sampling stage;

[0072] In the sampling stage, when the i-th target signal is detected, the counter value is determined as the i-th candidate value, and the counter is reset to the initial value and then continues to count, where i is an integer greater than 1, and the counter starts counting from the initial value when the first target signal is detected;

[0073] When i = 2, the i-th candidate value is determined as the i-th loading value, and the cumulative value corresponding to the i-th target signal is set to 1;

[0074] When i is greater than 2 and the absolute value of the difference between the i-th candidate value and the (i - 1)-th loading value is less than or equal to the error threshold, the average value of the i-th candidate value and the (i - 1)-th loading value is determined as the i-th loading value, and the cumulative value corresponding to the (i - 1)-th target signal is incremented by 1 and used as the cumulative value corresponding to the i-th data signal of the target key signal;

[0075] When i is greater than 2 and the absolute value of the difference between the i-th candidate value and the (i - 1)-th loading value is greater than the error threshold, the i-th candidate value is determined as the i-th loading value, and the cumulative value corresponding to the i-th target signal is set to 1;

[0076] When the cumulative value corresponding to the i-th target signal satisfies the set termination condition, the target value corresponding to the i-th target signal is determined as the signal period of the target signal, and the set termination condition includes: the cumulative value corresponding to the i-th target signal is greater than or equal to the cumulative threshold;

[0077] Wherein, in the case of satisfying the set switching condition, the sampling stage is switched to the locking stage, and the set switching condition includes: the signal period of the target signal is determined.

[0078] It should be understood that in the case where there are multiple key signals that need to be monitored for missing and filled in the video data stream, the set switching condition may be: for each key signal among the multiple key signals that need to be monitored for missing and filled, the signal period of each key signal has been determined.

[0079] In this embodiment, based on the repeated counting of the counter, combined with the setting of the error threshold and the cumulative threshold, the multiple counting differences of the target signal within the error range in the sampling stage are aggregated into the signal occurrence interval of the target signal, so as to ensure the accuracy of the calculated signal period of the target signal.

[0080] In one embodiment,

[0081] The locking stage is also used to fill in the missing target signals;

[0082] When the monitoring result indicates that the target signal is missing in the video data stream, based on a preset compensation signal, the missing target signal is complemented, including:

[0083] In the locking stage, when the target signal is not successfully received within the safe time period of the target signal, the compensation signal is determined as the target signal, and the reception of the target signal is stopped within the compensation time;

[0084] Among them, the start time of the safe time period of the target signal is: the time when the frame start signal of the target frame is detected, or the time when the line start signal of the target line is detected;

[0085] The start time of the compensation time is: the time when the compensation signal is determined as the target signal;

[0086] The target frame is one image frame corresponding to the target signal among multiple image frames of the video data stream; the target line is one line corresponding to the target signal within multiple lines included in the target frame. It should be noted that stopping the reception of the target signal within the compensation time can also be understood as: shielding the target signal within the compensation time. After the compensation time ends, the reception of the target key signal is immediately resumed, that is, the shielding of the target signal is lifted.

[0087] In one embodiment,

[0088] In the sampling stage, the time interval between the frame start signal and the frame end signal of the first frame is greater than or equal to the first threshold, and / or, the time interval between the line start signal and the line end signal of the first line is greater than or equal to the second threshold;

[0089] Among them, the first frame is one image frame corresponding to the target signal among multiple image frames of the video data stream; the first line is one line corresponding to the target signal within multiple lines included in the first frame;

[0090] When the target signal is the frame end signal, the duration of the compensation time is equal to the first threshold;

[0091] When the target signal is the line end signal, the duration of the compensation time is equal to the second threshold.

[0092] Based on the above settings, when the target signal is an end-of-frame signal or an end-of-line signal, by setting the duration of the compensation time to the first threshold or the second threshold, in the case where the first threshold and the second threshold are set improperly, it is possible to avoid entering the locking phase from the sampling phase, thereby facilitating relevant developers to timely discover the improper setting problems of the first threshold and / or the second threshold through the sampling phase with an overly long duration, and further avoiding the problem of improper compensation time caused by the improper setting of the first threshold and the second threshold, and ensuring the reliable execution of the key signal compensation and the corresponding key signal masking operation.

[0093] In one embodiment, after, in the case where the monitoring result indicates that a target signal is missing in the video data stream, based on a preset compensation signal, the missing target signal is complemented to obtain a target signal stream, the method further includes

[0094] In the locking phase, when the number of executions of the complementing operation is greater than or equal to a preset number threshold and an end-of-frame signal is detected, the locking phase is exited.

[0095] In this embodiment, based on the above settings, a perfect locking phase exit mechanism is established to ensure the complete reception of a frame of video, and further ensure the reliable transmission of the video data stream.

[0096] In application, it is preferably set that the number threshold is equal to the total number of signals of multiple key signals participating in the missing monitoring, so as to ensure the stable exit of the locking phase through the setting of a relatively small number threshold.

[0097] For easy understanding, the example is illustrated as follows:

[0098] This application also provides a video stream transmission system, as Figure 2 shown, the video stream transmission system includes a selector, a MIPI_CSI signal loss detection and compensation module, and an image processing or display module, and the camera serial interface data stream (MIPI_CSI) is transmitted among the three.

[0099] Among them, the selector is used to: receive the data streams of n video sources (n is a positive integer and less than or equal to 4) and transmit them to the MIPI_CSI signal loss detection and compensation module. For example, when the video stream transmission system is applied to a vehicle-mounted system, the n video sources can be understood as n vehicle-mounted cameras in the vehicle-mounted system.

[0100] The MIPI_CSI signal loss detection and compensation module is used to: perform setting operations on n video sources, and transmit the video stream processed by the setting operations to the image processing or display module for further processing. Among them, the setting operations include: detecting signal loss, and in the case of detecting loss, complementing the lost data signal / flag signal according to the pre-set default data / default value.

[0101] The image processing or display module is used to: perform a series of post-processing operations on the video stream, such as encoding and decoding, noise reduction, image quality compensation, target detection, image fusion, image display, etc.

[0102] The structure of the MIPI_CSI signal loss detection and compensation module can be as Figure 3 shown, and it can support the transmission of the camera serial interface data stream (MIPI_CSI) of multiple video sources, and can support the detection and compensation of signals for the data streams of up to four different video sources at most. Therefore, a virtual channel ID (vcid) signal is added as an identifier for distinguishing multiple parallel data streams. Each data stream is marked according to its assigned virtual channel ID (vcid) signal, so as to ensure that the data of different streams will not be confused with each other. Signals such as frame_start, frame_end, line_start, and line_end need to be aligned with the vcid signal to facilitate differentiation during the interleaved transmission of multiple groups of images.

[0103] Specifically, in the MIPI_CSI signal loss detection and compensation module, the vcid discrimination component is used to detect the vcid signal of the data stream, and accordingly distribute the data stream to the detection and compensation component of the corresponding vcid for the aforementioned setting processing. After the setting processing, the data stream passes through the data discrimination output module and is transmitted as an output to the image processing or display module. Among them, the register configuration module stores the relevant configuration parameters for performing the aforementioned setting processing.

[0104] The waveforms of some key signals of the camera serial interface data stream (MIPI_CSI) are as Figure 4 shown, and the register configuration module stores multiple enable signals, and can perform detection and compensation operations on any one or more of the following signals: frame end (frame_end), line start (line_start), line end (line_end), pixel data type (pix_data_type), pixel data valid (pix_data_valid), pixel data (pix_data), virtual channel ID (vcid) signal.

[0105] The register configuration module configures the corresponding enable signals to control whether the above signals are put into the detection compensation index respectively. For example: when the frame end detection enable signal (the first enable signal) configured in the register is valid, the frame_end signal will be put into the detection compensation index; similarly, the line start detection enable signal (the second enable signal) controls whether the line_start signal is detected and compensated; the line end detection enable signal (the third enable signal) controls whether the line_end signal is detected and compensated; the pixel data detection enable signal (the fourth enable signal) controls whether the pix_data_valid and pix_data signals are detected and compensated; there is no enable signal for the pix_data_type signal and the vcid signal, and these two signals are always put into the detection compensation index. After the above indexes are configured, finally, the total detection enable signal (the fifth enable signal) is turned on to determine the range of the signals to be detected and compensated.

[0106] In practical applications, the above enable signals can be flexibly configured to be enabled or not. This example takes the case where all the above enable signals are enabled for illustration.

[0107] In an ideal situation, after the video image parameters are configured, its field / frame period (such as Figure 4 shown as T1 in, the time interval between two consecutive frame_start signals with the same vcid value), line period (such as Figure 4 shown as T3 in, the time interval between two consecutive line_start signals with the same vcid value within the same frame) are fixed. In addition, the time interval between the start of the first line and the frame_start signal (such as Figure 4 shown as T2 in, the time interval between the frame_start signal and the line_start signal of the first line within the same frame with the same vcid value) is also fixed.

[0108] Set a state machine as shown in Figure 5 in the MIPI_CSI signal loss detection and compensation module. When the rising edge of the total detection enable signal of the register configuration module is detected (which can be manually triggered by the user or automatically triggered when a video source switch is detected), the state machine enters the sampling state from the idle state.

[0109] In the sampling state, when the sampling signal is at the time interval of the field / frame period (such as Figure 4 shown as T1 in) and line period (such as Figure 4The time interval shown in T3 in the middle), the time interval between the line start (line_start) signal of the first row and the frame start (frame_start) signal (such as Figure 4 The T2 shown in it) are sampled continuously a times (configured by the register, which can be understood as the aforementioned cumulative threshold) within the error.

[0110] At the same time, it is required that in the sampling state, before the next line start (line_start) signal appears after the line end (line_end) signal, and the time difference between the two signals is greater than the threshold configured by the register (the threshold needs to be greater than 1 clock cycle, and the default value of this example register is set to 8’h1f); before the next frame start (frame_start) signal appears after the frame end (frame_end) signal, and the time difference between the two signals is greater than the threshold configured by the register (the threshold needs to be greater than 1 clock cycle, and the default value of this example register is set to 8’h3f).

[0111] Then the sampling completion signal is triggered. After the frame end (frame_end) signal arrives, the state machine enters the locked state from the sampling state.

[0112] In the locked state, detect the frame end (frame_end), line start (line_start), line end (line_end), pixel data valid (pix_data_valid), and pixel data (pix_data) signals. If the above signals do not arrive within the safe time (that is, the aforementioned safe time period), a compensation operation is performed on the signal. When the cumulative number of compensation signals reaches a certain value (the cumulative error count configured by the register, and the default value of this example register is set to 8’h6, that is, 6 cumulative errors, which is the aforementioned number threshold), after the frame end (frame_end) signal arrives, the state machine jumps out of the locked state and returns to the idle state.

[0113] Specifically, in the locked state, each time a signal error compensation occurs, the cumulative error value is incremented by one. An error counter is built-in. Each time an error compensation occurs for the frame end (frame_end) signal, line end (line_end) signal, line start (line_start) signal, or data valid (pix_data_valid) signal, the corresponding error counter will increment by one. When the cumulative value of the error counter is greater than or equal to the error threshold configured by the register (the configuration needs to be greater than 1 time, and the default value of this example register is set to 8’h06, that is, the number of errors is greater than or equal to 6 times), and after the frame end (frame_end) signal appears, the state machine jumps out of the locked state, and the state machine returns to the idle state.

[0114] The relevant processing instructions for the state machine in the sampling state are as follows:

[0115] When the state machine enters the sampling state, the counter count_1 starts counting from the frame_start signal, records the position of the first first_line_start signal respectively and loads it as m2 (corresponding to T2 in Figure 4 ), records the next frame_start signal and loads it as m1 (corresponding to T1 in Figure 4 ), and then the counter count_1 starts counting again.

[0116] The counter count_2 starts counting from the first first_line_start signal, and after passing the position of the next first_line_start signal and loading it as m3 (corresponding to T3 in Figure 4 ), the counter count_2 starts counting again.

[0117] As Figure 6 shown is the flowchart of loading the sampling signal in the sampling state, taking the loading of the frame period m1 as an example. When the state machine enters the sampling state, count_1 starts counting from the frame_start signal. When it reaches the next frame_start signal, the count value n1 of count_1 at this time is directly assigned to the loading value m1 (i.e., m1 = n1), and then count_1 starts counting again.

[0118] Wait until the next frame start (frame_start) signal. At this time, the count value n1 of count_1 is compared with the load value m1: If the absolute value of the difference between n1 and m1 is within the error tolerance range (register configuration error threshold, which needs to be greater than 1 clock cycle. In this example, the default configuration of the first error threshold for the time interval register measuring the field / frame period is 8’h20, that is, 32 clock cycles in decimal, which is the aforementioned error threshold), then the average value of the count value n1 of count_1 and the load value m1 is assigned to the latest load value m1 (m1 = the integer part of (m1 + n1) / 2), and at the same time, the number of times a within consecutive errors (which can be understood as the aforementioned cumulative value) is incremented by 1. After that, count_1 starts counting again. If the absolute value of the difference between n1 and m1 is not within the error tolerance range (the same as above, the default value of this example register is 32 clock cycles), then the count value n1 of count_1 is directly assigned to the load value m1 (i.e., m1 = n1), and at the same time, the number of times a within consecutive errors is cleared. After that, count_1 starts counting again. Repeat the above operations until the number of times a within consecutive errors reaches the value configured in the register (the value of consecutive correct times configured in the register, which needs to be greater than 1 time. The default value of this example register is 8’h10), then the sampling of the time interval of the field / frame period (such as Figure 4 shown as T1) is completed.

[0119] Similarly, the time interval of the line period (such as Figure 4 shown as T3) and the time interval between the line start (line_start) signal of the first line and the frame start (frame_start) signal (such as Figure 4 shown as T2) also perform corresponding sampling operations as Figure 6 shown.

[0120] Among them, the register configuration error thresholds all need to be greater than 1 clock cycle. In this example, the first error threshold of the register for measuring the time interval of the field / frame period (i.e., T1) is default configured as 8’h20 (within 32 clock cycles), the third error threshold of the register for measuring the time interval of the line period (i.e., T3) is default configured as 8’h08 (within 8 clock cycles), and the second error threshold of the register for measuring the time interval between the line start (line_start) signal of the first line and the frame start (frame_start) signal (i.e., T2) is default configured as 8’h10 (within 16 clock cycles); the number of consecutive errors in the register configuration all need to be greater than 1 time. In this example, the number of consecutive errors of the registers for measuring the time interval of the field / frame period (i.e., T1) and the time interval between the line start (line_start) signal of the first line and the frame start (frame_start) signal (i.e., T2) are both default configured as 8’h10 (i.e., 16 frames within consecutive errors), and the number of consecutive errors of the register for measuring the time interval of the line period (i.e., T3) is default configured as 8’h64 (i.e., 100 lines within consecutive errors).

[0121] When the time intervals of the field / frame period (as shown by T1 in Figure 4 ), the line period (as shown by T3 in Figure 4 ), and the time interval between the line start (line_start) signal of the first line and the frame start (frame_start) signal (as shown by T2 in Figure 4 ) are all successfully sampled. And before the next line start (line_start) signal appears after the line end (line_end) signal in the sampling state, and the time difference between the two signals is greater than the second threshold configured in the register (the second threshold needs to be greater than 1 clock cycle, and the default value of this register in this example is set as 8’h1f); before the next frame start (frame_start) signal appears after the frame end (frame_end) signal, and the time difference between the two signals is greater than the first threshold configured in the register (the first threshold needs to be greater than 1 clock cycle, and the default value of this register in this example is set as 8’h3f). Then the sampling completion signal is triggered. After the frame end (frame_end) signal arrives, the state machine enters the locking state from the sampling state.

[0122] The state machine also loads the total number of lines in a field / frame, the number of pixel lines in a field / frame (excluding embedded and user-defined line numbers, only including image lines), the number of valid data within the pixel lines (i.e., Figure 4 the number of pix_data_valid within the pixel lines in Figure 4 ), and the data type of the pixel lines (i.e., ) of the pixel lines in Figure 4 in the sampling state to facilitate the use of signals required for compensation in the subsequent locking state.

[0123] The state machine performs corresponding signal detection operations in the locked state, and performs relevant signal compensation operations when relevant signals are found to be lost. It should be noted that in this example, no compensation operation is performed on the frame_start signal.

[0124] As Figure 7 shown is the frame end signal detection mechanism in the locked state. Since the field / frame period value (loaded value m1) has been loaded in the sampling state, if the frame_end signal is not detected within the first safety time (the time interval of the first safety time is equal to the loaded value m1 minus the first threshold configured in the register) in the locked state, the frame_end signal is compensated and output after the safety time ends. After compensating and outputting the frame_end signal, the frame_end signal in the mipi_csi input signal is immediately masked to prevent the frame_end signal from being output multiple times in one frame. The masking of the frame_end signal in the mipi_csi input signal is released after the next frame_start signal appears (that is, the frame_end signal is suppressed from being output again between the compensation time point and the next frame_start signal). The compensation time point of the frame_end signal is the field / frame period value (loaded value m1) minus the first threshold configured in the register (the same as the first threshold in the sampling state, and the configuration needs to be greater than 1 clock cycle. The default value of the register in this example is set to 8’h3f).

[0125] As Figure 8 shown is the line end signal detection mechanism in the locked state. Since the line period value (loaded value m3) has been loaded in the sampling state, if the line_end signal is not detected within the second safety time (the time interval of the second safety time is equal to the loaded value m3 minus the second threshold configured in the register) in the locked state, the line_end signal is compensated and output after the safety time ends. After compensating and outputting the line_end signal, the line_end signal in the mipi_csi input signal is immediately masked to prevent the line_end signal from being output multiple times in one line. The masking of the line_end signal in the mipi_csi input signal is released after the next line_start signal appears (that is, the line_end signal is suppressed from being output again between the compensation time point and the next line_start signal). The compensation time point of the line_end signal is the line period value (loaded value m3) minus the second threshold configured in the register (the same as the second threshold in the sampling state, and the configuration needs to be greater than 1 clock cycle. The default value of the register in this example is set to 8’h1f).

[0126] As Figure 9 shown is the detection mechanism of the first line start (line_start) signal in the locked state. Since the time interval (loaded value m2) between the frame start (frame_start) signal and the first line start (line_start) signal has been loaded in the sampling state, if it is detected in the locked state that the first line start (line_start) signal has not appeared within the third safety time (the time interval of the third safety time is equal to the loaded value m2 plus the third threshold configured in the register) after the safety time ends, the first line start (line_start) signal is compensated and output. After compensating and outputting the first line start (line_start) signal, the line start (line_start) signal in the mipi_csi input signal is immediately masked to prevent the line start (line_start) signal from being output multiple times in one line. The masking of the line start (line_start) signal in the mipi_csi input signal is released until the next line end (line_end) signal appears (i.e., the output of the line start signal is suppressed between the compensation time point and the next line end signal). The compensation time point of the first line start (line_start) signal is the loaded value m2 plus the third threshold configured in the register (the third threshold must be greater than the second error threshold configured in the register in the sampling state, and the default value of this register in this example is set to 8’h1f).

[0127] As Figure 10 shown is the detection mechanism of the non-first line start signal in the locked state. Since the line period value (loaded value m3) has been loaded in the sampling state, if it is detected in the locked state that the non-first line start (line_start) signal has not appeared within the fourth safety time (the time interval of the fourth safety time is equal to the loaded value m3 plus the fourth threshold configured in the register), and at the same time the current pixel row number (excluding the embedded and user-defined row numbers, only including the image row numbers) is less than the number of pixels in one field / frame loaded in the sampling stage, the non-first line start (line_start) signal is compensated and output after the fourth safety time ends. After compensating the non-first line start (line_start) signal, the line start (line_start) signal in the mipi_csi input signal is immediately masked to prevent the line start (line_start) signal from being output multiple times in one line. The masking of the line start (line_start) signal in the mipi_csi input signal is released until the next line end (line_end) signal appears (i.e., the output of the line start signal is suppressed between the compensation time point and the next line end signal). The compensation time point of the non-first line start (line_start) signal is the loaded value m3 plus the fourth threshold configured in the register (the fourth threshold must be greater than the third error threshold configured in the register in the sampling state, and the default value of this register in this example is set to 8’h0f).

[0128] In the locked state, the data valid (pix_data_valid) signal and data (pix_data) signal in the pixel row are also detected and compensated. Since in the sampling state, the number of pixel rows in one field / frame (excluding the embedded and user-defined number of rows, only including the number of image rows), the number of valid data within the pixel rows (i.e., Figure 3 the number of pix_data_valid within the pixel row in Figure 3 ), and the data type of the pixel rows (i.e.,

[0129] such as Figure 11 are loaded.

[0130] As shown in

[0131] Figure 11 is the pixel data signal detection mechanism in the locked state. After the end of the fifth safety time (the time interval of the fifth safety time is equal to the loaded value m3 minus the second threshold minus the fifth threshold minus the time interval of the continuous compensation interval), the compensation output of the pix_data_valid and pix_data (data filled with zeros) signals starts until the number of pix_data_valid is compensated completely. The time interval of the continuous compensation interval (i.e., the number of compensations, with the unit of one clock cycle) is the number of valid data within the pixel rows loaded in the sampling state minus the number of valid data output within the fifth safety time. After compensating the pix_data_valid and pix_data (data filled with zeros) signals, the pix_data_valid and pix_data signals in the mipi_csi input signal are immediately masked, and the masking of the pix_data_valid and pix_data signals in the mipi_csi input signal is released until the next line start (line_start) signal appears. The configuration of the fifth threshold needs to be greater than or equal to 1 clock cycle, and the default value of this example register is set to 8’h05.

[0130] Since the continuous compensation interval is dynamically changing, the fifth safety time is also dynamically changing. For the convenience of understanding, the following processing is carried out in this example:

[0131] In the locked state, a counter data_valid_cnt for the pix_data_valid signal is set. The data_valid_cnt is cleared when encountering the line start signal and incremented by one each time the pix_data_valid signal (including the compensated output pix_data_valid signal) is output. Another line period counter pix_cnt is set. The pix_cnt is incremented by one for each clock cycle after the line start (line_start) signal until it is cleared when encountering the line end (line_end) signal. The compensation conditions for the pix_data_valid and pix_data signals are that the counter data_valid_cnt is less than the number of valid data within the number of pixel rows loaded in the sampling state (temporarily denoted as load_pix_data_valid), and the counter pix_cnt is equal to the loaded value m3 minus the second threshold (temporarily denoted as y2) minus the fifth threshold (temporarily denoted as y5) minus the number of valid data within the number of pixel rows loaded in the sampling state (temporarily denoted as load_pix_data_valid) plus the counter data_valid_cnt. That is, the compensation conditions are that in the locked state, (data_valid_cnt < load_pix_data_valid) is satisfied and at the same time (pix_cnt == m3 - y2 - y5 - load_pix_data_valid + data_valid_cnt) is satisfied. When the continuous compensation interval starts to compensate the pix_data_valid and pix_data signals, the pix_cnt is still incremented by one, and the data_valid_cnt is also incremented by one, and the equation still holds until all pix_data_valid signals are filled.

[0132] See Figure 12 , Figure 12 which is a signal processing device for a video stream provided by an embodiment of the present invention. As Figure 12 shown, the signal processing device 1200 for the video stream includes:

[0133] A state machine 1201, configured to monitor the signal integrity of the video data stream during the reception of the video data stream and obtain a monitoring result;

[0134] The state machine 1201 is further configured to,

[0135] in the case where the monitoring result indicates that a target signal is missing in the video data stream, complete the missing target signal based on a preset compensation signal;

[0136] Among them, the video data stream includes a plurality of key signals, and the absence of the key signals will cause abnormalities in subsequent processing of the video data stream; the target signal is one of the plurality of key signals.

[0137] In one embodiment, the plurality of key signals includes one or more of a frame end signal, a line start signal, a line end signal, a pixel data type signal, a pixel data valid signal, a pixel data signal, and a virtual channel ID signal.

[0138] In one embodiment, the process of receiving the video data stream includes a sampling stage and a locking stage;

[0139] When the state machine 1201 is in the sampling stage, the state machine 1201 is configured to: generate a signal period of the target signal based on a plurality of signal sampling times corresponding to the target signal, where the signal period of the target signal is used to represent: the time interval between two consecutive target signals;

[0140] When the state machine 1201 is in the locking stage, the state machine 1201 is configured to: determine a safe time period of the target signal based on the signal period of the target signal, and the safe time period of the target signal is a time period for waiting to receive the target signal;

[0141] When the state machine 1201 is in the locking stage, the state machine 1201 is configured to: generate a monitoring result indicating the absence of the target signal in the video data stream when the target signal is not successfully received within the safe time period of the target signal;

[0142] When the state machine 1201 is in the locking stage, the state machine 1201 is configured to: generate a monitoring result indicating that the video data stream includes the target signal when the target signal is successfully received within the safe time period of the target signal;

[0143] Among them, the sampling stage is a time stage for determining the signal period of the target signal, the locking stage is a time stage for detecting whether the target signal is successfully received, and the sampling stage is before the locking stage.

[0144] In one embodiment, when it is detected that there is a risk of signal loss in the video data stream, the state machine 1201 enters the sampling stage;

[0145] In the sampling stage, the state machine 1201 is configured to: in the sampling stage, when the i-th target signal is detected, determine the counter value as the i-th candidate value, and reset the counter to the initial value and then continue counting, where i is an integer greater than 1, and the counter starts counting from the initial value when the first target signal is detected;

[0146] When i = 2, determine the i-th candidate value as the i-th loading value, and set the cumulative value corresponding to the i-th target signal to 1;

[0147] When i is greater than 2 and the absolute value of the difference between the i-th candidate value and the (i - 1)-th loading value is less than or equal to the error threshold, determine the average value of the i-th candidate value and the (i - 1)-th loading value as the i-th loading value, and increment the cumulative value corresponding to the (i - 1)-th target signal by 1 as the cumulative value corresponding to the i-th data signal of the target key signal;

[0148] When i is greater than 2 and the absolute value of the difference between the i-th candidate value and the (i - 1)-th loading value is greater than the error threshold, determine the i-th candidate value as the i-th loading value, and set the cumulative value corresponding to the i-th target signal to 1;

[0149] The state machine 1201 is further configured to: when the cumulative value corresponding to the i-th target signal meets the set termination condition, determine the target value corresponding to the i-th target signal as the signal period of the target signal, and the set termination condition includes: the cumulative value corresponding to the i-th target signal is greater than or equal to the cumulative threshold;

[0150] Wherein, when the set switching condition is met, the state machine switches from the sampling stage to the locking stage, and the set switching condition includes: the signal period of the target signal is determined.

[0151] In one embodiment, the performing signal integrity monitoring on the video data stream to obtain a monitoring result includes:

[0152] When it is determined that there is a risk of signal loss in the video data stream, perform signal integrity monitoring on the video data stream to obtain a monitoring result.

[0153] In one embodiment, when the video data stream is switched from a first data source to a second data source, it is determined that there is a risk of signal loss in the video data stream, where the first data source and the second data source are different data sources.

[0154] In one embodiment, the locking stage is further configured to complete missing target signals;

[0155] The state machine is further configured to:

[0156] When the monitoring result indicates that the target signal is missing in the video data stream, based on a preset compensation signal, completing the missing target signal includes:

[0157] In the locking stage, when the target signal is not successfully received within the safe time period of the target signal, determining the compensation signal as the target signal and stopping receiving the target signal within the compensation time;

[0158] Wherein, the start moment of the safe time period of the target signal is: the moment when the frame start signal of the target frame is detected, or the moment when the line start signal of the target line is detected;

[0159] The start moment of the compensation time is: the moment when the compensation signal is determined as the target signal;

[0160] The target frame is an image frame corresponding to the target signal among multiple image frames of the video data stream; the target line is a line corresponding to the target signal within multiple lines included in the target frame.

[0161] In one embodiment, the state machine is further configured to:

[0162] In the sampling stage, the time interval between the frame start signal and the frame end signal of the first frame is greater than or equal to a first threshold, and / or the time interval between the line start signal and the line end signal of the first line is greater than or equal to a second threshold;

[0163] Wherein, the first frame is an image frame corresponding to the target signal among multiple image frames of the video data stream; the first line is a line corresponding to the target signal within multiple lines included in the first frame;

[0164] When the target signal is a frame end signal, the duration of the compensation time is equal to the first threshold;

[0165] When the target signal is a line end signal, the duration of the compensation time is equal to the second threshold.

[0166] In one embodiment, the state machine is further configured to:

[0167] In the locking stage, when the execution times of the completion operation are greater than or equal to a preset number threshold and a frame end signal is detected, exiting the locking stage.

[0168] The signal processing device 1200 for a video stream provided by an embodiment of the present invention can implement each process in the embodiment of the above-mentioned signal processing method for a video stream. To avoid repetition, it will not be described in detail here.

[0169] According to an embodiment of the present invention, the present invention also provides an electronic device and a readable storage medium.

[0170] Figure 13 A schematic block diagram of an exemplary electronic device 1300 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0171] As Figure 13 shown, the device 1300 includes a computing unit 1301 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1302 or a computer program loaded from a storage unit 1308 into a random access memory (RAM) 1303. In the RAM 1303, various programs and data required for the operation of the device 1300 can also be stored. The computing unit 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0172] Multiple components in the device 1300 are connected to the I / O interface 1305, including: an input unit 1306, such as a keyboard, a mouse, etc.; an output unit 1307, such as various types of displays, speakers, etc.; a storage unit 1308, such as a magnetic disk, an optical disk, etc.; and a communication unit 1309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1309 allows the device 1300 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0173] The computing unit 1301 can be various general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1301 include, but are not limited to, a central processing unit (CPU), a graphic process unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1301 executes the various methods and processes described above, such as the signal processing method of the video stream. For example, in some embodiments, the signal processing method of the video stream can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 1308. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1300 via the ROM 1302 and / or the communication unit 1309. When the computer program is loaded into the RAM 1303 and executed by the computing unit 1301, one or more steps of the signal processing method of the video stream described above can be executed. Alternatively, in other embodiments, the computing unit 1301 can be configured to execute the signal processing method of the video stream in any other suitable manner (e.g., by means of firmware).

[0174] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs, which can be executed and / or interpreted on a programmable system including at least one programmable processor, the programmable processor can be a dedicated or general programmable processor, can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0175] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.

[0176] In the context of the present invention, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0177] As used herein, the term "machine-readable medium" refers to any computer program product, device, and / or apparatus (e.g., a disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0178] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or an LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the input received from the user can be in any form (including acoustic input, voice input, or tactile input).

[0179] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0180] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client - server relationship is created by computer programs running on respective computers and having a client - server relationship with each other. The server can be a cloud server, can also be a server of a distributed system, or a server incorporating a blockchain.

[0181] An embodiment of the present application also provides a computer program product, including computer instructions, which when executed by a processor, implement each process of the method embodiment shown above Figure 1 and can achieve the same technical effects. To avoid repetition, it will not be elaborated herein.

[0182] It should be understood that various forms of the flow shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in the present invention can be achieved. No limitation is imposed herein.

[0183] The above - mentioned specific embodiments do not constitute a limitation to the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A signal processing method for a video stream, characterized in that The method includes: During the process of receiving a video data stream, performing signal integrity monitoring on the video data stream to obtain a monitoring result; When the monitoring result indicates that a target signal is missing in the video data stream, based on a preset compensation signal, completing the missing target signal; Wherein, the video data stream includes multiple key signals, and the absence of the key signals will cause abnormalities in subsequent processing of the video data stream; The target signal is one of the multiple key signals.

2. The method according to claim 1, wherein The performing signal integrity monitoring on the video data stream to obtain a monitoring result includes: When it is determined that there is a risk of signal loss in the video data stream, performing signal integrity monitoring on the video data stream to obtain a monitoring result.

3. The method according to claim 2, wherein Before the performing signal integrity monitoring on the video data stream to obtain a monitoring result, the method further includes: When the video data stream is switched from a first data source to a second data source, determining that there is a risk of signal loss in the video data stream, where the first data source and the second data source are different data sources.

4. The method according to claim 1, wherein The multiple key signals include one or more of a frame end signal, a line start signal, a line end signal, a pixel data type signal, a pixel data valid signal, a pixel data signal, and a virtual channel ID signal.

5. The method according to claim 1, wherein The process of receiving the video data stream includes a sampling stage and a locking stage. The performing signal integrity monitoring on the video data stream to obtain a monitoring result includes: In the sampling stage, based on multiple signal sampling times corresponding to the target signal, generating a signal period of the target signal, where the signal period of the target signal is used to represent: the time interval between two consecutive target signals; In the locking stage, based on the signal period of the target signal, determining a safe time period of the target signal, where the safe time period of the target signal is a time period for waiting to receive the target signal; When the target signal is not successfully received within the safe time period of the target signal, generating a monitoring result indicating that the target signal is missing in the video data stream; When the target signal is successfully received within the safe time period of the target signal, generating a monitoring result indicating that the video data stream includes the target signal; Wherein, the sampling stage is a time stage for determining the signal period of the target signal, the locking stage is a time stage for detecting whether the target signal is successfully received, and the sampling stage is before the locking stage.

6. The method according to claim 5, wherein The generating the signal period of the target signal based on multiple signal sampling times corresponding to the target signal in the sampling stage includes: When it is detected that there is a risk of signal loss in the video data stream, entering the sampling stage; In the sampling stage, when the i-th target signal is detected, determining the counter value as the i-th candidate value, and resetting the counter to the initial value and then continuing to count, where i is an integer greater than 1, and the counter starts counting from the initial value when the first target signal is detected; When i = 2, determine the i-th candidate value as the i-th loading value, and set the cumulative value corresponding to the i-th target signal to 1; When i > 2 and the absolute value of the difference between the i-th candidate value and the (i - 1)-th loading value is less than or equal to the error threshold, determine the average value of the i-th candidate value and the (i - 1)-th loading value as the i-th loading value, and use the cumulative value corresponding to the (i - 1)-th target signal plus 1 as the cumulative value corresponding to the i-th data signal of the target key signal; When i > 2 and the absolute value of the difference between the i-th candidate value and the (i - 1)-th loading value is greater than the error threshold, determine the i-th candidate value as the i-th loading value, and set the cumulative value corresponding to the i-th target signal to 1; When the cumulative value corresponding to the i-th target signal meets the set termination condition, determine the target value corresponding to the i-th target signal as the signal period of the target signal, and the set termination condition includes: the cumulative value corresponding to the i-th target signal is greater than or equal to the cumulative threshold; Wherein, when the set switching condition is met, switch from the sampling stage to the locking stage, and the set switching condition includes: the signal period of the target signal is determined.

7. The method according to claim 5, characterized in that The locking stage is also used to complete the missing target signal; In the case that the monitoring result indicates that the target signal is missing in the video data stream, based on the preset compensation signal, complete the missing target signal, including: In the locking stage, when the target signal is not successfully received within the safe time period of the target signal, determine the compensation signal as the target signal, and stop receiving the target signal within the compensation time; Wherein, the start time of the safe time period of the target signal is: the time when the frame start signal of the target frame is detected, or the time when the line start signal of the target line is detected; The start time of the compensation time is: the time when the compensation signal is determined as the target signal; The target frame is one image frame corresponding to the target signal among multiple image frames of the video data stream; the target line is one line corresponding to the target signal among multiple lines included in the target frame.

8. The method according to claim 7, wherein: In the sampling stage, the time interval between the frame start signal and the frame end signal of the first frame is greater than or equal to the first threshold, and / or, the time interval between the line start signal and the line end signal of the first line is greater than or equal to the second threshold; Wherein, the first frame is one image frame corresponding to the target signal among multiple image frames of the video data stream; the first line is one line corresponding to the target signal among multiple lines included in the first frame; When the target signal is the frame end signal, the duration of the compensation time is equal to the first threshold; When the target signal is the line end signal, the duration of the compensation time is equal to the second threshold.

9. The method according to claim 5, characterized in that In the case that the monitoring result indicates the absence of a target signal in the video data stream, after completing the missing target signal based on a preset compensation signal, the method further includes In the locking phase, when the number of executions of the completion operation is greater than or equal to a preset number threshold and a frame end signal is detected, exit the locking phase.

10. A signal processing device for a video stream, characterized in that, The apparatus includes: A state machine for monitoring the signal integrity of the video data stream during the reception of the video data stream to obtain a monitoring result; The state machine is further configured to, in the case that the monitoring result indicates the absence of a target signal in the video data stream, complete the missing target signal based on a preset compensation signal; Wherein, the video data stream includes a plurality of key signals, each of the key signals includes a plurality of data signals and a plurality of flag signals, and the absence of a key signal will cause an abnormality in the subsequent processing of the video data stream; The target signal is one of the plurality of key signals.

11. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the signal processing method of the video stream according to any one of claims 1 to 9 are implemented.

Images