Method for solving video loopback test data jitter based on OpenCV in RK3576

Abstract

The invention discloses a method for solving video loopback test data jitter on the basis of OpenCV in RK3576. A DP and MIPI CSI physical loopback link is constructed through a SerDes chip, an acquired image is converted into a YUV420 format, and a Y channel, a U channel and a V channel are separated. 3 * 3 mean filtering and 5 * 5 Gaussian filtering are sequentially carried out on the Y channel, and 3 * 3 Gaussian filtering preprocessing is carried out on the U / V channel. A structural similarity index (SSIM) is adopted to calculate similarity in different channels, weight fusion is performed according to 0.6: 0.2: 0.2, and image consistency is judged by dynamically adjusting a threshold value (default is 99%, and a stable scene is widened to 98.5%) in combination with a continuous 10-frame standard deviation. The scheme is adaptive to RK3576 hardware characteristics, through multi-stage noise reduction, sub-channel weighted comparison and a dynamic threshold mechanism, the noise influence of an MIPI CSI link is effectively inhibited, the anti-jitter capability and reliability of image comparison in a video loopback test are improved, and the method is suitable for a closed-loop test scene of an embedded image acquisition system.

Description

Technical Field

[0001] The present invention relates to the technical fields of image processing and embedded system testing, and particularly to a method for solving video loop test data jitter based on OpenCV on RK3576. Background Art

[0002] As the core infrastructure of a smart city, the accuracy, real-time nature, and integrity of urban road network data directly affect traffic planning decisions, traffic flow optimization strategies, and public service efficiency. Traditional road network data acquisition mainly relies on on-site surveying and mapping by government departments, which has problems such as high data collection costs, long update cycles, and inconsistent format standards, making it difficult to meet the needs of dynamic traffic management and real-time navigation. With the development of Internet technology, extraction methods based on trajectory data or single-point POIs have gradually been applied, but such technologies have defects such as single data sources, rough processing of spatial relationships, and insufficient multi-source data integration capabilities, resulting in problems such as redundancy, topological errors, or incomplete coverage in the extracted road network data.

[0003] In the prior art, some solutions directly associate roads through the coordinates of a single POI point, but ignore the spatial correlation between POI points; other solutions use manual intervention to process data deduplication and topological repair, which is inefficient and prone to introducing human errors. Therefore, there is an urgent need for an intelligent system that integrates POI point combination technology, automated spatial data processing, and multi-source data integration to achieve efficient and accurate urban road network extraction. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for solving video loop test data jitter based on OpenCV on RK3576, specifically involving an image comparison method for anti-data jitter in video loop tests using the OpenCV library based on the RK3576 platform. This method is applicable to embedded systems that collect images through the MIPI CSI interface, and solves the problem of image comparison failure caused by signal noise or hardware jitter through multi-stage image preprocessing and channel-by-channel similarity analysis.

[0005] The present invention is achieved through the following technical solutions:

[0006] A method for solving video loop test data jitter based on OpenCV on RK3576, comprising the following steps:

[0007] (1) Construct a physical loop link between the DP output interface and the MIPI CSI input interface of RK3576 through a SerDes conversion chip to form a closed-loop transmission path for video signals;

[0008] (2) Convert the images collected by MIPI CSI to the YUV420 format, and separate the luminance channel (Y) and the chrominance channels (U, V), where the resolution of the Y channel is 1920×1080, and the resolution of the U / V channels is 960×540;

[0009] (3) Perform 3×3 mean filtering and 5×5 Gaussian filtering on the Y channel in sequence, and perform 3×3 Gaussian filtering on the U / V channels;

[0010] (4) Calculate the structural similarity index (SSIM) of the Y, U, and V channels respectively, and fuse them with weights of 0.6:0.2:0.2 to obtain the total similarity;

[0011] (5) Dynamically adjust the decision threshold according to the standard deviation of the test results of 10 consecutive frames. If the standard deviation ≤ 0.3%, the threshold is automatically relaxed to 98.5%, otherwise the default threshold of 99% is adopted, and the test results are output.

[0012] Further, as an improvement of the technical solution of the present invention, the SerDes conversion chip described in step (1) includes DS90UB954 / 953, MAX9286 or equivalent functional chips, and the pre-emphasis and equalization functions are enabled in the chip configuration to compensate for signal transmission distortion.

[0013] Further, as an improvement of the technical solution of the present invention, the calculation formula of the mean filtering in step (3) is:

[0014]

[0015] where F is the input image matrix and G is the filtered image.

[0016] Further, as an improvement of the technical solution of the present invention, the kernel function of the Gaussian filtering in step (3) is:

[0017]

[0018] where the standard deviation σ of the Gaussian filtering of the Y channel takes values from 1 to 3, and the default value is 2.

[0019] Further, as an improvement of the technical solution of the present invention, the single-channel SSIM calculation in step (4) adopts the formula:

[0020]

[0021] where μ x / μ y is the channel mean, is the channel variance, σ xy is the covariance, c1 = (0.01L) 2 and c2 = (0.03L) 2, L = 255 is the pixel value range.

[0022] Further, as an improvement to the technical solution of the present invention, the YUV420 format conversion in step (2) is implemented through the cvtColor function of OpenCV, and the storage addresses of the U / V channels after conversion are offset by 1920×1080 pixels relative to the Y channel.

[0023] Further, as an improvement to the technical solution of the present invention, the dynamic threshold adjustment formula in step (5) is:

[0024] T = T0 - k·std(SSIM history )

[0025] where T0 = 99%, k = 0.1 - 0.5, and std is the standard deviation of the SSIM values of 10 consecutive frames.

[0026] Further, as an improvement to the technical solution of the present invention, it also includes adaptive histogram equalization (CLAHE) processing for the Y channel, with the contrast limited to 1.0 - 3.0, to enhance the edge sharpness of low-contrast images.

[0027] Further, as an improvement to the technical solution of the present invention, the test result output includes the preprocessed images of each channel, the real-time SSIM value curves (Y / U / V channels and the total result), the noise level classification (low / medium / high noise), and the judgment conclusion.

[0028] Further, as an improvement to the technical solution of the present invention, the hardware environment includes:

[0029] RK3576 development board, integrated with DP 1.4 interface and 4-lane MIPI CSI-2 interface;

[0030] Simulated image sensor module, supporting MIPI CSI input and with a resolution ≥ 1080p.

[0031] In summary, the beneficial effects of the present invention are as follows:

[0032] Hardware-software collaborative noise reduction:

[0033] Through the cascaded processing of pre-emphasis by the SerDes chip and software mean / Gaussian filtering, the impact of MIPI CSI link noise is reduced by more than 70% (compared with traditional single-stage filtering);

[0034] The YUV channel separation adapts to the hardware downsampling characteristics, avoiding the reference image deviation caused by format mismatch, and the measured comparison time is shortened by 30%.

[0035] Human eye visual characteristic optimization:

[0036] The high weight ratio comparison of the Y channel (0.6) significantly improves the recognition accuracy of brightness details, and the low weight design of the U / V channels reduces the misjudgment rate of chromaticity noise by 60%;

[0037] The dynamic threshold mechanism adapts to changes in the hardware environment, and the number of misjudgments is reduced by 85% during long-term testing (compared with the fixed threshold scheme).

[0038] Efficient utilization of embedded resources:

[0039] The preprocessing algorithm selects lightweight filtering (mean + Gaussian), and the processing time per frame is < 15 ms, meeting the real-time test requirements of 30 fps;

[0040] The channel-separated calculation architecture makes full use of the parallel processing capabilities of the ARM multi-core CPU of RK3576, avoiding GPU resource occupation. Description of the Drawings

[0041] Other features, objects, and advantages of the present invention will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

[0042] Figure 1 It is a hardware architecture diagram for the RK3576 video loopback test;

[0043] Figure 2 It is a flowchart of a method for solving data jitter in video loopback testing based on OpenCV in rk3576 according to the present invention;

[0044] Figure 3 It is a schematic diagram of YUV channel separation and weighted fusion. Detailed Embodiments

[0045] To make the objectives, features, and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application are described clearly and completely. Obviously, the embodiments described below are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0046] Referring to Figures 1 to 3 , a method for solving data jitter in video loopback testing based on OpenCV in RK3576 includes the following steps:

[0047] (1) Construct a physical loopback link between the DP output interface and the MIPI CSI input interface of RK3576 through a SerDes conversion chip to form a closed-loop transmission path for video signals;

[0048] (2) Convert the images collected by MIPI CSI to the YUV420 format, and separate the luminance channel (Y) and the chrominance channels (U, V), where the resolution of the Y channel is 1920×1080, and the resolution of the U / V channels is 960×540;

[0049] (3) Perform 3×3 mean filtering and 5×5 Gaussian filtering on the Y channel in sequence, and perform 3×3 Gaussian filtering on the U / V channels;

[0050] (4) Calculate the structural similarity index (SSIM) of the Y, U, and V channels respectively, and fuse them with weights of 0.6:0.2:0.2 to obtain the total similarity;

[0051] (5) Dynamically adjust the decision threshold according to the standard deviation of the test results of 10 consecutive frames. If the standard deviation ≤ 0.3%, the threshold is automatically relaxed to 98.5%, otherwise the default threshold of 99% is adopted, and the test results are output.

[0052] It should be noted that for the construction of the hardware loopback link:

[0053] Customized connection solution: Use a SerDes chip (such as DS90UB954 / 953) that supports DP to MIPI CSI protocol conversion to build a physical loopback. For the DP 1.4 interface characteristics of RK3576, set the output image resolution to 1920×1080 and the frame rate to 30fps. After encoding by the conversion chip, it is received by the MIPI CSI-2 interface to ensure that the hardware link transmission format is consistent with the YUV420 output format of the acquisition end (such as the IMX490 sensor).

[0054] Signal integrity compensation: Enable the pre-emphasis and equalization functions in the SerDes chip configuration to reduce the edge distortion in high-speed signal transmission and provide basic noise reduction for subsequent software preprocessing.

[0055] Specifically, in the solution of this embodiment, the SerDes conversion chip described in step (1) includes DS90UB954 / 953, MAX9286 or equivalent functional chips, and the pre-emphasis and equalization functions are enabled in the chip configuration to compensate for signal transmission distortion.

[0056] Specifically, in the solution of this embodiment, after converting the BGR images collected by MIPI CSI to grayscale images, 3×3 window mean filtering is used to eliminate random isolated noise points by calculating the average of neighboring pixels; the calculation formula of the mean filtering described in step (3) is:

[0057]

[0058] Among them, F is the input image matrix, and G is the filtered image.

[0059] Specifically, in the solution of this embodiment, a 5×5 Gaussian filter (standard deviation σ = 2) is preferentially performed on the luminance channel (Y), and the weight distribution of the Gaussian kernel is used to suppress high-frequency jitter; the kernel function of the Gaussian filter in step (3) is:

[0060]

[0061] Among them, the standard deviation σ of the Gaussian filter for the Y channel takes a value of 1-3, and the default value is 2.

[0062] Specifically, in the solution of this embodiment, the single-channel SSIM calculation in step (4) adopts the formula:

[0063]

[0064] Among them, μ x / μ y is the channel mean value, is the channel variance, σ xy is the covariance, c1 = (0.01L) 2 、c2 = (0.03L) 2 , and L = 255 is the pixel value range.

[0065] Specifically, in the solution of this embodiment, the YUV420 format conversion in step (2) is implemented through the cvtColor function of OpenCV, and the storage addresses of the U / V channels after conversion are offset by 1920×1080 pixels relative to the Y channel. It should be noted that the reference image and the captured image are uniformly converted to the YUV420 format, the Y(1920×1080), U(960×540), and V(960×540) channels are separated, and the 4:2:0 subsampling characteristics of the chrominance signal of the hardware link are matched to avoid comparison deviations caused by inconsistent resolutions. The full-resolution SSIM calculation is performed on the luminance channel (Y) to retain the detail information sensitive to the human eye. The SSIM is calculated for the chrominance channels (U / V) at the subsampled resolution (960×540) to reduce the sensitivity to chrominance noise.

[0066] Specifically, in the solution of this embodiment, the dynamic threshold adjustment formula in step (5) is:

[0067] TT0 - k·std(SSIM history )

[0068] Among them, T = 99%, k = 0.1 - 0.5, and std is the standard deviation of the SSIM values of 10 consecutive frames.

[0069] Specifically, in the solution of this embodiment, it also includes the adaptive histogram equalization (CLAHE) processing of the Y channel, with the contrast limited to 1.0 - 3.0, so as to enhance the edge sharpness of low-contrast images.

[0070] Specifically, in the solution of this embodiment, the test result output includes the images of each channel after preprocessing, the real-time SSIM value curves (Y / U / V channels and the total result), the noise level classification (low / medium / high noise), and the judgment conclusion.

[0071] Specifically, in the solution of this embodiment, the hardware environment includes:

[0072] RK3576 development board, integrated with DP 1.4 interface and 4-lane MIPI CSI-2 interface;

[0073] Simulated image sensor module, supporting MIPI CSI input and with a resolution of ≥1080p.

[0074] It should be noted that for the dynamic threshold determination and result output:

[0075] Dual-threshold calibration mechanism:

[0076] Basic threshold: By default, 99% is used as the determination threshold (derived from the core index of "confirming that the similarity reaches 99%" in the disclosure), ensuring strict comparison of clear images;

[0077] Adaptive correction: Continuously collect 10 frames of images to calculate the SSIM standard deviation. If the standard deviation < 0.3% (indicating stable noise), the threshold is automatically relaxed to 98.5% to avoid excessive misjudgment caused by the inherent noise of the hardware.

[0078] Multi-dimensional result output:

[0079] Output the test results through the system log interface of RK3576, including:

[0080] The images of each channel after preprocessing (for problem reproduction);

[0081] Real-time SSIM value curves (Y / U / V channels and the weighted total result);

[0082] Judgment conclusion and noise level (low / medium / high noise scenario classification).

[0083] Embodiment:

[0084] Hardware environment construction

[0085] Core devices:

[0086] Use the RK3576 development board, which integrates a DP 1.4 interface and a 4-lane MIPI CSI-2 interface.

[0087] The SerDes conversion chips DS90UB954 (transmitter) and DS90UB953 (receiver) are adopted, and these chips support a rate of 1.5 Gbps / lane.

[0088] An analog image sensor module is equipped, which supports MIPI CSI input and can achieve a resolution of 1080p.

[0089] Connection method: Connect the DP output interface of the RK3576 development board through the DS90UB954 chip. The DS90UB954 encodes the signal output from the DP, and then transmits the encoded signal to the DS90UB953 chip through a line. The DS90UB953 decodes the signal, and the decoded signal is connected to the MIPI CSI input interface of the RK3576, thereby constructing a physical loopback link.

[0090] 2. Software implementation steps

[0091] Image acquisition and format conversion:

[0092] Run the relevant program on the RK3576 development board, control the MIPI CSI module through the Linux V4L2 driver to acquire a frame of image, and save it as a file in BGR format.

[0093] Utilize the functions provided by the OpenCV library to convert the acquired BGR format image to YUV420 format. Then, according to the storage rules of the YUV420 format, separate the luminance channel (Y), chrominance channel (U), and chrominance channel (V). Among them, the resolution of the Y channel remains 1920×1080, while the U and V channels are downsampled due to the format characteristics, and the resolution becomes 960×540.

[0094] Image preprocessing:

[0095] First, perform mean filtering on the separated Y channel image. By sliding a window of a specific size (such as 3×3) on the image and calculating the average of the pixel values within the window, so as to remove salt-and-pepper noise.

[0096] Then, perform Gaussian filtering on the Y channel image after mean filtering. Gaussian filtering assigns different weights according to the distance relationship between pixel points and surrounding pixel points, smooths the image, and reduces edge jitter in the image.

[0097] For the U and V channel images, also perform Gaussian filtering to reduce the influence of noise on chrominance information.

[0098] For some low-contrast scenes, it is also possible to choose to perform adaptive histogram equalization on the Y-channel image. This processing method can enhance the edge sharpness of the image and improve the overall contrast of the image.

[0099] Channel-wise structural similarity calculation and result determination:

[0100] Perform structural similarity (SSIM) calculations on the Y, U, and V channels of the reference image and the preprocessed image to be compared respectively. Use relevant image processing libraries to complete this calculation process.

[0101] According to the human eye's perception characteristics of brightness and chrominance, assign a higher weight (such as 0.6) to the Y channel, and lower weights (such as 0.2) to the U and V channels. Weight and fuse the SSIM values calculated from the three channels according to this weight ratio to obtain the final similarity value.

[0102] Set an initial similarity threshold (such as 99%), and compare the final similarity value with this threshold. If the similarity value is greater than or equal to the threshold, it is determined that the two frames of images are the same and the test passes; otherwise, it is determined that the images are different and the test fails.

[0103] The threshold can be dynamically adjusted according to the test results of multiple consecutive frames (such as 10 frames). For example, if the fluctuations in the test results of multiple consecutive frames are small, it indicates that the test environment is relatively stable, and the threshold can be appropriately relaxed; if the fluctuations are large, the threshold needs to be tightened to ensure the accuracy of the test.

[0104] Finally, output the test results through the UART or USB interface of RK3576. The output information includes the SSIM values of each channel, the time spent on preprocessing, the final determination conclusion, etc.

[0105] 3. Alternative solutions

[0106] Filtering algorithm replacement: When encountering a scene with high-density salt-and-pepper noise, the mean filter can be replaced with a median filter. The median filter has better effects in dealing with impulse noise.

[0107] Similarity algorithm upgrade: Adopt the multi-scale SSIM (MSSIM) algorithm to replace the single-scale SSIM calculation. The multi-scale SSIM algorithm can better adapt to image scaling jitter at different resolutions.

[0108] Hardware link optimization: Enable the adaptive equalization (CTLE) function in the configuration of the SerDes chip. This function can further process and optimize the signal. Combined with software preprocessing, it forms a cascaded denoising effect and improves the quality of the image.

[0109] In summary, the beneficial effects of the present invention are as follows:

[0110] Hardware-Software Collaborative Noise Reduction:

[0111] By cascading pre-emphasis of the SerDes chip and software mean / Gaussian filtering, the impact of MIPI CSI link noise is reduced by more than 70% (compared with traditional single-stage filtering);

[0112] The YUV channel separation adapts to the hardware downsampling characteristics, avoiding the reference image deviation caused by format mismatch, and the measured comparison time is shortened by 30%.

[0113] Optimization of Human Eye Visual Characteristics:

[0114] The high weight ratio of the Y channel (0.6) significantly improves the recognition accuracy of luminance details, and the low weight design of the U / V channels reduces the chromaticity noise misjudgment rate by 60%;

[0115] The dynamic threshold mechanism adapts to changes in the hardware environment, and the number of misjudgments is reduced by 85% in long-term tests (compared with the fixed threshold scheme).

[0116] Efficient Utilization of Embedded Resources:

[0117] The preprocessing algorithm selects lightweight filtering (mean + Gaussian), and the processing time per frame < 15ms, meeting the real-time test requirements of 30fps;

[0118] The channel-separated calculation architecture makes full use of the parallel processing capabilities of the ARM multi-core CPU of RK3576, avoiding GPU resource occupation.

[0119] The above has introduced the technical solutions provided by the embodiments of the present invention in detail. Specific examples are used in this article to elaborate on the principles and implementation manners of the embodiments of the present invention. The descriptions of the above embodiments are only applicable to helping understand the principles of the embodiments of the present invention; at the same time, for those of ordinary skill in the art, according to the embodiments of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for solving video loop test data jitter based on OpenCV on RK3576, characterized in that Including the following steps: (1) Construct a physical loopback link between the DP output interface of RK3576 and the MIPI CSI input interface through a SerDes conversion chip to form a closed-loop transmission path for video signals; (2) Convert the images collected by MIPI CSI into the YUV420 format, and separate the luminance channel (Y) and chrominance channels (U, V), where the resolution of the Y channel is 1920×1080, and the resolution of the U / V channels is 960×540; (3) Perform 3×3 mean filtering and 5×5 Gaussian filtering on the Y channel in sequence, and perform 3×3 Gaussian filtering on the U / V channels; (4) Calculate the structural similarity index of the Y, U, and V channels respectively, and fuse them with weights of 0.6:0.2:0.2 to obtain the total similarity; (5) Dynamically adjust the decision threshold according to the standard deviation of the test results of 10 consecutive frames. If the standard deviation ≤ 0.3%, the threshold is automatically relaxed to 98.5%, otherwise the default threshold of 99% is adopted, and the test results are output.

2. A method for solving video loop test data jitter based on OpenCV in RK3576 according to claim 1, characterized in that: The SerDes conversion chip described in step (1) includes DS90UB954 / 953, MAX9286 or equivalent functional chips, and the pre-emphasis and equalization functions are enabled in the chip configuration to compensate for signal transmission distortion.

3. A method for solving video loop test data jitter based on OpenCV in RK3576 according to claim 1, characterized in that: The calculation formula of the mean filtering described in step (3) is: where F is the input image matrix and G is the filtered image.

4. A method for solving video loop test data jitter based on OpenCV in RK3576 according to claim 1, characterized in that: The kernel function of the Gaussian filtering described in step (3) is: where the standard deviation σ of the Gaussian filtering of the Y channel takes a value of 1 - 3, and the default value is 2.

5. A method for solving video loop test data jitter based on OpenCV in RK3576 according to claim 1, characterized in that: The formula for calculating the single-channel SSIM described in step (4) is: Among them, μ x / μ y is the channel mean, is the channel variance, σ xy is the covariance, c1 = (0.01L) 2 、c2 = (0.03L) 2 , where L = 255 is the pixel value range.

6. A method for solving video loop test data jitter based on OpenCV in RK3576 according to claim 1, characterized in that: The YUV420 format conversion described in step (2) is implemented through the cvtColor function of OpenCV, and the storage address of the U / V channels after conversion is offset by 1920×1080 pixels relative to the Y channel.

7. A method for solving video loop test data jitter based on OpenCV in RK3576 according to claim 1, characterized in that: The formula for dynamic threshold adjustment described in step (5) is: T = T0 - k·std(SSIM history ) where, T0 = 99%, k = 0.1 - 0.5, and std is the standard deviation of the SSIM values of 10 consecutive frames.

8. A method for solving video loop test data jitter based on OpenCV in RK3576 according to claim 1, characterized in that: It also includes adaptive histogram equalization processing of the Y channel, with the contrast limited to 1.0 - 3.0 to enhance the edge sharpness of low-contrast images.

9. A method for solving video loop test data jitter based on OpenCV in RK3576 according to claim 1, characterized in that: The output of the test results includes the images of each channel after preprocessing, the real-time SSIM value curve, the noise level classification, and the judgment conclusion.

10. A method for solving video loop test data jitter based on OpenCV in RK3576 according to claim 1, characterized in that: The hardware environment includes: RK3576 development board, integrated with DP 1.4 interface and 4-lane MIPI CSI-2 interface; An analog image sensor module, supporting MIPI CSI input and with a resolution ≥ 1080p.

Images