A gmsl2 camera data synchronization method and system based on real-time and security

Abstract

The application relates to a GMSL2 camera data synchronization method and system based on real-time and security, which comprises the following steps: U1. A vehicle travels on a road, real-time data information of images collected by multiple cameras is acquired based on a vehicle-mounted multi-camera system, and time stamps are marked to obtain time-stamp-marked data information of images collected by the multiple cameras; U2. Based on the time-stamp-marked data information of images collected by the multiple cameras, a similarity algorithm of images based on a Spearman correlation coefficient is adopted to represent the similarity of images collected by the multiple cameras, and data information of the similarity of images collected by the multiple cameras is obtained. The application can accurately process each frame of images of the camera, guarantee the integrity of data synchronization, and the data synchronization process is not affected by man-in-the-middle attacks or electromagnetic interference, and the security of the synchronized data is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of visual synchronization technology, and in particular to a GMSL2 camera data synchronization method and system based on real-time and security. Background Technology

[0002] With the rapid development of autonomous driving, industrial machine vision, and other fields, the synchronization and security of multi-camera systems have become core requirements. GMSL2 (Gigabit Multimedia Serial Link 2) is widely used in automotive and industrial camera systems due to its high bandwidth (6Gbps), long-distance transmission (15-20 meters), and anti-interference capabilities. However, existing solutions have bottlenecks in the following aspects:

[0003] Real-time performance: Traditional time synchronization solutions rely on the gPTP network protocol (accuracy <1ms) or GPS PPS signals, which are subject to network delays or signal blockage risks.

[0004] Security: Time synchronization data is vulnerable to man-in-the-middle attacks or electromagnetic interference, which can lead to synchronization failure or data tampering.

[0005] Dynamic adaptability: In hot-swappable camera scenarios, existing solutions struggle to quickly resynchronize and lack a link health monitoring mechanism. Summary of the Invention

[0006] In view of the above problems, the present invention provides a real-time and secure GMSL2 camera data synchronization method and system, which can not only accurately process each frame of the camera image to ensure the integrity of data synchronization, but also ensure the security of the synchronized data by preventing man-in-the-middle attacks or electromagnetic interference.

[0007] To achieve the above and other related objectives, the present invention provides the following technical solution:

[0008] A real-time and security-based GMSL2 camera data synchronization method, the method comprising:

[0009] U1. When a vehicle is driving on the road, it acquires data information of images collected by multiple cameras in real time based on the vehicle-mounted multi-camera system, and timestamps the data information of images collected by multiple cameras after timestamping.

[0010] U2. Based on the data information of images acquired by multiple cameras after the timestamp, the similarity of images acquired by multiple cameras is characterized by an image similarity algorithm based on Spearman correlation coefficient, and the data information of the similarity of images acquired by multiple cameras is obtained.

[0011] U3. Based on the similarity data of the images acquired by the multiple cameras, a time-series-based frame synchronization algorithm is used to synchronize the image data of the multiple cameras to obtain the synchronized image data information of the multiple cameras;

[0012] U4. Based on the image data information of the multiple cameras after synchronization processing, construct an image synchronization detection function Q, detect the synchronization result of the image, and obtain the data information of the detection value of the image synchronization result.

[0013] Furthermore, the synchronization detection function Q of the image is,

[0014] ,

[0015] Where, x i Let α1, α2, and α3 be the eigenvalues ​​of the pixel matrix of the i-th frame image in the image data information of multiple cameras after synchronous processing, where n is the sample size and α1, α2, and α3 are weighting factors.

[0016] Furthermore, the weighting factors α1, α2, and α3 are,

[0017] ,

[0018] ,

[0019] ,

[0020] Where, x i Let be the eigenvalues ​​of the pixel matrix of the i-th frame image in the image data information of multiple cameras after synchronous processing, and n be the sample size.

[0021] Furthermore, the method also includes:

[0022] U5. Based on the data information of the detection value of the synchronization result of the image, a preset threshold is set. If the detection value of the synchronization result of the image is less than the preset threshold, the requirement is not met, and the process returns to step U2. If the detection value of the synchronization result of the image is greater than the preset threshold, the requirement is met, and the data synchronization of multiple cameras is completed.

[0023] Furthermore, in step U2, the characterization of the similarity of images acquired by multiple cameras using an image similarity algorithm based on Spearman correlation coefficient includes:

[0024] U21. Based on the data information of the images acquired by the multiple cameras after the timestamp, construct a sequence of pixel matrices of the images acquired by the multiple cameras to obtain the data information of the sequence of pixel matrices of the images acquired by the multiple cameras;

[0025] U22. Based on the sequence data information of the pixel matrix of the images acquired by the multiple cameras, construct the Spearman correlation function W of the image pixel matrix sequence.

[0026] ,

[0027] Among them, y i y is the data information of the i-th sequence of the pixel matrix of images acquired by multiple cameras. i+1 y is the data information of the (i+1)th sequence of the pixel matrix of images acquired by multiple cameras. i+2 This refers to the (i+2)th sequence of pixel matrix data from images acquired by multiple cameras, where m is a positive integer and β... j These are the weighting coefficients;

[0028] U23. Based on the Spearman correlation function W of the image pixel matrix sequence, the similarity of images acquired by multiple cameras is characterized to obtain data information on the similarity of images acquired by multiple cameras.

[0029] Furthermore, the weighting coefficient β j The constraints are as follows:

[0030] ,

[0031] Where m is a positive integer.

[0032] Furthermore, in step U3, the synchronization processing of image data from multiple cameras using a time-series-based frame synchronization algorithm includes:

[0033] U31. Based on the similarity data of the images acquired by the multiple cameras, extract the image with the highest similarity in each frame and construct the time series data of the pixel matrix with the highest similarity among the images of the multiple cameras;

[0034] U32. Based on the time-series data of the pixel matrix with the highest similarity among the images from the multiple cameras, a prediction function G for the pixel matrix of the next frame image from the multiple cameras is established.

[0035] ,

[0036] Where z is the time series data information of the pixel matrix with the highest similarity among multiple camera images, and δ1, δ2 and δ3 are any constant parameters between 0 and 1. The pixel matrix of the next frame of the multiple camera images is predicted to obtain the data information of the pixel matrix of the next frame of the predicted multiple camera images.

[0037] U33. Based on the pixel matrix data of the next frame of the predicted multi-camera images and the time series data of the pixel matrix with the highest similarity among the multi-camera images, an image frame fusion synchronization function H is established.

[0038] ,

[0039] Where g is the pixel matrix data of the next frame of the predicted multi-camera images, h is the time series data of the pixel matrix with the highest similarity among the multi-camera images, and μ1, μ2 and μ3 are the fusion factors of the image pixel matrix. The image data of the multi-cameras are processed synchronously to obtain the image data information of the multi-cameras after synchronous processing.

[0040] Furthermore, the fusion factors μ1, μ2, and μ3 of the image pixel matrix are,

[0041] ,

[0042] ,

[0043] ,

[0044] Where g represents the pixel matrix data of the next frame of the predicted multi-camera images, and h represents the time series data of the pixel matrix with the highest similarity among the multi-camera images.

[0045] To achieve the above and other related objectives, the present invention also provides a real-time and security-based GMSL2 camera data synchronization system, including a computer device programmed or configured to perform the steps of any of the real-time and security-based GMSL2 camera data synchronization methods described above.

[0046] To achieve the above and other related objectives, the present invention also provides a computer-readable storage medium storing a computer program programmed or configured to perform any of the GMSL2 camera data synchronization methods based on real-time and security as described above.

[0047] The present invention has the following positive effects:

[0048] 1. This invention uses an image similarity algorithm based on Spearman correlation coefficient to characterize the similarity of images acquired by multiple cameras, and combines it with a time-series-based frame synchronization algorithm to synchronize the image data of multiple cameras. This not only accurately processes the similarity of images from multiple cameras, ensuring the accuracy of synchronization of each frame, but also allows for comprehensive judgment by combining the image data of previous and subsequent frames during the synchronization process, thereby improving the accuracy of data synchronization.

[0049] 2. This invention constructs an image synchronization detection function Q to detect the synchronization results of images and resynchronizes images that do not meet the synchronization requirements. This not only solves the problem of data synchronization being protected from man-in-the-middle attacks or electromagnetic interference, ensuring the security of synchronized data, but also ensures the robustness of the entire process by dynamically adjusting parameters to guarantee the integrity of the data synchronization results. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0051] Figure 2 This is a flowchart illustrating the image similarity algorithm based on Spearman correlation coefficient of the present invention.

[0052] Figure 3 This is a flowchart illustrating the time-series-based frame synchronization algorithm of the present invention. Detailed Implementation

[0053] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0054] Example 1: As Figure 1 As shown, a GMSL2 camera data synchronization method based on real-time and security is described, the method comprising:

[0055] U1. When a vehicle is driving on the road, it acquires data information of images collected by multiple cameras in real time based on the vehicle-mounted multi-camera system, and timestamps the data information of images collected by multiple cameras after timestamping.

[0056] U2. Based on the data information of images acquired by multiple cameras after the timestamp, the similarity of images acquired by multiple cameras is characterized by an image similarity algorithm based on Spearman correlation coefficient, and the data information of the similarity of images acquired by multiple cameras is obtained.

[0057] U3. Based on the similarity data of the images acquired by the multiple cameras, a time-series-based frame synchronization algorithm is used to synchronize the image data of the multiple cameras to obtain the synchronized image data information of the multiple cameras;

[0058] U4. Based on the image data information of the multiple cameras after synchronization processing, construct an image synchronization detection function Q, detect the synchronization result of the image, and obtain the data information of the detection value of the image synchronization result.

[0059] In this embodiment, as Figure 2 As shown, in step U2, the method of characterizing the similarity of images acquired by multiple cameras using an image similarity algorithm based on Spearman correlation coefficient includes:

[0060] U21. Based on the data information of the images acquired by the multiple cameras after the timestamp, construct a sequence of pixel matrices of the images acquired by the multiple cameras to obtain the data information of the sequence of pixel matrices of the images acquired by the multiple cameras;

[0061] U22. Based on the sequence data information of the pixel matrix of the images acquired by the multiple cameras, construct the Spearman correlation function W of the image pixel matrix sequence.

[0062] ,

[0063] Among them, y i y is the data information of the i-th sequence of the pixel matrix of images acquired by multiple cameras. i+1 y is the data information of the (i+1)th sequence of the pixel matrix of images acquired by multiple cameras. i+2 This refers to the (i+2)th sequence of pixel matrix data from images acquired by multiple cameras, where m is a positive integer and β... j These are the weighting coefficients;

[0064] U23. Based on the Spearman correlation function W of the image pixel matrix sequence, the similarity of images acquired by multiple cameras is characterized to obtain data information on the similarity of images acquired by multiple cameras.

[0065] In this embodiment, the weighting coefficient β j The constraints are as follows:

[0066] ,

[0067] Where m is a positive integer.

[0068] In this embodiment, as Figure 3 As shown, in step U3, the synchronization processing of image data from multiple cameras using a time-series-based frame synchronization algorithm includes:

[0069] U31. Based on the similarity data of the images acquired by the multiple cameras, extract the image with the highest similarity in each frame and construct the time series data of the pixel matrix with the highest similarity among the images of the multiple cameras;

[0070] U32. Based on the time-series data of the pixel matrix with the highest similarity among the images from the multiple cameras, a prediction function G for the pixel matrix of the next frame image from the multiple cameras is established.

[0071] ,

[0072] Where z is the time series data information of the pixel matrix with the highest similarity among multiple camera images, and δ1, δ2 and δ3 are any constant parameters between 0 and 1. The pixel matrix of the next frame of the multiple camera images is predicted to obtain the data information of the pixel matrix of the next frame of the predicted multiple camera images.

[0073] U33. Based on the pixel matrix data of the next frame of the predicted multi-camera images and the time series data of the pixel matrix with the highest similarity among the multi-camera images, an image frame fusion synchronization function H is established.

[0074] ,

[0075] Where g is the pixel matrix data of the next frame of the predicted multi-camera images, h is the time series data of the pixel matrix with the highest similarity among the multi-camera images, and μ1, μ2 and μ3 are the fusion factors of the image pixel matrix. The image data of the multi-cameras are processed synchronously to obtain the image data information of the multi-cameras after synchronous processing.

[0076] In this embodiment, the fusion factors μ1, μ2, and μ3 of the image pixel matrix are,

[0077] ,

[0078] ,

[0079] ,

[0080] Where g represents the pixel matrix data of the next frame of the predicted multi-camera images, and h represents the time series data of the pixel matrix with the highest similarity among the multi-camera images.

[0081] Example 2: Based on the GMSL2 camera data synchronization method based on real-time and security in Example 1, the present invention will be further explained and described below.

[0082] like Figure 1 As shown, a GMSL2 camera data synchronization method based on real-time and security is described, the method comprising:

[0083] U1. When a vehicle is driving on the road, it acquires data information of images collected by multiple cameras in real time based on the vehicle-mounted multi-camera system, and timestamps the data information of images collected by multiple cameras after timestamping.

[0084] U2. Based on the data information of images acquired by multiple cameras after the timestamp, the similarity of images acquired by multiple cameras is characterized by an image similarity algorithm based on Spearman correlation coefficient, and the data information of the similarity of images acquired by multiple cameras is obtained.

[0085] U3. Based on the similarity data of the images acquired by the multiple cameras, a time-series-based frame synchronization algorithm is used to synchronize the image data of the multiple cameras to obtain the synchronized image data information of the multiple cameras;

[0086] U4. Based on the image data information of the multiple cameras after synchronization processing, construct an image synchronization detection function Q, detect the synchronization result of the image, and obtain the data information of the detection value of the image synchronization result.

[0087] In this embodiment, the synchronization detection function Q of the image is,

[0088] ,

[0089] Where, x i Let α1, α2, and α3 be the eigenvalues ​​of the pixel matrix of the i-th frame image in the image data information of multiple cameras after synchronous processing, where n is the sample size and α1, α2, and α3 are weighting factors.

[0090] In this embodiment, the weighting factors α1, α2, and α3 are,

[0091] ,

[0092] ,

[0093] ,

[0094] Where, x i Let be the eigenvalues ​​of the pixel matrix of the i-th frame image in the image data information of multiple cameras after synchronous processing, and n be the sample size.

[0095] In this embodiment, the method further includes:

[0096] U5. Based on the data information of the detection value of the synchronization result of the image, a preset threshold is set. If the detection value of the synchronization result of the image is less than the preset threshold, the requirement is not met, and the process returns to step U2. If the detection value of the synchronization result of the image is greater than the preset threshold, the requirement is met, and the data synchronization of multiple cameras is completed.

[0097] In this embodiment, the present invention provides a real-time and security-based GMSL2 camera data synchronization system, including a computer device programmed or configured to perform the steps of any of the GMSL2 camera data synchronization methods described above.

[0098] In this embodiment, the present invention provides a computer-readable storage medium storing a computer program programmed or configured to perform any of the GMSL2 camera data synchronization methods based on real-time and security as described above.

[0099] Any references to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0100] In summary, this invention not only accurately processes each frame of the camera image to ensure the integrity of data synchronization, but also ensures the security of the synchronized data by preventing man-in-the-middle attacks or electromagnetic interference during the data synchronization process.

[0101] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A GMSL2 camera data synchronization method based on real-time performance and security, characterized in that, The method includes: U1. When a vehicle is driving on the road, it acquires data information of images collected by multiple cameras in real time based on the vehicle-mounted multi-camera system, and timestamps the data information of images collected by multiple cameras after timestamping. U2. Based on the data information of images acquired by multiple cameras after the timestamp, the similarity of images acquired by multiple cameras is characterized by an image similarity algorithm based on Spearman correlation coefficient, and the data information of the similarity of images acquired by multiple cameras is obtained. U3. Based on the similarity data of the images acquired by the multiple cameras, a time-series-based frame synchronization algorithm is used to synchronize the image data of the multiple cameras to obtain the synchronized image data information of the multiple cameras; U4. Based on the image data information of the multiple cameras after synchronization processing, construct an image synchronization detection function Q, detect the synchronization result of the image, and obtain the data information of the detection value of the image synchronization result; The synchronous detection function Q of the image is, ， Where, x i Let α1, α2, and α3 be the eigenvalues ​​of the pixel matrix of the i-th frame image in the image data information of multiple cameras after synchronous processing, where n is the sample size and α1, α2, and α3 are weighting factors.

2. The GMSL2 camera data synchronization method based on real-time performance and security according to claim 1, characterized in that: The weighting factors α1, α2, and α3 are, ， ， ， Where, x i Let be the eigenvalues ​​of the pixel matrix of the i-th frame image in the image data information of multiple cameras after synchronous processing, and n be the sample size.

3. The GMSL2 camera data synchronization method based on real-time performance and security according to claim 1, characterized in that, The method further includes: U5. Based on the data information of the detection value of the synchronization result of the image, a preset threshold is set. If the detection value of the synchronization result of the image is less than the preset threshold, the requirement is not met, and the process returns to step U2. If the detection value of the synchronization result of the image is greater than the preset threshold, the requirement is met, and the data synchronization of multiple cameras is completed.

4. The GMSL2 camera data synchronization method based on real-time performance and security according to claim 1, characterized in that, In step U2, the process of characterizing the similarity of images acquired by multiple cameras using an image similarity algorithm based on Spearman correlation coefficient includes: U21. Based on the data information of the images acquired by the multiple cameras after the timestamp, construct a sequence of pixel matrices of the images acquired by the multiple cameras to obtain the data information of the sequence of pixel matrices of the images acquired by the multiple cameras; U22. Based on the sequence data information of the pixel matrix of the images acquired by the multiple cameras, construct the Spearman correlation function W of the image pixel matrix sequence. ， Among them, y i y is the data information of the i-th sequence of the pixel matrix of images acquired by multiple cameras. i+1 y is the data information of the (i+1)th sequence of the pixel matrix of images acquired by multiple cameras. i+2 This refers to the (i+2)th sequence of pixel matrix data from images acquired by multiple cameras, where m is a positive integer and β... j These are the weighting coefficients; U23. Based on the Spearman correlation function W of the image pixel matrix sequence, the similarity of images acquired by multiple cameras is characterized to obtain data information on the similarity of images acquired by multiple cameras.

5. The GMSL2 camera data synchronization method based on real-time performance and security according to claim 4, characterized in that: The weighting coefficient β j The constraints are as follows: ， Where m is a positive integer.

6. The GMSL2 camera data synchronization method based on real-time performance and security according to claim 1, characterized in that, In step U3, the synchronization processing of image data from multiple cameras using a time-series-based frame synchronization algorithm includes: U31. Based on the similarity data of the images acquired by the multiple cameras, extract the image with the highest similarity in each frame and construct the time series data of the pixel matrix with the highest similarity among the images of the multiple cameras; U32. Based on the time-series data of the pixel matrix with the highest similarity among the images from the multiple cameras, a prediction function G for the pixel matrix of the next frame image from the multiple cameras is established. ， Where z is the time series data information of the pixel matrix with the highest similarity among multiple camera images, and δ1, δ2 and δ3 are any constant parameters between 0 and 1. The pixel matrix of the next frame of the multiple camera images is predicted to obtain the data information of the pixel matrix of the next frame of the predicted multiple camera images. U33. Based on the pixel matrix data of the next frame of the predicted multi-camera images and the time series data of the pixel matrix with the highest similarity among the multi-camera images, an image frame fusion synchronization function H is established. ， Where g is the pixel matrix data of the next frame of the predicted multi-camera images, h is the time series data of the pixel matrix with the highest similarity among the multi-camera images, and μ1, μ2 and μ3 are the fusion factors of the image pixel matrix. The image data of the multi-cameras are processed synchronously to obtain the image data information of the multi-cameras after synchronous processing.

7. The GMSL2 camera data synchronization method based on real-time performance and security according to claim 6, characterized in that: The fusion factors μ1, μ2, and μ3 of the image pixel matrix are, ， ， ， Where g represents the pixel matrix data of the next frame of the predicted multi-camera images, and h represents the time series data of the pixel matrix with the highest similarity among the multi-camera images.

8. A GMSL2 camera data synchronization system based on real-time and security, comprising computer equipment, characterized in that, The computer device is programmed or configured to perform the steps of the GMSL2 camera data synchronization method based on real-time and security as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is programmed or configured to perform the GMSL2 camera data synchronization method based on real-time and security as described in any one of claims 1 to 7.

Images