GMSL2 camera data synchronization method and system based on real-time and security

Abstract

The invention relates to a GMSL2 camera data synchronization method and system based on real-time and safety, and the method comprises the steps: U1, enabling a vehicle to run on a road, obtaining the data information of images collected by a plurality of cameras in real time based on a vehicle-mounted multi-camera system, carrying out the marking of timestamps, obtaining data information of the images acquired by the plurality of cameras after the timestamps are marked; and U2, on the basis of the data information of the images acquired by the plurality of cameras after the timestamps are marked, representing the similarity of the images acquired by the plurality of cameras by adopting an image similarity algorithm based on a Spearman correlation coefficient to obtain the data information of the similarity of the images acquired by the plurality of cameras. According to the method, each frame of image of the camera can be accurately processed, the integrity of data synchronization is ensured, the data synchronization process is not attacked by a man-in-the-middle or electromagnetic interference, and the security of the synchronized data is ensured.

Description

Technical Field

[0001] The present invention relates to the field of visual synchronization technology, and in particular to a real-time and security-based GMSL2 camera data synchronization method and system. Background Art

[0002] With the rapid development of autonomous driving, industrial machine vision, and other fields, the synchronization and security of multi-camera systems have become core requirements. GMSL2 (Gigabit Multimedia Serial Link 2) is widely used in automotive and industrial camera systems due to its high bandwidth (6Gbps), long transmission distance (15-20 meters), and anti-interference capabilities. However, existing solutions have bottlenecks in the following areas: Real-time: Traditional timing solutions rely on the gPTP network protocol (accuracy <1ms) or GPS PPS signals, which may lead to network delays or signal obstruction risks.

[0003] Security: Timing data is vulnerable to man-in-the-middle attacks or electromagnetic interference, which can lead to synchronization failure or data tampering.

[0004] Dynamic adaptability: In hot-swappable camera scenarios, existing solutions are difficult to quickly resynchronize and lack a link health monitoring mechanism. Summary of the Invention

[0005] In view of the above problems, the present invention provides a real-time and secure GMSL2 camera data synchronization method and system, which can not only accurately process each frame of the camera's image to ensure the integrity of data synchronization, but also the data synchronization process is not subject to man-in-the-middle attacks or electromagnetic interference, ensuring the security of the synchronized data.

[0006] In order to achieve the above-mentioned and other related purposes, the present invention provides the following technical solutions: A GMSL2 camera data synchronization method based on real-time and security, the method comprising: U1. A vehicle is traveling on a road. The vehicle's multi-camera system acquires image data from multiple cameras in real time and timestamps the images. The timestamped images are then obtained. U2. Based on the data information of the images captured by the multiple cameras after the timestamp mark, the similarity of the images captured by the multiple cameras is characterized by an image similarity algorithm based on the Spearman correlation coefficient to obtain data information of the similarity of the images captured by the multiple cameras; U3 based on the similarity of the data information of the images collected by the multiple cameras, the image data of the multiple cameras are synchronized using a time series-based frame synchronization algorithm to obtain the image data information of the multiple cameras after synchronization; U4. Based on the image data information of the multiple cameras after the synchronization processing, construct an image synchronization detection function Q, detect the image synchronization result, and obtain data information of the detection value of the image synchronization result.

[0007] Furthermore, the synchronization detection function Q of the image is, , Among them, x i is the eigenvalue of the pixel matrix of the i-th frame image in the image data information of multiple cameras after synchronous processing, n is the sample capacity, and α1, α2 and α3 are weight factors.

[0008] Furthermore, the weight factors α1, α2 and α3 are, , , , Among them, x i is the eigenvalue of the pixel matrix of the i-th frame image in the image data information of multiple cameras after synchronous processing, and n is the sample size.

[0009] Furthermore, the method further comprises: U5. Based on the data information of the detection value of the synchronization result of the image, a preset threshold is set. If the detection value of the synchronization result of the image is less than the preset threshold, the requirement is not met, and the process returns to step U2. If the detection value of the synchronization result of the image is greater than the preset threshold, the requirement is met, and the data synchronization of multiple cameras is completed.

[0010] Furthermore, in step U2, the use of an image similarity algorithm based on the Spearman correlation coefficient to characterize the similarity of images captured by multiple cameras includes: U21. Based on the data information of the images collected by multiple cameras after the timestamp mark, a sequence of pixel matrices of images collected by multiple cameras is constructed to obtain data information of a sequence of pixel matrices of images collected by multiple cameras; U22. Based on the data information of the sequence of pixel matrices of the images collected by the multiple cameras, construct a Spearman correlation function W of the image pixel matrix sequence, , Among them, y i The data information of the i-th sequence of the pixel matrix of the image collected by multiple cameras, y i+1 The data information of the i+1th sequence of the pixel matrix of the image collected by multiple cameras, y i+2 is the data information of the i+2th sequence of the pixel matrix of the image collected by multiple cameras, m is a positive integer, βj is the weight coefficient; U23. Based on the Spearman correlation function W of the image pixel matrix sequence, the similarity of the images captured by multiple cameras is characterized to obtain data information on the similarity of the images captured by multiple cameras.

[0011] Furthermore, the weight coefficient β j The constraints are, , Wherein, m is a positive integer.

[0012] Furthermore, in step U3, the synchronous processing of the image data of multiple cameras using a time series-based frame synchronization algorithm includes: U31 based on the data information of the similarity of the images collected by the multiple cameras, extract the image with the highest similarity in each frame, and construct a time series data information of the pixel matrix with the highest similarity of multiple camera images; U32. Based on the time series data information of the pixel matrix with the highest similarity among the multiple camera images, a prediction function G of the pixel matrix of the next frame image of multiple cameras is established. , Where z is the time series data information of the pixel matrix with the highest similarity among multiple camera images, δ1, δ2, and δ3 are any constant parameters between 0 and 1, and the pixel matrix of the next frame of the multiple camera images is predicted to obtain the predicted pixel matrix data information of the next frame of the multiple camera images; U33. Based on the data information of the pixel matrix of the next frame of the predicted multiple camera images and the time series data information of the pixel matrix with the highest similarity among the multiple camera images, establish an image frame fusion synchronization function H. , Among them, g is the data information of the pixel matrix of the next frame of the predicted multiple camera images, h is the time series data information of the pixel matrix with the highest similarity among the multiple camera images, μ1, μ2 and μ3 are the fusion factors of the image pixel matrix, and the image data of multiple cameras are synchronously processed to obtain the image data information of multiple cameras after synchronous processing.

[0013] Furthermore, the fusion factors μ1, μ2 and μ3 of the image pixel matrix are: , , , Among them, g is the data information of the pixel matrix of the next frame of the predicted multiple camera images, and h is the time series data information of the pixel matrix with the highest similarity among the multiple camera images.

[0014] In order to achieve the above-mentioned and other related objectives, the present invention also provides a GMSL2 camera data synchronization system based on real-time and security, including a computer device programmed or configured to perform any one of the steps of the GMSL2 camera data synchronization method based on real-time and security.

[0015] In order to achieve the above objectives and other related objectives, the present invention also provides a computer-readable storage medium, which stores a computer program programmed or configured to execute any one of the real-time and security-based GMSL2 camera data synchronization methods.

[0016] The present invention has the following positive effects: 1. The present invention characterizes the similarity of images captured by multiple cameras by using an image similarity algorithm based on the Spearman correlation coefficient, and synchronizes the image data of multiple cameras by using a frame synchronization algorithm based on time series. This not only accurately processes the similarity of images from multiple cameras to ensure the accuracy of synchronization of each frame of the image, but also combines the image data of the previous and next frames for comprehensive judgment during the synchronization process, thereby improving the accuracy of data synchronization.

[0017] 2. The present invention constructs an image synchronization detection function Q to detect the image synchronization results and resynchronize the images that do not meet the synchronization requirements. This not only protects the data synchronization process from man-in-the-middle attacks or electromagnetic interference, thereby ensuring the security of the synchronized data, but also makes the entire process highly robust and dynamically adjusts parameters to ensure the integrity of the data synchronization results. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the method flow of the present invention; Figure 2 Schematic diagram of the process of the image similarity algorithm based on the Spearman correlation coefficient of the present invention; Figure 3 Schematic diagram of the flow of the time series-based frame synchronization algorithm of the present invention. DETAILED DESCRIPTION

[0019] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0020] Example 1: Figure 1 As shown, a GMSL2 camera data synchronization method based on real-time and security, the method comprising: U1. A vehicle is traveling on a road. The vehicle's multi-camera system acquires image data from multiple cameras in real time and timestamps the images. The timestamped images are then obtained. U2. Based on the data information of the images captured by the multiple cameras after the timestamp mark, the similarity of the images captured by the multiple cameras is characterized by an image similarity algorithm based on the Spearman correlation coefficient to obtain data information of the similarity of the images captured by the multiple cameras; U3 based on the similarity of the data information of the images collected by the multiple cameras, the image data of the multiple cameras are synchronized using a time series-based frame synchronization algorithm to obtain the image data information of the multiple cameras after synchronization; U4. Based on the image data information of the multiple cameras after the synchronization processing, construct an image synchronization detection function Q, detect the image synchronization result, and obtain data information of the detection value of the image synchronization result.

[0021] In this embodiment, if Figure 2 As shown, in step U2, the use of the image similarity algorithm based on the Spearman correlation coefficient to characterize the similarity of images collected by multiple cameras includes: U21. Based on the data information of the images collected by multiple cameras after the timestamp mark, a sequence of pixel matrices of images collected by multiple cameras is constructed to obtain data information of a sequence of pixel matrices of images collected by multiple cameras; U22. Based on the data information of the sequence of pixel matrices of the images collected by the multiple cameras, construct a Spearman correlation function W of the image pixel matrix sequence, , Among them, y i The data information of the i-th sequence of the pixel matrix of the image collected by multiple cameras, y i+1 The data information of the i+1th sequence of the pixel matrix of the image collected by multiple cameras, y i+2is the data information of the i+2th sequence of the pixel matrix of the image collected by multiple cameras, m is a positive integer, β j is the weight coefficient; U23. Based on the Spearman correlation function W of the image pixel matrix sequence, the similarity of the images captured by multiple cameras is characterized to obtain data information on the similarity of the images captured by multiple cameras.

[0022] In this embodiment, the weight coefficient β j The constraints are, , Wherein, m is a positive integer.

[0023] In this embodiment, if Figure 3 As shown, in step U3, the synchronous processing of the image data of multiple cameras using the time series-based frame synchronization algorithm includes: U31 based on the data information of the similarity of the images collected by the multiple cameras, extract the image with the highest similarity in each frame, and construct a time series data information of the pixel matrix with the highest similarity of multiple camera images; U32. Based on the time series data information of the pixel matrix with the highest similarity among the multiple camera images, a prediction function G of the pixel matrix of the next frame image of multiple cameras is established. , Where z is the time series data information of the pixel matrix with the highest similarity among multiple camera images, δ1, δ2, and δ3 are any constant parameters between 0 and 1, and the pixel matrix of the next frame of the multiple camera images is predicted to obtain the predicted pixel matrix data information of the next frame of the multiple camera images; U33. Based on the data information of the pixel matrix of the next frame of the predicted multiple camera images and the time series data information of the pixel matrix with the highest similarity among the multiple camera images, establish an image frame fusion synchronization function H. , Among them, g is the data information of the pixel matrix of the next frame of the predicted multiple camera images, h is the time series data information of the pixel matrix with the highest similarity among the multiple camera images, μ1, μ2 and μ3 are the fusion factors of the image pixel matrix, and the image data of multiple cameras are synchronously processed to obtain the image data information of multiple cameras after synchronous processing.

[0024] In this embodiment, the fusion factors μ1, μ2 and μ3 of the image pixel matrix are: , , , Among them, g is the data information of the pixel matrix of the next frame of the predicted multiple camera images, and h is the time series data information of the pixel matrix with the highest similarity among the multiple camera images.

[0025] Example 2: Based on the real-time and security-based GMSL2 camera data synchronization method in Example 1, the present invention is further illustrated and described below.

[0026] like Figure 1 As shown, a GMSL2 camera data synchronization method based on real-time and security, the method comprising: U1. A vehicle is traveling on a road. The vehicle's multi-camera system acquires image data from multiple cameras in real time and timestamps the images. The timestamped images are then obtained. U2. Based on the data information of the images captured by the multiple cameras after the timestamp mark, the similarity of the images captured by the multiple cameras is characterized by an image similarity algorithm based on the Spearman correlation coefficient to obtain data information of the similarity of the images captured by the multiple cameras; U3 based on the similarity of the data information of the images collected by the multiple cameras, the image data of the multiple cameras are synchronized using a time series-based frame synchronization algorithm to obtain the image data information of the multiple cameras after synchronization; U4. Based on the image data information of the multiple cameras after the synchronization processing, construct an image synchronization detection function Q, detect the image synchronization result, and obtain data information of the detection value of the image synchronization result.

[0027] In this embodiment, the synchronization detection function Q of the image is: , Among them, x i is the eigenvalue of the pixel matrix of the i-th frame image in the image data information of multiple cameras after synchronous processing, n is the sample capacity, and α1, α2 and α3 are weight factors.

[0028] In this embodiment, the weight factors α1, α2 and α3 are, , , , Among them, x i is the eigenvalue of the pixel matrix of the i-th frame image in the image data information of multiple cameras after synchronous processing, and n is the sample size.

[0029] In this embodiment, the method further includes: U5. Based on the data information of the detection value of the synchronization result of the image, a preset threshold is set. If the detection value of the synchronization result of the image is less than the preset threshold, the requirement is not met, and the process returns to step U2. If the detection value of the synchronization result of the image is greater than the preset threshold, the requirement is met, and the data synchronization of multiple cameras is completed.

[0030] In this embodiment, the present invention provides a real-time and security-based GMSL2 camera data synchronization system, including a computer device programmed or configured to perform any one of the steps of the real-time and security-based GMSL2 camera data synchronization method.

[0031] In this embodiment, the present invention provides a computer-readable storage medium storing a computer program programmed or configured to execute any one of the above-described methods for GMSL2 camera data synchronization based on real-time and security.

[0032] Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0033] In summary, the present invention can not only accurately process each frame of the camera's image to ensure the integrity of data synchronization, but also the data synchronization process is not subject to man-in-the-middle attacks or electromagnetic interference, thereby ensuring the security of the synchronized data.

[0034] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A GMSL2 camera data synchronization method based on real-time and security, characterized in that: The method comprises: U1. A vehicle is traveling on a road. The vehicle's multi-camera system acquires image data from multiple cameras in real time and timestamps the images. The timestamped images are then obtained. U2. Based on the data information of the images captured by the multiple cameras after the timestamp mark, the similarity of the images captured by the multiple cameras is characterized by an image similarity algorithm based on the Spearman correlation coefficient to obtain data information of the similarity of the images captured by the multiple cameras; U3 based on the similarity of the data information of the images collected by the multiple cameras, the image data of the multiple cameras are synchronized using a time series-based frame synchronization algorithm to obtain the image data information of the multiple cameras after synchronization; U4. Based on the image data information of the multiple cameras after the synchronization processing, construct an image synchronization detection function Q, detect the image synchronization result, and obtain data information of the detection value of the image synchronization result.

2. The GMSL2 camera data synchronization method based on real-time and security according to claim 1, characterized in that: The synchronization detection function Q of the image is, ， Among them, x i is the eigenvalue of the pixel matrix of the i-th frame image in the image data information of multiple cameras after synchronous processing, n is the sample capacity, and α1, α2 and α3 are weight factors.

3. The GMSL2 camera data synchronization method based on real-time and security according to claim 2, characterized in that: The weight factors α1, α2 and α3 are, ， ， ， Among them, x i is the eigenvalue of the pixel matrix of the i-th frame image in the image data information of multiple cameras after synchronous processing, and n is the sample size.

4. The GMSL2 camera data synchronization method based on real-time and security according to claim 1, characterized in that: The method further comprises: U5. Based on the data information of the detection value of the synchronization result of the image, a preset threshold is set. If the detection value of the synchronization result of the image is less than the preset threshold, the requirement is not met, and the process returns to step U2. If the detection value of the synchronization result of the image is greater than the preset threshold, the requirement is met, and the data synchronization of multiple cameras is completed.

5. The GMSL2 camera data synchronization method based on real-time and security according to claim 1, characterized in that: In step U2, the use of an image similarity algorithm based on the Spearman correlation coefficient to characterize the similarity of images captured by multiple cameras includes: U21. Based on the data information of the images collected by multiple cameras after the timestamp mark, a sequence of pixel matrices of images collected by multiple cameras is constructed to obtain data information of a sequence of pixel matrices of images collected by multiple cameras; U22. Based on the data information of the sequence of pixel matrices of the images collected by the multiple cameras, construct a Spearman correlation function W of the image pixel matrix sequence, ， Among them, y i The data information of the i-th sequence of the pixel matrix of the image collected by multiple cameras, y i+1 The data information of the i+1th sequence of the pixel matrix of the image collected by multiple cameras, y i+2 is the data information of the i+2th sequence of the pixel matrix of the image collected by multiple cameras, m is a positive integer, β j is the weight coefficient; U23. Based on the Spearman correlation function W of the image pixel matrix sequence, the similarity of the images captured by multiple cameras is characterized to obtain data information on the similarity of the images captured by multiple cameras.

6. The GMSL2 camera data synchronization method based on real-time and security according to claim 5, characterized in that: The weight coefficient β j The constraints are, ， Wherein, m is a positive integer.

7. The GMSL2 camera data synchronization method based on real-time and security according to claim 1, characterized in that: In step U3, the synchronous processing of the image data of multiple cameras using a time series-based frame synchronization algorithm includes: U31 based on the data information of the similarity of the images collected by the multiple cameras, extract the image with the highest similarity in each frame, and construct a time series data information of the pixel matrix with the highest similarity of multiple camera images; U32. Based on the time series data information of the pixel matrix with the highest similarity among the multiple camera images, a prediction function G of the pixel matrix of the next frame image of the multiple cameras is established. ， Where z is the time series data information of the pixel matrix with the highest similarity among multiple camera images, δ1, δ2, and δ3 are any constant parameters between 0 and 1, and the pixel matrix of the next frame of the multiple camera images is predicted to obtain the predicted pixel matrix data information of the next frame of the multiple camera images; U33. Based on the data information of the pixel matrix of the next frame of the predicted multiple camera images and the time series data information of the pixel matrix with the highest similarity among the multiple camera images, establish an image frame fusion synchronization function H. ， Among them, g is the data information of the pixel matrix of the next frame of the predicted multiple camera images, h is the time series data information of the pixel matrix with the highest similarity among the multiple camera images, μ1, μ2 and μ3 are the fusion factors of the image pixel matrix, and the image data of multiple cameras are synchronously processed to obtain the image data information of multiple cameras after synchronous processing.

8. The real-time and security-based GMSL2 camera data synchronization method according to claim 7, characterized in that: The fusion factors μ1, μ2 and μ3 of the image pixel matrix are: ， ， ， Among them, g is the data information of the pixel matrix of the next frame of the predicted multiple camera images, and h is the time series data information of the pixel matrix with the highest similarity among the multiple camera images.

9. A GMSL2 camera data synchronization system based on real-time and security, comprising a computer device, characterized in that: The computer device is programmed or configured to execute the steps of the real-time and security-based GMSL2 camera data synchronization method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program programmed or configured to execute the real-time and security-based GMSL2 camera data synchronization method according to any one of claims 1 to 8.

Images