Automatic Time Alignment Method and Application of Target Association-Based Radar and Image Data

Abstract

This invention belongs to the field of data processing technology and provides a method and application for automatic time alignment of radar and image data based on target association. The method includes: creating a time search window on the time axis of the image data stream for each frame of radar data in the radar data stream; projecting each radar target in the radar dynamic target list onto the image coordinate system; for each frame of image within the time search window, counting the number of projection points falling into the image detection box to generate a target association score; selecting the radar frame and image frame with the highest target association score as a synchronization frame pair; calculating the timestamp offset based on the timestamp of the synchronization frame pair; and selecting a corresponding time alignment strategy to automatically align the radar data stream and image data stream based on the statistical stability of the calculated timestamp offsets. This invention can solve the problem of timestamp asynchrony between independently recorded millimeter-wave radar point cloud data and camera image data; and can achieve automatic offline time alignment.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and application for automatic time alignment of radar and image data based on target association. Background Technology

[0002] In real-world testing of autonomous vehicles, millimeter-wave radar and cameras typically acquire and record data independently. Due to internal processing delays of each sensor, response times of the data acquisition card, and slight differences in system clocks, an unknown timestamp offset exists between the two data streams during data playback and analysis. This millisecond-level asynchrony can cause a sharp decline in the performance of fusion algorithms when the target is moving at high speed, resulting in ghosting or misjudgments.

[0003] Currently, the mainstream time synchronization method in the industry is hardware synchronization, such as using the PTP precision clock protocol to provide a unified clock source for all sensors, or using hardware trigger cables to ensure instantaneous synchronization during data acquisition. However, this hardware synchronization has the following drawbacks:

[0004] 1. High cost: It requires sensors to support specific hardware interfaces and protocols, and additional synchronization devices are needed;

[0005] 2. Poor flexibility: The system wiring is complex, which is not conducive to later installation or modification of test vehicles;

[0006] 3. Unable to repair historical data: Hardware synchronization solutions are powerless to recover data that has already been recorded independently.

[0007] Based on this, relevant technical personnel in the industry have developed some time synchronization methods based on software algorithms. However, the time synchronization accuracy and flexibility of these methods are still not high enough. For example, patent CN113965289B discloses a time synchronization method and device based on multi-sensor data. The method includes: classifying the acquired sensor data according to the sensor type to obtain different categories of sensor data, and each category of sensor data comes from at least one sensor; and performing time synchronization processing on each category of sensor data using a buffer pool. This method improves the efficiency of data time synchronization processing by classifying the sensor data to facilitate time synchronization processing on the sensor data corresponding to each type of sensor. Furthermore, synchronizing sensor data on a separate basis reduces the amount of data that needs to be synchronized, improves the accuracy of data time synchronization, and avoids situations where data interruption and time asynchrony occur due to slow sensor startup or mid-process failure. However, this method relies on the sensor's own timestamp and the system's receiving time, and sorts and interpolates the data in the buffer pool by time. This is a time-related synchronization, and when the sensor clock has unknown drift, initialization deviation, or brief anomalies, the synchronization result of this classification buffer may be incorrect. Furthermore, this method cannot achieve millisecond-level time alignment accuracy. Summary of the Invention

[0008] In view of the shortcomings of the prior art, the present invention provides a method and application for automatic time alignment of radar and image data based on target association, which can solve the problem of time stamp asynchrony between independently recorded millimeter-wave radar point cloud data and camera image data; and does not rely on dedicated hardware, can be used in the data playback stage, and can achieve automatic offline time alignment.

[0009] To achieve the above and related objectives, the present invention adopts the following technical solution:

[0010] The first aspect of this invention provides a method for automatic temporal alignment of radar and image data based on target association, comprising the following steps:

[0011] Step S100: Acquire radar data stream and image data stream respectively, and create a time search window on the time axis of image data stream for each frame of radar data in radar data stream;

[0012] Step S200: Preprocess the radar data stream and image data stream respectively to obtain the radar dynamic target list and image detection box; project each radar target in the radar dynamic target list onto the image coordinate system; for each frame of image within the time search window, count the number of projection points falling into the image detection box and generate a target correlation score.

[0013] Step S300: Select the radar frame and image frame with the highest target correlation score as the synchronization frame pair, and calculate the timestamp offset based on the timestamp of the synchronization frame pair.

[0014] Step S400: Based on the statistical stability of the calculated multiple timestamp offsets, select the corresponding time alignment strategy to automatically align the radar data stream and image data stream in time. The time alignment strategy includes an overall translation strategy and an interpolation generation strategy.

[0015] Furthermore, step S100 also includes: taking the timestamp of the current frame radar data as the center, and extending forward and backward on the time axis of the image data stream with a preset maximum allowable delay time as the extension value, to obtain a time search window.

[0016] Further, in step S200, the radar data stream and image data stream are preprocessed separately to obtain the radar dynamic target list and image detection boxes, including:

[0017] Point cloud clustering is performed on the radar data stream to obtain multiple target point clusters, and their positions in the vehicle coordinate system are calculated to obtain a dynamic radar target list.

[0018] The target detection algorithm is run on the image data stream to identify dynamic targets, obtain their bounding boxes, and obtain the image detection boxes.

[0019] Further, in step S300, the timestamp offset includes:

[0020] (Formula 1),

[0021] In formula 1, Indicates the timestamp offset; Indicates the timestamp of the current frame of radar data; This represents the timestamp of the synchronization frame pair.

[0022] Furthermore, in step S400, the statistical stability based on the calculated multiple timestamp offsets includes:

[0023] Select the timestamp offset corresponding to a preset proportion of radar data frames from the radar data stream and calculate its variance; by comparing the variance with the preset stability threshold, determine whether the time offset is constant.

[0024] Furthermore, in step S400, the overall translation strategy includes: when the time offset is constant, adding the timestamp offset to all timestamps of the image data stream to complete automatic time alignment.

[0025] Furthermore, in step S400, the interpolation generation strategy includes: when the time offset is time-varying, taking the timestamp of the radar data stream as a reference, and generating a time-aligned virtual image frame for each radar data frame according to the timestamp offset corresponding to each radar data frame through an interpolation algorithm.

[0026] A second aspect of the present invention provides an automatic time alignment system for radar and image data based on target association, comprising:

[0027] The acquisition and creation module is used to acquire radar data stream and image data stream respectively, and to create a time search window on the time axis of image data stream for each frame of radar data in radar data stream;

[0028] The target correlation calculation module is used to preprocess the radar data stream and image data stream respectively to obtain the radar dynamic target list and image detection box; project each radar target in the radar dynamic target list onto the image coordinate system; for each frame of image within the time search window, count the number of projection points falling into the image detection box and generate the target correlation score.

[0029] The offset calculation module is used to select the radar frame and image frame with the highest target correlation score as a synchronization frame pair, and calculate the timestamp offset based on the timestamp of the synchronization frame pair.

[0030] The time alignment execution module is used to automatically align the radar data stream and image data stream by selecting the corresponding time alignment strategy based on the statistical stability of multiple timestamp offsets obtained from the calculation. The time alignment strategy includes an overall translation strategy and an interpolation generation strategy.

[0031] A third aspect of the present invention provides a computer-readable storage medium having computer-readable instructions stored thereon, which, when executed by a computer processor, cause the computer to execute the aforementioned automatic time alignment method for radar and image data based on target association.

[0032] A fourth aspect of the present invention provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described automatic time alignment method for radar and image data based on target association.

[0033] The beneficial technical effects of this invention are as follows:

[0034] This invention is based on dynamic target projection matching, which finds the frame with the best matching scene content within a time search window, thereby improving the accuracy and anti-interference of time alignment. It calculates the timestamp offset based on the timestamps of synchronized frame pairs, which can maximize scene consistency and ensure the reliability of the time alignment benchmark. It selects the corresponding time alignment strategy based on stability judgment, which can achieve scene adaptation. The overall translation strategy is suitable for stable scenes, while the interpolation generation strategy is suitable for time-varying scenes, thus covering different hardware / environment requirements and improving the versatility of the method.

[0035] This invention can achieve millisecond-level time alignment accuracy, meeting the fusion requirements in high-speed scenarios; this invention does not require additional hardware synchronization equipment, and can be implemented using existing sensors and computing resources, which has the advantage of low cost.

[0036] This invention is highly flexible and particularly suitable for post-processing and analysis of test data. It can repair data collected due to synchronization problems. Moreover, this invention requires no manual intervention and can automatically complete the entire process from data association to timestamp compensation, achieving full automation.

[0037] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0038] The accompanying drawings, incorporated in and forming part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without inventive effort. In the drawings:

[0039] Figure 1 This is a flowchart of the automatic temporal alignment method for radar and image data based on target association, as described in this application.

[0040] Figure 2 Here is a flowchart of another exemplary automatic time alignment method of this application;

[0041] Figure 3 This is a framework diagram of the target association-based automatic temporal alignment system for radar and image data in this application.

[0042] Figure 4 A schematic diagram of the structure of a computer system suitable for an embodiment of this application is shown. Detailed Implementation

[0043] Unless otherwise defined, all technical and / or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should be understood that certain features of the invention (described in the context of separate embodiments for clarity) may also be provided in a single embodiment. Conversely, multiple features of the invention (described in the context of a single embodiment for brevity) may also be provided separately or in any suitable combination or, where appropriate, in any other described embodiment of the invention. Certain features described in the context of various embodiments will not be considered essential features of those embodiments unless the embodiment is inoperable without those elements. The invention is further illustrated below by specific examples; however, it should be noted that the specific process conditions and results described in the embodiments of the invention are merely illustrative and should not be construed as limiting the scope of protection of the invention. All equivalent changes or modifications made in accordance with the spirit and essence of the invention should be covered within the scope of protection of the invention.

[0044] Please see Figure 1 The flowchart of the automatic temporal alignment method for radar and image data based on target association in this application is described in detail below:

[0045] Step S100: Acquire radar data stream and image data stream respectively, and create a time search window on the time axis of image data stream for each frame of radar data in radar data stream.

[0046] Specifically, in combination Figure 2 This application first acquires independently recorded raw radar data streams and camera data streams, and then performs data parsing and feature extraction on each data stream to obtain radar data streams and image data streams. Next, using the timestamp of the current frame of radar data as the center, this application expands forward and backward along the time axis of the image data stream, using a preset maximum allowable delay time as the expansion value, to obtain a time search window. The preset maximum allowable delay time is estimated and set based on system conditions, for example, 200 milliseconds. The expression for the time search window in this application is: ,in, Indicates the timestamp of the current frame of radar data. This indicates the preset maximum allowable delay time.

[0047] Specifically, this application can narrow the search scope and improve efficiency by creating a time search window.

[0048] Step S200: Preprocess the radar data stream and image data stream respectively to obtain a radar dynamic target list and image detection boxes; project each radar target in the radar dynamic target list onto the image coordinate system; for each frame of image within the time search window, count the number of projection points falling into the image detection boxes to generate a target correlation score.

[0049] Specifically, in combination Figure 2 This application first performs point cloud clustering on the radar data stream to obtain multiple target point clusters and calculates their positions in vehicle coordinates to obtain a radar dynamic target list. Each target point cluster represents a dynamic target, such as a vehicle or pedestrian. A target detection algorithm is then run on the image data stream to identify the dynamic targets and obtain their bounding boxes, resulting in image detection boxes. Next, using initial calibration parameters (known inputs from factory or offline calibration), this application projects each radar target in the radar dynamic target list onto the image coordinate system. It automatically finds radar-image target pairs whose projection points fall within the image detection boxes and counts the number of successfully matched target pairs, which is the target correlation score between the image data frame and the current radar data frame. In this way, the target correlation score quantifies the scene consistency between the radar frame and the image frame. A higher score indicates a greater likelihood that the radar frame and the image frame are seeing the same group of objects, i.e., a greater likelihood of temporal synchronization.

[0050] Step S300: Select the radar frame and image frame with the highest target correlation score as a synchronization frame pair, and calculate the timestamp offset based on the timestamp of the synchronization frame pair.

[0051] Specifically, this application determines that the synchronization frame pair with the highest target correlation score has high scene consistency, meaning that the radar frame and the image frame see the same scene. Then, the timestamp offset of this best-matching synchronization frame pair is directly calculated, as follows:

[0052] (Formula 1),

[0053] In formula 1, Indicates the timestamp offset; Indicates the timestamp of the current frame of radar data; This represents the timestamp of the synchronization frame pair.

[0054] Step S400: Based on the statistical stability of the calculated multiple timestamp offsets, select the corresponding time alignment strategy to automatically align the radar data stream and image data stream in time. The time alignment strategy includes an overall translation strategy and an interpolation generation strategy.

[0055] Specifically, the statistical stability of the calculated timestamp offsets in this application includes: selecting a preset proportion of radar data frames from the radar data stream corresponding to the timestamp offsets and calculating their variance; determining whether the time offset is constant by comparing the variance with a preset stability threshold. The preset proportion can be the first 1 / 10 of the radar data stream, and the preset stability threshold is set according to actual needs. When the variance α ≥ the preset stability threshold β, the time offset is considered constant; when the variance α < the preset stability threshold β, the time offset is considered time-varying.

[0056] Specifically, the overall translation strategy of this application includes: when the time offset is constant, adding the timestamp offset to all timestamps of the image data stream to achieve automatic time alignment and efficiently complete the time axis correction of the entire data stream.

[0057] Specifically, the interpolation generation strategy of this application includes: when the time offset is time-varying, taking the timestamp of the radar data stream as a reference, and based on the timestamp offset corresponding to each radar data frame and the timestamp of adjacent radar data frames, generating a time-aligned virtual image frame for each radar data frame through an interpolation algorithm, so as to dynamically and finely compensate for the time-varying timestamp deviation.

[0058] Optionally, when calculating the timestamp offset, this application can integrate multiple matching results and find the most likely offset through statistical histograms or clustering to resist interference. The interpolation algorithm of this application can be linear interpolation or motion model interpolation.

[0059] Optionally, the object detection algorithm in this application can be any existing model, such as YOLO, SSD, Faster R-CNN, etc.

[0060] Optionally, in addition to associating dynamic targets, this application can also use static targets such as road edges and traffic signs for association. These static targets remain in place for a short period of time. By accurately matching the point cloud of static targets detected by radar with the image features of static targets identified by the camera, it can be determined whether two frames of data depict the same scene, and thus infer whether the time is aligned. At the same time, associating static targets also requires more accurate positioning and map information.

[0061] Please see Figure 3 This is a framework diagram of the radar and image data time automatic alignment system 300 based on target association, as described in this application, including:

[0062] The acquisition and creation module 310 is used to acquire radar data stream and image data stream respectively, and to create a time search window on the time axis of image data stream for each frame of radar data in radar data stream;

[0063] The target correlation calculation module 320 is used to preprocess the radar data stream and image data stream respectively to obtain the radar dynamic target list and image detection box; project each radar target in the radar dynamic target list onto the image coordinate system; for each frame of image in the time search window, count the number of projection points falling into the image detection box and generate the target correlation score.

[0064] The offset calculation module 330 is used to select the radar frame and image frame with the highest target correlation score as a synchronization frame pair, and calculate the timestamp offset based on the timestamp of the synchronization frame pair.

[0065] The time alignment execution module 340 is used to automatically align the radar data stream and image data stream by selecting the corresponding time alignment strategy based on the statistical stability of multiple timestamp offsets calculated from the calculation. The time alignment strategy includes an overall translation strategy and an interpolation generation strategy.

[0066] Optionally, the method and system of this application are also applicable to data time alignment between LiDAR and cameras, and between different cameras. The method of this application can be used not only for offline processing, but also integrated into online systems as a supplementary verification and fine-tuning channel for hardware synchronization.

[0067] It should be noted that the target-association-based automatic time alignment system for radar and image data provided in the above embodiments and the target-association-based automatic time alignment method for radar and image data provided in the above embodiments belong to the same concept. The specific methods by which each module and unit performs operations have been described in detail in the method embodiments and will not be repeated here. In practical applications, the target-association-based automatic time alignment system for radar and image data provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0068] Embodiments of this application also provide a computer device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the computer device to implement the target association-based automatic time alignment method for radar and image data provided in the above embodiments.

[0069] Figure 4 A schematic diagram of the structure of a computer system suitable for an embodiment of this application is shown. It should be noted that... Figure 4 The computer system 400 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0070] like Figure 4 As shown, the computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage section 408 into a random access memory (RAM) 403, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in the RAM 403. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404. The following components are connected to the I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (local area network) card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A driver 410 is also connected to the I / O interface 405 as needed. Removable media 411, such as disks, optical discs, magneto-optical discs, semiconductor memories, etc., are installed on drive 410 as needed so that computer programs read from them can be installed into storage section 408 as needed.

[0071] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer tool programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs various functions defined in the system of this application.

[0072] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, flash memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. Computer programs contained on computer-readable media can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0073] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0074] The units described in the embodiments of this application can be implemented by tools or by hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the unit itself.

[0075] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a computer's processor, causes the computer to perform the target association-based automatic time alignment method for radar and image data as described above. This computer-readable storage medium may be included in the computer device described in the above embodiments, or it may exist independently and not incorporated into the computer device.

[0076] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the target-association-based automatic time alignment method for radar and image data provided in the various embodiments described above.

[0077] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for automatic temporal alignment of radar and image data based on target association, characterized in that, Includes the following steps: Step S100: Acquire radar data stream and image data stream respectively, and create a time search window on the time axis of image data stream for each frame of radar data in radar data stream; Step S200: Preprocess the radar data stream and the image data stream respectively to obtain a radar dynamic target list and an image detection box; project each radar target in the radar dynamic target list onto the image coordinate system; for each frame of the image within the time search window, count the number of projection points falling into the image detection box to generate a target correlation score. Step S300: Select the radar frame and image frame with the highest target correlation score as a synchronization frame pair, and calculate the timestamp offset based on the timestamp of the synchronization frame pair; Step S400: Based on the statistical stability of the calculated timestamp offsets, a corresponding time alignment strategy is selected to automatically align the radar data stream and the image data stream in time. The time alignment strategy includes an overall translation strategy and an interpolation generation strategy.

2. The automatic time alignment method according to claim 1, characterized in that, Step S100 further includes: taking the timestamp of the current frame radar data as the center, and extending forward and backward on the time axis of the image data stream with a preset maximum allowable delay time as the extension value, to obtain the time search window.

3. The automatic time alignment method according to claim 1, characterized in that, In step S200, preprocessing the radar data stream and the image data stream to obtain the radar dynamic target list and image detection box includes: Point cloud clustering is performed on the radar data stream to obtain multiple target point clusters, and their positions in vehicle coordinates are calculated to obtain the radar dynamic target list; A target detection algorithm is run on the image data stream to identify dynamic targets, obtain their bounding boxes, and thus obtain the image detection boxes.

4. The automatic time alignment method according to claim 2, characterized in that, In step S300, the timestamp offset includes: (Official 1), In formula 1, This represents the timestamp offset; This indicates the timestamp of the current frame of radar data; This represents the timestamp of the synchronization frame pair.

5. The automatic time alignment method according to claim 1, characterized in that, In step S400, the statistical stability based on the calculated timestamp offsets includes: The timestamp offset corresponding to a preset proportion of radar data frames is selected from the radar data stream, and its variance is calculated; by comparing the variance with a preset stability threshold, it is determined whether the time offset is constant.

6. The automatic time alignment method according to claim 5, characterized in that, In step S400, the overall translation strategy includes: when the time offset is constant, adding the timestamp offset to all timestamps of the image data stream to complete automatic time alignment.

7. The automatic time alignment method according to claim 5, characterized in that, In step S400, the interpolation generation strategy includes: when the time offset is time-varying, using the timestamp of the radar data stream as a reference, and based on the timestamp offset corresponding to each radar data frame, generating a time-aligned virtual image frame for each radar data frame through an interpolation algorithm.

8. A radar and image data automatic time alignment system based on target association, characterized in that, include: The acquisition and creation module is used to acquire radar data stream and image data stream respectively, and to create a time search window on the time axis of the image data stream for each frame of radar data in the radar data stream. The target correlation calculation module is used to preprocess the radar data stream and the image data stream respectively to obtain a radar dynamic target list and an image detection box; project each radar target in the radar dynamic target list onto the image coordinate system; for each frame of the image in the time search window, count the number of projection points falling into the image detection box to generate a target correlation score. The offset calculation module is used to select the radar frame and image frame with the highest target correlation score as a synchronization frame pair, and calculate the timestamp offset based on the timestamp of the synchronization frame pair. The time alignment execution module is used to automatically align the radar data stream and the image data stream by selecting a corresponding time alignment strategy based on the statistical stability of the calculated timestamp offsets. The time alignment strategy includes an overall translation strategy and an interpolation generation strategy.

9. A computer-readable storage medium, characterized in that, It stores computer-readable instructions, which, when executed by the computer's processor, cause the computer to perform the target association-based automatic time alignment method for radar and image data as described in any one of claims 1 to 7.

10. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the steps of the automatic time alignment method for radar and image data based on target association as described in any one of claims 1 to 7.

Images