Fusion space-time alignment method for multi-source dynamic data of parking lot in service area

Abstract

The invention discloses a fusion space-time alignment method for multi-source dynamic data of a parking lot in a service area, and the method comprises the following steps: generating a synchronous pulse signal based on a hardware trigger circuit of an FPGA (Field Programmable Gate Array); the panoramic camera array and the radar are driven through the low-voltage differential signal interface and the transistor-transistor logic level respectively; based on collaborative design of a hardware-level timestamp marking unit and a closed-loop clock taming mechanism, timestamp injection of a message is carried out through collaborative design of a physical layer and a protocol stack, master-slave clock deviation is calculated, and dynamic compensation is carried out according to the obtained master-slave clock deviation; and performing space-time alignment of cross-modal data based on hardware-level time synchronization, data buffering and interpolation compensation. The beneficial effects of the invention are that the programmable trigger circuit based on the FPGA generates independent pulse signals with adjustable phases, the independent pulse signals drive the camera array and the radar sensor respectively, global clock synchronization is realized in combination with a PTP protocol, and time sequence accumulated deviation among multiple devices is effectively eliminated.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and in particular, to a method for fusing spatio-temporal alignment of multi-source dynamic data in a service area parking lot. Background Art

[0002] With the deep integration of intelligent transportation, parking lot management systems based on the fusion of multi-source sensor data are accelerating their penetration into scenarios such as highway service areas. The current mainstream solutions rely on high-computing power cloud servers or local industrial control computers, and realize the perception of vehicle and parking space states through visual detection of panoramic camera image stitching or radar-vision cross-modal fusion algorithms, which can achieve large-scale coverage management of service area parking lots. However, the centralized data processing architecture of this method faces the following problems: bandwidth pressure caused by the full transmission of sensor data, response latency caused by the cloud processing link, and interference of service area network fluctuations on real-time decision-making.

[0003] In this context, the industry explores the introduction of edge computing devices into the service area parking management system. By locally deploying lightweight AI models, core tasks such as vehicle detection and positioning are directly completed at the data generation end, giving full play to the advantages of millisecond-level response, sharp reduction in bandwidth requirements, and autonomous operation in weak network environments, and building a collaborative mode of "edge real-time decision-making + cloud optimization". However, with the advancement of large-scale deployment of edge nodes, the collaborative mode also has the following defects: Existing radar-vision fusion detection schemes rely on software-level time synchronization of operating systems or middleware. Affected by thread scheduling, memory copying, etc., the timestamp alignment error between image and radar data is usually high. Due to the inherent delay between sensors, there is a certain difference between the radar rotation period and the camera exposure time, which will cause the "ghost" phenomenon in dynamic target tracking. And because the spatio-temporal alignment of the original sensor data is not achieved, when the camera's view angle misses a vehicle due to occlusion, the radar data cannot reverse-correct the image detection result, resulting in an increase in the misjudgment rate of vehicle states. Summary of the Invention

[0004] One of the purposes of this application is to provide a method for fusing spatio-temporal alignment of multi-source dynamic data in a service area parking lot that can solve at least one of the defects in the above background art.

[0005] To achieve at least one of the above purposes, the technical solution adopted in this application is: a method for fusing spatio-temporal alignment of multi-source dynamic data in a service area parking lot, including the following steps: S100: Based on the hardware trigger circuit of FPGA, generate a synchronous pulse signal; match the period of the synchronous pulse signal with the image acquisition frame rate, and drive the panoramic camera array and radar respectively through low-voltage differential signal interfaces and transistor-transistor logic levels; S200: Based on the co - design of the hardware - level timestamp marking unit and the closed - loop clock taming mechanism, the timestamp of the message is injected through the cooperation of the physical layer and the protocol stack, and the master - slave clock deviation is calculated. Dynamic compensation is performed according to the obtained master - slave clock deviation. S300: Based on hardware - level time synchronization, data buffering and interpolation compensation, spatio - temporal alignment of cross - modal data is carried out.

[0006] Preferably, in step S200, the timestamp injection includes the following process: Construct a global clock source, and drive the time counter through a high - stability crystal oscillator; Deploy the hardware timestamp marking unit at the data link layer; When the frame start delimiter SFD of the synchronization protocol message is detected, immediately latch the current value of the time counter and embed the timestamp into the message correction field.

[0007] Preferably, the update logic of the time counter satisfies: ; where ns_count represents the nanosecond count value of the time counter, represents the nanosecond increment, , f clk represents the FPGA master clock frequency.

[0008] Preferably, in step S200, the calculation formula of the master - slave clock deviation Δt is: Δt = ((t 2 -t 1 )-(t 4 -t 3 )) / 2; The compensation of the master - slave clock deviation is carried out by generating a frequency adjustment amount Δf(k) through a PID controller. The calculation formula of the frequency adjustment amount Δf(k) is as follows: ; where t 1 and t 2 respectively represent the master - clock synchronization message sending and slave - clock receiving timestamps, t 3 and t 4 respectively represent the slave - clock delay request and master - clock response timestamps, e(k) represents the master - slave clock deviation of the k - th sampling, ΔT represents the control period, K p 、K i and K d respectively represent the proportional, integral and differential coefficients.

[0009] Preferably, temperature - frequency compensation is performed on the master - slave clock deviation, and the compensation amount Δf comp is directly superimposed on the frequency adjustment amount output by the PID controller; The calculation formula of the compensation amount Δf comp is as follows: Δf comp =α(T - T ref )+β×dT / dt; Among them, α and β respectively represent device characteristic coefficients, T represents the temperature acquisition amount, and T ref represents the reference temperature.

[0010] Preferably, the hardware-level time synchronization process in step S300 is as follows: Generate a global synchronization clock signal based on the PTP protocol, and distribute it to each data acquisition module through the dedicated global wiring resources of the FPGA, so that the phases of the clock signals of each data acquisition module are consistent; The timestamp marking unit adopts a two-stage latch structure to be used for latching the external trigger signal and the synchronous system clock domain respectively; Integrate a programmable trigger synchronization module, and dynamically generate image acquisition and radar trigger pulses through a numerically controlled oscillator.

[0011] Preferably, in step S300, in order to adapt to the dynamic switching of the panoramic camera picture resolution and the radar scanning frequency, perform calibration based on timing compensation on the panoramic camera timestamp and the radar timestamp. The specific calibration formula is as follows: t cam_corrected =t trigger +A×R width ×R height ×t pixel ; t radar_corrected =t trigger +B / f radar ; Among them, t cam_corrected and t radar_corrected respectively represent the timestamps after calibration of the panoramic camera and the radar, and t trigger represents the timestamp of the original trigger signal; R width and R height respectively represent the image horizontal and vertical resolution parameters; t pixel represents the single-pixel processing time, A and B represent the corresponding calibration coefficients, and f radar represents the scanning frequency of the radar.

[0012] Preferably, when switching the panoramic camera picture resolution and the radar scanning frequency, the update calculation formulas for the calibration coefficients A and B are as follows: A=A base ((R width ×R height ) / ( R base_width ×R base_height )) 0.8 ; B=B fix +ω / f radar ; Among them, A base represents the calibration value of the coefficient, which can be determined according to the calibration test specifically; R base_ represents the reference resolution, Bfix represents a fixed delay, and ω represents a frequency-related term.

[0013] Preferably, in the data buffering and interpolation compensation process in step S300: buffer areas are set for the panoramic camera image data and the radar data first; a ping-pong operation strategy is adopted for the image data buffer area, so that when one buffer area is being written, the other buffer area is being read; priority queue management is implemented for the radar buffer area based on timestamps, and expired data with a survival time exceeding a set period is dynamically deleted; when it is detected that the timestamp and position differences between the panoramic camera image and the radar are too large, linear interpolation compensation is started to generate a fused target position, and thus the spatio-temporal benchmarks of the panoramic camera and the radar are aligned.

[0014] Preferably, the timestamp and position differences between the panoramic camera image and the radar are represented by an error function E(t); the error function E(t) and the interpolation function P comp (t) for generating the fused target position are respectively expressed as: ; ; where a and b respectively represent the contribution ratio weight coefficients for adjusting the time error and the space error, P cam (t) and P radar (t) respectively represent the spatial position coordinates of the image data and the radar data, E th represents the error judgment threshold, P comp (t) represents the compensated spatial position, P(t k ) represents the position in the k-th frame of data, and t k represents the calibrated alignment timestamp.

[0015] Compared with the prior art, the beneficial effects of the present application are as follows: Through a hardware-level synchronous trigger and dynamic compensation mechanism, a high-precision spatio-temporal synchronization system for the panoramic camera and the lidar is constructed. A programmable trigger circuit based on FPGA generates independent pulse signals with adjustable phases, which respectively drive the camera array and the radar sensor, and global clock synchronization is achieved in combination with the PTP protocol, effectively eliminating the timing cumulative deviation between multiple devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic diagram of the working process of the present application.

[0017] Figure 2 is a schematic diagram of the specific implementation process of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] Next, in combination with specific embodiments, the present application will be further described. It should be noted that in the description of this specification, the descriptions with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic descriptions of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification.

[0019] In the description of the present application, it should be noted that for orientation terms, such as terms "center", "horizontal", "longitudinal", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc., the orientation and position relationships indicated are based on the orientation or position relationship shown in the drawings. It is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and should not be construed as limiting the specific protection scope of the present application.

[0020] It should be noted that the terms "first", "second", etc. in the description and claims of the present application are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence.

[0021] In the present application, unless otherwise clearly specified and defined, terms such as "install", "connect", "connect", "fix", etc. should be understood in a broad sense. For example, it can be a connection, a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0022] In this application, unless otherwise clearly specified or limited, the first feature being "on" or "under" the second feature may include direct contact between the first and second features, or may include indirect contact between the first and second features through additional features therebetween. Moreover, the first feature being "above", "over" and "on top of" the second feature includes the first feature being directly above and obliquely above the second feature, or simply means that the horizontal height of the first feature is higher than that of the second feature. The first feature being "under", "beneath" and "underneath" the second feature includes the first feature being directly below and obliquely below the second feature, or simply means that the horizontal height of the first feature is less than that of the second feature.

[0023] The terms "comprising" and "having" and any variations thereof in the description and claims of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that comprises a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0024] One preferred embodiment of this application is as Figure 1 shown. A method for spatio-temporal alignment of multi-source dynamic data in a service area parking lot includes the following steps: S100: Based on the hardware trigger circuit of FPGA, generate a synchronous pulse signal; match the period of the synchronous pulse signal with the image acquisition frame rate, and drive the panoramic camera array and the radar through a low-voltage differential signal (LVDS) interface and a transistor-transistor logic (TTL) level respectively.

[0025] S200: Based on the co-design of the hardware-level timestamp marking unit and the closed-loop clock taming mechanism, inject the timestamp of the message through the cooperation of the physical layer and the protocol stack and calculate the master-slave clock deviation, and perform dynamic compensation according to the obtained master-slave clock deviation.

[0026] S300: Based on the hardware-level time synchronization, data buffering and interpolation compensation, perform spatio-temporal alignment of cross-modal data.

[0027] It should be known that the specific structure and working principle of the hardware trigger circuit of FPGA are well-known technologies to those skilled in the art. For the convenience of understanding, the logic of its pulse signal generation will be briefly described below. FPGA realizes the period control of the pulse signal through a hardware counter, and the counter bit width is dynamically configured according to the image acquisition frame rate. The pulse period T satisfies T = 1 / f frame , where f frame represents the preset image frame rate. Then, a trigger pulse with a fixed width is generated according to the programmable pulse width counter to ensure that the pulse edge is steep to meet the corresponding timing requirements of the sensor.

[0028] It should be noted that for the drive of the panoramic camera array, an LVDS interface is adopted. The LVDS interface integrates a current-mode driver, uses differential signal transmission, and supports long-distance anti-interference communication. Generally, the impedance matching of the LVDS interface is 100Ω, and the signal swing amplitude is 350mV. For the drive of the radar, a TTL interface is adopted. The TTL interface uses a CMOS push-pull output stage, and the output level is generally 3.3V. Through shaping by a Schmitt trigger, signal jitter can be effectively eliminated.

[0029] It can be understood that the technical solution of this application constructs a high-precision spatio-temporal synchronization system for the panoramic camera and the lidar through a hardware-level synchronous trigger and a dynamic compensation mechanism. The programmable trigger circuit based on FPGA generates independent pulse signals with adjustable phases, drives the camera array and the radar sensor respectively, and realizes global clock synchronization in combination with the PTP protocol, which can effectively eliminate the timing cumulative deviation between multiple devices.

[0030] In this embodiment, as Figure 2 shown, the mechanisms adopted for cross-device clock synchronization mainly include a timestamp generation mechanism and a clock deviation dynamic compensation mechanism. For the convenience of understanding, the two mechanisms will be described in detail below.

[0031] 1. Timestamp generation mechanism.

[0032] Its core lies in the collaborative design of a hardware-level timestamp marking unit and a closed-loop clock taming mechanism, and realizes nanosecond-level time synchronization through the collaboration of the physical layer and the protocol stack. Specifically, a global clock source is constructed, and a time counter is driven by a high-stability crystal oscillator; the hardware timestamp marking unit is deployed at the data link layer; when the frame start delimiter SFD of the synchronization protocol message PTP Sync is detected, the current time counter value is immediately latched, and the timestamp is embedded in the message correction field. This process is controlled by a dedicated hardware state machine, and the timestamp injection delay can be controlled ≤5ns.

[0033] It should be known that the specific structure and working principle of the time counter are well-known technologies to those skilled in the art. The time counter generally uses 64 bits, with its high 32 bits for second counting and its low 32 bits for nanosecond counting. The update logic of the time counter satisfies: . Among them, ns_count represents the nanosecond count value of the time counter, represents the nanosecond increment, , f clk represents the FPGA main clock frequency.

[0034] 2. Clock deviation dynamic compensation mechanism.

[0035] Based on the timestamp generation mechanism, the master-slave clock deviation can be calculated according to the timestamp message exchange, and then dynamic compensation can be performed according to the calculated master-slave clock deviation to ensure that the panoramic camera and the radar can achieve spatio-temporal synchronization. For the master-slave clock deviation compensation, a master-slave control architecture is adopted, that is, the corresponding frequency adjustment amount Δf(k) is generated through a data PID controller according to the calculated master-slave time deviation Δt. The specific calculation formulas for the master-slave time deviation Δt and the frequency adjustment amount Δf(k) are as follows: Δt = [(t 2 - t 1 ) - (t 4 - t 3 )] / 2.

[0036] .

[0037] Among them, t 1 and t 2 respectively represent the master clock synchronization message sending and slave clock receiving timestamps, t 3 and t 4 respectively represent the slave clock delay request and master clock response timestamps, e(k) represents the master-slave clock deviation of the kth sampling, ΔT represents the control period, K p , K i and K d respectively represent the proportional, integral and differential coefficients.

[0038] It can be understood that when the panoramic camera and the radar are working, the temperature inside them will also rise, and the rise in temperature will also affect the working performance of the device, and may further affect the timestamp synchronization of the panoramic camera and the radar. Therefore, in this embodiment, the timestamp synchronization process of the panoramic camera and the radar can be further optimized by integrating a temperature-frequency drift compensation module. For the convenience of understanding, the specific working process of the temperature-frequency drift compensation module will be described in detail below.

[0039] Specifically, based on the chip temperature collected by the temperature sensor built into the panoramic camera-radar spatio-temporal synchronization architecture, temperature-frequency compensation is performed on the master-slave clock deviation, and nonlinear correction compensation is performed on the output of the voltage-controlled oscillator according to the pre-stored temperature-frequency characteristic curve. The compensation amount Δf comp is directly superimposed on the frequency adjustment amount Δf(k) output by the PID controller to form a composite control signal. The calculation formula for the compensation amount Δf comp is as follows: Δf comp = α(T - T ref ) + β × dT / dt.

[0040] Among them, α and β respectively represent device characteristic coefficients, and their specific values are selected according to the device type; T represents the temperature acquisition amount; T ref represents the reference temperature, and the specific value of T ref can be set by those skilled in the art according to actual needs.

[0041] In this embodiment, as Figure 2 shown, after the construction of the panoramic camera-radar spatio-temporal synchronization architecture is completed, spatio-temporal alignment design can be carried out on the picture resolution of the panoramic camera and the scanning frequency of the radar. The specific design mainly includes the hardware-level time synchronization process and the data buffering and interpolation compensation process; for the convenience of understanding, the following will describe these two processes in detail.

[0042] I. Regarding the hardware-level time synchronization process.

[0043] Generate a global synchronization clock signal based on the PTP protocol and distribute it to each data acquisition module through the dedicated global wiring resources of the FPGA to ensure the phase consistency of the clock signal. The timestamp marking unit adopts a two-stage latch structure; the first stage latches the external trigger signal, and the second stage is synchronized in the system clock domain, and a double flip-flop chain is used to eliminate the timing deviation caused by metastability. For dynamic adjustment requirements, a programmable trigger synchronization module is integrated, and image acquisition and radar trigger pulses are dynamically generated through a numerically controlled oscillator (NCO).

[0044] To adapt to the dynamic switching of the panoramic camera picture resolution and the radar scanning frequency, the timestamp marking unit integrates an acquisition duration compensation module. The panoramic camera and the radar share the same PTP clock source to ensure that the timestamps after compensation and correction are based on the unified time axis. The correction formulas for the panoramic camera picture timestamp and the radar timestamp inheriting the trigger and clock parameters are respectively: t cam_corrected =t trigger +A×R width ×R height ×t pixel .

[0045] t radar_corrected =t trigger +B / f radar .

[0046] Among them, t cam_corrected and t radar_corrected respectively represent the timestamps after correction of the panoramic camera and the radar, t trigger represents the timestamp of the original trigger signal; R width and R height respectively represent the horizontal and vertical resolution parameters of the image; t pixel represents the single-pixel processing time, A and B represent the corresponding correction coefficients, f radarIndicates the scanning frequency of the radar.

[0047] It should be noted that the timestamps of the panoramic camera and the radar are dynamically loaded through the AXI-Lite interface. After the trigger signal is generated, the corresponding calibration coefficients A and B are automatically called to achieve the timing compensation for the adaptive resolution of the panoramic camera image and the radar scanning frequency.

[0048] It should be noted that when switching the resolution of the panoramic camera image and the radar scanning frequency, the calibration coefficients A and B corresponding to the timestamp calibration of the panoramic camera and the radar need to be updated. The specific update calculation formula is as follows: A = A base ((R width ×R height ) / (R base_width ×R base_height )) 0.8 , B = B fix + ω / f radar .

[0049] Among them, A base represents the calibration value of the coefficient, which can be determined according to the calibration test specifically; R base_ represents the reference resolution, B fix represents the fixed delay, and ω represents the frequency-related term.

[0050] II. Regarding the data buffering and interpolation compensation process.

[0051] Design a double-buffer architecture to solve the problems of data rate difference and instantaneous jitter. First, set up buffers for the panoramic camera image and radar data. First, adopt the ping-pong operation strategy for the image buffer (Buffer A), that is, when writing to one buffer, the other buffer can be read, and the two are used alternately to avoid data conflicts or waits caused by simultaneously reading and writing the same buffer; this design allows the write and read operations to seamlessly switch between the double buffers, thereby effectively eliminating the data flow breakpoints.

[0052] The specific address (addr) mapping logic is: addr = Buffer_index × S frame + y × W × x; where Buffer_index represents the buffer search function, S frame represents the single-frame data capacity, W represents the image width, and y and x represent the current pixel row number and column number respectively.

[0053] Then, implement the timestamp-based priority queue management for the radar buffer (Buffer B), that is, by dynamically deleting the expired data whose survival time exceeds the set period, thereby ensuring the real-time nature of the data in the radar buffer.

[0054] After completing the timestamp design of the image buffer and the radar buffer, the timestamps of the two can be compared. When the differences in the timestamps and positions of the panoramic camera image and the radar are not obvious, it can be determined that the timestamps of the panoramic camera image and the radar are in an aligned state; while when the differences in the timestamps and positions of the panoramic camera image and the radar are too large, linear interpolation compensation can be initiated to generate the fused target position, and then the spatio-temporal benchmarks of the panoramic camera and the radar can be aligned.

[0055] The differences in the timestamps and positions of the panoramic camera image and the radar can be represented by the error function E(t); the expression of the error function E(t) is: 。

[0056] Where a and b respectively represent the contribution ratio weight coefficients for adjusting the time error and the space error, E th represents the error judgment threshold. When the error function satisfies the error judgment threshold condition, the panoramic camera and the radar will perform spatio-temporal benchmark alignment. P cam (t) and P radar (t) respectively represent the spatial position coordinates of the image data and the radar data.

[0057] For the interpolation function P comp (t) used to generate the fused target position, the expression is: 。

[0058] Where P comp (t) represents the compensated spatial position, P(t k ) represents the position in the k-th frame of data, t k represents the corrected aligned timestamp, t represents the current time point for interpolation, and they form the interpolation weights and are pre-stored in the lookup table to avoid real-time division operations.

[0059] The above describes the basic principle, main features and advantages of the present application. Those skilled in the art of this industry should understand that the present application is not limited by the above embodiments. What is described in the above embodiments and the specification is only the principle of the present application. Without departing from the spirit and scope of the present application, the present application will have various changes and improvements, and these changes and improvements all fall within the scope of the present application claimed. The scope of protection required by the present application is defined by the appended claims and their equivalents.

Claims

1. A fusion spatiotemporal alignment method for multi-source dynamic data of a service area parking lot, characterized in that: The steps include: S100: FPGA-based hardware trigger circuit, generating synchronous pulse signal; Match the period of the synchronization pulse signal with the image acquisition frame rate, and drive the panoramic camera array and radar through a low-voltage differential signal interface and transistor-transistor logic level respectively; S200: Based on the collaborative design of the hardware-level timestamp marking unit and the closed-loop clock taming mechanism, the timestamp of the message is injected and the master-slave clock deviation is calculated through the collaboration of the physical layer and the protocol stack, and dynamic compensation is performed based on the obtained master-slave clock deviation; S300: Perform spatiotemporal alignment of cross-modal data based on hardware-level time synchronization, data buffering, and interpolation compensation.

2. The fusion spatiotemporal alignment method for multi-source dynamic data of a service area parking lot as claimed in claim 1, characterized in that: In step S200, timestamp injection includes the following process: Build a global clock source and drive the time counter through a high-stability crystal oscillator; Deploy the hardware timestamp unit at the data link layer; When the frame start delimiter SFD of the synchronous protocol message is detected, the current time counter value is immediately latched and the timestamp is embedded in the message correction field.

3. The fusion spatiotemporal alignment method for multi-source dynamic data of a service area parking lot as claimed in claim 2, characterized in that: The update logic of the time counter satisfies: ; Among them, ns_count represents the nanosecond count value of the time counter, represents nanosecond increments, , f clk Indicates the FPGA main clock frequency.

4. The fusion spatiotemporal alignment method for multi-source dynamic data of a service area parking lot as claimed in claim 1, characterized in that: In step S200, the calculation formula of the master-slave clock deviation Δt is: Δt=((t2-t1)-(t4-t3)) / 2; The master-slave clock deviation is compensated by generating a frequency adjustment value Δf(k) through a PID controller. The calculation formula of the frequency adjustment value Δf(k) is as follows: ； Where t1 and t2 represent the timestamps of the master clock synchronization message sending and the slave clock receiving, respectively; t3 and t4 represent the slave clock delay request and the master clock response timestamp, respectively; e(k) represents the master-slave clock deviation of the kth sampling; ΔT represents the control period; K p , K i and K d They represent proportional, integral and differential coefficients respectively.

5. The fusion spatiotemporal alignment method for multi-source dynamic data of a service area parking lot as claimed in claim 3, characterized in that: Temperature-frequency compensation for master-slave clock deviation, compensation amount Δf comp Directly superimpose the frequency adjustment amount output by the PID controller; compensation amount Δf comp The calculation formula is as follows: Δf comp =α(T-T ref )+β×dT / dt; Among them, α and β represent the device characteristic coefficients, T represents the temperature acquisition amount, and T ref Indicates the reference temperature.

6. The method for fusion spatiotemporal alignment of multi-source dynamic data of a service area parking lot according to any one of claims 1 to 4, characterized in that: The hardware-level time synchronization process in step S300 is as follows: Generate a global synchronous clock signal based on the PTP protocol and distribute it to each data acquisition module through the dedicated global wiring resources of the FPGA to make the phase of the clock signal of each data acquisition module consistent; The timestamp unit adopts a two-level latch structure to latch the external trigger signal and the synchronous system clock domain respectively; Integrated programmable trigger synchronization module to dynamically generate image acquisition and radar trigger pulses through a digitally controlled oscillator.

7. The fusion spatiotemporal alignment method for multi-source dynamic data of a service area parking lot as claimed in claim 5, characterized in that: In step S300, in order to adapt to the dynamic switching of the panoramic camera image resolution and the radar scanning frequency, the panoramic camera timestamp and the radar timestamp are corrected based on timing compensation. The specific correction formula is as follows: t cam_corrected =t trigger +A×R width ×R height ×t pixel ; t radar_corrected =t trigger +B / f radar ; Among them, t cam_corrected and t radar_corrected Respectively represent the timestamps after panoramic camera and radar correction, t trigger Indicates the timestamp of the original trigger signal; R width and R height Respectively represent the horizontal and vertical resolution parameters of the image; t pixel represents the single pixel processing time, A and B represent the corresponding correction coefficients, f radar Indicates the scanning frequency of the radar.

8. The fusion spatiotemporal alignment method for multi-source dynamic data of a service area parking lot as claimed in claim 7, characterized in that: When the panoramic camera image resolution and radar scanning frequency are switched, the update calculation formulas of the correction coefficients A and B are as follows: A=A base ((R width ×R height ) / ( R base_width ×R base_height )) 0.8 ; B=B fix +ω / f radar ; Among them, A base Represents the nominal value of the coefficient, R base_ Indicates the base resolution, B fix represents a fixed delay and ω represents a frequency-dependent term.

9. The fusion spatiotemporal alignment method for multi-source dynamic data of a service area parking lot as claimed in claim 6, characterized in that: The data buffering and interpolation compensation process in step S300 is as follows: Set up a buffer for the panoramic camera image data and radar data first; A ping-pong operation strategy is adopted for the image data buffer so that when one buffer is being written, the other buffer is being read; Implement priority queue management based on timestamps for radar buffers, and dynamically delete expired data whose survival time exceeds the set period; When the timestamp and position differences between the panoramic camera image and the radar are detected to be too large, linear interpolation compensation is started to generate the fusion target position, and then the time and space references of the panoramic camera and radar are aligned.

10. The fusion spatiotemporal alignment method for multi-source dynamic data of a service area parking lot as claimed in claim 9, characterized in that: The timestamp and position difference between the panoramic camera image and the radar are represented by the error function E(t); the error function E(t) and the interpolation function P used to generate the fusion target position comp The expressions of (t) are respectively: ； ； Among them, a and b represent the contribution ratio weight coefficients of adjusting the time error and space error respectively, P cam (t) and P radar (t) represent the spatial position coordinates of image data and radar data, respectively, E th represents the error judgment threshold, P comp (t) represents the spatial position after compensation, P(t k ) represents the position in the k-th frame data, t k Represents the corrected alignment timestamp.

Images