FPGA accelerated fusion method for dynamic resource data of parking lot in service area

Abstract

The invention discloses an FPGA acceleration fusion method for dynamic resource data of a parking lot in a service area, and the method comprises the following steps: constructing a panorama camera-radar space-time synchronization architecture based on FPGA hardware acceleration, splicing panorama camera images in real time, and aligning the panorama camera images with a timestamp of a radar point cloud; cooperatively deploying a vehicle detection model and a radar tracking algorithm on an edge computing platform, and constructing a heterogeneous cooperative architecture of an FPGA and the edge computing platform; and constructing a traffic flow prediction model of a convolutional neural network-long and short-term memory network, and carrying out adaptive collaborative optimization design of an adjustment mechanism on the obtained traffic flow prediction model and hardware parameters of the panoramic camera in the service area. The beneficial effects of the invention are that the programmable trigger circuit based on the FPGA generates independent pulse signals with adjustable phases, the independent pulse signals drive the camera array and the radar sensor respectively, global clock synchronization is realized in combination with a PTP protocol, and time sequence accumulated deviation among multiple devices is effectively eliminated.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and particularly to an FPGA acceleration fusion method for dynamic resource data of a service area parking lot. Background Art

[0002] With the deep integration of intelligent transportation, parking lot management systems based on multi-source sensor data fusion are accelerating their penetration into scenarios such as highway service areas. The current mainstream solutions rely on high-computing power cloud servers or local industrial computers, and realize vehicle and parking space status perception through visual detection of panoramic camera image stitching or radar-vision cross-modal fusion algorithms, which can achieve large-scale coverage management of service area parking lots. However, this centralized data processing architecture of this method faces the following problems: bandwidth pressure caused by full-volume transmission of sensor data, response latency caused by the cloud processing link, and interference of service area network fluctuations on real-time decision-making.

[0003] In this context, the industry explores the introduction of edge computing devices into service area parking management systems. By locally deploying lightweight AI models, core tasks such as vehicle detection and positioning are directly completed at the data generation end, giving full play to the advantages of millisecond-level response, sharp reduction in bandwidth requirements, and autonomy in weak network environments, and building a collaborative mode of "edge real-time decision-making + cloud optimization". However, with the advancement of large-scale deployment of edge nodes, the collaborative mode also has the following defects: (1) Existing radar-vision fusion detection schemes rely on software-level time synchronization of operating systems or middleware. Affected by thread scheduling, memory copying, etc., the timestamp alignment error between image and radar data is usually relatively high. Due to the inherent delay between sensors, there is a certain difference between the radar rotation period and the camera exposure time, which will cause the "ghost" phenomenon in dynamic target tracking. And because the spatio-temporal alignment of the original sensor data is not achieved, when the camera's view is blocked and a vehicle is missed in detection, the radar data cannot reverse-correct the image detection result, resulting in an increase in the vehicle status misjudgment rate.

[0004] (2) Under the existing technical framework, panoramic image stitching and radar data processing need to rely on GPU computing power to be executed step by step according to a fixed process (such as image preprocessing - point cloud feature extraction based on a parallel computing framework), forming a serial process. Although this mode can complete basic detection tasks, it causes significant accumulation of task processing delays between algorithm layers. If the time delay is compressed by enhancing the hardware computing power, it will in turn cause an increase in the power consumption level of edge devices, which may lead to a decrease in the power supply stability and device durability of the devices, and it is difficult to meet the all-weather operation requirements of service areas.

[0005] (3) The current management system uses fixed parameter configurations of sensors such as cameras and radars, and does not establish a dynamic coupling mechanism between sensing resources and traffic flow intensity. During peak traffic hours, fixed frame rate acquisition and computing resource quotas are prone to target missed detection and trajectory interruption; while during low traffic hours, continuous operation of high-power consumption detection algorithms leads to wasted computing power. Summary of the Invention

[0006] One object of the present application is to provide an FPGA acceleration fusion method for dynamic resource data of service area parking lots that can solve at least one defect in the above-mentioned background technology.

[0007] To achieve at least one of the above objects, the technical solution adopted by the present application is: an FPGA acceleration fusion method for dynamic resource data of service area parking lots, including the following steps: S100: Construct a panoramic camera-radar spatio-temporal synchronization architecture based on FPGA hardware acceleration, and align the real-time stitching of service area panoramic camera images with the timestamps of radar point clouds through FPGA parallel computing; S200: Collaboratively deploy a vehicle detection model based on visual detection and a radar tracking algorithm on an edge computing platform, and construct a heterogeneous collaborative architecture between the FPGA and the edge computing platform; S300: Construct a traffic flow prediction model of a convolutional neural network-long short-term memory network, and perform adaptive collaborative optimization design of the adjustment mechanism between the obtained traffic flow prediction model and the hardware parameters of panoramic cameras in the service area.

[0008] Preferably, the construction of the panoramic camera-radar spatio-temporal synchronization architecture in step S100 includes the following processes: Based on the hardware trigger circuit of the FPGA, generate a synchronization pulse signal; match the period of the synchronization pulse signal with the image acquisition frame rate, and drive the panoramic camera array and the radar respectively through a low-voltage differential signal interface and a transistor-transistor logic level; based on the collaborative design of a hardware-level timestamp marking unit and a closed-loop clock taming mechanism, inject timestamps into messages through the collaboration of the physical layer and the protocol stack; calculate the master-slave clock deviation based on the timestamp message exchange and dynamically compensate the master-slave clock deviation.

[0009] Preferably, the timestamp injection includes the following processes: Construct a global clock source, drive a time counter through a high-stability crystal oscillator; deploy the hardware timestamp marking unit at the data link layer; when detecting the start frame delimiter SFD of the synchronization protocol message, immediately latch the current value of the time counter and embed the timestamp into the message correction field.

[0010] Preferably, the calculation formula for the master-slave clock deviation Δt is: Δt = ((t 2 - t 1 ) - (t 4 - t 3)) / 2; The compensation for the master-slave clock deviation is carried out by generating a frequency adjustment amount Δf(k) through a PID controller. The calculation formula for the frequency adjustment amount Δf(k) is as follows: ; Where, t 1 and t 2 respectively represent the master clock synchronization message sending and slave clock receiving timestamps, t 3 and t 4 respectively represent the slave clock delay request and master clock response timestamps, e(k) represents the master-slave clock deviation of the k-th sampling, ΔT represents the control period, K p , K i and K d respectively represent the proportional, integral and differential coefficients.

[0011] Preferably, based on the chip temperature collected by the temperature sensor built in the panoramic camera-radar spatio-temporal synchronization architecture, temperature-frequency compensation is performed on the master-slave clock deviation, and the compensation amount Δf comp is directly superimposed on the frequency adjustment amount output by the PID controller; the calculation formula for the compensation amount Δf comp is as follows: Δf comp = α(T - T ref ) + β × dT / dt; Where, α and β respectively represent the device characteristic coefficients, T represents the temperature acquisition amount, and T ref represents the reference temperature.

[0012] Preferably, in step S100, the timestamp alignment between the panoramic camera and the radar includes a hardware-level time synchronization process, and the specific design process is as follows: Generate a global synchronization clock signal based on the PTP protocol, and distribute it to each data acquisition module through the dedicated global wiring resources of the FPGA, so that the phase of the clock signals of each data acquisition module is consistent; The timestamp marking unit adopts a two-stage latch structure to respectively latch the external trigger signal and the synchronization system clock domain; Integrate a programmable trigger synchronization module to dynamically generate image acquisition and radar trigger pulses through a numerically controlled oscillator.

[0013] Preferably, in step S100, in order to adapt to the dynamic switching of the panoramic camera screen resolution and radar scanning frequency, correction based on timing compensation is performed on the panoramic camera timestamp and the radar timestamp, and the specific correction formula is as follows: t cam_corrected = t trigger + A × R width × R height × t pixel ; t radar_corrected = t trigger + B / fradar ; wherein, t cam_corrected and t radar_corrected respectively represent the timestamps after panoramic camera and radar calibration, and t trigger represents the timestamp of the original trigger signal; R width and R height respectively represent the horizontal and vertical resolution parameters of the image; t pixel represents the single-pixel processing time, A and B represent the corresponding calibration coefficients, and f radar represents the scanning frequency of the radar.

[0014] Preferably, the alignment of the timestamps of the panoramic camera and the radar in step S100 further includes a data buffering and interpolation compensation process, and the specific design process is as follows: set buffers for the panoramic camera image data and the radar data first; adopt a ping-pong operation strategy for the image data buffer so that while one buffer is being written, the other buffer is being read; perform priority queue management on the radar buffer based on the timestamp, and dynamically delete the expired data whose survival time exceeds the set period; when it is detected that the timestamps and position differences between the panoramic camera image and the radar are too large, start linear interpolation compensation to generate the fused target position, and then align the spatio-temporal benchmarks of the panoramic camera and the radar.

[0015] Preferably, the timestamp and position differences between the panoramic camera image and the radar are represented by an error function E(t); the expressions of the error function E(t) and the interpolation function P comp (t) for generating the fused target position are respectively represented as: ; ; wherein, a and b respectively represent the contribution ratio weight coefficients for adjusting the time error and the space error, P cam (t) and P radar (t) respectively represent the spatial position coordinates of the image data and the radar data, E th represents the error judgment threshold, P comp (t) represents the compensated spatial position, P(t k ) represents the position in the k-th frame of data, and t k represents the calibrated alignment timestamp.

[0016] Preferably, in step S200, the construction of the vehicle detection model includes the following process: adopt an improved YOLO v11 model as the basic detection framework, and introduce depthwise separable convolution to reconstruct the backbone network; integrate a cross-scale attention mechanism in the feature pyramid layer, and then at the P4 / P5 output nodes, fuse the high-resolution feature map F high with the low-resolution feature map Flow Perform interactive fusion to enhance small-scale target features; the enhanced feature F out has the following expression: ; ; where W represents the channel weight vector, denotes element-wise addition, GAP represents global average pooling, σ represents the Sigmoid activation function, MLP represents the multi-layer perceptron, denotes channel-wise multiplication, denotes upsampling of the low-resolution feature map.

[0017] Preferably, in step S200, the radar tracking algorithm includes the following working process: eliminate the influence of the radar's own motion on the point cloud coordinates through the motion compensation algorithm, and perform coordinate transformation on the original point cloud based on the vehicle's real-time speed and heading angle; adopt an improved density clustering algorithm to calculate the average distance d of each radar point cloud data point to its nearest k neighbors i_avg ; adaptively adjust the neighborhood radius ε according to the point cloud distribution density i , and the neighborhood radius ε i has the following dynamic setting formula: ε i =max(ε min , μ, d i_avg ); where ε min represents the preset minimum radius, and μ represents the density sensitivity coefficient.

[0018] Preferably, the collaborative deployment of the vehicle detection model and the radar tracking algorithm includes the following process: in the multi-target tracking stage, predict the target state through the extended Kalman filter combined with the uniform motion model, and construct the data association cost matrix through the joint constraint of the Mahalanobis distance and the Doppler velocity, and then perform cross-modal matching of the radar and visual trajectories; project the radar point cloud to the image coordinate system based on the pre-calibrated external parameter matrix through the spatio-temporal alignment mechanism, and combine the hardware-level synchronous timestamp to generate a synchronous pulse signal through the FPGA, and then trigger the camera exposure and the radar scan simultaneously.

[0019] Preferably, calculate the joint matching cost C ij through the radar trajectory and the visual trajectory to construct the cost matrix, and the calculation formula of the joint matching cost C ij is as follows: ; where D Mahalanobis represents the Mahalanobis distance, λ represents the weight coefficient, represents the radial velocity of the i-th radar trajectory, represents the projection velocity of the j-th visual target in the radar radial direction.

[0020] Preferably, in the heterogeneous collaborative architecture of step S200, the edge computing platform and the FPGA are interconnected through a high-speed bus, and a shared memory zero-copy transmission mechanism is adopted between the two, so that the FPGA writes the preprocessed radar data into a fixed physical address area through direct memory access, and at the same time, the Jetson side directly accesses this area through memory mapping technology.

[0021] Preferably, for the heterogeneous collaborative architecture in step S200, the FPGA adopts a double-buffer storage strategy, alternately performing data writing and reading, and then continuously processing the radar data in a pipeline; the Jetson side manages the computing tasks through a priority scheduler, responds to the interrupt signal of the FPGA in real time, and preferentially processes the data ready for radar preprocessing.

[0022] Preferably, in step S300, the construction of the traffic flow prediction model includes the following process: using a multi-node spatio-temporal correlation matrix as the input, integrating the historical traffic flow data of the target area and adjacent highly correlated monitoring points in the horizontal dimension, and constructing the temporal characteristics of continuous time segments in the vertical dimension to form a two-dimensional spatio-temporal feature vector; the model architecture adopts a CNN-LSTM dual-channel structure, with multiple convolutional kernels configured in the first layer for spatial feature extraction, and the feature dimension is compressed through a max-pooling layer; the spatio-temporal feature vector is input into the LSTM network after being transformed by a Flatten layer, and a memory unit is used to capture bidirectional temporal dependencies, and a gated recurrent mechanism is established in the time dimension; a fully connected network is introduced in the output layer for multi-scale prediction, and the traffic flow values at future prediction times are output synchronously.

[0023] Preferably, in step S300, the adaptive collaborative optimization design includes the following process: according to the panoramic camera resolution and radar scanning frequency configuration, the predicted traffic flow is divided into three levels: low, medium, and high; a hysteresis threshold is introduced so that when the prediction level changes, it is necessary to satisfy that the traffic flow fluctuation exceeds the preset hysteresis threshold to trigger an adjustment; when performing a mode switch based on the change in the traffic flow gear level, resolution and frequency adjustment instructions are generated, and then the panoramic camera-radar spatio-temporal synchronization architecture calculates the hardware timing parameters in real time according to the new panoramic camera resolution and radar frequency and performs timestamp alignment.

[0024] Preferably, when calculating the hardware timing parameters, the calculation formula for the trigger pulse parameters is: N reload_cam =f clk / f frame ; T cam =k cam ×N reload_cam / f clk ; N reload_radar= f clk / f radar ; T radar = k radar × N reload_radar / f clk ; Wherein, N reload_cam and N reload_radar respectively represent the counter reload values of the panoramic camera and the radar; k cam and k radar respectively represent the programmable phase offset coefficient and the radar phase offset coefficient; f frame represents the image frame rate, and f clk represents the FPGA main clock frequency.

[0025] Compared with the prior art, the beneficial effects of the present application are as follows: (1) Through the hardware-level synchronous trigger and dynamic compensation mechanism, a high-precision spatio-temporal synchronization system for the panoramic camera and the lidar is constructed. The programmable trigger circuit based on FPGA generates independent pulse signals with adjustable phases, which respectively drive the camera array and the radar sensor. Combining with the PTP protocol to achieve global clock synchronization, effectively eliminating the timing cumulative deviation between multiple devices.

[0026] (2) Based on the FPGA-Jetson heterogeneous collaborative architecture, the deep combination of algorithm acceleration and resource optimization is realized. Thus, effectively improving the end-to-end low-latency processing and high detection accuracy of multi-modal data in complex traffic scenarios, and significantly enhancing the real-time performance and energy efficiency ratio of the system.

[0027] (3) Through the CNN-LSTM dual-channel spatio-temporal feature fusion model, the future traffic flow is accurately predicted. Combining with the dynamic parameter mapping rule and the FPGA programmable timing compensation mechanism, the real-time switching of the resolution / frequency parameters is realized at the hardware layer. Thus, while ensuring that the spatio-temporal synchronization error ≤ 1 ms, the ineffective energy consumption is reduced through the sensor energy efficiency control strategy, achieving the dynamic balance between the sensing accuracy and the hardware resource consumption. Brief Description of the Drawings

[0028] Figure 1 is the schematic diagram of the working process of the present application.

[0029] Figure 2 is the schematic diagram of the specific implementation process of the present application. Detailed Embodiments

[0030] Next, in combination with specific embodiments, the present application will be further described. It should be noted that in the description of this specification, the descriptions with reference to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic descriptions of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification.

[0031] In the description of the present application, it should be noted that for orientation terms, if there are terms such as "center", "horizontal", "longitudinal", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc., indicating the orientation and position relationship is based on the orientation or position relationship shown in the drawings. This is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and should not be understood as limiting the specific protection scope of the present application.

[0032] It should be noted that the terms "first", "second", etc. in the description and claims of the present application are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence.

[0033] In the present application, unless otherwise clearly specified and defined, terms such as "installation", "connection", "connection", "fixation", etc. should be understood in a broad sense. For example, it can be a connection, a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0034] In this application, unless otherwise clearly defined and limited, the first feature being "on" or "under" the second feature may include direct contact between the first and second features, or may include the first and second features not being in direct contact but being in contact through additional features therebetween. Moreover, the first feature being "above", "over" and "on top of" the second feature includes the first feature being directly above and obliquely above the second feature, or merely indicating that the horizontal height of the first feature is higher than that of the second feature. The first feature being "under", "beneath" and "underneath" the second feature includes the first feature being directly below and obliquely below the second feature, or merely indicating that the horizontal height of the first feature is less than that of the second feature.

[0035] The terms "comprise" and "have" and any variations thereof in the description and claims of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that comprises a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these process, method, product or device.

[0036] One preferred embodiment of this application, as Figure 1 shown, a method for FPGA-accelerated fusion of dynamic resource data of a service area parking lot, comprising the following steps: S100: Construct a panoramic camera-radar spatio-temporal synchronization architecture based on FPGA hardware acceleration, and align the timestamps of real-time stitching of panoramic camera images in the service area with radar point clouds through FPGA parallel computing.

[0037] It can be understood that in this embodiment, a high-precision spatio-temporal synchronization system for the panoramic camera and lidar is constructed through a hardware-level synchronization trigger and dynamic compensation mechanism. And an independent pulse signal with adjustable phase is generated by a programmable trigger circuit based on FPGA (Field Programmable Gate Array) to drive the camera array and radar sensor respectively, and then global clock synchronization can be achieved by combining with the PTP protocol, so as to effectively eliminate the timing cumulative deviation between multiple devices. Compared with the traditional scheme, this application can solve the timing misalignment problem caused by software layer scheduling in the traditional scheme, thus greatly shortening the timestamp error of images and radar data during fusion to avoid or weaken the generation of "ghost" phenomena.

[0038] S200: Co-deploy a vehicle detection model based on visual detection and a radar tracking algorithm on an edge computing platform, and construct a heterogeneous collaborative architecture between the FPGA and the edge computing platform.

[0039] It can be understood that the edge computing platform is a distributed computing system designed to push computing, storage, and application capabilities closer to the data generation source, rather than relying on a centralized data center or cloud computing. By processing data near the data source, this platform significantly reduces data transmission latency, improves data processing efficiency, and meets the requirements in aspects such as real-time services, application intelligence, security, and privacy protection. There are various types of common edge computing platforms, and in this embodiment, Jetson will be taken as an example for detailed description.

[0040] S300: Build a traffic flow prediction model of Convolutional Neural Network (CNN) - Long Short-Term Memory Network (LSTM), and perform an adaptive collaborative optimization design of the adjustment mechanism between the obtained traffic flow prediction model and the hardware parameters of the panoramic camera in the service area.

[0041] Specifically, build a traffic flow prediction model of Convolutional Neural Network + Long Short-Term Memory Network (CNN + LSTM), and adjust the panoramic camera resolution and lidar scanning frequency in real time according to the traffic flow prediction results, dynamically optimize each hardware resource, so as to reduce the use of ineffective energy consumption.

[0042] It should be known that the construction of the panoramic camera - radar spatio-temporal synchronization architecture in step S100 mainly includes two parts: hardware synchronization trigger design and cross-device clock synchronization. For the convenience of understanding, the following will describe these two parts in detail respectively.

[0043] In this embodiment, as Figure 2 shown, the specific process of the hardware synchronization trigger design is as follows: Based on the hardware trigger circuit of FPGA, generate a synchronization pulse signal; match the period of the synchronization pulse signal with the image acquisition frame rate, and drive the panoramic camera array and radar through the Low-Voltage Differential Signaling (LVDS) interface and Transistor-Transistor Logic (TTL) level respectively.

[0044] It can be understood that the specific structure and working principle of the hardware trigger circuit of FPGA are well-known technologies to those skilled in the art. For the convenience of understanding, the following will briefly describe the logic of its pulse signal generation. FPGA realizes the period control of the pulse signal through a hardware counter, and the counter bit width is dynamically configured according to the image acquisition frame rate. The pulse period T satisfies T = 1 / f frame , where f frame represents the preset image frame rate. Then, generate a trigger pulse with a fixed width according to the programmable pulse width counter to ensure that the pulse edge is steep to meet the corresponding timing requirements of the sensor.

[0045] It should be noted that for the drive of the panoramic camera array, an LVDS interface is adopted. The LVDS interface integrates a current-mode driver, uses differential signal transmission, and supports long-distance anti-interference communication. Generally, the impedance matching of the LVDS interface is 100Ω, and the signal swing amplitude is 350mV. For the drive of the radar, a TTL interface is adopted. The TTL interface uses a CMOS push-pull output stage, and the output level is generally 3.3V. Through shaping by a Schmitt trigger, signal jitter can be effectively eliminated.

[0046] In this embodiment, as Figure 2 shown, the mechanisms adopted for cross-device clock synchronization mainly include a timestamp generation mechanism and a clock deviation dynamic compensation mechanism. For the convenience of understanding, the two mechanisms will be described in detail below.

[0047] I. Timestamp generation mechanism.

[0048] Its core lies in the collaborative design of a hardware-level timestamp marking unit and a closed-loop clock taming mechanism, and nanosecond-level time synchronization is achieved through the collaboration of the physical layer and the protocol stack. Specifically, a global clock source is constructed, and a high-stability crystal oscillator is used to drive a time counter; the hardware timestamp marking unit is deployed at the data link layer; when the frame start delimiter SFD of the synchronization protocol packet PTP Sync is detected, the current value of the time counter is immediately latched, and the timestamp is embedded in the correction field of the packet. This process is controlled by a dedicated hardware state machine, and the timestamp injection delay can be controlled ≤5ns.

[0049] It should be known that the specific structure and working principle of the time counter are both well-known technologies to those skilled in the art. The time counter generally uses 64 bits, with its high 32 bits used for second counting and its low 32 bits used for nanosecond counting. The update logic of the time counter satisfies: . Among them, ns_count represents the nanosecond count value of the time counter, represents the nanosecond increment, , f clk represents the FPGA main clock frequency.

[0050] II. Clock deviation dynamic compensation mechanism.

[0051] Based on the timestamp generation mechanism, the master-slave clock deviation can be calculated according to the exchange of timestamp packets, and then dynamic compensation can be performed according to the calculated master-slave clock deviation to ensure that the panoramic camera and the radar can achieve spatio-temporal synchronization. For the master-slave clock deviation compensation, a master-slave control architecture is adopted, that is, the corresponding frequency adjustment amount Δf(k) is generated through a data PID controller according to the calculated master-slave time deviation Δt. The specific calculation formulas for the master-slave time deviation Δt and the frequency adjustment amount Δf(k) are as follows: Δt=((t 2 -t1 )-(t 4 -t 3 )) / 2。

[0052] 。

[0053] Among them, t 1 and t 2 respectively represent the master clock synchronization message sending and slave clock receiving timestamps, t 3 and t 4 respectively represent the slave clock delay request and master clock response timestamps, e(k) represents the master-slave clock deviation of the k-th sampling, ΔT represents the control period, K p 、K i and K d respectively represent the proportional, integral and differential coefficients.

[0054] It can be understood that when the panoramic camera and the radar are working, the internal temperature will also rise, and the rise in temperature will also affect the working performance of the device, which may further affect the timestamp synchronization of the panoramic camera and the radar. Therefore, in this embodiment, the temperature-frequency drift compensation module can be integrated to further optimize the timestamp synchronization process of the panoramic camera and the radar. For the convenience of understanding, the specific working process of the temperature-frequency drift compensation module will be described in detail below.

[0055] Specifically, based on the chip temperature collected by the temperature sensor built in the panoramic camera-radar spatio-temporal synchronization architecture, temperature-frequency compensation is performed on the master-slave clock deviation, and non-linear correction compensation is performed on the output of the voltage-controlled oscillator according to the pre-stored temperature-frequency characteristic curve. The compensation amount Δf comp is directly superimposed with the frequency adjustment amount Δf(k) output by the PID controller to form a composite control signal. The calculation formula of the compensation amount Δf comp is as follows: Δf comp =α(T-T ref )+β×dT / dt.

[0056] Among them, α and β respectively represent the device characteristic coefficients, and the specific values are selected according to the device type; T represents the temperature acquisition amount; T ref represents the reference temperature, and the specific value of T ref can be set by those skilled in the art according to their actual needs.

[0057] In this embodiment, such as Figure 2As shown in the figure, after the construction of the panoramic camera-radar spatio-temporal synchronization architecture is completed, the spatio-temporal alignment design can be carried out for the picture resolution of the panoramic camera and the scanning frequency of the radar. The specific design mainly includes the hardware-level time synchronization process and the data buffering and interpolation compensation process; for the convenience of understanding, the following will describe these two processes in detail.

[0058] I. Regarding the hardware-level time synchronization process.

[0059] Generate a global synchronization clock signal based on the PTP protocol and distribute it to each data acquisition module through the dedicated global wiring resources of the FPGA to ensure the phase consistency of the clock signal. The timestamp marking unit adopts a two-stage latch structure; the first stage latches the external trigger signal, and the second stage is synchronized in the system clock domain, using a dual trigger chain to eliminate the timing deviation caused by metastability. For dynamic adjustment requirements, a programmable trigger synchronization module is integrated, and the image acquisition and radar trigger pulses are dynamically generated through a numerically controlled oscillator (NCO).

[0060] To adapt to the dynamic switching of the panoramic camera picture resolution and the radar scanning frequency, the timestamp marking unit integrates an acquisition duration compensation module. The panoramic camera and the radar share the same PTP clock source to ensure that the timestamps after compensation and correction are based on the unified time axis. The correction formulas for the panoramic camera picture timestamp and the radar timestamp inheriting the trigger and clock parameters are respectively: t cam_corrected =t trigger +A×R width ×R height ×t pixel 。

[0061] t radar_corrected =t trigger +B / f radar 。

[0062] Among them, t cam_corrected and t radar_corrected respectively represent the timestamps after correction of the panoramic camera and the radar, and t trigger represents the timestamp of the original trigger signal; R width and R height respectively represent the horizontal and vertical resolution parameters of the image; t pixel represents the single-pixel processing time, A and B represent the corresponding correction coefficients, and f radar represents the scanning frequency of the radar.

[0063] It should be known that the timestamps of the panoramic camera and the radar are dynamically loaded through the AXI-Lite interface, and the corresponding correction coefficients A and B are automatically called after the trigger signal is generated to achieve the adaptive timing compensation of the panoramic camera picture resolution and the radar scanning frequency.

[0064] II. For the data buffering and interpolation compensation process.

[0065] Design a dual-buffer architecture to solve the data rate difference and instantaneous jitter problems, and first set up buffers for panoramic camera images and radar data. First, adopt the ping-pong operation strategy for the image buffer (Buffer A), that is, when writing to one buffer, the other buffer can be read, and the two are used alternately to avoid data conflicts or waiting caused by simultaneous reading and writing of the same buffer; this design allows the write and read operations to seamlessly switch between the two buffers, thus effectively eliminating the data flow breakpoints.

[0066] The specific address (addr) mapping logic is: addr = Buffer_index × S frame + y × W × x; where Buffer_index represents the buffer search function, S frame represents the single-frame data capacity, W represents the image width, and y and x represent the current pixel row number and column number respectively.

[0067] Then implement the priority queue management based on timestamps for the radar buffer (Buffer B), that is, ensure the real-time nature of the data in the radar buffer by dynamically deleting expired data whose survival time exceeds the set period.

[0068] After completing the timestamp design of the image buffer and the radar buffer, the timestamps of the two can be compared. When the detected timestamps and position differences between the panoramic camera image and the radar are not obvious, it can be determined that the timestamps of the panoramic camera image and the radar are in an aligned state; while when the detected timestamps and position differences between the panoramic camera image and the radar are too large, linear interpolation compensation can be started to generate the fused target position, and then align the spatio-temporal benchmarks of the panoramic camera and the radar.

[0069] The timestamps and position differences between the panoramic camera image and the radar can be represented by the error function E(t); the expression of the error function E(t) is: .

[0070] Among them, a and b respectively represent the contribution ratio weight coefficients for adjusting the time error and the space error, E th represents the error judgment threshold, and the spatio-temporal benchmarks of the panoramic camera and the radar are aligned only when the error function meets the conditions of the error judgment threshold, P cam (t) and P radar (t) respectively represent the spatial position coordinates of the image data and the radar data.

[0071] For the interpolation function P comp (t) used to generate the fused target position, the expression is: 。

[0072] Among them, P comp (t) represents the compensated spatial position, and P(t k ) represents the position in the k-th frame of data, and t k represents the calibrated alignment timestamp, t represents the current time point for interpolation, and they are combined to form interpolation weights and pre-stored in the lookup table to avoid real-time division operations.

[0073] In this embodiment, for step S200, real-time perception in the edge computing scenario can be achieved through the deep fusion of vision and radar algorithms and the efficient scheduling of hardware resources. The core of the algorithm includes the collaborative deployment of the YOLO v11 vision detection model + Bytetrack tracking algorithm and the radar tracking algorithm, as well as the heterogeneous collaborative architecture of the FPGA and the edge computing platform. For easy understanding, specific descriptions will be given below.

[0074] Specifically, the vehicle detection model mainly tracks vehicles through vision algorithms. The specific construction process of the vehicle detection model is as follows: An improved YOLO v11 model is used as the basic detection framework, and depthwise separable convolutions are introduced to reconstruct the backbone network, so that the 3×3 standard convolution layer can be replaced in the Darknet backbone network, thereby reducing the computational complexity. At the same time, a cross-scale attention mechanism is integrated in the feature pyramid layer, and then at the P4 / P5 output nodes, the high-resolution feature map F high and the low-resolution feature map F low are interactively fused to enhance the feature extraction ability for small-scale targets (distant vehicles).

[0075] For specific feature fusion, it can be represented by the channel weight vector W, that is, by aligning the upsampling of the low-resolution feature map F low and the upsampling of the high-resolution feature map F high in terms of size, so that the channel weight vector W can be generated. After obtaining the corresponding channel weight vector W, the required enhanced feature F out can be calculated through the corresponding feature enhancement calculation formula. The calculation formulas for the channel weight vector W and the enhanced feature F out are as follows: 。

[0076] 。

[0077] Among them, represents element-wise addition, GAP represents global average pooling, σ represents the Sigmoid activation function, MLP represents a multi-layer perceptron, represents channel-wise multiplication, represents the upsampling of the low-resolution feature map.

[0078] It should be noted that during the normal operation of the vehicle detection model, in order to meet the real-time requirements of the edge computing platform Jetson, the vehicle detection model can be optimized and deployed based on the TensorRT engine. The convolution, batch normalization, and activation functions are merged into a single computing unit through layer fusion technology to reduce the kernel call overhead. At the same time, the INT8 quantization strategy is adopted, and the calibration dataset covering day and night and harsh weather scenarios is used to determine the dynamic range of activation values, significantly reducing the computing resource occupancy while ensuring the detection accuracy. Finally, combined with the Bytetrack multi-object tracking algorithm for the YOLO detection results, the Hungarian algorithm is used for matching based on the intersection over union (IOU) of the detection boxes and the Kalman filter results of motion prediction, thus completing the trajectory association of the targets.

[0079] Specifically, the radar tracking algorithm mainly performs multi-stage processing on the point cloud data of the lidar. First, the influence of the radar's own motion on the point cloud coordinates is eliminated through the motion compensation algorithm, and the original point cloud is subjected to coordinate transformation based on the vehicle's real-time speed and heading angle. Then, an improved density clustering algorithm is adopted to calculate the average distance d of each radar point cloud data point to its nearest k neighbors i_avg ; thus, the neighborhood radius ε can be adaptively adjusted according to the point cloud distribution density i , realizing stable target separation in a dynamic environment. The dynamic setting formula for the neighborhood radius ε i is: ε i = max(ε min , μ, d i_avg ); where, ε min represents the preset minimum radius, and μ represents the density sensitivity coefficient.

[0080] Specifically, the collaborative deployment based on the vehicle detection model and the radar tracking algorithm includes the following process: In the multi-object tracking stage, the target state is predicted through the extended Kalman filter combined with the uniform motion model, and the data association cost matrix is constructed by jointly constraining the Mahalanobis distance and the Doppler velocity, and then the cross-modal matching of the radar and visual trajectories is performed. The cost matrix is constructed by calculating the joint matching cost C ij between the radar trajectory and the visual trajectory. The calculation formula for the joint matching cost C ij is as follows: .

[0081] Among them, D Mahalanobis represents the Mahalanobis distance, λ represents the weight coefficient, represents the radial velocity of the i-th radar trajectory, represents the projection velocity of the j-th visual target in the radar radial direction.

[0082] After completing the calculation of the cost matrix, based on the spatio-temporal alignment mechanism, the radar point cloud is projected into the image coordinate system through the pre-calibrated external parameter matrix. Combining with the hardware-level synchronous timestamp, an FPGA generates a synchronous pulse signal, thereby triggering the camera exposure and radar scanning simultaneously, so as to eliminate the acquisition time difference between the panoramic camera and the lidar sensors.

[0083] In this embodiment, step S200 constructs an FPGA-Jetson heterogeneous cooperative architecture to improve the overall system efficiency through hardware resource division of labor and zero-copy data transmission for efficient operation of the algorithm. The FPGA side focuses on low-latency and high-deterministic radar data preprocessing tasks, including motion compensation calculation, point cloud filtering, and coordinate system conversion, and uses the parallel processing ability of programmable logic to accelerate fixed-point matrix operations. At the same time, the FPGA generates a hardware synchronization trigger signal to control the acquisition timings of the panoramic camera and the radar, ensuring the timing consistency of multi-sensor data. The Jetson platform undertakes visual detection, tracking algorithms, and multi-modal data fusion tasks, runs the optimized YOLO v11 model through the TensorRT inference engine accelerated by the GPU, and manages the target trajectory status based on the Bytetrack algorithm.

[0084] Specifically, as Figure 2 shown, the two platforms are interconnected through a PCIe high-speed bus and adopt a shared memory zero-copy transmission mechanism; that is, the FPGA writes the preprocessed radar data into a fixed physical address area through direct memory access (DMA), and the Jetson side directly accesses this area through memory mapping technology, avoiding the data copy delay through the CPU in the traditional scheme. The data interaction protocol defines a unified data header structure, including a timestamp, a data type identifier, and a physical address pointer, ensuring the parsing consistency of both ends.

[0085] In this embodiment, the heterogeneous cooperative architecture of step S200 dynamically allocates computing resources according to the real-time requirements of tasks in terms of the cooperative scheduling mechanism. The FPGA side adopts a double-buffer storage strategy, alternately writing and reading data to ensure continuous processing of the radar data pipeline, solidifying the computationally intensive and low-latency radar processing tasks in the FPGA, and releasing the Jetson resources for complex vision algorithms. The Jetson side manages computing tasks through a priority scheduler, responds to the interrupt signal of the FPGA in real time, and preferentially processes the radar preprocessed ready data.

[0086] Specifically, as Figure 2As shown, when working in the FPGA-Jetson heterogeneous collaborative architecture, a physical connection is achieved between the FPGA side and the Jetson side through a PCIe Gen3 x8 interface, supporting high-bandwidth data transmission. The FPGA side uses a hardware description language to implement a preprocessing pipeline and synchronization control logic, and the Jetson side integrates multi-threaded task scheduling and algorithm modules based on the ROS2 framework. Compared with the traditional method, this application provides low-latency and highly reliable multi-modal perception capabilities for intelligent transportation scenarios through the deep combination of algorithm optimization and hardware collaboration.

[0087] Generally speaking, step S2002 is based on the FPGA-Jetson heterogeneous collaborative architecture, achieving a deep combination of algorithm acceleration and resource optimization. The core advantages are based on hardware-level division of labor (the FPGA is responsible for radar data preprocessing and synchronous triggering, and the Jetson is equipped with an improved YOLO v11 model accelerated by TensorRT and the Bytetrack tracking algorithm) and the PCIe zero-copy transmission mechanism. Combining cross-scale attention feature fusion, radar dynamic density clustering, and Doppler-Mahalanobis distance joint tracking algorithms, while ensuring the spatio-temporal alignment accuracy, the computing resources are dynamically allocated through a double-buffer pipeline and a priority scheduling strategy. Using INT8 quantization and hardware collaborative optimization, end-to-end low-latency processing and high detection accuracy of multi-modal data in complex traffic scenarios are achieved, significantly improving the real-time performance and energy efficiency ratio of the system.

[0088] In this embodiment, step S300 accurately predicts future traffic flow by constructing a CNN-LSTM dual-channel spatio-temporal feature fusion model. Combining the dynamic parameter mapping rule and the FPGA programmable timing compensation mechanism, the real-time switching of resolution / frequency parameters is realized at the hardware layer, so as to ensure that the spatio-temporal synchronization error ≤ 1ms, and at the same time, the invalid energy consumption is reduced through the sensor energy efficiency control strategy, achieving a dynamic balance between perception accuracy and hardware resource consumption.

[0089] Specifically, as Figure 2 shown, the construction of the traffic flow prediction model in step S300 includes the following process: using a multi-node spatio-temporal correlation matrix as the input, integrating the historical traffic flow data of the target area and adjacent highly correlated monitoring points in the horizontal dimension, and constructing the temporal features of continuous time segments in the vertical dimension, so as to form a two-dimensional spatio-temporal feature vector. The model architecture adopts a CNN-LSTM dual-channel structure. The first layer is configured with multiple convolutional kernels for spatial feature extraction, and the feature dimension is compressed through a max-pooling layer. After being transformed by the Flatten layer, the spatio-temporal feature vector is input into the LSTM network, and the memory unit is used to capture bidirectional temporal dependencies, and a gated recurrent mechanism is established in the time dimension. The output layer introduces a fully connected network for multi-scale prediction, and simultaneously outputs the traffic flow value at the future prediction moment.

[0090] It should be noted that the number of convolution kernels configured in the first layer can be set according to the actual needs of those skilled in the art, for example, set to 10, 20, or 30. The number of memory units in the LSTM network can also be set according to the actual needs of those skilled in the art, for example, set to 40, 50, or 60.

[0091] Specifically, the adaptive collaborative optimization design for step S300 includes the following process: According to the panoramic camera resolution and radar scanning frequency configuration, the predicted traffic flow is divided into three levels: low, medium, and high. A hysteresis threshold is introduced so that when the predicted level changes, it is necessary to satisfy that the traffic flow fluctuation exceeds the preset hysteresis threshold to trigger an adjustment. When performing a mode switch based on the change in the traffic flow gear level, resolution and frequency adjustment instructions are generated, and the adjustment instructions are transmitted to the configuration register of the FPGA through the PCIe interface. Then, the panoramic camera-radar spatio-temporal synchronization architecture calculates the hardware timing parameters in real time according to the new panoramic camera resolution and radar frequency and performs timestamp alignment, thereby ensuring the timing consistency under dynamic switching of hardware resources.

[0092] It should be noted that based on the panoramic camera resolution and radar scanning frequency configuration, the gear division of the predicted traffic flow can be set according to the actual needs of those skilled in the art. For example, it can be divided into low gear: 720P / 10Hz, medium gear: 1080P / 20Hz, and high gear: 4K / 30Hz. The preset hysteresis threshold can be set according to the actual needs of those skilled in the art. For example, the hysteresis threshold can be set to ±10%, that is, when the predicted level changes, it is necessary to satisfy that the traffic flow fluctuation exceeds ±10% to trigger an adjustment, avoiding invalid mode switching caused by short-term fluctuations. Compared with the traditional method, the present application dynamically adjusts the panoramic camera resolution and radar frequency through the CNN+LSTM traffic flow prediction result to optimize the hardware resource call, and when performing dynamic resource scheduling, uses the programmability of the FPGA to adjust parameters to maintain time synchronization.

[0093] Specifically, when calculating the hardware timing parameters, the calculation formula for the trigger pulse parameters is: N reload_cam =f clk / f frame ; T cam =k cam ×N reload_cam / f clk .

[0094] N reload_radar =f clk / f radar ; T radar =k radar ×N reload_radar / f clk .

[0095] Among them, N reload_cam and N reload_radar respectively represent the counter reload values of the panoramic camera and the radar; k cam and k radar respectively represent the programmable phase offset coefficient and the radar phase offset coefficient; f frame represents the image frame rate, and f clk represents the FPGA main clock frequency.

[0096] In addition, when performing mode switching, the correction coefficients A and B corresponding to the timestamp correction of the panoramic camera and the radar need to be updated. The specific update calculation formulas are as follows: A = A base ((R width ×R height ) / (R base_width ×R base_height )) 0.8 , B = B fix + ω / f radar .

[0097] Among them, A base represents the calibration value of the coefficient, which can be determined according to the calibration test specifically; R base_ represents the reference resolution, B fix represents the fixed delay, and ω represents the frequency-related term.

[0098] The above describes the basic principle, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited by the above embodiments. What is described in the above embodiments and the specification is only the principle of the present application. Without departing from the spirit and scope of the present application, the present application will have various changes and improvements, and these changes and improvements all fall within the scope of the present application claimed. The scope of protection claimed by the present application is defined by the appended claims and their equivalents.

Claims

1. A FPGA accelerated fusion method for dynamic resource data of a service area parking lot, characterized in that: The steps include: S100: Build a panoramic camera-radar spatiotemporal synchronization architecture based on FPGA hardware acceleration, and align the timestamps of the service area panoramic camera images with the radar point cloud in real time through FPGA parallel computing; S200: Co-deploy the vehicle detection model based on visual detection and the radar tracking algorithm on the edge computing platform, and build a heterogeneous collaborative architecture of FPGA and edge computing platform; S300: Construct a traffic flow prediction model of convolutional neural network-long short-term memory network, and adaptively coordinate and optimize the adjustment mechanism of the obtained traffic flow prediction model and the hardware parameters of the panoramic camera in the service area.

2. The FPGA accelerated fusion method for dynamic resource data of a service area parking lot as claimed in claim 1, characterized in that: The construction of the panoramic camera-radar spatiotemporal synchronization architecture in step S100 includes the following processes: The period of the synchronization pulse signal generated by the FPGA hardware trigger circuit is matched with the image acquisition frame rate, and the panoramic camera array and radar are driven through the low voltage differential signal interface and transistor-transistor logic level respectively; Based on the collaborative design of the hardware-level timestamp marking unit and the closed-loop clock taming mechanism, the timestamp injection of the message is carried out through the collaboration of the physical layer and the protocol stack; the master-slave clock deviation is calculated according to the timestamp message exchange and dynamic compensation is performed.

3. The FPGA accelerated fusion method for dynamic resource data of a service area parking lot as claimed in claim 2, characterized in that: The calculation formula of the master-slave clock deviation Δt is: Δt=((t2-t1)-(t4-t3)) / 2; The master-slave clock deviation is compensated by generating a frequency adjustment value Δf(k) through a PID controller. The calculation formula of the frequency adjustment value Δf(k) is as follows: ； Where t1 and t2 represent the timestamps of the master clock synchronization message sending and the slave clock receiving, respectively; t3 and t4 represent the slave clock delay request and the master clock response timestamp, respectively; e(k) represents the master-slave clock deviation of the kth sampling; ΔT represents the control period; K p , K i and K d They represent proportional, integral and differential coefficients respectively.

4. The FPGA accelerated fusion method for dynamic resource data of a service area parking lot as claimed in claim 3, characterized in that: Perform temperature-frequency compensation on the master-slave clock deviation, compensation amount Δf comp Directly superimpose the frequency adjustment value output by the PID controller; compensation value Δf comp The calculation formula is as follows: Δf comp =α(T-T ref )+β×dT / dt; Among them, α and β represent the device characteristic coefficients, T represents the temperature acquisition amount, and T ref Indicates the reference temperature.

5. The FPGA accelerated fusion method for dynamic resource data of a service area parking lot as claimed in claim 1, characterized in that: In step S100, in order to adapt to the dynamic switching of the panoramic camera image resolution and the radar scanning frequency, the panoramic camera timestamp and the radar timestamp are corrected based on timing compensation. The specific correction formula is as follows: t cam_corrected =t trigger +A×R width ×R height ×t pixel ; t radar_corrected =t trigger +B / f radar ; Among them, t cam_corrected and t radar_corrected Respectively represent the timestamps after panoramic camera and radar correction, t trigger Indicates the timestamp of the original trigger signal; R width and R height Respectively represent the horizontal and vertical resolution parameters of the image; t pixel represents the single pixel processing time, A and B represent the corresponding correction coefficients, f radar Indicates the scanning frequency of the radar.

6. The FPGA accelerated fusion method for dynamic resource data of a service area parking lot as claimed in claim 1, characterized in that: When performing the timestamp alignment of the panoramic camera and the radar in step S100, if it is detected that the timestamp and position difference between the panoramic camera image and the radar is too large, the linear interpolation compensation is started to generate the fusion target position, and then the time and space references of the panoramic camera and the radar are aligned; the timestamp and position difference between the panoramic camera image and the radar are represented by the error function E(t); the error function E(t) and the interpolation function P for generating the fusion target position are comp The expressions of (t) are respectively: ； ； Among them, a and b represent the contribution ratio weight coefficients of adjusting the time error and space error respectively, P cam (t) and P radar (t) represent the spatial position coordinates of image data and radar data, respectively, E th represents the error judgment threshold, P comp (t) represents the spatial position after compensation, P(t k ) represents the position in the k-th frame data, t k Represents the corrected alignment timestamp.

7. The FPGA accelerated fusion method for dynamic resource data of a service area parking lot as claimed in claim 1, characterized in that: In step S200, the construction of the vehicle detection model includes the following process: The improved YOLO v11 model is used as the basic detection framework, and a deep separable convolutional reconstruction backbone network is introduced; Integrate the cross-scale attention mechanism at the feature pyramid layer, and then use the P4 / P5 output node to calculate the high-resolution feature map F of adjacent scales. high With the low-resolution feature map F low Interactive fusion is performed to enhance the small-scale target features; the enhanced features F out The expression is as follows: ； ； Among them, W represents the channel weight vector, represents element-by-element addition, GAP represents global average pooling, σ represents Sigmoid activation function, MLP represents multi-layer perceptron, represents channel-level multiplication, Represents low-resolution feature map upsampling.

8. The FPGA accelerated fusion method for dynamic resource data of a service area parking lot according to any one of claims 1 to 7, characterized in that: The coordinated deployment of the vehicle detection model and radar tracking algorithm includes the following processes: In the multi-target tracking stage, the target state is predicted by combining the extended Kalman filter with the uniform motion model, and the data association cost matrix is ​​constructed by the joint constraint of the Mahalanobis distance and Doppler velocity, so as to achieve cross-modal matching of radar and visual trajectories. Based on the spatiotemporal alignment mechanism, the radar point cloud is projected into the image coordinate system through a pre-calibrated external parameter matrix. Combined with the hardware-level synchronization timestamp, a synchronization pulse signal is generated through the FPGA, so that the camera exposure and radar scanning are triggered simultaneously.

9. The FPGA accelerated fusion method for dynamic resource data of a service area parking lot as claimed in claim 8, characterized in that: Calculate the joint matching cost C by using the radar trajectory and the visual trajectory ij To construct the cost matrix, the joint matching cost C ij The calculation formula is as follows: ； Among them, D Mahalanobis represents the Mahalanobis distance, λ represents the weight coefficient, represents the radial velocity of the ith radar track, Represents the projection speed of the jth visual target in the radar radial direction.

10. The FPGA accelerated fusion method for dynamic resource data of a service area parking lot according to claim 1, characterized in that: In step S300, the construction of the traffic flow prediction model includes the following process: Using a multi-node spatiotemporal correlation matrix as input, the historical traffic flow data of the target area and adjacent high-correlation monitoring points are integrated in the horizontal dimension, and the time series features of continuous time segments are constructed in the vertical dimension, thus forming a two-dimensional spatiotemporal feature vector; The model architecture adopts a CNN-LSTM dual-channel structure. The first layer is configured with multiple convolution kernels for spatial feature extraction, and the feature dimension is compressed through the maximum pooling layer. The spatiotemporal feature vector is converted by the Flatten layer and then input into the LSTM network. The memory unit is used to capture the bidirectional temporal dependency and a gated loop mechanism is established in the time dimension. The output layer introduces a fully connected network for multi-scale prediction and simultaneously outputs the traffic flow value at the future prediction time.

11. The FPGA accelerated fusion method for dynamic resource data of a service area parking lot according to claim 1, characterized in that: In step S300, the adaptive collaborative optimization design includes the following process: According to the panoramic camera resolution and radar scanning frequency configuration, the predicted traffic flow is divided into three levels: low, medium and high. A hysteresis threshold is introduced so that when the predicted level changes, the adjustment is triggered only when the traffic flow fluctuation exceeds the preset hysteresis threshold. When switching modes based on changes in vehicle flow levels, resolution and frequency adjustment instructions are generated, and the panoramic camera-radar spatiotemporal synchronization architecture calculates hardware timing parameters and performs timestamp alignment in real time based on the new panoramic camera resolution and radar frequency.

12. The FPGA accelerated fusion method for dynamic resource data of a service area parking lot according to claim 11, characterized in that: When calculating the hardware timing parameters, the calculation formula for the trigger pulse parameters is: N reload_cam =f clk / f frame ; T cam =k cam ×N reload_cam / f clk ; N reload_radar =f clk / f radar ; T radar =k radar ×N reload_radar / f clk ; Among them, N reload_cam and N reload_radar Respectively represent the counter reload values ​​of the panoramic camera and radar; k cam and k radar Respectively represent the programmable phase offset coefficient and radar phase offset coefficient; f frame represents the image frame rate, f radar represents the scanning frequency of the radar, f clk Indicates the FPGA main clock frequency.

Images