An automatic driving perception decision decoupling FPGA resource scheduling method
By decoupling the perception and decision-making links in autonomous driving systems through FPGA resource scheduling methods and using dynamic maps of driving conditions for data synchronization and fusion, the latency and hardware adaptability issues caused by the coupling of perception and decision-making in traditional platforms are solved, achieving efficient sensor configuration adaptation and system robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UNIV OF SCI & TECH OF CHINA
- Filing Date
- 2026-04-23
- Publication Date
- 2026-07-14
AI Technical Summary
In existing autonomous driving computing platforms, the perception model and decision control algorithm are deeply coupled, which leads to a surge in system latency in high-concurrency scenarios, making it difficult to meet the millisecond-level real-time requirements. Furthermore, when the sensor configuration is changed, recompilation and restart are required, resulting in a lack of hardware flexibility.
By employing FPGA resource scheduling methods, a reconfigurable sensor memory interface module and on-chip high-speed storage resources are constructed to achieve hardware-level decoupling of the perception and decision-making links. Data synchronization and fusion are performed using dynamic maps of driving conditions, and local logic is refreshed when sensor configurations change.
It achieves complete decoupling of the perception and decision-making links, improves the system's adaptability and hardware flexibility under varying sensor configurations, reduces interaction latency, and enhances the system's robustness and cross-vehicle portability.
Smart Images
Figure CN122086632B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of autonomous driving technology, and in particular to an FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving. Background Technology
[0002] In the field of autonomous driving technology, the onboard heterogeneous computing platform is the physical core for realizing high-level autonomous driving. With the evolution of autonomous driving technology, the types and number of sensors on vehicles have increased dramatically, placing extremely high demands on the underlying computing power for the spatiotemporal alignment and fusion processing of multi-source heterogeneous sensor data. Currently, most mainstream autonomous driving computing platforms adopt a system-on-a-chip (SoC) architecture based on a central processing unit (CPU) and a graphics processing unit (GPU).
[0003] However, these traditional architectures have significant limitations. The perception model and decision-making control algorithm are deeply coupled within the software framework. Coordinate transformation and spatiotemporal alignment of heterogeneous data consume a large amount of central processing unit (CPU) computing power, leading to a surge in system latency under high concurrency scenarios, making it difficult to meet millisecond-level real-time requirements. Furthermore, most underlying hardware resources adopt a static allocation strategy. When the vehicle's sensor configuration undergoes physical changes, the entire system software stack typically needs to be recompiled, flashed, and restarted. This lack of underlying hardware flexibility poses challenges to system robustness and R&D / maintenance costs.
[0004] Therefore, how to completely decouple the perception and decision-making links at the underlying hardware level and improve adaptability under varying sensor configurations has become a pressing technical challenge. Summary of the Invention
[0005] The main objective of this invention is to provide an FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving, aiming to completely decouple the perception and decision-making links at the underlying hardware level and improve adaptability under varying sensor configurations.
[0006] To achieve the above objectives, this invention proposes an FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving, comprising the following steps:
[0007] Step S1: Receive multi-source heterogeneous sensor data using the reconfigurable sensor memory interface module of the FPGA front-end dynamic logic area, perform spatiotemporal synchronization of the underlying signals using the built-in hardware timestamp synchronizer, and normalize and project the multi-source heterogeneous sensor data into a unified standard format driving situation dynamic map data frame.
[0008] Step S2: Construct a dual-port buffer using the cognitive fusion and working memory module of the FPGA on-chip high-speed storage resources to maintain and refresh the driving situation dynamic map in real time. The driving situation dynamic map is constructed as a multi-channel hierarchical spatiotemporal tensor as a medium for on-chip data exchange.
[0009] Step S3: Using the cognitive decision module deployed in the static logic area of the FPGA backend, the driving situation dynamic map is read directly from the cognitive fusion and working memory module at a fixed time sequence, and logical reasoning is performed in combination with the prior knowledge retrieved from the long-term memory interface to generate control commands.
[0010] Step S4: Define the dynamic driving situation map as the standardized boundary of the software and hardware interaction; when a sensor configuration change is detected, trigger a reconstruction interrupt signal, use dynamic partial reconstruction technology to refresh only the logic circuit corresponding to the sensory memory interface module, and keep the operating state and resource allocation of the cognitive decision module unchanged.
[0011] Preferably, in step S1, performing spatiotemporal synchronization of the underlying signals includes: setting the global heartbeat clock period inside the FPGA to be... The time difference between two adjacent sampling times is obtained using the vehicle's inertial measurement unit. pose transformation matrix within Translation vector Asynchronous sensor data is aligned to the current heartbeat moment using a linear interpolation algorithm based on vehicle motion compensation. The alignment formula is shown in equation (1):
[0012]
[0013] in, and These are the observation data at adjacent sampling times, The time interval for observing the data. Based on the current moment exist The time weighting coefficient for position calculation within the interval is calculated using the formula shown in equation (2):
[0014] .
[0015] Preferably, in step S2, the dual-port buffer is constructed as a ping-pong operation dual-buffered queue using the UltraRAM resources inside the FPGA; the ping-pong operation dual-buffered queue includes a first address space repository Bank1 and a second address space repository Bank2; by alternating the flipping of the read and write pointers in adjacent time frames, the operation is achieved in the first... While writing the perception result to the repository Bank1 in the first address space in each time frame, the system reads the perception result from the repository Bank2 in the second address space. Decisions are made based on map data from each time frame, where It is a natural number greater than 1.
[0016] Preferably, in step S2, the driving situation dynamic map is constructed as a multi-channel hierarchical spatiotemporal tensor centered on the vehicle. ,in and The preset bird's-eye view grid size, The number of feature channels includes: geometric feature channels representing the obstacle occupancy state and velocity vector, semantic feature channels representing the distribution of lane lines and passable areas, and cognitive potential field channels representing the distribution of driving risks.
[0017] Preferably, the method further includes: targeting the grid in the geometric feature channel. , perform Bayesian occupying grid update; the update process adopts log odds form, and the calculation formula is shown in equation (3):
[0018]
[0019] in, The logarithmic odds at the current moment. The logarithmic odds of the previous moment. For the current observation value, The grid occupancy state is defined; and a block RAM-based lookup table is pre-built in the FPGA to convert the floating-point logarithmic operations in the log probability update formula into fixed-point lookup table operations.
[0020] Preferably, the method further includes: generating a velocity-adaptive asymmetric two-dimensional Gaussian risk potential field in the cognitive potential field channel; for the center coordinates being For dynamic obstacles, calculate the risk value they generate in the surrounding grid. The calculation formula is shown in equation (4):
[0021]
[0022] in, For magnitude weighting, The heading angle of the obstacle's movement. For the horizontal variance, The longitudinal variance, which dynamically adjusts with speed, is calculated using the formula shown in equation (5):
[0023]
[0024] in, Based on the variance, For speed sensitivity coefficient, and Estimate the velocity components for obstacles.
[0025] Preferably, step S3 includes: the cognitive decision-making module extracts local high-precision map data through a long-term memory interface, and uses it as a base mask to stitch the driving situation dynamic map in the cognitive fusion and working memory module together using feature channels.
[0026] Preferably, in step S3, the generation of the control command includes: sampling and generating a candidate trajectory cluster in the space in front of the vehicle; and using the DSP slice array on the back end of the FPGA to perform parallel calculations of each candidate trajectory. Total cost The calculation formula is shown in equation (6):
[0027]
[0028] in, For the sake of smoothness, Cost of deviation from reference path The cognitive potential risk cost is the risk value of all dynamic map grids covered by the predicted trajectory. The sum of the sums, , , The preset weight coefficients are used; the optimal trajectory with the lowest cost is selected using a hardware comparator tree, and the optimal trajectory is subjected to amplitude limiting filtering in combination with the vehicle dynamics limit rule library; and when the optimal trajectory triggers the physical limit threshold, a degradation instruction is output through a hardware circuit breaker mechanism.
[0029] Preferably, in step S4, performing on-chip resource dynamic scheduling includes: using a hardware watchdog timer to monitor valid signals of sensor channel data, and triggering a reconstruction interrupt signal when an abnormal heartbeat packet is detected or a sensor configuration change is caused by hardware hot-plugging; and using a resource scheduler to execute a prefetch strategy to prefetch the spare interface logic bit stream from external memory to the on-chip cache, and using the on-chip hard-core AES engine to perform decryption.
[0030] Preferably, after the reconstruction interrupt signal is triggered, the following steps are performed: under the premise of maintaining the normal operation of the power supply and clock distribution network of the cognitive fusion and working memory module and the cognitive decision module, the logic circuit of the sensory memory interface module is erased and rewritten through the internal configuration access interface using an isolation design process.
[0031] The above technical solution has the following advantages:
[0032] This invention achieves complete decoupling of the perception and decision-making links at the hardware level by constructing a reconfigurable sensory memory interface module in the dynamic logic area of the FPGA front end and using on-chip high-speed storage resources to build a dynamic driving situation map as a standardized boundary for software and hardware interaction. This architecture design forces the front-end perception output to follow a unified tensor format, enabling the back-end decision-making module to read data in a fixed timing sequence. Thus, when sensor configuration changes or hardware hot-swapping occurs, only the front-end logic needs to be partially refreshed using dynamic partial reconstruction technology, while keeping the back-end decision-making logic and resource allocation unchanged, significantly improving the system's cross-vehicle portability and hardware flexibility. Simultaneously, by using on-chip UltraRAM resources to construct a dual-port buffer, zero-copy pipelined operation for perception writing and decision reading is achieved, eliminating the bandwidth bottleneck caused by data transport in traditional architectures and ensuring extremely low interaction latency between perception and decision. Furthermore, by constructing a multi-channel hierarchical spatiotemporal tensor, geometric, semantic, and cognitive potential field information is integrated, providing the decision-making module with a high-dimensional environmental representation with spatiotemporal continuity, enhancing the system's safety and robustness in complex traffic scenarios. Attached Figure Description
[0033] The present invention will now be described in detail with reference to specific embodiments and accompanying drawings, wherein:
[0034] Figure 1 This is a flowchart of an FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving, provided as an embodiment of the present invention.
[0035] Figure 2 This invention provides an FPGA overall hardware architecture and functional module division diagram for autonomous driving perception and decision decoupling in an embodiment of the present invention.
[0036] Figure 3 This is a schematic diagram of a ping-pong operation double-buffered queue and a dynamic map tensor structure for driving status, provided as an embodiment of the present invention. Detailed Implementation
[0037] The technical solution provided by the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described below are intended to explain the technical concept of the present invention through specific examples, and are not intended to limit its scope of protection in any way. Any equivalent substitutions or improvements made by those skilled in the art regarding specific technical details without departing from the concept of the present invention fall within the scope of protection of the present invention.
[0038] Example 1
[0039] like Figures 1 to 3As shown, this embodiment provides an FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving. Its core lies in constructing a dynamic, cognitively intelligent environmental representation medium—a dynamic driving situation map—on-chip within the FPGA through a standardized hardware architecture design. This achieves complete decoupling of the perception and decision-making links at the hardware level. This method effectively addresses the pain points of traditional heterogeneous computing platforms, such as difficulties in cross-vehicle portability due to deep coupling between perception and decision-making, the enormous CPU computational power consumption for spatiotemporal alignment of heterogeneous sensor data, and the difficulty of adapting static on-chip resource allocation to dynamic sensor changes.
[0040] During system initialization or operation, step S1 is executed first, utilizing the reconfigurable sensor memory interface module in the FPGA front-end dynamic logic area to receive data from multiple heterogeneous sensors. This sensor data includes, but is not limited to, image pixel streams acquired by high-resolution cameras, sparse point cloud data output by LiDAR, and target lists output by millimeter-wave radar. To eliminate spatial position offsets caused by inconsistent sampling frequencies of different sensors, the sensor memory interface module uses a built-in hardware timestamp synchronizer to perform spatiotemporal synchronization of the underlying signals. The system sets the global heartbeat clock period within the FPGA to [value missing]. For example, it can be set to 20ms. For asynchronously arriving sensor data, the system uses the vehicle's inertial measurement unit to obtain the time difference between two adjacent sampling times. pose transformation matrix within Translation vector Subsequently, a linear interpolation algorithm based on vehicle motion compensation was used to align the asynchronous sensor data to the current heartbeat moment. The specific alignment formula is shown in equation (1):
[0041]
[0042] in, and These are the observation data at adjacent sampling times, The time interval for observing the data. Based on the current moment exist The time weighting coefficient for position calculation within the interval is calculated using the formula shown in equation (2):
[0043]
[0044] The synchronized data is logically normalized and projected into a unified standard format driving situation dynamic map data frame through coordinate transformation. This process transforms image features into bird's-eye view space through inverse perspective mapping and processes point clouds through voxelization operations to ensure that the output data contains only environmental features and not sensor physical attributes.
[0045] Next, step S2 is executed, utilizing the cognitive fusion and working memory modules of the FPGA's on-chip high-speed storage resources to construct a dual-port buffer area, maintaining and refreshing the dynamic driving situation map in real time. To achieve parallel pipelined operations for perception writing and decision reading, the dual-port buffer area utilizes the FPGA's internal on-chip UltraRAM resources to construct a ping-pong dual-buffer queue. This queue includes a first address space repository Bank1 and a second address space repository Bank2. By alternately flipping the read / write pointers in adjacent time frames, the system achieves the following: While writing the perception results to the first address space's repository Bank1 in the first time frame, the second time frame is read from the second address space's repository Bank2. Decisions are made based on map data from each time frame, where It is a natural number greater than 1.
[0046] In this step, the dynamic map of driving situation is constructed as a multi-channel hierarchical spatiotemporal tensor centered on the vehicle. ,in and The preset grid size for the bird's-eye view is, for example, a range of 100m in front and 80m in the side. This refers to the number of feature channels. These feature channels include geometric feature channels representing obstacle occupancy states and velocity vectors, semantic feature channels representing lane lines and passable area distributions, and cognitive potential field channels representing driving risk distributions. The grid within the geometric feature channels is then considered. The system performs a Bayesian occupancy grid update to filter out noise. The update process uses a log-probability form, and the calculation formula is shown in equation (3):
[0047]
[0048] in, The logarithmic odds at the current moment. The logarithmic odds of the previous moment. For the current observation value, This represents the grid occupancy state. To improve real-time performance, a lookup table based on block RAM is pre-built in the FPGA, converting floating-point logarithmic operations into fixed-point lookup table operations. Furthermore, the system generates a velocity-adaptive asymmetric two-dimensional Gaussian risk potential field in the cognitive potential field channel. This is for a center coordinate of... For dynamic obstacles, calculate the risk value they generate in the surrounding grid. The calculation formula is shown in equation (4):
[0049]
[0050] in, For magnitude weighting, The heading angle of the obstacle's movement. For the horizontal variance, The longitudinal variance, which dynamically adjusts with speed, is calculated using the formula shown in equation (5):
[0051]
[0052] in, Based on the variance, This is the speed sensitivity coefficient. This asymmetric potential field can simulate human prediction of oncoming high-speed vehicles.
[0053] Then, step S3 is executed, utilizing the cognitive decision module deployed in the static logic area of the FPGA backend to directly read the dynamic driving situation map from the cognitive fusion and working memory module at a fixed timing. The cognitive decision module extracts local high-precision map data through the long-term memory interface, uses it as a base mask, and stitches it with the dynamic map using feature channels. The generation of control commands includes sampling the space in front of the vehicle to generate candidate trajectory clusters, and using the DSP slice array on the FPGA backend to compute each candidate trajectory in parallel. Total cost The calculation formula is shown in equation (6):
[0054]
[0055] in, For the sake of smoothness, Cost of deviation from reference path The cognitive potential risk cost is the risk value of all dynamic map grids covered by the predicted trajectory. The system uses a hardware comparator tree to select the optimal trajectory with the lowest cost, and combines this with a vehicle dynamics limit rule library to perform amplitude limiting filtering on the optimal trajectory. When the optimal trajectory triggers the physical limit threshold, a degradation instruction is output through a hardware circuit breaker mechanism.
[0056] Finally, step S4 is executed, defining the dynamic driving situation map as the standardized boundary for software and hardware interaction. The resource scheduler uses a hardware watchdog timer to monitor valid signals from sensor channels and triggers a reconstruction interrupt signal when it detects abnormal heartbeat packets or sensor configuration changes caused by hardware hot-plugging. Dynamic partial reconstruction technology is used to refresh only the logic circuits corresponding to the sensory memory interface module. During reconstruction, the system uses the resource scheduler to execute a prefetch strategy to prefetch the backup interface logic bitstream from external memory to the on-chip cache and uses the on-chip hard-core Advanced Encryption Standard (AES) engine for decryption. While maintaining the normal operation of the power supply and clock distribution networks for the cognitive fusion and working memory modules and the cognitive decision module, an isolated design process is adopted to erase and rewrite the logic circuits of the sensory memory interface module through the internal configuration access interface. This mechanism ensures that even if the front-end sensors change or are damaged, the back-end decision logic and resource allocation remain unchanged, achieving smooth system degradation and rapid recovery.
[0057] Building upon Example 1, this example further elaborates on the implementation details of the underlying signal processing logic and storage architecture. Example 2 primarily involves a spatiotemporal synchronization enhancement mechanism for sensor data and efficient hardware-based double-buffer management, aiming to solve the perception ghosting problem caused by differences in sampling frequencies of multi-source heterogeneous data in high-speed driving scenarios.
[0058] Specifically, in Embodiment 2, when performing spatiotemporal synchronization of the underlying signals, the sensory memory interface module first obtains the high-frequency pose information output by the vehicle's inertial measurement unit through the system bus. The system sets the global heartbeat clock period inside the FPGA to be... For example, a typical value is 20ms. Since the LiDAR sampling frequency is usually 10Hz, while the camera sampling frequency may be 30Hz or 60Hz, the arrival timestamps of the data from each sensor exhibit significant asynchronous characteristics. To eliminate this asynchronous bias, the system utilizes the vehicle's inertial measurement unit to obtain the timestamps of two adjacent sampling moments. and Time difference between pose transformation matrix within Translation vector Subsequently, a linear interpolation algorithm based on vehicle motion compensation was used to align the asynchronous sensor data to the current system heartbeat moment. This process ensures that the spatial position error of the perceived target is reduced even at high speeds, i.e., when the vehicle speed reaches 120 km / h. The calculation formula used in the alignment process is shown in equation (1):
[0059]
[0060] in, and These are the observation data at adjacent sampling times, The time interval for observing the data, and the time weighting coefficient. Based on the current time In the observation interval The relative position within is determined, and its calculation formula is shown in equation (2):
[0061]
[0062] At the data storage and exchange level, the cognitive fusion and working memory module utilizes the on-chip UltraRAM resources within the FPGA to construct a ping-pong double-buffered queue. This design aims to completely eliminate resource contention between writing at the sensing end and reading at the decision end. This double-buffered queue is physically divided into a first address space (Bank1) and a second address space (Bank2). The read / write pointers, implemented through hardware logic, alternately flip in adjacent time frames, enabling the system to operate in a ping-pong manner. While writing the latest perception results to the first address space's repository Bank1 in each time frame, the cognitive decision-making module can simultaneously read the latest perception results from the second address space's repository Bank2. Logical reasoning is performed using map data from each time frame. This zero-copy on-chip exchange mechanism significantly reduces the perceived latency of decision-making interactions, far exceeding the latency levels of traditional board-level processors that transfer data via a bus.
[0063] Example 3 focuses on the high-dimensional representation structure of the dynamic map of driving situation and its cognitive update algorithm to further improve the safety of the system in complex traffic flow environments.
[0064] In this embodiment, the dynamic map of driving situation is constructed as a multi-channel hierarchical spatiotemporal tensor centered on the vehicle. .in, and The preset bird's-eye view grid size is set, for example, corresponding to 800 grids horizontally and 1500 grids vertically, with the physical resolution set to 0.1m per grid. This represents the number of feature channels. These channels are assigned different physical semantic logics. Specifically, geometric feature channels are used to characterize the occupancy state and dynamic velocity vector of obstacles; semantic feature channels are used to characterize lane line positions, zebra crossing distribution, and constraints on passable areas; while cognitive potential field channels are used to characterize the distribution of potential collision risks.
[0065] For the geometric feature channels, in order to handle the probabilistic noise present in the sensor, the system performs a specific procedure for each grid cell. Perform a Bayesian occupancy grid update. This process uses a logarithmic probability approach for recursive calculation to avoid overflow issues caused by floating-point multiplication. The specific update formula is shown in equation (3):
[0066]
[0067] In the above formula, This represents the logarithmic probability that the current grid cell is occupied, while This refers to the historical state at the previous moment. This represents the observation value of the grid at the current moment. To achieve massive grid updates in a single clock cycle in the FPGA, the system pre-builds a lookup table based on block RAM, directly mapping the logarithmic operations in the formula to fixed-point table lookup operations.
[0068] Simultaneously, within the cognitive potential field channel, the system generates a velocity-adaptive asymmetric two-dimensional Gaussian risk potential field. This design aims to simulate the intuitive avoidance psychology of human drivers towards hazards. For the center coordinates (... For dynamic obstacles, the system calculates the risk value they generate in the surrounding grid. The formula for calculating this risk value is shown in equation (4):
[0069]
[0070] In this formula, For the amplitude weights related to the obstacle hazard level, The heading angle of the obstacle. The variance is lateral. The variance is longitudinal. It will dynamically adjust according to the speed of the obstacle's movement, and its calculation formula is shown in equation (5):
[0071]
[0072] in, The basic variance is the extent of influence under static conditions. This is the speed sensitivity coefficient. This asymmetrical design causes high-speed obstacles to project a longer risk range in front of them, thus forcing the decision-making module to make avoidance decisions in advance.
[0073] Example 4 describes in detail the reasoning process of the cognitive decision-making module and the dynamic resource scheduling mechanism when the sensor configuration changes.
[0074] During the generation of control commands, the cognitive decision-making module first extracts static data from a local high-precision map through a long-term memory interface and uses it as a base mask to perform channel stitching with the dynamic map in working memory. Subsequently, the decision-making module samples the space in front of the vehicle to generate a cluster of candidate trajectories. To optimize a large number of candidate trajectories in a very short time, the system utilizes a DSP slice array on the FPGA backend to perform parallel computation on each candidate trajectory. Total cost The formula for calculating the total cost is shown in equation (6):
[0075]
[0076] In this formula, The smoothness of the trajectory represents a trade-off, designed to ensure ride comfort; To reference the cost of path deviation, vehicles are limited from excessively deviating from the lane centerline; while The core risk cost is the risk value of all dynamic map grids covered by the predicted trajectory. The summation is calculated. After the calculation is completed, the system uses a hardware comparator tree to select the optimal trajectory with the minimum cost over multiple clock cycles, and combines it with a physical limit rule library for amplitude limiting filtering.
[0077] To ensure system robustness in the event of sensor hardware failure or changes, this embodiment implements dynamic scheduling of on-chip resources. The resource scheduler monitors for heartbeat anomalies or hardware hot-plugging in real time. Upon detecting a change in sensor configuration, it immediately triggers a reconstruction interrupt signal. At this point, the system utilizes dynamic partial reconstruction technology to refresh only the logic circuitry of the sensor memory interface module located in the front-end dynamic logic area. During reconstruction, the resource scheduler uses a prefetch strategy to read the backup interface logic bitstream from external non-volatile memory (Flash) into the on-chip cache and performs decryption using the on-chip hard-core Advanced Encryption Standard (AES) engine to ensure secure bitstream transmission. After the reconstruction interrupt signal is triggered, the system, while maintaining the normal operation of the power supply and clock distribution network for the cognitive fusion and working memory modules and the cognitive decision module, erases and rewrites the sensor memory interface module through the internal configuration access interface. This process enables online updates of the front-end hardware interface. Furthermore, due to the existence of the standard boundary of the dynamic driving situation map, the back-end cognitive decision module can smoothly connect to new perception data without restarting or recompiling, achieving deep decoupling of perception and decision-making at the physical architecture level.
[0078] During the on-chip resource dynamic scheduling process, the resource scheduler monitors the online status of each sensor's physical interface in real time via the system bus. When drastic changes occur in the external environment, such as strong light causing camera overexposure and failure, or severe weather causing a decrease in the detection performance of a certain type of sensor and triggering degradation logic, the built-in hardware watchdog timer will immediately generate and trigger a reconstruction interrupt signal if it detects an abnormal heartbeat packet or a sensor configuration change caused by hardware hot-plugging. To minimize the system's downtime during logic reconfiguration, this invention employs a prefetch strategy. The resource scheduler uses a direct memory access controller to prefetch batches of spare interface logic bitstreams stored in external non-volatile memory into the on-chip cache. To ensure the security of the vehicle system, these configuration bitstreams are encrypted before transmission. The system calls the on-chip integrated hard-core symmetric encryption engine, namely the Advanced Encryption Standard (AES) engine, to perform single-cycle streaming decryption on the prefetched bitstreams. The decrypted data is injected into the front-end dynamic logic area through the internal configuration access interface, namely ICAP. During the refactoring process, the system strictly adheres to the Isolation Design Flow (IDF) to ensure that the logical erasure and rewriting operations on the sensory memory interface module do not interfere with the normal power supply and clock distribution network of the cognitive fusion and working memory modules and the cognitive decision-making module. After refactoring, the scheduler sends a soft reset command to the newly loaded interface logic. Once the verification flag passes, the new logic immediately begins writing dynamic map tensors conforming to standard protocols to the on-chip high-speed storage resources, achieving a seamless, zero-aware transition of the backend cognitive decision-making module to changes in the frontend sensory hardware.
[0079] It should also be noted that the various technical features mentioned in the above embodiments can be combined in various ways, and as long as these combinations do not contradict each other and can be implemented by those skilled in the art, they all fall within the protection scope of this invention. For example, under different sensor configurations, the weight coefficients of each parameter in the cognitive potential field channel can be dynamically adjusted according to the needs of the actual driving scenario. This invention constructs a standardized dynamic map of driving situation as a standardized boundary for software and hardware interaction, and utilizes the hardware reconfigurability of field-programmable gate arrays to shield the physical differences of heterogeneous sensors. This mechanism not only eliminates the bandwidth bottleneck of general-purpose processors when handling large-scale spatiotemporal tensors, but also provides a high-dimensional environmental representation with spatiotemporal continuity through Bayesian occupancy updates and dynamic risk potential field generation, significantly enhancing the robustness of the decision module in complex traffic flow scenarios.
[0080] In the process of cognitive decision-making, for any predicted trajectory in the candidate trajectory cluster... The system utilizes hardware logic to perform parallel computation of the total cost. Its more refined mathematical representation is shown in equation (6):
[0081]
[0082] In the formula, the first term on the right side of the equation is the ride comfort cost, obtained by integrating the square of the trajectory jerk to ensure ride comfort; the second term is the cognitive potential field risk cost, which is the risk value in all dynamic map grids covered by the predicted trajectory. The first term is the discrete cumulative sum; the second term is the reference path deviation cost, which calculates the sum of squared lateral distances from each trajectory point to the center line of the high-precision map. This multi-dimensional evaluation mechanism ensures that the output control commands are both safe and maneuverable.
[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. An FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving, characterized in that, Includes the following steps: Step S1: Receive multi-source heterogeneous sensor data using the reconfigurable sensor memory interface module of the FPGA front-end dynamic logic area, perform spatiotemporal synchronization of the underlying signals using the built-in hardware timestamp synchronizer, and normalize and project the multi-source heterogeneous sensor data into a unified standard format driving situation dynamic map data frame. Step S2: Construct a dual-port buffer using the cognitive fusion and working memory module of the FPGA on-chip high-speed storage resources to maintain and refresh the driving situation dynamic map in real time. The driving situation dynamic map is constructed as a multi-channel hierarchical spatiotemporal tensor as a medium for on-chip data exchange. Step S3: Using the cognitive decision module deployed in the static logic area of the FPGA backend, the driving situation dynamic map is read directly from the cognitive fusion and working memory module at a fixed time sequence, and logical reasoning is performed in combination with the prior knowledge retrieved from the long-term memory interface to generate control commands. Step S4: Define the dynamic driving situation map as the standardized boundary of the software and hardware interaction; when a sensor configuration change is detected, trigger a reconstruction interrupt signal, use dynamic partial reconstruction technology to refresh only the logic circuit corresponding to the sensory memory interface module, and keep the operating state and resource allocation of the cognitive decision module unchanged.
2. The FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving according to claim 1, characterized in that, In step S1, performing spatiotemporal synchronization of the underlying signals includes: setting the global heartbeat clock period inside the FPGA to be... The time difference between two adjacent sampling times is obtained using the vehicle's inertial measurement unit. pose transformation matrix within Translation vector Asynchronous sensor data is aligned to the current heartbeat moment using a linear interpolation algorithm based on vehicle motion compensation. The alignment formula is shown in equation (1): in, and These are the observation data at adjacent sampling times, The time interval for observing the data. Based on the current moment exist The time weighting coefficient for position calculation within the interval is calculated using the formula shown in equation (2): 。 3. The FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving according to claim 1, characterized in that, In step S2, the dual-port buffer is constructed as a ping-pong operation dual-buffer queue using the UltraRAM resources inside the FPGA; the ping-pong operation dual-buffer queue includes a first address space repository Bank1 and a second address space repository Bank2; by alternating flipping of the read and write pointers in adjacent time frames, the operation is achieved in the first... While writing the perception result to the repository Bank1 in the first address space in each time frame, the system reads the perception result from the repository Bank2 in the second address space. Decisions are made based on map data from each time frame, where It is a natural number greater than 1.
4. The FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving according to claim 1, characterized in that, In step S2, the driving situation dynamic map is constructed as a multi-channel hierarchical spatiotemporal tensor centered on the vehicle. ,in and The preset bird's-eye view grid size, The number of feature channels; The feature channels include: a geometric feature channel characterizing the obstacle occupancy state and velocity vector, a semantic feature channel characterizing the distribution of lane lines and passable areas, and a cognitive potential field channel characterizing the distribution of driving risks.
5. The FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving according to claim 4, characterized in that, The method further includes: targeting the grid in the geometric feature channel. , perform Bayesian occupying grid update; the update process adopts log odds form, and the calculation formula is shown in equation (3): in, The logarithmic odds at the current moment. The logarithmic odds of the previous moment. For the current observation value, The grid occupancy state is defined; and a block RAM-based lookup table is pre-built in the FPGA to convert the floating-point logarithmic operations in the log probability update formula into fixed-point lookup table operations.
6. The FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving according to claim 4, characterized in that, The method further includes: generating a velocity-adaptive asymmetric two-dimensional Gaussian risk potential field in the cognitive potential field channel; for the center coordinates being... For dynamic obstacles, calculate the risk value they generate in the surrounding grid. The calculation formula is shown in equation (4): in, For magnitude weighting, The heading angle of the obstacle's movement. For the horizontal variance, The longitudinal variance, which dynamically adjusts with speed, is calculated using the formula shown in equation (5): in, Based on the variance, For speed sensitivity coefficient, and Estimate the velocity components for obstacles.
7. The FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving according to claim 1, characterized in that, Step S3 includes: the cognitive decision-making module extracts local high-precision map data through the long-term memory interface, and uses it as a base mask to stitch the driving situation dynamic map in the cognitive fusion and working memory module with feature channels.
8. The FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving according to claim 6, characterized in that, In step S3, the generation of the control command includes: sampling and generating a candidate trajectory cluster in the space in front of the vehicle; and using the DSP slice array on the back end of the FPGA to calculate each candidate trajectory in parallel. Total cost The calculation formula is shown in equation (6): in, For the sake of smoothness, Cost of deviation from reference path The cognitive potential risk cost is the risk value of all dynamic map grids covered by the predicted trajectory. The sum of the sums, , , The preset weight coefficients are used; the optimal trajectory with the lowest cost is selected using a hardware comparator tree, and the optimal trajectory is subjected to amplitude limiting filtering in combination with the vehicle dynamics limit rule library; and when the optimal trajectory triggers the physical limit threshold, a degradation instruction is output through a hardware circuit breaker mechanism.
9. The FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving according to claim 1, characterized in that, In step S4, performing on-chip resource dynamic scheduling includes: using a hardware watchdog timer to monitor valid signals of sensor channel data, and triggering a reconstruction interrupt signal when a heartbeat packet abnormality or sensor configuration change caused by hardware hot-plugging is detected; and using a resource scheduler to execute a prefetch strategy to prefetch the spare interface logic bit stream from external memory to the on-chip cache, and using the on-chip hard core AES engine to perform decryption.
10. The FPGA resource scheduling method for decoupling perception and decision-making in autonomous driving according to claim 9, characterized in that, After the reconstruction interrupt signal is triggered, the following steps are executed: Under the premise of maintaining the normal operation of the power supply and clock distribution network of the cognitive fusion and working memory module and the cognitive decision module, the logic circuit of the sensory memory interface module is erased and rewritten through the internal configuration access interface using an isolation design process.