An edge device-oriented three-dimensional target detection neural network acceleration method

Abstract

An edge device-oriented three-dimensional target detection neural network acceleration method. Through the algorithm level innovation of orthogonal dictionary convolution and the hardware level reconstruction of FPGA memory mapping, the contradiction between delay, power consumption and accuracy of edge three-dimensional point cloud target detection is solved. The steps are as follows: based on the orthogonal dictionary convolution optimization paradigm, a micro neural network model for road scene is constructed, and a three-dimensional point cloud feature extraction framework is generated; the trained micro neural network is quantized and compiled, and the hardware module scheduling of heterogeneous parallel computing is performed on FPGA; the road scene dedicated tensor layout transformation is executed, the memory reconstruction is realized through the bottom conversion circuit, and the conflict-free data transmission stream is generated; based on the prior parameters of road environment, the dynamic fusion feature extraction of multi-modal data is executed; the end-to-end low-delay forward inference is completed at the edge, and the context smoothing algorithm is adopted to output the final three-dimensional bounding box and target category.

Description

Technical Field

[0001] This invention belongs to the field of information data processing technology, and specifically relates to a method for accelerating three-dimensional target detection neural networks for edge devices. Background Technology

[0002] With the widespread adoption of intelligent transportation and advanced driver assistance systems (ADAS), extremely high demands are being placed on 3D target perception in complex road environments (such as urban roads and snow-covered or icy roads). 3D target detection algorithms deployed on edge devices such as roadside sensing units (RSUs) or vehicle-mounted domain controllers must output 3D bounding boxes of targets such as pedestrians, vehicles, and cones in real time and with high accuracy under strict power consumption and computing power constraints.

[0003] Currently, mainstream architectures of 3D point cloud object detection networks (such as VoxelNet, PointPillars, and CenterPoint) largely rely on standard 3D convolutional neural networks (3D-CNN) or dense self-attention mechanisms to extract spatiotemporal features. However, migrating these models to edge computing environments faces insurmountable underlying technological gaps: First, there is the von Neumann bottleneck in model parameter size and memory bandwidth. The number of parameters in a standard 3D convolution kernel grows exponentially in spatial dimension, resulting in extremely high memory bandwidth consumption. The read speed of external memory (such as DDR) on edge devices (such as low-end FPGAs or embedded GPUs) is much lower than the computation speed of logic computing units, causing extremely high memory access latency (typically greater than 100ms), which cannot meet real-time requirements.

[0004] Second, traditional lightweighting methods lead to severe collapse of 3D features. To adapt to edge computing, dimensionality reduction techniques such as network pruning, channel reduction, or knowledge distillation are often used. However, road point cloud data itself has extremely high sparsity, especially in degraded scenarios such as ice and snow or backlighting. Conventional lightweighting methods will directly destroy the basic features of the 3D spatial geometry, resulting in a sharp increase in the false negative rate of small targets (such as pedestrians in the distance).

[0005] Third, there is a structural mismatch between software and hardware memory access mapping. Existing deep learning frameworks use continuous linear tensor arrangements in memory (such as NCHW or NHWC), while the multiply-accumulate tree arrays (MAC arrays) at the bottom layer of edge computing hardware often need to retrieve block data in high parallelism. This mismatch between the algorithm data structure and the hardware physical addressing logic leads to a low cache hit rate, with computing power utilization typically less than 30%.

[0006] In summary, existing technologies cannot simultaneously achieve "extremely low computing power consumption, extremely low memory access latency, and high-dimensional feature preservation" on edge devices, which severely restricts the engineering implementation of 3D road perception under complex weather conditions. Summary of the Invention

[0007] The main objective of this invention is to provide a neural network acceleration method for 3D target detection on edge devices. Through algorithmic innovation of orthogonal dictionary convolution and hardware-level reconstruction of FPGA memory access mapping, it completely solves the contradiction between latency, power consumption and accuracy in 3D point cloud target detection at the edge.

[0008] The method includes the following steps: S1. Construct a micro neural network model for road surface scenarios based on the orthogonal dictionary convolution optimization paradigm to generate a 3D point cloud feature extraction framework; S2. Quantize and compile the trained micro neural network, and schedule the hardware modules for heterogeneous parallel computing on the FPGA accelerator. S3. Perform a special tensor layout transformation for the road scene, and realize memory access reconstruction through the underlying conversion circuit to generate a conflict-free data transmission stream; S4. Based on prior parameters of the road surface environment, perform dynamic fusion feature extraction of multimodal data; S5. Perform end-to-end low-latency forward inference at the edge and use a context smoothing algorithm to output the final 3D bounding box and target category.

[0009] Furthermore, the micro neural network model for road surface scenarios is constructed based on the orthogonal dictionary convolution optimization paradigm as follows: The standard 3D convolution in the micro neural network model used for road surface scenarios is decomposed into a two-level mapping structure of dictionary convolution and encoding convolution; the input feature map of dictionary convolution is... The number of dictionary bases is The dictionary convolution kernel matrix is ​​represented as Extracted basis feature map The calculation formula is: ; Encoding convolution reassembles basis features through pointwise linear combination, and the encoding matrix is... Output feature map The calculation expression is: .

[0010] Furthermore, during the training phase of the micro neural network model, an orthogonal regularization term with adaptive modulus length is incorporated. Embedded total loss function In this context, its mathematical expression is: ; in, For regularization weights, The total number of dictionary convolutional layers. For the first The layer dictionary convolution is flattened into a two-dimensional matrix form; It is a diagonal matrix, and its diagonal elements are the feature magnitude parameters of the network's adaptive learning. ; This represents the bounding box regression and classification loss for object detection.

[0011] Furthermore, the quantization compilation employs a symmetric linear quantization method, with the quantization formula as follows: , ; in, Indicates the quantization scaling factor. and These represent the maximum and minimum values ​​of the full-precision tensor to be quantized within the statistical range, respectively. This represents the tensor data in signed integer format that is the final output after quantization. This represents a numerical truncation function. This function represents the rounding function that rounds the nearest integer to the nearest whole number. This represents the original full precision before quantization.

[0012] Furthermore, the hardware module scheduling for heterogeneous parallel computing is as follows: at the FPGA bottom layer, the global instruction scheduler performs heterogeneous allocation of the two-level convolution. For dictionary convolution, the systolic multiply-accumulate array is scheduled to perform high-throughput computation; for coded convolution, the low-latency register pipeline unit is scheduled to perform element-wise multiply-accumulate computation.

[0013] Furthermore, step S3 specifically involves: deploying a dedicated direct memory access conversion circuit at the front end of the FPGA memory control interface to pipeline the data read / write cycle. arrive Linear speed conversion; This represents the one-dimensional physical address of the feature tensor in memory before the transformation. This represents the one-dimensional physical address of the transformed feature tensor in memory. ; Among them, among them, The channel index of the feature map. The width direction index representing a feature in three-dimensional space. Indicates the height orientation index of a three-dimensional spatial feature. Depth direction index representing three-dimensional spatial features. Indicates crossing a size of The memory address step required for a complete channel block; and These represent the depth dimension under a blocky, interwoven layout. or height dimension Each increment of 1 corresponds to the physical address span; This indicates the modulo operation.

[0014] Furthermore, step S4 specifically includes: Dynamically calculate the fusion weight allocation of lidar modes and visible light modes : ; The cross-modal feature fusion process is as follows: ; in, and These represent the environmental feature weight coefficients for adaptive learning in the network. and These are snow particle density and road surface reflectivity, respectively. For bias term parameters, and These represent the feature space mapping weight matrices for the visible light mode and the laser mode, respectively. The Sigmoid activation function is used. This represents a two-dimensional image feature map extracted from the visible light modes. This represents the spatial feature map of the three-dimensional point cloud extracted from the lidar mode.

[0015] Furthermore, the context smoothing algorithm specifically involves introducing a smoothing optimization mechanism based on Kalman filtering and temporal confidence decay to correlate the trajectory features of the target's 3D bounding box in a continuous multi-frame video stream, and to perform state estimation and filtering correction for temporal position jitter caused by sudden environmental noise, ultimately outputting the road target detection result.

[0016] The beneficial effects of the proposed method are as follows: This invention, through its pioneering "dictionary + encoding" dual-level convolution and adaptive orthogonal regularization, fully preserves the 3D basis features while compressing over 80% of the model parameters, effectively solving the problem of easy missed detection of small targets. Simultaneously, combined with a dedicated block-interleaved tensor layout and DMA hardware-level address translation, this method perfectly matches the parallel addressing logic of edge devices, increasing DDR bandwidth utilization to 88% and completely breaking through the underlying memory access bottleneck. Based on this, by combining INT8 dynamic quantization and FPGA heterogeneous operator separation scheduling, an ultra-low latency inference of only 18.5 milliseconds across the entire link and per frame is achieved. Furthermore, this invention introduces a multi-modal dynamic fusion mechanism that perceives weather parameters such as snow particle density in real time, which can adaptively suppress extreme environmental noise, significantly reducing the target missed detection rate by 30% in harsh scenarios such as snow and ice. Overall, this solution perfectly balances the extreme lightweight nature, ultra-low latency, and high perception robustness of 3D target detection under stringent edge computing resource constraints. Attached Figure Description

[0017] Figure 1 This is an overall flowchart of the neural network acceleration method for 3D target detection in edge devices according to the present invention; Figure 2 This is a schematic diagram of the orthogonal dictionary convolutional two-level mapping mechanism and adaptive regularization network structure of the present invention; Figure 3 This is a schematic diagram of the heterogeneous parallel computing architecture and operator separation scheduling principle based on FPGA of the present invention. Figure 4 This is a schematic diagram illustrating the mapping relationship between the tensor block-interleaved layout for road surface scenarios in this invention and the underlying memory access and reorganization. Figure 5 This is a schematic diagram of the dynamic fusion and feature adaptive adjustment mechanism for multimodal heterogeneous data in snow and ice scenarios according to the present invention. Detailed Implementation

[0018] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0019] Example 1 This embodiment provides a method for accelerating 3D target detection neural networks for edge devices, such as... Figure 1 The diagram shown is the overall flowchart. The method includes the following steps: Step S10: Construct a micro neural network model for road surface scenarios based on the orthogonal dictionary convolution optimization paradigm to generate a 3D point cloud feature extraction framework.

[0020] like Figure 2 As shown, unlike standard 3D convolution which learns high-dimensional weights directly on the full input channels, this invention proposes to decompose standard 3D convolution into a two-level mapping structure of dictionary convolution and encoding convolution, which significantly compresses the number of parameters while expanding the search space. Dictionary Convolution: Learns a compact set of 3D basis feature vectors to construct a dictionary space of road surface geometry. Let the input feature map be... The number of dictionary bases is Then the dictionary convolution kernel matrix is ​​represented as Extracted basis feature map The calculation formula is: ; .

[0021] : Represents the input 3D feature map tensor. : Indicates the number of channels in the input feature map (InputChannels). : Indicates the number of feature map channels expected to be output by the convolutional layer; , , : These represent the depth, height, and width of the three-dimensional spatial features, respectively.

[0022] : Represents the weight matrix of the dictionary convolution kernel, : Indicates the set number of dictionary bases (which here acts as the number of output channels for this convolutional layer). : Indicates the number of input channels, , , : These represent the dimensions of the 3D convolution kernel in the depth, height, and width directions, respectively.

[0023] : Represents the basis feature map extracted after dictionary convolution. : refers to the dictionary convolution kernel weights defined above. : This is the input feature map for the previous layer.

[0024] Coding Convolution: This involves linearly combining points... Convolution reorganizes the basis features. Let the encoding matrix be... Output feature map The calculation expression is: ; Through the above decomposition, the computational complexity of convolution is reduced from... Decrease to The number of parameters decreased by more than 80%.

[0025] Adaptive module length orthogonal regularization: To prevent feature basis linear correlation (feature collapse) caused by lightweighting, an orthogonal regularization term with an adaptive module length is embedded into the total loss function during the model training phase. In the middle. Its mathematical expression is: ; ; : Represents the adaptive modulus orthogonal regularization loss. The summation upper bound of the formula represents the total number of dictionary convolutional layers in the network that apply this orthogonal regularization.

[0026] in, The bounding box regression and classification loss are used for object detection; For regularization weights; For the first The layer dictionary convolution is flattened into a two-dimensional matrix form; It is a diagonal matrix, whose diagonal elements are the feature magnitude parameters of the network's adaptive learning (as opposed to traditional strictly orthogonal matrices). This allows the model to retain certain scale variation characteristics based on the data distribution.

[0027] Step S20: Quantize and compile the trained micro neural network, and schedule the hardware modules for heterogeneous parallel computing on the FPGA accelerator.

[0028] Low-bit dynamic quantization: The weights and activation values ​​of the trained full-precision (FP32) model are converted into 8-bit integers (INT8). A symmetric linear quantization algorithm is used, and the quantization formula is as follows: ; ; in, This is the quantization scaling factor. and These represent the maximum and minimum values ​​of the full-precision tensor to be quantized (such as a weight matrix or activation feature map) within the statistical range, respectively. and These are their absolute values. By taking the maximum value of the two, the maximum numerical range of dynamic symmetric quantization can be determined. : Represents the final output tensor data in 8-bit signed integer format after quantization calculation; : Represents the original full-precision (32-bit floating-point format) tensor input value before quantization; : This function rounds the nearest integer to the nearest integer, and is used to map floating-point values ​​to discrete integers; : Represents the numeric truncation (saturation) function, which forces the rounded result to be limited to the valid representation range of the INT8 type. Within this range, to prevent overflow errors in the quantized values.

[0029] Heterogeneous hardware operator mapping: such as Figure 3 As shown, at the FPGA's underlying layer, the global instruction scheduler heterogeneously allocates the two levels of convolutions mentioned above. This is for dictionary convolutions, which are computationally intensive but have a small channel size. A dedicated systolic multiply-accumulate array (Systolic MAC Array) performs high-throughput computations; it is used for parameter-dense encoded convolutions. It schedules low-latency register pipeline logic to perform element-wise multiply-accumulate calculations, achieving efficient utilization of hardware resources.

[0030] Step S30: Perform a special tensor layout transformation for the road surface scene, and realize memory access reconstruction through the underlying conversion circuit to generate a conflict-free data transmission stream.

[0031] To overcome memory access bottlenecks, this invention reconstructs the physical address mapping relationship of three-dimensional voxels in DDR memory: Let the logical coordinates of a traditional 3D feature tensor in memory be... Its one-dimensional physical address The calculation formula is usually as follows: ; in, This represents the channel index of the feature map. Depth Index, representing the depth orientation of three-dimensional spatial features. Height Index represents the height orientation of a three-dimensional spatial feature. Width Index represents the width direction of a three-dimensional spatial feature. These represent the total dimensions (upper limit of dimensions) of the three-dimensional feature tensor in the depth, height, and width directions, respectively. This represents the total number of elements within a single channel. In contiguous memory layouts, it indicates the physical address span required to traverse a complete channel.

[0032] In this invention, the parallel data acquisition characteristics of FPGA (the length of each burst read is...) are taken into account. (e.g., elements), such as Figure 4 As shown, a block-interleaved tensor layout is designed. Its physical address... The reconstruction formula is: ; in, Indicates crossing a complete channel block (size is) The required memory address step size; , : These represent the depth dimension under a blocky, interwoven layout. or height dimension Each increment of 1 corresponds to the physical address span; This represents the modulo (remainder) operation. The meaning is to calculate the current channel index. Divide by The remainder after that represents the local channel offset of the target data within the current data block. Similarly, This indicates the local width offset of the target data within the current data block.

[0033] By deploying a dedicated direct memory access (DMA) conversion circuit at the front end of the FPGA memory control interface, data read and write operations are pipelined within the data read / write cycle. arrive Linear-speed conversion. This avoids CPU involvement in formatting and reduces cache miss rate by over 90%.

[0034] Step S40: Based on prior parameters of the road surface environment, perform dynamic fusion feature extraction of multimodal data.

[0035] The aforementioned steps S10 to S30 constitute the basic underlying architecture of this acceleration method, fundamentally solving the problems of slow edge extraction and memory access difficulties in 3D point clouds, and obtaining an efficient point cloud feature base. However, in extreme snow and ice scenarios, a single point cloud modality is prone to local feature failure due to snow particle noise. Therefore, this step introduces a visual (Camera) modality for complementarity, based on the aforementioned hardware-accelerated point cloud backbone. Figure 5 As shown, the scene perception unit acquires current weather characteristics (such as snow particle density) in real time. With road surface reflectivity The multimodal fusion module dynamically calculates the fusion weight allocation between the LiDAR mode and the visible light (Camera) mode using the following formula. : ; The mathematical expression for the cross-modal feature fusion process is as follows: ; : Represents the final multimodal fusion feature map output after cross-modal cross-attention gating calculation and dynamic weight allocation; and : These represent the environmental feature weight coefficients for network adaptive learning, used to dynamically measure the impact of "snow particle density" and "road surface reflectivity" on the reliability of the current mode. These are the bias term parameters for the dynamically weighted network. and These represent the feature space mapping weight matrices for the visible light (Camera) mode and the LiDAR mode, respectively. Their function is to align the heterogeneous multimodal feature maps into a unified feature subspace for subsequent cross-attention gating calculations. : Represents the two-dimensional image feature map extracted from the visible light (Camera) mode. : Represents the spatial feature map of the three-dimensional point cloud extracted by the LiDAR mode (usually converted and aligned to a unified bird's-eye view or voxel space view).

[0036] in, The Sigmoid activation function is used. This represents an element-wise multiplication attention-gated operation. This mechanism enables attention gating in extreme blizzard conditions (…). Large) or icy reflection ( When the noise level is high, the network can adaptively suppress the noise of the failure mode and amplify the characteristics of the reliable mode.

[0037] Step S50: Complete end-to-end low-latency forward inference at the edge and output the final 3D bounding box and target category using the context smoothing algorithm.

[0038] like Figure 5 As shown, the fused features are input into the detection head, and the output is a 3D prediction tensor. A smoothing optimization mechanism based on Kalman filtering and temporal confidence decay is introduced. This mechanism correlates the trajectory features of the target's 3D bounding box in multiple consecutive video frames and performs state estimation and filtering correction for temporal position jitter caused by sudden environmental noise, ultimately outputting high-precision road target detection results. Represents the absolute coordinates of the target's center point in a three-dimensional coordinate system; These represent the physical width, length, and height dimensions of the target's 3D bounding box, respectively. This indicates the target's yaw angle (or rotational orientation) on the horizontal plane. The classification prediction result for the target's category (such as pedestrian, vehicle, etc.); This is the final overall confidence score for the prediction box.

[0039] Example 2 This embodiment further defines Embodiment 1, and uses specific application examples to illustrate Embodiment 1.

[0040] Case 1: Training and Model Compression of the Miniature Neural Network XTinyNet Network Framework Construction: A point cloud object detection network is built based on PyTorch. The 2D backbone network in the traditional PointPillars is replaced with XTinyNet based on orthogonal dictionary convolutions. The network core contains three downsampling stages, each containing two dense linear residual modules. The feature mappings within each module all use a two-layer structure of "dictionary convolution + encoding convolution" from step S10 instead of standard convolutions.

[0041] Orthogonal regularization training: Using publicly available autonomous driving datasets (such as KITTI or a self-built icy road surface dataset), the input voxel size is set to... An adaptive modulus orthogonal regularization term is introduced into the loss function calculation, and the regularization coefficient is... Set as The AdamW optimizer was used, with an initial learning rate of... Implement a one-cycle learning rate decay strategy.

[0042] Model Quantization and Accuracy Comparison: After training, the optimal model parameters are extracted. INT8 static quantization is performed using a calibration algorithm based on KL divergence (Kullback-Leibler Divergence).

[0043] Experimental data comparison: Under the same accuracy requirements, the backbone parameters of the standard PointPillars are approximately 4.8MB, while the parameters of the XTinyNet constructed using this embodiment are only 895KB. Vehicle target detection on the test set... It achieved 85.3% accuracy (only 0.7% lower than the full-precision standard model), but the volume was reduced by 81.3%.

[0044] Case 2: FPGA-based Hardware Circuit Architecture Mapping and Memory Access Control Hardware operator separation scheduling: Select Xilinx Zynq UltraScale+ (such as ZU3EG) or equivalent domestic FPGA chips as acceleration carriers. Configure heterogeneous computing engines in the FPGA's programmable logic area (PL).

[0045] Dictionary convolution requires fewer input channels but has high reusability; mapping it to... A large-scale DSP48E2 pulse array with a system clock configuration of 300MHz.

[0046] Encoding convolution involves cross-channel global summation, which is mapped to a register pipeline accumulation tree of depth 8 levels.

[0047] Low-level memory access reorganization and block-based interleaving execution: Based on the memory access formula proposed in step S30, the parameters for FPGA hardware to handle burst reads are set: the memory channel bandwidth is set to 256-bit (i.e., 32 INT8 data bytes can be read at a time). Block parameters are defined. .

[0048] When the ARM processor stores the preprocessed voxel data of the point cloud into DDR memory, the data is arranged continuously in standard C format. When the FPGA's DMA engine is activated to read the data, the front-end hard-connected "format conversion circuitry" contains two sets of Ping-Pong BRAMs. Simultaneously with the data flowing into the BRAM, the read controller strictly follows the... The physical address timing generated by the formula will pop the data into the computing array.

[0049] Experimental data: Tested with the Vivado hardware emulator, this dedicated tensor layout boosts the total DDR bandwidth utilization from 35% in the conventional design to 88%, eliminating the long latency overhead caused by row buffer misses.

[0050] Case 3: Anti-interference application and system-level verification in multimodal ice and snow scenarios The vehicle-mounted edge computing system was validated in actual icy and snowy road sections in the north.

[0051] Dynamic weight capture: The vehicle-mounted domain controller is equipped with a 128-line LiDAR and a 1080P visible light camera. During a sudden blizzard, the contrast of the image sensor feed drops drastically, and the LiDAR point cloud shows a large number of suspended snow particles. The system's scene perception unit calculates the snow particle density. It reached a high threshold (0.85).

[0052] Hardware-level attention intervention: Based on the fusion formula in step S40, the control registers inside the FPGA dynamically adjust the activation response of the LiDAR branch feature map ( The value was suppressed to 0.3, while the attention weight for extracting features from camera branch edges was increased.

[0053] Performance evaluation metrics: Running end-to-end inference on an edge terminal equipped with the above acceleration method: Inference latency: The average end-to-end inference time per frame of the system has been reduced to 18.5 milliseconds (i.e., frame rate of more than 50 FPS), which is far lower than the 95 milliseconds required by general-purpose GPUs (such as Jetson Nano) to run standard networks.

[0054] Robustness and accuracy: In snow and ice degradation scenarios, the detection accuracy of pedestrians and snow-covered vehicles at a distance of 50 meters remains above 82.5%, which significantly reduces the false negative rate by 30% compared to traditional lightweight algorithms that do not use the architecture of this invention.

Claims

1. A neural network acceleration method for 3D target detection in edge devices, characterized in that, The method includes the following steps: S1. Construct a micro neural network model for road surface scenarios based on the orthogonal dictionary convolution optimization paradigm to generate a 3D point cloud feature extraction framework; S2. Quantize and compile the trained micro neural network, and schedule the hardware modules for heterogeneous parallel computing on the FPGA accelerator. S3. Perform a special tensor layout transformation for the road scene, and realize memory access reconstruction through the underlying conversion circuit to generate a conflict-free data transmission stream; S4. Based on prior parameters of the road surface environment, perform dynamic fusion feature extraction of multimodal data; S5. Perform end-to-end low-latency forward inference at the edge and use a context smoothing algorithm to output the final 3D bounding box and target category.

2. The method for accelerating 3D target detection neural networks for edge devices according to claim 1, characterized in that, The specific steps for constructing a micro neural network model for road surface scenarios based on the orthogonal dictionary convolution optimization paradigm are as follows: The standard 3D convolution in the micro neural network model used for road surface scenarios is decomposed into a two-level mapping structure of dictionary convolution and encoding convolution; the input feature map of dictionary convolution is... The number of dictionary bases is The dictionary convolution kernel matrix is ​​represented as Extracted basis feature map The calculation formula is: ; Encoding convolution reassembles basis features through pointwise linear combination, and the encoding matrix is... Output feature map The calculation expression is: .

3. The method for accelerating 3D target detection neural networks for edge devices according to claim 2, characterized in that, During the training phase of the micro neural network model, an orthogonal regularization term with adaptive module length is used. Embedded total loss function In this context, its mathematical expression is: ； in, For regularization weights, The total number of dictionary convolutional layers, For the first The layer dictionary convolution is flattened into a two-dimensional matrix form; It is a diagonal matrix, and its diagonal elements are the feature magnitude parameters of the network's adaptive learning. ; This represents the bounding box regression and classification loss for object detection.

4. The method for accelerating 3D target detection neural networks for edge devices according to claim 3, characterized in that, The quantization compilation employs a symmetric linear quantization method, and the quantization formula is as follows: ， ； in, Indicates the quantization scaling factor. and These represent the maximum and minimum values ​​of the full-precision tensor to be quantized within the statistical range, respectively. This represents the tensor data in signed integer format that is the final output after quantization. This represents a numerical truncation function. This function represents the rounding function that rounds the nearest integer to the nearest whole number. This represents the original full precision before quantization.

5. The method for accelerating 3D target detection neural networks for edge devices according to claim 4, characterized in that, The hardware module scheduling for heterogeneous parallel computing is as follows: At the FPGA bottom layer, the global instruction scheduler performs heterogeneous allocation of the two levels of convolution. For dictionary convolution, the systolic multiply-accumulate array is scheduled to perform high-throughput computation; for coded convolution, the low-latency register pipeline unit is scheduled to perform element-wise multiply-accumulate computation.

6. The method for accelerating 3D target detection neural networks for edge devices according to claim 5, characterized in that, Step S3 specifically involves deploying a dedicated direct memory access (DMI) conversion circuit at the front end of the FPGA memory control interface to pipeline the data read / write cycle. arrive Linear speed conversion; This represents the one-dimensional physical address of the feature tensor in memory before the transformation. This represents the one-dimensional physical address of the transformed feature tensor in memory. ； Among them, among them, The channel index of the feature map. The width direction index representing a feature in three-dimensional space. Indicates the height orientation index of a three-dimensional spatial feature. Depth direction index representing three-dimensional spatial features. Indicates crossing a size of The memory address step required for a complete channel block; and These represent the depth dimension under a blocky, interwoven layout. or height dimension Each increment of 1 corresponds to the physical address span; This indicates the modulo operation.

7. The method for accelerating 3D target detection neural networks for edge devices according to claim 6, characterized in that, Step S4 is as follows: Dynamically calculate the fusion weight allocation of lidar modes and visible light modes : ； The cross-modal feature fusion process is as follows: ； in, and These represent the environmental feature weight coefficients for adaptive learning in the network. and These are snow particle density and road surface reflectivity, respectively. For bias term parameters, and These represent the feature space mapping weight matrices for the visible light mode and the laser mode, respectively. The Sigmoid activation function is used. This represents a two-dimensional image feature map extracted from the visible light modes. This represents the spatial feature map of the three-dimensional point cloud extracted from the lidar mode.

8. The method for accelerating 3D target detection neural networks for edge devices according to claim 7, characterized in that, The context smoothing algorithm specifically introduces a smoothing optimization mechanism based on Kalman filtering and temporal confidence decay to associate the trajectory features of the target's three-dimensional bounding box in a continuous multi-frame video stream, and performs state estimation and filtering correction for temporal position jitter caused by sudden environmental noise, finally outputting the road target detection result.

Images