A hardware acceleration system and processing method for a lightweight object detection network

Abstract

The application discloses a hardware acceleration system and processing method for a lightweight target detection network, and the method comprises the following steps: step S1: obtaining an improved YOLOv8n detection model; step S2: deploying the improved YOLOv8n detection model to an FPGA hardware acceleration unit, performing accelerated calculation on each convolution layer by using a reconfigurable computing array, and obtaining a convolution output feature map; step S3: in the convolution layer output and feature fusion point, performing parallel accumulation on the sum in the convolution output feature map by using a pipeline addition tree, and obtaining accumulated feature data; and step S4: performing a maximum value pooling operation on the accumulated feature data, and obtaining the maximum value of a two-dimensional pooling window as a pooling result output. The application can improve small target detection precision, improve the calculation efficiency and resource utilization rate of the FPGA through the reconfigurable computing array, the pipeline addition tree and the orthogonal decomposition pooling mechanism, and meet the real-time detection requirements of edge devices.

Description

Technical Field

[0001] This invention relates to the fields of image processing and neural network acceleration technology, specifically to a hardware acceleration system and processing method for lightweight target detection networks. Background Technology

[0002] In the fields of image processing and computer vision, object detection technology based on convolutional neural networks has been widely applied to various visual tasks. However, in the layer-by-layer downsampling process of existing convolutional neural networks, small objects account for a very low percentage of pixels in the spatial dimension. As the network depth increases, the structural features of small objects, such as edges and texture details, are easily lost or obscured by complex background noise during multiple downsampling and feature extraction processes. This phenomenon significantly weakens the representation ability of small objects in deep feature maps, thus affecting the overall performance of the detection model.

[0003] YOLOv8n, as a lightweight object detection model, maintains high detection speed while having a small number of parameters, with the smallest backbone network depth and width among its series of models. However, due to its shallow network depth and relatively small receptive field, the model's ability to extract features from small objects is insufficient, making it difficult to fully capture the spatial distribution and local texture information of small objects. This can easily lead to missed detections and false detections in actual detection processes. Therefore, how to enhance the representation ability of small object features within a lightweight model framework while maintaining low computational overhead has become a pressing technical problem in the field of object detection. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, one of the objectives of this invention is to provide a hardware acceleration processing method for lightweight target detection networks. This method can improve the detection accuracy of small targets while enhancing the computational efficiency and resource utilization of FPGAs through reconfigurable computing arrays, pipelined addition trees, and row-column orthogonal decomposition pooling mechanisms, thereby meeting the real-time detection needs of edge devices.

[0005] The second objective of this invention is to provide a hardware acceleration system for lightweight target detection networks, which can improve the detection accuracy of small targets while improving the computing efficiency and resource utilization of FPGAs through reconfigurable computing arrays, pipelined addition trees and row-column orthogonal decomposition pooling mechanisms, thereby meeting the real-time detection needs of edge devices.

[0006] To achieve one of the objectives of this invention, the following solution is adopted: A hardware acceleration method for lightweight target detection networks includes the following steps: Step S1: Obtain the improved YOLOv8n detection model; the improved YOLOv8n detection model includes: A weighted bidirectional feature pyramid network is used in the neck network for cross-scale feature fusion, and learnable weights are assigned to different input feature maps. A high-resolution detection branch is added to the neck network to form a multi-scale detection head with the original detection branch. A dynamic non-monotonic focusing mechanism is introduced into the regression loss to adaptively adjust the gradient weights according to the overlap of the bounding boxes of the samples. A mixed precision quantization strategy is adopted to perform non-uniform quantization on shallow feature maps with a higher quantization precision than that on deep feature maps. Step S2: Deploy the improved YOLOv8n detection model to the FPGA hardware acceleration unit, build a reconfigurable computing array based on the FPGA, and dynamically adjust the computing scale and computing mode of the reconfigurable computing array according to the parameters of the current convolutional layer during the entire forward inference process of the improved YOLOv8n detection model to accelerate the execution of all convolution operations in the model and obtain the convolution output feature map. Step S3: At the point where the convolutional layer output meets the feature fusion point, a pipelined addition tree is used to perform parallel accumulation of the partial sums in the convolutional output feature map to obtain the accumulated feature data. Step S4: Perform max pooling on the accumulated feature data. Use row-column orthogonal decomposition. First, pre-compress the features in the horizontal direction and compare them in a pipeline to obtain the maximum value of the in-row window. Then, compare them in the vertical direction in a pipeline to obtain the maximum value of the two-dimensional pooling window as the pooling result output.

[0007] Furthermore, the weighted bidirectional feature pyramid network adopts a fast normalization fusion strategy, assigning learnable weights to each input feature map, and the output feature map is equal to the sum of each input feature map multiplied by its corresponding weight and then divided by the sum of the weights, wherein the weights are updated through network training.

[0008] Furthermore, in the dynamic non-monotonic focusing mechanism, the distance overlap loss is used as the evaluation index of sample quality. The focusing coefficient is determined according to the ratio of the distance overlap loss to the learnable adjustable parameter. When the detected target is a small target with an area of ​​less than 32×32 pixels, the focusing coefficient is multiplied by the enhancement coefficient to amplify the outlier. The hybrid precision quantization strategy employs asymmetric quantization, and the quantization formula is as follows: Where q is the quantized integer. For raw floating-point numbers, is the scaling factor, and z is the zero-point offset; wherein, shallow feature maps are quantized using INT16, and deep semantic features are quantized using alternating precision INT8 and INT4.

[0009] Furthermore, in step S2, dynamically adjusting the computing scale and computing mode of the reconfigurable computing array includes inter-layer configuration scheduling, intra-block configuration scheduling, and architecture-deep configuration scheduling. The inter-layer configuration scheduler switches the array structure between different convolutional layers. When performing 3×3 convolution, the feature data is split into three rows of cache and accumulated. When performing 1×1 convolution, the cache resources are released for data prefetching. The intra-block configuration scheduling is dynamically adjusted within the same convolutional layer according to different computation blocks. When processing the C2f module, the large array is split into two independent sub-arrays for parallel processing. The architecture's deep configuration scheduling switches the resident data based on the downsampling depth, resident pixel data in shallow downsampling, and resident weight data in deep downsampling.

[0010] Furthermore, the reconfigurable computing array is a pulsating array, and its processing unit has five configurable modes, including: MAC mode for convolution operations, single-multiplication pass-through mode for 1×1 convolution, branch mode for C2f modules, weight-resident mode for deep downsampling, and feature map-resident mode for shallow downsampling; each mode is switched by the configuration register according to the arbitration of the control signal.

[0011] Furthermore, in step S3, the pipelined adder tree decomposes the N-way accumulation into log2N-stage parallel additions, inserts a pipeline register at each stage, and uses the output of the previous stage adder tree as the input of the next stage adder tree. This cascading method adapts to the accumulation requirements of different numbers of processing units.

[0012] Furthermore, in step S4, the row-column orthogonal decomposition method is specifically as follows: Multi-channel input data is decomposed into multiple independent pixel streams, and multiple parallel computing units are constructed using the FPGA's generate syntax. In each computing unit, a 1×5 sliding window is obtained through a cascaded shift register group. Five consecutive pixels are sent to a tree comparator for pairwise comparison. After multi-stage pipeline comparison, the maximum value of the current row's 1×5 window is output and cached in the FIFO chain. When the maximum value of the 1×5 window is filled in four rows, a vertical dimension tree comparison is triggered, which is compared with the maximum value output in the current row in a 5×1 comparison, and finally the 5×5 maximum value pooling result is output.

[0013] Furthermore, the improved YOLOv8n detection model is implemented in pure Verilog language on the FPGA, and the scheduler is configured to adapt to all types of convolution calculations in the model. The hardware architecture sets up a bit control register to sense the quantization parameters of the algorithm layer, and the switching precision is triggered by a single-cycle instruction without reconfiguring the bit stream.

[0014] To achieve the second objective of this invention, the following solution is adopted: A hardware acceleration system for lightweight object detection networks includes: An improved YOLOv8n detection module is used to obtain an improved YOLOv8n detection model; the improved YOLOv8n detection model includes: A weighted bidirectional feature pyramid network is used in the neck network for cross-scale feature fusion, and learnable weights are assigned to different input feature maps. A high-resolution detection branch is added to the neck network to form a multi-scale detection head with the original detection branch. A dynamic non-monotonic focusing mechanism is introduced into the regression loss to adaptively adjust the gradient weights according to the overlap of the bounding boxes of the samples. A mixed precision quantization strategy is adopted to perform non-uniform quantization on shallow feature maps with a higher quantization precision than that on deep feature maps. A reconfigurable computing array module is used to deploy the improved YOLOv8n detection model to an FPGA hardware acceleration unit. Based on the FPGA, a reconfigurable computing array is constructed. During the entire forward inference process of the improved YOLOv8n detection model, the computing scale and computing mode of the reconfigurable computing array are dynamically adjusted according to the parameters of the current convolutional layer to accelerate the execution of all convolution operations in the model and obtain the convolution output feature map. The pipelined addition tree accumulation module is used to perform parallel accumulation of partial sums in the convolutional output feature map at the point where the convolutional layer output and feature fusion are located, in order to obtain the accumulated feature data. The row-column orthogonal decomposition pooling module is used to perform maximum pooling on the accumulated feature data. It adopts a row-column orthogonal decomposition method, first pre-compressing the features in the horizontal direction and pipelined comparison to obtain the maximum value of the in-row window, and then pipelined comparison in the vertical direction to obtain the maximum value of the two-dimensional pooling window as the pooling result output.

[0015] Furthermore, the reconfigurable computing array module includes: The hardware scheduler is used to parse the current layer instructions and allocate on-chip SRAM resident buffers; Configuration registers are used to store mode codes and arbitrate different modes; The processing unit array has five configurable modes, namely MAC mode, single-multiplication pass-through mode, branch mode, weight-resident mode and feature map-resident mode; the processing unit array is dynamically reconfigured at three levels: inter-layer, intra-block and architecture depth, according to the control signals of the configuration register.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. In terms of detection accuracy, this invention effectively solves the problem of small target features being masked by deep redundant information in traditional feature pyramids by employing a weighted bidirectional feature pyramid network for cross-scale feature fusion and assigning learnable weights to different input feature maps. By adding a high-resolution detection branch, the invention enhances the preservation of details such as edges and textures in shallow feature maps. Furthermore, by introducing a dynamic non-monotonic focusing mechanism, the gradient weights are adaptively adjusted based on the overlap of sample bounding boxes, avoiding gradient bias caused by uneven training sample quality. Simultaneously, a hybrid precision quantization strategy is adopted to impart higher quantization precision to shallow feature maps, further preserving sensitive information about small targets. These improvements work synergistically, enabling this invention to increase detection accuracy from 88.5% of the original YOLOv8n to 92.6% and mAP from 85.2% to 89.7%, an improvement of 4.5 percentage points, with only a 2.7M increase in parameters. This significantly reduces the false negatives and false positives for small targets. 2. In terms of computational efficiency, this invention constructs a reconfigurable computing array based on FPGA, which can dynamically adjust the computing scale and computing mode of the array according to the parameters of the current convolutional layer. This overcomes the "computational hole" problem caused by the uneven utilization of processing units due to the change in feature map scale in traditional fixed-size systolic arrays, significantly improves the effective working efficiency of the multiply-accumulate unit, avoids frequent reading of weights and feature maps from off-chip DDR or BRAM, reduces data transfer power consumption, and improves inference throughput in edge computing scenarios. 3. In terms of hardware resources and operating frequency, this invention replaces the traditional serial accumulation chain with a pipelined addition tree, decomposing multi-path accumulation into multi-level parallel addition and inserting it into the pipeline register. This shortens the critical path, reduces wiring congestion caused by deep carry chains, and improves the logic convergence speed during large-scale feature fusion. Furthermore, it employs a row-column orthogonal decomposition method to achieve maximum pooling. Through horizontal feature pre-compression and a tree-structured comparator, the comparison logic of multiple inputs is decomposed into multi-level pipelines. This avoids the problems of excessive BRAM resource consumption and excessively deep combinational logic levels caused by caching multiple rows of original pixels in traditional implementations, enabling the system to operate stably in high-frequency environments above 200MHz and meeting the throughput requirements of real-time detection. Attached Figure Description

[0017] Figure 1 This is a flowchart of a hardware acceleration processing method for lightweight target detection networks in an embodiment of the present invention; Figure 2 This is a schematic diagram of the weighted bidirectional feature pyramid network (BiFPN) in an embodiment of the present invention; Figure 3 This is a diagram of the improved YOLOv8n architecture in an embodiment of the present invention; Figure 4This is a first detection result diagram of the original YOLOv8n in an embodiment of the present invention; Figure 5 This is a second detection effect diagram of the original YOLOv8n in an embodiment of the present invention; Figure 6 This is a first detection effect diagram of the improved YOLOv8n in an embodiment of the present invention; Figure 7 This is a diagram showing the improved YOLOv8n second detection effect in an embodiment of the present invention; Figure 8 This is a schematic diagram of the two-dimensional pulsating array matrix multiplication in an embodiment of the present invention; Figure 9 This is a schematic diagram of the dynamic partitioning structure of the pulsating array in an embodiment of the present invention; Figure 10 This is an internal structure diagram of the dynamically reconfigurable processing unit (PE) in an embodiment of the present invention; Figure 11 This is a schematic diagram of the pipeline tree-shaped adder in an embodiment of the present invention; Figure 12 This is a schematic diagram of the 5×5 maximum value pooling row and column orthogonal decomposition operation in an embodiment of the present invention; Figure 13 This is a block diagram of a hardware acceleration system for lightweight target detection networks in an embodiment of the present invention. Detailed Implementation

[0018] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0019] Example 1 like Figure 1-12 As shown, this embodiment of the invention provides a hardware acceleration processing method for lightweight target detection networks, including the following steps: Step S1: Obtain the improved YOLOv8n detection model; the improved YOLOv8n detection model includes: A weighted bidirectional feature pyramid network is used in the neck network for cross-scale feature fusion, and learnable weights are assigned to different input feature maps. A high-resolution detection branch is added to the neck network to form a multi-scale detection head with the original detection branch. A dynamic non-monotonic focusing mechanism is introduced into the regression loss to adaptively adjust the gradient weights according to the overlap of the bounding boxes of the samples. A mixed precision quantization strategy is adopted to perform non-uniform quantization on shallow feature maps with a higher quantization precision than that on deep feature maps.

[0020] Step S2: Deploy the improved YOLOv8n detection model to the FPGA hardware acceleration unit, build a reconfigurable computing array based on the FPGA, and dynamically adjust the computing scale and computing mode of the reconfigurable computing array according to the parameters of the current convolutional layer during the entire forward inference process of the improved YOLOv8n detection model to accelerate the execution of all convolution operations in the model and obtain the convolution output feature map.

[0021] Step S3: At the point where the convolutional layer output meets the feature fusion point, a pipelined addition tree is used to perform parallel accumulation of the partial sums in the convolutional output feature map to obtain the accumulated feature data.

[0022] Step S4: Perform max pooling on the accumulated feature data. Use row-column orthogonal decomposition. First, pre-compress the features in the horizontal direction and compare them in a pipeline to obtain the maximum value of the in-row window. Then, compare them in the vertical direction in a pipeline to obtain the maximum value of the two-dimensional pooling window as the pooling result output.

[0023] The hardware acceleration processing method for lightweight target detection networks according to embodiments of the present invention will be further described in detail below.

[0024] To address the issue that traditional feature pyramids in YOLOv8n cannot dynamically adjust feature fusion weights according to target scale, leading to the obscuring of small target features by deep redundant information, this invention first introduces a weighted bidirectional feature pyramid network. This network constructs a bidirectional cross-scale fusion link in the YOLOv8n neck network. Secondly, a small target detection head is added to enhance feature fusion for small objects, resolving the small target problem caused by shallow feature loss in the original YOLOv8n. By introducing a dynamic non-monotonic focusing mechanism, the gradient weights of small target samples are adaptively adjusted during the regression process, addressing the issues of uneven training sample quality and inaccurate bounding box regression. To address the high computational cost of the model, this invention employs hybrid multi-precision quantization, implementing non-uniform quantization on different residual blocks. For the shallow layers (P2 / P3) sensitive to small targets, INT16 is used to preserve texture information, while deep semantic features are compressed using INT8 and INT4.

[0025] The hardware acceleration processing method for lightweight target detection networks in this embodiment of the invention improves precision from 88.5% to 92.6% and mAP from 85.2% to 89.7%, an increase of 4.5%, with only a 2.7M increase in the number of parameters. The detection performance for small targets is significantly improved.

[0026] The hardware acceleration processing method for lightweight target detection networks according to embodiments of the present invention is as follows: The weighted bidirectional feature pyramid network employs a Fast Normalized Fusion strategy, assigning a learnable weight w to each input feature map, as shown in the following formula: The weighted bidirectional feature fusion module BiFPN is introduced to replace the PANet structure used in the original YOLOv8 neck. Skip connections are added to the neck part, so that the high-resolution information of the lower layer can be directly passed to the upper layer, which enhances the feature extraction capability of small targets.

[0027] In this embodiment, the addition of the small target detection head specifically involves adding a detection branch for the P2 scale (downsampling factor of 8x) feature map in the Neck. The P2 feature map has a higher resolution, thus preserving more detailed textures and edge information of small targets during downsampling of the P2 scale feature map. This enhances the detection of small targets in the 8×8 to 32×32 pixel range. Combined with detection at the P3, P4, and P5 scales, a multi-scale detection head is formed, predicting feature maps at more levels and providing a richer feature base for small target detection.

[0028] In this embodiment, the dynamic non-monotonic focusing mechanism specifically involves introducing a distance overlap loss R. WIoU And the focusing coefficient β of the accompanying regulatory factor. R WIoU This is the WIoU distance term, which is the normalized center distance plus the aspect ratio. When β is small, R is small, reducing the gradient contribution of high-quality samples; when β is large, R is large, maintaining attention to low-quality samples. When the detected target is a small target (area less than 32×32 pixels), multiplying β by the enhancement coefficient amplifies the outlier of the small target, thereby obtaining a higher focusing coefficient, making the model pay more attention to the optimization of small targets.

[0029] In this embodiment, the hybrid multi-precision quantization specifically involves: employing quantization-aware training and dynamically allocating hardware bit width based on the contribution of feature maps to detection accuracy. Small object detection heavily relies on shallow positional information; therefore, the input layer of YOLOv8n and the feature fusion node of BiFPN are set as sensitive layers, using INT16. The mid-to-deep semantic features of the backbone network are rich but texture is redundant; therefore, alternating precision quantization using INT8 and INT4 is employed. A bit control register is set up so that the hardware architecture can be aware of the quantization parameters of the algorithm layer, eliminating the need to reconfigure the bit stream when switching precisions, requiring only a single-cycle instruction trigger. To reduce the bias error in small object detection, the hardware layer must support asymmetric quantization. In each cascade of the tree adder, an offset addition operator is reserved to directly cancel the influence of Z in the pipeline. The formula is as follows: Where q is the quantized integer. For raw floating-point numbers, Let z be the scaling factor and z be the zero offset. Taking 8-bit activation value quantization as an example, first, the maximum and minimum values ​​of the input data are recursively found, and then... Calculate the scaling factor by Calculate the zero point, mapping the floating-point number 0 to a certain position in the quantization interval, instead of forcibly mapping it to 0, and finally execute the quantized activation value.

[0030] Understandably, the feature fusion path in YOLOv8n becomes more complex after the introduction of BiFPN, with frequent changes in feature map size. The YOLOv8n architecture involves multiple convolution operations of different sizes, and with the addition of a small target head (160×160), the input size of the feature map increases. Traditional FPGA-based YOLOv8n acceleration methods (such as nested loops or general-purpose DSP arrays) suffer from insufficient parallelism and slow computation speed when processing convolution operations, requiring repeated readings of weights and feature maps from off-chip DDR or BRAM. Frequent data transfer leads to a significant increase in power consumption, and external bandwidth limits computational throughput. Using a fixed-size systolic array results in uneven PE (processing unit) utilization when processing feature maps of different scales, making it difficult to maintain full-load operation of the computing units and thus affecting edge computing efficiency. Traditional FPGA pooling implementations typically require caching N-1 rows of raw image data; for a 5×5 window, this requires caching 4 rows of raw pixels. When processing multi-channel, high-resolution images, on-chip BRAM resources are consumed significantly. In addition, the traditional 25-way comparator tree logic hierarchy is deep, which limits the system's operating frequency.

[0031] To address the heterogeneous computational demands of the improved YOLOv8n at different scale feature fusion stages, and the low data reuse rate and computational efficiency of traditional FPGA-based YOLOv8n acceleration implementations, this invention proposes a processing matrix that dynamically adjusts the row-column mapping ratio based on layer parameters. Leveraging the deep pipeline and spatial parallelism of a systolic array, the hardware scheduling engine dynamically adjusts the array's parallel dimension according to the parameters of different YOLOv8n convolutional layers, fully releasing the parallel computation potential of YOLOv8n convolutional layers and significantly reducing inference latency. Furthermore, at the convolutional layer output and BiFPN feature fusion point, a cascaded pipelined addition tree is adopted instead of a serial accumulation chain, improving the parallel computing capabilities of the FPGA. In max pooling, a row-column orthogonal decomposition and feature pre-compression are used to calculate the maximum value of a 5×5 window. This cleverly avoids the massive combinational logic tree caused by 25 inputs, and divides the critical path delay into different clock cycles. Combined with a pipelined tree comparator, this allows the module to easily operate in high-frequency environments above 200MHz-300MHz, meeting the throughput requirements of YOLOv8n real-time detection. Compared to the redundant raw sampling points, the feature pre-compression method greatly optimizes the slicing utilization of the FPGA's internal BRAM.

[0032] This invention solves the "computational gap" problem caused by traditional fixed structures when handling different numbers of channels and feature map sizes by dynamically adjusting the parallel dimension of the array, significantly improving the effective working efficiency of the multiply-accumulate unit. The tree-like comparator and cascaded pipelined addition tree in the 5×5 max pooling reduce wiring congestion caused by deep carry chains and improve the logic convergence speed during large-scale feature fusion.

[0033] This invention is based on pure Verilog language, sets up a scheduler, and has a hardware architecture that can be adapted to all types of convolution calculations in YOLOv8n without the need to design dedicated accelerators for different layers.

[0034] The hardware acceleration component of the hardware acceleration method for lightweight target detection networks in this embodiment of the invention is detailed below: A systolic array consists of a large number of identical processing units (PEs) arranged in a regular grid. Each PE communicates only with its adjacent PEs, avoiding global interconnection. Data flows "systolicly" within the array, achieving high-level pipelined parallelism. Simultaneously, memory accesses are reduced through multiple data reuses within the array. Systolic arrays operate by converting convolutions into matrix multiplication (GEMM). Figure 8As shown, taking a 4×4 convolution as an example, Y = X×W, where X is a 4×4 input matrix, W is a 4×4 weight matrix, and Y is a 4×4 output matrix. The processing units (PEs) of the systolic array are responsible for multiply-accumulate (MAC) operations. Each PE passes its partially calculated sum to the right / downwards neighboring PEs, while simultaneously receiving new X and W data to begin a new round of multiply-accumulate computation. In each clock cycle, all PEs perform multiply-accumulate operations simultaneously, achieving a high degree of parallelism.

[0035] In this embodiment, the dynamically reconfigurable pulsating array specifically refers to the method of changing the data transmission between PEs by means of a hardware scheduler and control signals, based on the traditional static pulsating array, without reprogramming the FPGA firmware.

[0036] This invention employs a hardware scheduler to parse the current layer instructions. Based on the YOLOv8n input image size and number of channels, and the cross-layer skip connection requirements of BiFPN, it pre-allocates resident buffers in on-chip SRAM and dynamically adjusts the computation mode of the streaming systolic array. The dynamic configuration scheduling strategy includes three levels: inter-layer configuration scheduling, switching array structures between different convolutional layers; intra-block configuration scheduling, dynamically adjusting within the same convolutional layer according to different computation blocks; and architecture depth configuration scheduling, switching the data residing in the PE block for downsampling at different depths.

[0037] In this embodiment, the inter-layer configuration scheduling is specifically as follows: YOLOv8n has 3×3 convolution and 1×1 convolution. When performing 3×3 convolution (taking single-channel convolution as an example), the PE within the block receives the feature data from the left side, splits it into 3 rows and caches it, then multiplies it with the input weights one by one to obtain the product of each PE block in a single column. The result of the single column is cached and accumulated with the result of the previous level to obtain the final 3×3 window convolution result. When performing 1×1 convolution, the calculation mode of the PE unit within the block changes. It no longer needs to be accumulated level by level, but is directly output after multiplication. Therefore, it is not necessary to pre-cache for accumulation. The resources of the originally cached part are released for data prefetching. While processing the current data, the hardware can load the next batch of data from DDR in advance, which greatly improves the throughput.

[0038] In this embodiment, the intra-block configuration scheduling specifically refers to the following: When inferring networks containing cross-layer cascaded structures (such as the C2f module of YOLOv8), traditional single systolic arrays face the problem of computational and data loss mismatch. In this module, the input feature map is divided into two branches along the channel dimension: branch A performs identity mapping (direct connection), and branch B enters the Bottleneck for convolutional transformation. For static systolic arrays, due to the lack of dynamic reconstruction capability of computational topology, they can only process these two branches sequentially in a time-slice round-robin manner. That is, the array is scheduled to execute the dense convolutional computation of branch B first, and after the result of branch B is generated, the concatenation operation of branch A is processed. During this process, the data of branch A is idle and waiting in the off-chip or on-chip buffer, while the weight loading and computation of branch B also monopolize bus resources, resulting in an increase in overall inference latency, and the array utilization can only reach 50% of the theoretical peak within the actual computation window. To address the aforementioned issues, this invention introduces a hardware-aware dynamic array partitioning mechanism. When processing high-channel-count convolutional layers (such as 128 or 256 channels) in the backbone, all PEs are concatenated into a large 16×16 array, and multiplication and accumulation operations are performed on 16 channels in parallel to maximize single-channel throughput. When processing the C2f module, the 16×16 array is split into two independent 8×16 sub-arrays. One half processes convolution while the other half processes concatenation operations in parallel, reducing overall inference latency by reconstructing the array topology.

[0039] In this embodiment, the architecture depth configuration scheduling specifically involves the following: In the shallow backbone layer, the input feature image size is large and the number of channels is small. As the architecture depth increases, the feature image becomes smaller, the number of channels increases, and the corresponding weight data increases, sometimes far exceeding the feature image data. If only one array mode is fixed, there will be differences in data reuse, leading to frequent DDR access due to insufficient resident data. Therefore, by controlling the data reuse type signal, in shallow downsampling, pixel data is resident, allowing intermediate results to remain in the PE, reducing frequent writes to memory; in deep downsampling, weight data is resident, reducing frequent weight reads and writes, allowing pixels to pass through each PE level as a data stream.

[0040] The dynamically reconfigurable PE designed in this invention has five configurable functions, and the corresponding use cases and mode codes are shown in the table below. Different modes are arbitrated by the configuration register, and the control signal determines whether to perform the corresponding calculation. The internal structure diagram of a single PE is shown below. Figure 10 As shown.

[0041] In this embodiment, the tree-structured adder utilizes cascaded registers by breaking down the N-way accumulation into log2N stages of parallel addition, inserting a pipeline register at each stage. This shortens the critical path to a "single-stage adder carry chain," distributing the large-bit-width addition pressure across multiple clock cycles. For example... Figure 11 As shown, the 16 inputs are divided into a 4-level binary tree structure (log216=4), with each level performing 2 additions and outputting a partial sum. By cascading the "output of the previous addition tree to the input of the next addition tree", it adapts to the accumulation requirements of different PE numbers of the pulsating array.

[0042] In this embodiment, the 5×5 max pooling (pooling step size of 1) specifically involves: firstly, decomposing the multi-channel input data (this invention uses 8 output channels) and restoring it into multiple independent pixel streams. M parallel computing units are constructed using the FPGA's generate syntax to achieve a throughput of processing M pixels in a single clock cycle; subsequently, horizontal feature pre-compression is performed: after the data enters the computing unit of each channel, it first undergoes a 1×5 window sliding acquisition through a cascaded shift register group, and the obtained 5 consecutive pixels are synchronously sent to a high-performance tree comparator. The tree-structured comparator employs a pairwise comparison logic, distributing the comparisons of N inputs across log₂N. Each group of pixels enters the first-stage comparator array in parallel for numerical discrimination. For the 5-to-1 asymmetric structure, redundant branch data is time-delayed and time-stamped via a synchronization register, ensuring strict temporal alignment with the local maxima generated at each iteration. Through multi-stage pipelined comparisons, the complex combinational logic path is divided into multiple independent synchronous timing segments. Finally, the maximum value of the current row's 1×5 window is latched at the output and buffered in a synchronous read-write FIFO chain. When the maximum values ​​of four rows' 1×5 windows are full, a control signal triggers a vertical tree-structured comparison, performing a 5×1 comparison with the current row's output maximum value, and finally outputting the 5×5 maximum pooling result. The single-channel 5×5 maximum pooling operation is as follows: Figure 12 As shown.

[0043] Example 2 like Figure 13 As shown, this embodiment of the invention also provides a hardware acceleration system for lightweight target detection networks, comprising: An improved YOLOv8n detection module is used to obtain an improved YOLOv8n detection model; the improved YOLOv8n detection model includes: A weighted bidirectional feature pyramid network is used in the neck network for cross-scale feature fusion, and learnable weights are assigned to different input feature maps. A high-resolution detection branch is added to the neck network to form a multi-scale detection head with the original detection branch. A dynamic non-monotonic focusing mechanism is introduced into the regression loss to adaptively adjust the gradient weights according to the overlap of the bounding boxes of the samples. A mixed precision quantization strategy is adopted to perform non-uniform quantization on shallow feature maps with a higher quantization precision than that on deep feature maps.

[0044] A reconfigurable computing array module is used to deploy the improved YOLOv8n detection model to an FPGA hardware acceleration unit. Based on the FPGA, a reconfigurable computing array is constructed. During the entire forward inference process of the improved YOLOv8n detection model, the computing scale and computing mode of the reconfigurable computing array are dynamically adjusted according to the parameters of the current convolutional layer to accelerate the execution of all convolution operations in the model and obtain convolution output feature maps.

[0045] The pipelined addition tree accumulation module is used to perform parallel accumulation of partial sums in the convolutional output feature map at the point where the convolutional layer output and feature fusion are located, using a pipelined addition tree to obtain the accumulated feature data.

[0046] The row-column orthogonal decomposition pooling module is used to perform maximum pooling on the accumulated feature data. It adopts a row-column orthogonal decomposition method, first pre-compressing the features in the horizontal direction and pipelined comparison to obtain the maximum value of the in-row window, and then pipelined comparison in the vertical direction to obtain the maximum value of the two-dimensional pooling window as the pooling result output.

[0047] Furthermore, the reconfigurable computing array module includes: The hardware scheduler is used to parse the current layer instructions and allocate on-chip SRAM resident buffers; Configuration registers are used to store mode codes and arbitrate different modes; The processing unit array has five configurable modes, namely MAC mode, single-multiplication pass-through mode, branch mode, weight-resident mode and feature map-resident mode; the processing unit array is dynamically reconfigured at three levels: inter-layer, intra-block and architecture depth, according to the control signals of the configuration register.

[0048] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A hardware acceleration processing method for lightweight target detection networks, characterized in that, Includes the following steps: Step S1: Obtain the improved YOLOv8n detection model; the improved YOLOv8n detection model includes: A weighted bidirectional feature pyramid network is used in the neck network for cross-scale feature fusion, and learnable weights are assigned to different input feature maps. A high-resolution detection branch is added to the neck network to form a multi-scale detection head with the original detection branch. A dynamic non-monotonic focusing mechanism is introduced into the regression loss to adaptively adjust the gradient weights according to the overlap of the bounding boxes of the samples. A mixed precision quantization strategy is adopted to perform non-uniform quantization on shallow feature maps with a higher quantization precision than that on deep feature maps. Step S2: Deploy the improved YOLOv8n detection model to the FPGA hardware acceleration unit, build a reconfigurable computing array based on the FPGA, and dynamically adjust the computing scale and computing mode of the reconfigurable computing array according to the parameters of the current convolutional layer during the entire forward inference process of the improved YOLOv8n detection model to accelerate the execution of all convolution operations in the model and obtain the convolution output feature map. Step S3: At the point where the convolutional layer output meets the feature fusion point, a pipelined addition tree is used to perform parallel accumulation of the partial sums in the convolutional output feature map to obtain the accumulated feature data. Step S4: Perform max pooling on the accumulated feature data. Use row-column orthogonal decomposition. First, pre-compress the features in the horizontal direction and compare them in a pipeline to obtain the maximum value of the in-row window. Then, compare them in the vertical direction in a pipeline to obtain the maximum value of the two-dimensional pooling window as the pooling result output.

2. The hardware acceleration processing method for lightweight target detection networks according to claim 1, characterized in that, The weighted bidirectional feature pyramid network adopts a fast normalization fusion strategy, assigning learnable weights to each input feature map. The output feature map is equal to the sum of each input feature map multiplied by its corresponding weight and then divided by the sum of the weights. The weights are updated through network training.

3. The hardware acceleration processing method for lightweight target detection networks according to claim 1, characterized in that, In the dynamic non-monotonic focusing mechanism, the distance overlap loss is used as the evaluation index of sample quality. The focusing coefficient is determined according to the ratio of the distance overlap loss to the learnable adjustable parameter. When the detected target is a small target with an area of ​​less than 32×32 pixels, the focusing coefficient is multiplied by the enhancement coefficient to amplify the outlier. The hybrid precision quantization strategy employs asymmetric quantization, and the quantization formula is as follows: Where q is the quantized integer. For raw floating-point numbers, is the scaling factor, and z is the zero-point offset; among them, shallow feature maps are quantized using INT16, and deep semantic features are quantized using alternating precision INT8 and INT4.

4. The hardware acceleration processing method for lightweight target detection networks according to claim 1, characterized in that, In step S2, dynamically adjusting the computing scale and computing mode of the reconfigurable computing array includes inter-layer configuration scheduling, intra-block configuration scheduling, and architecture-deep configuration scheduling. The inter-layer configuration scheduler switches the array structure between different convolutional layers. When performing 3×3 convolution, the feature data is split into three rows of cache and accumulated. When performing 1×1 convolution, the cache resources are released for data prefetching. The intra-block configuration scheduling is dynamically adjusted within the same convolutional layer according to different computation blocks. When processing the C2f module, the large array is split into two independent sub-arrays for parallel processing. The architecture's deep configuration scheduling switches the resident data based on the downsampling depth, resident pixel data in shallow downsampling, and resident weight data in deep downsampling.

5. The hardware acceleration processing method for lightweight target detection networks according to claim 4, characterized in that, The reconfigurable computing array is a systolic array, and its processing unit has five configurable modes, including: MAC mode for convolution operations, single-multiplication pass-through mode for 1×1 convolution, branch mode for C2f modules, weight-resident mode for deep downsampling, and feature map-resident mode for shallow downsampling; each mode is switched by the configuration register according to the arbitration of the control signal.

6. The hardware acceleration processing method for lightweight target detection networks according to claim 1, characterized in that, In step S3, the pipelined adder tree decomposes the N-way accumulation into log2N-stage parallel additions, inserts a pipeline register at each stage, and uses the output of the previous stage adder tree as the input of the next stage adder tree. The cascading method adapts to the accumulation requirements of different numbers of processing units.

7. The hardware acceleration processing method for lightweight target detection networks according to claim 1, characterized in that, In step S4, the orthogonal decomposition method is specifically used as follows: Multi-channel input data is decomposed into multiple independent pixel streams, and multiple parallel computing units are constructed using the FPGA's generate syntax. In each computing unit, a 1×5 sliding window is obtained through a cascaded shift register group. Five consecutive pixels are sent to a tree comparator for pairwise comparison. After multi-stage pipeline comparison, the maximum value of the current row's 1×5 window is output and cached in the FIFO chain. When the maximum value of the 1×5 window is filled in four rows, a vertical dimension tree comparison is triggered, which is compared with the maximum value output in the current row in a 5×1 comparison, and finally the 5×5 maximum value pooling result is output.

8. The hardware acceleration processing method for lightweight target detection networks according to claim 1, characterized in that, The improved YOLOv8n detection model is implemented in pure Verilog language on FPGA. It adapts all types of convolution calculations in the model by configuring the scheduler. The hardware architecture sets up a bit control register to sense the quantization parameters of the algorithm layer. When switching precision, it is triggered by a single-cycle instruction without reconfiguring the bit stream.

9. A hardware acceleration system for lightweight target detection networks, characterized in that, include: An improved YOLOv8n detection module is used to obtain an improved YOLOv8n detection model; The improved YOLOv8n detection model includes: A weighted bidirectional feature pyramid network is used in the neck network for cross-scale feature fusion, and learnable weights are assigned to different input feature maps. A high-resolution detection branch is added to the neck network to form a multi-scale detection head with the original detection branch. A dynamic non-monotonic focusing mechanism is introduced into the regression loss to adaptively adjust the gradient weights according to the overlap of the bounding boxes of the samples. A mixed precision quantization strategy is adopted to perform non-uniform quantization on shallow feature maps with a higher quantization precision than that on deep feature maps. A reconfigurable computing array module is used to deploy the improved YOLOv8n detection model to an FPGA hardware acceleration unit. Based on the FPGA, a reconfigurable computing array is constructed. During the entire forward inference process of the improved YOLOv8n detection model, the computing scale and computing mode of the reconfigurable computing array are dynamically adjusted according to the parameters of the current convolutional layer to accelerate the execution of all convolution operations in the model and obtain the convolution output feature map. The pipelined addition tree accumulation module is used to perform parallel accumulation of partial sums in the convolutional output feature map at the point where the convolutional layer output and feature fusion are located, in order to obtain the accumulated feature data. The row-column orthogonal decomposition pooling module is used to perform maximum pooling on the accumulated feature data. It adopts a row-column orthogonal decomposition method, first pre-compressing the features in the horizontal direction and pipelined comparison to obtain the maximum value of the in-row window, and then pipelined comparison in the vertical direction to obtain the maximum value of the two-dimensional pooling window as the pooling result output.

10. The hardware acceleration system for lightweight target detection networks according to claim 9, characterized in that, The reconfigurable computing array module includes: The hardware scheduler is used to parse the current layer instructions and allocate on-chip SRAM resident buffers; Configuration registers are used to store mode codes and arbitrate different modes; The processing unit array has five configurable modes, namely MAC mode, single-multiplication pass-through mode, branch mode, weight-resident mode and feature map-resident mode; the processing unit array is dynamically reconfigured at three levels: inter-layer, intra-block and architecture depth, according to the control signals of the configuration register.

Images