Low latency non-maximum suppression efficient digital circuit

Abstract

The application discloses a low-delay non-maximum suppression efficient digital circuit, comprising a plurality of parallel preprocessing modules, an arbitration module, a preprocessing result buffer module and a plurality of serial NMS units. The circuit does not need to sort all initial candidate frames in the processing flow, reduces the starting delay, parallelizes the sorting process of the intermediate cache frame and the suppression process of the cache frame, so that the hardware implementing the algorithm can accept new input candidate frames without blocking, further reduces the processing delay, and further reduces the overall delay of the target recognition neural network. The application can reduce the resources and area occupied by the non-maximum suppression digital circuit under the condition of ensuring that the algorithm accuracy is almost unchanged, and significantly reduces the processing delay of the non-maximum suppression algorithm.

Description

Technical Field

[0001] This invention belongs to the field of target recognition technology. Specifically, this invention relates to a low-latency, non-maximum suppression, high-efficiency digital circuit. Background Technology

[0002] Object recognition neural networks are widely used in systems such as robotics and autonomous driving. Processing latency is a key metric for evaluating the performance of these systems. Research on accelerating object detection neural networks typically focuses on speeding up the convolutional neural network portion, while neglecting latency optimization in the post-processing stage.

[0003] Non-Maximum Suppression (NMS) is a common post-processing algorithm. Among them, Greedy NMS is a widely used NMS algorithm. However, due to its quadratic algorithm complexity and frequent and random memory accesses, it has become a bottleneck for target recognition neural networks. In general-purpose graphics processing units (GPGPUs), the latency of Greedy NMS accounts for 22% to 39% of the total execution latency of the target detection neural network.

[0004] The patents "A Non-Maximum Suppression Method for Point Cloud Networks Based on FPGA" and "An FPGA Acceleration Method Based on Non-Maximum Suppression Algorithm" propose an FPGA-based Greedy NMS digital circuit. Since Greedy NMS requires sorting all initial candidate boxes, this acceleration method, after obtaining each output box, still needs to sort all remaining candidate boxes by confidence. It can only reduce the single sorting delay by increasing computational resources, offering limited optimization.

[0005] The paper "AFast and Power-Efficient Hardware Architecture for Non-Maximum Suppression" proposes an efficient digital circuit for Greedy NMS. However, it does not involve optimization at the core algorithm level. Its performance improvement is mainly achieved by simplifying the cross-parallel ratio calculation and introducing a bit-mapped table circuit to accelerate the memory indexing process of candidate boxes. The reduction in NMS latency is limited.

[0006] The paper "PSRR-MaxpoolNMS: Pyramid Shifted MaxpoolNMS with RelationshipRecovery" proposes PSRR-MaxpoolNMS, a highly parallelizable NMS algorithm. Building upon this, the paper "Scalable Hardware Acceleration of Non-Maximum Suppression" presents a circuit implementation scheme for PSRR-MaxpoolNMS, significantly reducing NMS processing latency. However, due to the increased parallelism and redundant operations in PSRR-MaxpoolNMS, the required circuit area is significantly increased, resulting in a high cost for this scheme.

[0007] The patent "Low-Latency Non-Maximum Suppression Method and Device Based on FPGA" proposes an efficient digital circuit for NMS optimization targeting startup delay. This method does not require pre-sorting of all initial candidate boxes, but after an input candidate box, it needs to sort all intermediate buffer boxes. Before the sorting is completed, the hardware system cannot accept new input candidate boxes, causing blocking and ultimately resulting in a limited reduction in NMS delay. Summary of the Invention

[0008] This invention addresses the problems of high latency and large area overhead in existing NMS digital circuits by proposing a low-latency, high-efficiency non-maximum suppression digital circuit. This circuit eliminates the need to sort all initial candidate boxes, reducing startup latency. It parallelizes the sorting and suppression processes of intermediate buffer boxes, allowing the hardware implementing the algorithm to accept new input candidate boxes without blocking, further reducing processing latency and consequently lowering the overall latency of the target recognition neural network.

[0009] The technical solution of the present invention is as follows:

[0010] A non-maximum suppression high-efficiency digital circuit is characterized by comprising: multiple parallel preprocessing modules, an arbitration module, a preprocessing result buffer module, and multiple series-connected NMS units.

[0011] The preprocessing module fully utilizes the sparsity of confidence scores in the candidate boxes generated by the target recognition neural network to pre-screen candidate boxes with high parallelism, eliminating a large number of candidate boxes with extremely low confidence scores, reducing the workload of subsequent hardware circuits, and thus reducing the processing latency of the entire system.

[0012] The arbitration module is responsible for resolving conflicts when multiple preprocessing modules write to a preprocessing result buffer module at the same time. By analyzing the storage utilization rate inside the preprocessing modules, it allows the preprocessing modules with higher storage utilization to write their results to the preprocessing result buffer module first.

[0013] The preprocessing result buffer module is responsible for solving the problem of speed mismatch between the preprocessing module and the subsequent hardware circuit. It uses FIFO as a buffer between the two, which greatly reduces the probability of system blocking.

[0014] An NMS unit is responsible for generating a final output box.

[0015] The preprocessing module includes: a comparator module and a FIFO module;

[0016] Comparator module: Compares the confidence level of the current candidate box with the confidence threshold. If the confidence level of the candidate box is greater than the confidence threshold, a write request signal is sent to the FIFO module to write the current candidate box information into the FIFO module. Otherwise, the current candidate box information is discarded.

[0017] FIFO module: Accepts read request signals and write request signals, and issues FIFO empty signal and FIFO full signal to cache candidate box information.

[0018] The arbitration module is responsible for controlling the data reading process of the FIFO in the preprocessing module. It receives the FIFO empty signal and FIFO full signal from the FIFO and sends a read request signal to the FIFO module. When any FIFO is in a "non-empty" state, the arbitration module will send a read request signal to it in numerical order. When any FIFO is in a "full" state, it will send a read request signal to it first.

[0019] The preprocessing result buffer module includes: a multiplexer and a FIFO module;

[0020] The multiplexer selects one data output path from multiple preprocessing modules based on the read request signal sent by the arbitration module to the FIFO in the preprocessing module, and connects it to the FIFO of the preprocessing result buffer module.

[0021] The FIFO module is responsible for caching the preprocessed candidate box information and outputting it to the subsequent NMS unit.

[0022] The NMS unit includes: a candidate box storage module, an intersection-over-union (IoU) calculation module, and a candidate box storage update control module;

[0023] The candidate box storage module is responsible for caching candidate box information and writing or reading data according to the control signals issued by the candidate box storage update control module.

[0024] The intersection-union ratio (IUU) calculation module receives the current input candidate box data sent by the predecessor NMS module and the temporary selection box data read from the candidate box storage module. It calculates the IUU ratio between the temporary selection box transmitted by the candidate box storage module and the current input candidate box, which is the ratio of the intersection area of ​​the two candidate boxes to the union area of ​​the two candidate boxes, and transmits the result to the candidate box storage update control module.

[0025] The candidate box storage update control module receives the current input candidate box data sent by the predecessor NMS module, the temporary selection box data read from the candidate box storage module, and the sorting box data read from the candidate box storage module, generates a status signal, and generates read / write control signals for the candidate box storage module based on the intersection-union ratio and the status signal.

[0026] The candidate box storage module includes: an I storage unit, an S storage unit, and a C storage unit;

[0027] The I-cell storage unit caches candidate box information for output to subsequent NMS units. The cached candidate boxes are the input candidate boxes of the subsequent NMS units. The cached information includes the candidate box coordinates, confidence score, and a flag indicating that the candidate box has been saved.

[0028] The S storage unit is a redundant storage unit that caches the coordinates and confidence score of a candidate bounding box. Its function is to assist the candidate bounding box storage update control module in sorting candidate bounding boxes in the C storage units of multiple NMS units according to their confidence scores.

[0029] The C storage unit caches the coordinates, confidence score, and cell validity flag of an output box. When no candidate box has passed through all NMS cells, it stores the temporary results of the output box.

[0030] The status signals include: S storage unit valid signal, I storage unit input box saved signal, cross-over ratio greater than threshold signal, and confidence comparison signal;

[0031] The valid signal for the S memory cell is high when the confidence level of the candidate box stored in the S memory cell of the current NMS cell is greater than 1, and low otherwise.

[0032] The signal indicating that the candidate box has been saved in the I memory cell is high when the candidate box has been saved flag is high, and low otherwise.

[0033] The signal is high when the cross-parallel ratio (CPR) output by the CPR calculation module is greater than the CPR threshold, and low otherwise.

[0034] The confidence comparison signal is high when the confidence of the candidate box stored in the I memory cell of the predecessor NMS cell is greater than the confidence of the candidate box stored in the C memory cell of the current NMS cell, and low otherwise.

[0035] The read / write control signals of the candidate box storage module include I-unit read / write control signals, S-unit read / write control signals, and C-unit read / write control signals.

[0036] The I-cell read / write control signal controls the reading and writing of the I-cells of the current NMS cell;

[0037] The S-cell read / write control signal controls the read and write operations of the S-cell of the current NMS cell.

[0038] The C cell read / write control signal controls the reading and writing of the C memory cell in the current NMS cell.

[0039] The steps for generating the I-cell read / write control signal are as follows:

[0040] Step 8-1: When the saved signal in the I storage cell input box is low, the cross-parallel ratio greater than the threshold signal is high, and the confidence comparison signal is low, a control signal is generated to clear the data in the I storage cell. Otherwise, the control signal is generated to write the data in the I storage cell of the predecessor NMS cell into the I storage cell of the current NMS cell.

[0041] The steps for generating the S-unit read / write control signal are as follows:

[0042] Step 9-1: If the signal saved in the I memory cell input box is high and the valid signal in the S memory cell is high, then jump to step 9-7;

[0043] Step 9-2: If the signal saved in the I storage cell input box is high, the valid signal in the S storage cell is low, and the signal with a cross-parallel ratio greater than the threshold is high, then jump to step 9-7;

[0044] Step 9-3: If the signal saved in the I storage cell input box is high, the valid signal in the S storage cell is low, and the signal with a cross-parallel ratio greater than the threshold is low, then jump to step 9-8;

[0045] Step 9-4: If the signal that the I storage cell input box has been saved is low, the signal that the cross-parallel ratio is greater than the threshold is high, and the confidence comparison signal is high, then jump to step 9-7;

[0046] Step 9-5: If the signal that the I storage cell input box has been saved is low, the signal that the cross-parallel ratio is greater than the threshold is high, and the confidence comparison signal is low, then jump to step 9-8;

[0047] Step 9-6: If the signal saved in the I storage cell input box is low, and the signal with an intersection-to-parallel ratio greater than the threshold is low, then jump to step 9-7;

[0048] Step 9-7: Generate a control signal to clear the data in the S memory cell of the current NMS cell, and jump to step 9-9;

[0049] Step 9-8: Generate a control signal to write the data in the C memory cell of the current NMS cell into the S memory cell of the current NMS cell, set the cell valid flag to high level, and jump to step 9-9;

[0050] Step 9-9: Generation of S-unit read / write control signals ends.

[0051] The steps for generating the C unit read / write control signal are as follows:

[0052] Step 10-1: If the signal saved in the input box of storage cell I is high, the valid signal of storage cell S is high, and the signal with cross-parallel ratio greater than the threshold is high, then jump to step 10-11;

[0053] Step 10-2: If the signal saved in the I storage cell input box is high, the valid signal in the S storage cell is high, and the signal with the cross-parallel ratio greater than the threshold is low, then jump to step 10-10;

[0054] Step 10-3: If the signal saved in the I memory cell input box is high and the valid signal in the S memory cell is low, then jump to step 10-9;

[0055] Step 10-4: If the signal that the I storage cell input box has been saved is low, the signal that the cross-parallel ratio is greater than the threshold is high, and the confidence comparison signal is high, then jump to step 10-8;

[0056] Step 10-5: If the signal saved in the I storage cell input box is low, the signal that the cross-parallel ratio is greater than the threshold is high, and the confidence comparison signal is low, then jump to step 10-10;

[0057] Step 10-6: If the signal that the I storage cell input box has been saved is low, the signal that the cross-parallel ratio is greater than the threshold is low, and the confidence comparison signal is high, then jump to step 10-8;

[0058] Step 10-7: If the signal saved in the I storage cell input box is low, the signal that the cross-parallel ratio is greater than the threshold is low, and the confidence comparison signal is low, then jump to step 10-10;

[0059] Step 10-8: Generate a control signal to write the data in the I memory cell of the predecessor NMS cell into the C memory cell of the current NMS cell, set the cell valid flag to high level, and jump to step 10-12;

[0060] Step 10-9: Generate a control signal to write the data in the S memory cell of the predecessor NMS cell into the C memory cell of the current NMS cell, set the cell valid flag to high level, and jump to step 10-12;

[0061] Step 10-10: Generate control signals, without any read or write operations, then jump to step 10-12;

[0062] Step 10-11: Set the cell valid flag to low level, then jump to step 10-12;

[0063] Steps 10-12: The generation of read / write control signals for Unit C is complete.

[0064] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0065] 1) In fields such as robotics and autonomous driving, the real-time performance requirements for target recognition technology are high; lower target recognition latency translates to higher security and reliability. This invention reduces the latency of the overall target recognition neural network, thereby improving the performance and reliability of the application.

[0066] 2) Compared with the circuit implementation scheme proposed in the article "Scalable Hardware Acceleration of Non-Maximum Suppression", the present invention reduces the circuit area by 75%.

[0067] 3) Compared with the circuit implementation scheme proposed in the article "Scalable Hardware Acceleration of Non-Maximum Suppression", the present invention reduces the processing latency of each candidate box by 76%. Attached Figure Description

[0068] Figure 1 This is a flowchart of the Greedy NMS algorithm in the existing technology.

[0069] Figure 2 This is the top-level block diagram of the NMS high-efficiency digital circuit of the present invention. Detailed Implementation

[0070] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0071] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes or methods.

[0072] The specific steps of implementing this invention on the ASIC platform will be further described below with reference to the accompanying drawings.

[0073] This implementation is based on the YOLOv3 network and the PASCAL VOC dataset. According to the network accuracy and application requirements, the initial number of candidate boxes is set to 10647, the upper limit of output boxes per image is 16, the confidence threshold is 0.5, the intersection-over-union (IoU) threshold is 0.4, the number of preprocessing modules is 20, the number of NMS units is 20, the FIFO depth of the preprocessing modules is 8, and the FIFO depth of the preprocessing result buffer module is 8.

[0074] This invention mainly includes the following steps:

[0075] Step 1: Construct 16 parallel preprocessing modules. The specific steps for constructing the preprocessing modules are as follows:

[0076] Step 1-1: Construct a comparator module to compare the confidence level of the current candidate box with the confidence threshold. If the confidence level of the candidate box is greater than the confidence threshold, send a write request signal to the FIFO module to write the current candidate box information into the FIFO module; otherwise, discard the current candidate box information.

[0077] Steps 1-2: Construct a FIFO module to accept read request signals and write request signals, and issue FIFO empty signal and FIFO full signal to cache candidate box information.

[0078] Step 2: Construct an arbitration module to control the data reading process of the FIFO in the preprocessing module, receive FIFO empty and FIFO full signals from the FIFO, and send read request signals to the FIFO module. When any FIFO is in a "non-empty" state, the arbitration module will send read request signals to it in numerical order. When any FIFO is in a "full" state, it will send read request signals to it first.

[0079] Step 3: Construct a preprocessing result buffer module. The specific steps for constructing the preprocessing result buffer module are as follows:

[0080] Step 3-1: Construct a multiplexer. Based on the read request signal sent by the arbitration module to the FIFO in the preprocessing module, select one of the multiple data output paths of the preprocessing module and connect it to the FIFO of the preprocessing result buffer module.

[0081] Step 3-2: Construct a FIFO module, which is responsible for caching the preprocessed candidate box information and outputting it to the subsequent NMS unit.

[0082] Step 4: Construct 20 cascaded NMS units. The specific steps for constructing the NMS units are as follows:

[0083] Step 4-1: Construct a candidate box storage module, responsible for caching candidate box information. Based on the control signals issued by the candidate box storage update control module, it performs data writing or reading. The specific steps are as follows:

[0084] Step 4-1-1: Construct an I-memory unit to cache candidate box information output to the subsequent NMS unit. The cached candidate box is the input candidate box of the subsequent NMS unit. The cached information includes the candidate box coordinates, confidence score, and a candidate box saving flag.

[0085] Step 4-1-2: Construct the S storage unit, a redundant storage unit capable of caching the coordinates and confidence score of a candidate bounding box. Its function is to assist the candidate bounding box storage update control module in sorting the candidate bounding boxes in the C storage unit of multiple NMS units according to their confidence scores;

[0086] Step 4-1-3: C storage unit caches the coordinates, confidence score, and cell validity flag of an output box. When all candidate boxes have not passed through all NMS units, it stores the temporary results of the output boxes.

[0087] Step 4-2: Construct an intersection-union ratio (IUU) calculation module, which receives the current input candidate box data sent by the predecessor NMS module and the temporary selection box data read from the candidate box storage module, calculates the IUU ratio between the temporary selection box transmitted by the candidate box storage module and the current input candidate box, that is, the ratio of the intersection area of ​​the two candidate boxes to the union area of ​​the two candidate boxes, and transmits the result to the candidate box storage update control module.

[0088] Step 4-3: Construct a candidate box storage update control module, which receives the current input candidate box data sent by the predecessor NMS module, the temporary selection box data read from the candidate box storage module, and the sorting box data read from the candidate box storage module. It generates status signals and, based on the intersection-over-union ratio (IoU) and the status signals, generates read / write control signals for the candidate box storage module. The specific steps are as follows:

[0089] Step 4-3-1: Generate the valid signal of the S storage cell, the signal already saved in the input box of the I storage cell, the signal with an intersection-to-exchange ratio greater than the threshold, and the confidence comparison signal. The specific steps are as follows:

[0090] Step 4-3-1-1: Generate an effective signal for the S memory cell. When the confidence level of the candidate box stored in the S memory cell of the current NMS cell is greater than 1, the signal is high; otherwise, it is low.

[0091] Step 4-3-1-2: Generate the saved signal for the input box of the I memory cell. When the candidate box is saved flag high in the current NMS cell I memory cell, this signal is high; otherwise, it is low.

[0092] Step 4-3-1-3: Generate a signal that the cross-parallel ratio (CPR) is greater than the threshold. When the CPR output by the CPR calculation module is greater than the CPR threshold, the signal is high; otherwise, it is low.

[0093] Step 4-3-1-4: Generate a confidence comparison signal. When the confidence of the candidate box stored in the I memory cell of the predecessor NMS cell is greater than the confidence of the candidate box stored in the C memory cell of the current NMS cell, the signal is high; otherwise, it is low.

[0094] Step 4-3-2: Generate read / write control signals for the candidate box storage module. The specific steps are as follows:

[0095] Step 4-3-2-1: I-cell read / write control signal, controls the read and write of the I-cell memory of the current NMS cell, the specific steps are as follows:

[0096] Step 4-3-2-1-1: When the saved signal in the I memory cell input box is low, the cross-parallel ratio greater than the threshold signal is high, and the confidence comparison signal is low, a control signal is generated to clear the data in the I memory cell. Otherwise, the control signal is generated to write the data in the I memory cell of the predecessor NMS cell into the I memory cell of the current NMS cell.

[0097] Step 4-3-2-2: S-cell read / write control signal, controls the read / write of the S-cell of the current NMS cell, the specific steps are as follows:

[0098] Step 4-3-2-2-1: If the signal saved in the I memory cell input box is high and the valid signal in the S memory cell is high, then jump to step 4-3-2-2-7;

[0099] Step 4-3-2-2-2: If the signal saved in the input box of storage cell I is high, the valid signal of storage cell S is low, and the signal with cross-parallel ratio greater than the threshold is high, then jump to step 4-3-2-2-7;

[0100] Step 4-3-2-2-3: If the signal saved in the input box of storage cell I is high, the valid signal of storage cell S is low, and the signal with cross-parallel ratio greater than the threshold is low, then jump to step 4-3-2-2-8;

[0101] Step 4-3-2-2-4: If the signal saved in the I storage cell input box is low, the signal that the cross-parallel ratio is greater than the threshold is high, and the confidence comparison signal is high, then jump to step 4-3-2-2-7;

[0102] Step 4-3-2-2-5: If the signal saved in the I storage cell input box is low, the signal that the cross-parallel ratio is greater than the threshold is high, and the confidence comparison signal is low, then jump to step 4-3-2-2-8;

[0103] Step 4-3-2-2-6: If the signal saved in the I storage cell input box is low, and the signal with an intersection-to-parallel ratio greater than the threshold is low, then jump to step 4-3-2-2-7;

[0104] Step 4-3-2-2-7: Generate a control signal to clear the data in the S memory cell of the current NMS cell, and jump to step 4-3-2-2-9;

[0105] Step 4-3-2-2-8: Generate a control signal to write the data in the C memory cell of the current NMS cell into the S memory cell of the current NMS cell, set the cell valid flag to high level, and jump to step 4-3-2-2-9.

[0106] Step 4-3-2-2-9: Generation of S-unit read / write control signals ends.

[0107] Step 4-3-2-3: C cell read / write control signal, controls the read / write of the C memory cell of the current NMS cell, the specific steps are as follows:

[0108] Step 4-3-2-3-1: If the signal saved in the input box of storage cell I is high, the valid signal of storage cell S is high, and the signal with cross-parallel ratio greater than the threshold is high, then jump to step 4-3-2-3-11;

[0109] Step 4-3-2-3-2: If the signal saved in the input box of storage cell I is high, the valid signal of storage cell S is high, and the signal with cross-parallel ratio greater than the threshold is low, then jump to step 4-3-2-3-10;

[0110] Step 4-3-2-3-3: If the signal saved in the I memory cell input box is high and the valid signal in the S memory cell is low, then jump to step 4-3-2-3-9;

[0111] Step 4-3-2-3-4: If the signal saved in the I storage cell input box is low, the signal that the cross-parallel ratio is greater than the threshold is high, and the confidence comparison signal is high, then jump to step 4-3-2-3-8;

[0112] Step 4-3-2-3-5: If the signal saved in the I storage cell input box is low, the signal that the cross-parallel ratio is greater than the threshold is high, and the confidence comparison signal is low, then jump to step 4-3-2-3-10;

[0113] Step 4-3-2-3-6: If the signal saved in the I storage cell input box is low, the signal that the cross-parallel ratio is greater than the threshold is low, and the confidence comparison signal is high, then jump to step 4-3-2-3-8;

[0114] Step 4-3-2-3-7: If the signal saved in the I storage cell input box is low, the signal that the cross-parallel ratio is greater than the threshold is low, and the confidence comparison signal is low, then jump to step 4-3-2-3-10;

[0115] Step 4-3-2-3-8: Generate control signal to write the data of the I memory cell in the predecessor NMS cell to the C memory cell of the current NMS cell, set the cell valid flag to high level, and jump to step 4-3-2-3-12;

[0116] Step 4-3-2-3-9: Generate control signal to write the data of the S memory cell in the predecessor NMS cell into the C memory cell of the current NMS cell, set the cell valid flag to high level, and jump to step 4-3-2-3-12;

[0117] Step 4-3-2-3-10: Generate control signals, without any read or write operations, and jump to step 4-3-2-3-12;

[0118] Step 4-3-2-3-11: Set the cell valid flag to low level, and jump to step 4-3-2-3-12;

[0119] Step 4-3-2-3-12: The generation of read / write control signals for Unit C is complete.

[0120] To verify the delay and area of ​​the NMS high-efficiency digital circuit of this invention, the evaluation experiment used a 40nm process library to perform logic synthesis on the circuit, and the performance indicators are shown in Table 1 below.

[0121] Table 1. Performance Indicators of NMS High-Efficiency Digital Circuits

[0122] Frequency (MHz) Power consumption (mW) <![CDATA[Area (μm 2 )]]> Average processing latency for candidate boxes (ns) 180 8.398 76239 0.35

[0123] To verify the accuracy of the NMS high-efficiency digital circuit of this invention, an accuracy test was conducted to compare and evaluate the differences between the results obtained by the CPU and the results obtained by the digital circuit. In the experiment, the NMS algorithm's intersection-over-union (IoU) threshold was set to 0.4, the confidence threshold was set to 0.5, and the accuracy was calculated using the following formula: In the formula p d p represents the cumulative pixel difference in the candidate box coordinates. total The total number of pixels in the image is represented by , h represents the number of output boxes that can be matched, and m represents the total number of output boxes when the CPU executes. The results are shown in Table 2. In the table, X1 represents the horizontal coordinate of the top left vertex of the output box, X2 represents the horizontal coordinate of the top right vertex of the output box, Y1 represents the vertical coordinate of the top left vertex of the output box, and Y2 represents the vertical coordinate of the top right vertex of the output box.

[0124] Table 2. Impact of high-efficiency digital circuits on the accuracy of nonmaximum suppression algorithms

[0125] Output box information X1 X2 Y1 Y2 accuracy 98.8％ 98.8％ 98.7％ 98.7％

[0126] It can be seen that the results obtained by the high-efficiency digital circuit are basically consistent with the results obtained by the CPU.

[0127] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A low-latency non-maximum suppression efficient digital circuit, characterized by, The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method.

2. The non-maximum suppression high-efficiency digital circuit according to claim 1, characterized in that, The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method.

3. The non-maximum suppression high-efficiency digital circuit according to claim 1, wherein, The application relates to a neural network processing system and method.

4. The non-maximum suppression high-efficiency digital circuit according to claim 1, wherein, The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing system and method. The application relates to a neural network processing The multiplexer selects one of the data output channels of the plurality of preprocessing modules according to a read request signal sent by the arbitration module to the FIFO in the preprocessing module, and accesses the FIFO in the preprocessing result buffer module; The FIFO module is responsible for caching the candidate frame information after preprocessing and outputting it to the subsequent NMS unit.

5. The non-maximum suppression high-efficiency digital circuit according to claim 1, wherein, The candidate frame storage module comprises an I storage unit, an S storage unit and a C storage unit. The I storage unit caches a candidate frame information output to the subsequent NMS unit, the cached candidate frame is the input candidate frame of the subsequent NMS unit, and the cached information comprises the coordinates and confidence of the candidate frame and a candidate frame saved flag. The S storage unit is a redundant storage unit and can cache the coordinates and confidence of a candidate frame, which assists the candidate frame storage update control module in sorting the candidate frames in the C storage units of the plurality of NMS units according to the confidence. The C storage unit caches the coordinates, confidence and unit valid flag of an output frame, and the saved result is the temporary result of the output frame when all the candidate frames pass through all the NMS units.

6. The non-maximum suppression high-efficiency digital circuit of claim 1, wherein, The state signals comprise an S storage unit valid signal, an I storage unit input frame saved signal, an intersection-over-union greater than threshold signal and a confidence comparison signal. The S storage unit valid signal is high when the confidence of the candidate frame saved in the S storage unit of the current NMS unit is greater than 1, and is low otherwise. The I storage unit input frame saved signal is high when the candidate frame saved flag in the I storage unit of the current NMS unit is high, and is low otherwise. The intersection-over-union greater than threshold signal is high when the intersection-over-union output by the intersection-over-union calculation module is greater than the intersection-over-union threshold, and is low otherwise. The confidence comparison signal is high when the confidence of the candidate frame saved in the I storage unit of the previous NMS unit is greater than the confidence of the candidate frame saved in the C storage unit of the current NMS unit, and is low otherwise.

7. The non-maximum suppression high-efficiency digital circuit according to claim 1, wherein, The read-write control signals of the candidate frame storage module comprise: The I unit read-write control signal is used for controlling the reading and writing of the I storage unit of the current NMS unit. The S unit read-write control signal is used for controlling the reading and writing of the S storage unit of the current NMS unit. The C unit read-write control signal is used for controlling the reading and writing of the C storage unit of the current NMS unit.

8. The non-maximum suppression high-efficiency digital circuit according to claim 7, wherein, The I unit read-write control signal is generated in the following steps: Step 8-1: when the I storage unit input frame saved signal is low, the intersection-over-union greater than threshold signal is high and the confidence comparison signal is low, a control signal is generated to clear the data in the I storage unit, and otherwise, the control signal is generated to write the data in the I storage unit of the previous NMS unit into the I storage unit of the current NMS unit.

9. The non-maximum suppression high-efficiency digital circuit according to claim 7, wherein, The S unit read-write control signal is generated in the following steps: Step 9-1: if the I storage unit input frame saved signal is high and the S storage unit valid signal is high, go to step 9-7. Step 9-2: If the I storage unit input frame has saved signal is high, S storage unit valid signal is low, and the ratio of intersection to union is greater than the threshold signal is high, then jump to step 9-7; Step 9-3: If the I storage unit input frame has saved signal is high, S storage unit valid signal is low, and the ratio of intersection to union is greater than the threshold signal is low, then jump to step 9-8; Step 9-4: If the I storage unit input frame has saved signal is low, the ratio of intersection to union is greater than the threshold signal is high, and the confidence comparison signal is high, then jump to step 9-7; Step 9-5: If the I storage unit input frame has saved signal is low, the ratio of intersection to union is greater than the threshold signal is high, and the confidence comparison signal is low, then jump to step 9-8; Step 9-6: If the I storage unit input frame has saved signal is low, the ratio of intersection to union is greater than the threshold signal is low, then jump to step 9-7; Step 9-7: Generate control signal to clear the data in the S storage unit in the current NMS unit, and jump to step 9-9; Step 9-8: Generate control signal to write the data in the C storage unit in the current NMS unit into the S storage unit in the current NMS unit, set the unit valid flag bit to high, and jump to step 9-9; Step 9-9: The generation of S unit read-write control signal is completed.

10. The non-maximum suppression high-efficiency digital circuit of claim 7, wherein, The steps of generating the C unit read-write control signal are as follows: Step 10-1: If the I storage unit input frame has saved signal is high, S storage unit valid signal is high, and the ratio of intersection to union is greater than the threshold signal is high, then jump to step 10-11; Step 10-2: If the I storage unit input frame has saved signal is high, S storage unit valid signal is high, and the ratio of intersection to union is greater than the threshold signal is low, then jump to step 10-10; Step 10-3: If the I storage unit input frame has saved signal is high, S storage unit valid signal is low, then jump to step 10-9; Step 10-4: If the I storage unit input frame has saved signal is low, the ratio of intersection to union is greater than the threshold signal is high, and the confidence comparison signal is high, then jump to step 10-8; Step 10-5: If the I storage unit input frame has saved signal is low, the ratio of intersection to union is greater than the threshold signal is high, and the confidence comparison signal is low, then jump to step 10-10; Step 10-6: If the I storage unit input frame has saved signal is low, the ratio of intersection to union is greater than the threshold signal is low, and the confidence comparison signal is high, then jump to step 10-8; Step 10-7: If the I storage unit input frame has saved signal is low, the ratio of intersection to union is greater than the threshold signal is low, and the confidence comparison signal is low, then jump to step 10-10; Step 10-8: Generate control signal to write the data in the I storage unit in the previous NMS unit into the C storage unit in the current NMS unit, set the unit valid flag bit to high, and jump to step 10-12; Step 10-9: Generate control signal to clear the data in the C storage unit in the current NMS unit, and jump to step 10-12; Step 10-10: Generate control signal to write the data in the I storage unit in the previous NMS unit into the C storage unit in the current NMS unit, set the unit valid flag bit to high, and jump to step 10-12; Step 10-11: Generate control signal to write the data in the S storage unit in the previous NMS unit into the C storage unit in the current NMS unit, set the unit valid flag bit to high, and jump to step 10-12; Step 10-12: The generation of C unit read-write control signal is completed. Step 10-9: generate control signal to write the data of S storage unit in the previous NMS unit into C storage unit in the current NMS unit, set the valid flag of the unit to high level, and jump to step 10-12; Step 10-10: generate control signal, and no read / write operation occurs, and jump to step 10-12; Step 10-11: set the valid flag of the unit to low level, and jump to step 10-12; Step 10-12: the generation of C unit read / write control signal is completed.

Images