A post-processing acceleration method, device and equipment for quantized neural network reasoning

Through the coordinated work of DSP and neural network parallel acceleration units, data filtering, inverse quantization and non-maximum suppression algorithms are used to optimize the post-processing process of neural networks, solving the problem of neglected post-processing performance and achieving efficient inference Efficiency and low power consumption improve the overall performance of neural networks.

CN118780374BActive Publication Date: 2024-11-26HUNAN GREAT WALL GALAXY TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411255126.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-09
Publication Date
2024-11-26
Estimated Expiration
2044-09-09

AI Technical Summary

Technical Problem

In the prior art, there is a lack of effective methods for post-processing acceleration of neural networks, which leads to the focus of optimization of inference performance on inference chips, while post-processing performance is ignored, resulting in high cost, large energy consumption and limited scope of application.

Method used

A post-processing acceleration method for quantizing neural network inference is adopted, and the digital signal processing unit (DSP) and the neural network parallel acceleration unit work together, and through data filtering, inverse quantization, sigmoid function conversion and non-maximum suppression algorithm, it is optimized. Processing process, improve inference efficiency and reduce power consumption.

Benefits of technology

It significantly improves the efficiency of neural network inference post-processing, shortens computing time, reduces system power consumption, enhances real-time processing capabilities, and provides efficient solutions for artificial intelligence applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118780374B_ABST
    Figure CN118780374B_ABST
Patent Text Reader

Abstract

The present invention relates to a post-processing acceleration method, device and equipment for quantized neural network reasoning. The method comprises: obtaining the data reasoning result of a neural network parallel acceleration unit through a digital signal processing unit DSP, filtering the data reasoning result to obtain a target data set. Dequantizing each data in the target data set to obtain a quantized target data set. Probability conversion of the quantized target data set is performed using a sigmoid function to obtain a target data probability matrix. Non-maximum suppression is used to calculate the target candidate box area of ​​the target data probability matrix, and the target reasoning data is output. The use of this method can improve the post-processing calculation efficiency of the neural network reasoning result, shorten the calculation time and reduce power consumption.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a post-processing acceleration method, device and equipment for quantized neural network reasoning. Background Art

[0002] A neural network is an algorithmic model inspired by the structure of the human brain. It processes information by imitating the way neurons in the human brain are connected and interact. This model is the basis of deep learning in the field of machine learning and is widely used in various tasks, including image recognition, speech recognition, natural language processing, predictive analysis, etc.

[0003] Quantization is a way to accelerate neural network inference, which involves converting the weights and activation values ​​in the neural network model from floating point numbers to integers or lower bit width values. This conversion can significantly reduce the computational complexity and memory usage of the model, thereby accelerating the inference process.

[0004] In real-world application scenarios, the business process of neural networks covers several key links: the first is data preprocessing, which converts the raw data collected from sensors into an input format acceptable to the neural network to ensure the quality and applicability of the data; the second is neural network reasoning, which uses a well-trained neural network model to analyze and predict the input data; the last is data post-processing, which further processes the output results of the neural network to make it conform to the specific format of business needs.

[0005] Target detection is the most widely used application scenario of neural networks. The output of neural networks is usually several matrices with specific meanings. Its post-processing mainly includes threshold screening, non-maximum suppression, category probability application, bounding box adjustment and output formatting. Generally, in application scenarios, neural network acceleration or pre-processing acceleration is mainly selected. In the actual application scenario of neural networks, data needs to be obtained from sensors, and a general computing unit is required for data processing. The processed data is sent to the neural network acceleration unit for inference, and the inferenced data is returned to the general computing unit for calculation. The final result is what the business needs. Because of the wide variety of post-processing acceleration, different network post-processing varies greatly, and the major manufacturers on the market do it separately. Those who make inference chips only care about inference performance, not post-processing performance. At present, hardware optimization requires specific hardware, the applicable chip range is limited, and the cost is expensive, and the computing energy consumption is very high. Summary of the invention

[0006] Based on this, it is necessary to provide a post-processing acceleration method, device and equipment for quantized neural network reasoning that can improve the efficiency of neural network reasoning post-processing in order to solve the above technical problems.

[0007] A post-processing acceleration method for quantized neural network reasoning is applied to an edge reasoning chip, wherein the edge reasoning chip includes: a neural network parallel acceleration unit and a digital signal processing unit DSP, wherein the neural network parallel acceleration unit is composed of operator acceleration hardware and general computing components. The method includes:

[0008] The data inference results of the neural network parallel acceleration unit are obtained through the digital signal processing unit DSP, and the data inference results are filtered to obtain the target data set.

[0009] Dequantize each data in the target data set to obtain a quantized target data set.

[0010] The sigmoid function is used to perform probability transformation on the quantized target data set to obtain the target data probability matrix.

[0011] The target candidate box area of ​​the target data probability matrix is ​​calculated using non-maximum suppression, and the target inference data is output.

[0012] In one embodiment, the method further includes: obtaining data inference results from a DDR module of a neural network parallel acceleration unit through a pointer of a digital signal processing unit DSP, dividing the data inference results into square arrays, and disassembling the data blocks of each square array to generate a prediction probability result sequence for each candidate area, and taking the prediction result corresponding to the maximum probability in the prediction probability result sequence to form a target data set.

[0013] In one embodiment, the method further includes: performing inverse quantization on each data in the target data set to obtain a quantized target data set:

[0014] ;

[0015] in, To quantize the target dataset, For the target data, is the offset value of 0 after the coordinate conversion, is the scaling factor.

[0016] In one embodiment, the method further includes: performing probability conversion on the quantized target data set using a sigmoid function to obtain a target data probability matrix:

[0017] ;

[0018] in, is the sigmoid function of the target data, For the target data, is the natural base of the objective function of Power.

[0019] In one of the embodiments, it also includes: using non-maximum suppression to screen the target candidate frames that meet the preset conditions in the target data probability matrix, judging the size of the intersection and union ratio of any two target candidate frames and the threshold, if the intersection and union ratio is not less than the threshold, the area of ​​the two currently selected target candidate frames is the same object, merging the two target candidate frames to obtain the target candidate area, and outputting the target reasoning data. Otherwise, the area of ​​the two currently selected target candidate frames is different objects, retaining one of the two currently selected target candidate frames, matching the next target candidate frame to perform the intersection and union ratio calculation, until all the target candidate frames in the target data probability matrix have completed the intersection and union ratio calculation, obtaining the target candidate area, and outputting the target reasoning data.

[0020] In one embodiment, the neural network parallel acceleration unit is composed of operator acceleration hardware and general computing components.

[0021] A post-processing acceleration device for quantized neural network reasoning, the device comprising:

[0022] A DSP filtering module, used to obtain the data inference result of the neural network parallel acceleration unit, and filter the data inference result to obtain a target data set;

[0023] A DSP dequantization module is used to dequantize each data in the target data set to obtain a quantized target data set;

[0024] A DSP Sigmoid processing module is used to perform probability conversion on the quantized target data set using a sigmoid function to obtain a target data probability matrix;

[0025] The non-maximum suppression processing module is used to calculate the target candidate box area of ​​the target data probability matrix using non-maximum suppression and output target inference data.

[0026] In one embodiment, the non-maximum suppression processing module is also used to use non-maximum suppression to screen the target candidate frames that meet the preset conditions in the target data probability matrix, and to determine the size of the intersection and union ratio of any two target candidate frames and the threshold. If the intersection and union ratio is not less than the threshold, the area of ​​the two currently selected target candidate frames is the same object, and the two target candidate frames are merged to obtain the target candidate area, and the target reasoning data is output. Otherwise, the area of ​​the two currently selected target candidate frames is different objects, and one of the two currently selected target candidate frames is retained, and the next target candidate frame is matched to perform the intersection and union ratio calculation until all target candidate frames in the target data probability matrix have completed the intersection and union ratio calculation, and the target candidate area is obtained, and the target reasoning data is output.

[0027] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0028] The data inference results of the neural network parallel acceleration unit are obtained through the digital signal processing unit DSP, and the data inference results are filtered to obtain the target data set.

[0029] Dequantize each data in the target data set to obtain a quantized target data set.

[0030] The sigmoid function is used to perform probability transformation on the quantized target data set to obtain the target data probability matrix.

[0031] The target candidate box area of ​​the target data probability matrix is ​​calculated using non-maximum suppression, and the target inference data is output.

[0032] The above-mentioned post-processing acceleration method, device and equipment for quantized neural network reasoning, first, the digital signal processing unit (DSP) is good at processing complex mathematical operations and large-scale data processing. The data reasoning results of the neural network parallel acceleration unit are obtained through the DSP, which can effectively share the burden of the central processing unit (CPU). The neural network parallel acceleration unit can process multiple data streams at the same time, improve the parallelism of data processing, reduce waiting time, and improve the overall computing efficiency. After obtaining the inference results, data filtering can eliminate unnecessary or invalid data, further reducing the amount of data to be processed. This step is particularly important for large-scale data sets because it can significantly reduce the amount of calculation and memory requirements for subsequent processing, thereby improving the overall processing speed and efficiency. Next, each data in the target data set is dequantized. This step restores the original range of the data, making the subsequent probability conversion more accurate. The dequantization process is relatively simple and efficient, suitable for implementation in DSP, and further improves the efficiency of data processing. Subsequently, the sigmoid function is used to perform probability conversion on the dequantized data to generate a probability matrix of the target data. The sigmoid function is a commonly used activation function with high computational efficiency and good numerical stability, and is suitable for fast calculation in DSP. This process maps the data to a range of 0 to 1, which is convenient for subsequent probability analysis and target detection. Finally, the target candidate box area in the target data probability matrix is ​​calculated by the non-maximum suppression (NMS) algorithm, which can effectively remove redundant candidate boxes and retain the optimal detection results. Due to the low computational complexity of the NMS algorithm, it can be efficiently implemented on the DSP, thereby greatly improving the computational efficiency while ensuring the detection accuracy. Combining the above steps, through the collaborative work of the DSP and the neural network parallel acceleration unit, the hardware level is optimized from data acquisition, processing to target detection. This optimization not only improves the efficiency of neural network inference post-processing, but also significantly reduces the computing time and power consumption of the system. The significant reduction in computing time and the reduction in system power consumption. This not only enhances the real-time processing capability of the system, but also provides a more efficient solution for a wide range of artificial intelligence applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of an edge inference chip framework in an embodiment;

[0034] Figure 2 A schematic diagram of a flow chart of a post-processing acceleration method for quantized neural network reasoning in one embodiment;

[0035] Figure 3 A schematic diagram of a prediction result data structure in one embodiment;

[0036] Figure 4A block diagram of a traditional quantized neural network reasoning acceleration processing module in one embodiment;

[0037] Figure 5 A structural block diagram of a post-processing acceleration device for quantized neural network reasoning in one embodiment;

[0038] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0040] The present invention provides a post-processing acceleration method for quantized neural network reasoning, which can be applied to Figure 1 In the edge inference chip shown in FIG. The edge inference chip includes: a neural network parallel acceleration unit and a digital signal processing unit DSP, wherein the neural network parallel acceleration unit is composed of operator acceleration hardware and general computing components.

[0041] In one embodiment, Figure 2 As shown in the figure, a post-processing acceleration method for quantized neural network inference is provided. Figure 1 The edge inference chip in the example is used to illustrate, including the following steps:

[0042] Step 202, obtaining the data inference result of the neural network parallel acceleration unit through the digital signal processing unit DSP, filtering the data inference result to obtain the target data set.

[0043] Specifically, the DSP obtains the result before sigmoid from the neural network parallel acceleration unit through pointers instead of data copying, and filters the result based on whether the target number exists.

[0044] Step 204 , dequantize each data in the target data set to obtain a quantized target data set.

[0045] Step 206: Use the sigmoid function to perform probability conversion on the quantized target data set to obtain a target data probability matrix.

[0046] Specifically, DSP performs sigmoid calculation on the inverse quantization result. Taking a 13*13*255 matrix as an example, the meaning of this matrix is: 13*13*255 can actually be decomposed into 13*13*3*85. That is, the image is divided into blocks of 13 rows and 13 columns, and each block predicts 3 results, each result is 85 numbers, the first one represents whether the result has a target, and the second to fifth are descriptions of the target area, such as Figure 3 As shown, the x and y coordinates of the fixed point in the upper left corner represent the length and width respectively. The last eighty numbers are the probability values ​​of each of the eighty candidates, and the one with the largest probability is output as the prediction result.

[0047] Step 208, non-maximum suppression is used to calculate the target candidate box area of ​​the target data probability matrix, and the target inference data is output.

[0048] Specifically, we filter out candidate boxes with targets and calculate the area of ​​the candidate boxes. We calculate the intersection-over-union ratio (the area ratio of the overlapping parts) of the candidate boxes one by one to determine whether they are repeated predictions of the same object. If they are the same object and the intersection-over-union ratio exceeds the threshold, we merge them to standardize the data output.

[0049] In the above-mentioned post-processing acceleration method of quantized neural network inference, firstly, the digital signal processing unit (DSP) is good at processing complex mathematical operations and large-scale data processing. By obtaining the data inference results of the neural network parallel acceleration unit through the DSP, the burden of the central processing unit (CPU) can be effectively shared. The neural network parallel acceleration unit can process multiple data streams at the same time, improve the parallelism of data processing, reduce waiting time, and improve the overall computing efficiency. After obtaining the inference results, data filtering can eliminate unnecessary or invalid data, further reducing the amount of data to be processed. This step is particularly important for large-scale data sets because it can significantly reduce the computational workload and memory requirements of subsequent processing, thereby improving the overall processing speed and efficiency. Next, each data in the target data set is dequantized. This step restores the original range of the data, making the subsequent probability conversion more accurate. The dequantization process is relatively simple and efficient, suitable for implementation in DSP, and further improves the efficiency of data processing. Subsequently, the sigmoid function is used to perform probability conversion on the dequantized data to generate a probability matrix of the target data. The sigmoid function is a commonly used activation function with high computational efficiency and good numerical stability, which is suitable for fast calculation in DSP. This process maps the data to a range of 0 to 1, which is convenient for subsequent probability analysis and target detection. Finally, the target candidate box area in the target data probability matrix is ​​calculated by the non-maximum suppression (NMS) algorithm, which can effectively remove redundant candidate boxes and retain the optimal detection results. Due to the low computational complexity of the NMS algorithm, it can be efficiently implemented on the DSP, thereby greatly improving the computational efficiency while ensuring the detection accuracy. Combining the above steps, through the collaborative work of the DSP and the neural network parallel acceleration unit, the hardware level is optimized from data acquisition, processing to target detection. This optimization not only improves the efficiency of neural network inference post-processing, but also significantly reduces the computing time and power consumption of the system. The significant reduction in computing time and the reduction in system power consumption. This not only enhances the real-time processing capability of the system, but also provides a more efficient solution for a wide range of artificial intelligence applications.

[0050] In one embodiment, the data inference result is obtained from the DDR module of the neural network parallel acceleration unit through the pointer of the digital signal processing unit DSP. The data inference result is divided into a square array, and the data block of each square array is disassembled to generate a prediction probability result sequence for each candidate area, and the prediction result corresponding to the maximum probability in the prediction probability result sequence is taken to form a target data set.

[0051] In one embodiment, each data in the target data set is dequantized to obtain a quantized target data set:

[0052] ;

[0053] in, To quantize the target dataset, For the target data, is the offset value of 0 after the coordinate conversion, is the scaling factor.

[0054] In one embodiment, a sigmoid function is used to perform probability conversion on the quantized target data set to obtain a target data probability matrix:

[0055] ;

[0056] in, is the sigmoid function of the target data, For the target data, is the natural base of the objective function of Power.

[0057] In one of the embodiments, non-maximum suppression is used to screen the target candidate frames that meet the preset conditions in the target data probability matrix, and the intersection and union ratio of any two target candidate frames and the threshold value are determined. If the intersection and union ratio is not less than the threshold value, the area of ​​the two currently selected target candidate frames is the same object, and the two target candidate frames are merged to obtain the target candidate area, and the target reasoning data is output. Otherwise, the area of ​​the two currently selected target candidate frames is different objects, and one of the two currently selected target candidate frames is retained, and the next target candidate frame is matched to perform the intersection and union ratio calculation, until all the target candidate frames in the target data probability matrix have completed the intersection and union ratio calculation, the target candidate area is obtained, and the target reasoning data is output.

[0058] In one embodiment, the neural network parallel acceleration unit is composed of operator acceleration hardware and general computing components.

[0059] In one embodiment, taking the inference model trained by YOLOv4-tiny coco dataset (currently the most widely used model, often used as a benchmark, i.e., the new model will compare the inference time and accuracy with yolov4-tiny using coco dataset) as an example, the traditional quantized neural network inference acceleration processing module is applied. Figure 4 As shown in the figure, in a normal, unoptimized, post-processing scenario, the result of neural network inference is two matrices, both of which are uint8 data types and have sizes of 13*13*255 and 26*26*255 respectively. The post-processing process is as follows:

[0060] Step 1: Each data needs to be dequantized first.

[0061] Step 2: You need to apply the sigmoid function to each element in each matrix to map the value between 0 and 1. The formula of the sigmoid function is , the sigmoid function is often used to convert the raw output (which may be unnormalized logits) into probabilities.

[0062] It is worth noting that if this step is processed in neural network reasoning, it will be calculated in the general computing component of the neural network parallel acceleration unit and then quantized into U8 data. If it is executed in the DSP, Sigmoid is directly performed without re-quantization and dequantization in step 3. For networks such as yolov5s (pytorch version) for target detection using pnna quantization, Sigmoid is placed in the network itself, and then calculated in the general computing component of the neural network parallel acceleration unit, and the quantized Sigmoid result is output. For some networks such as the darknet version of yolov-4-tiny for target detection, Sigmoid is placed in post-processing and calculated in the DSP.

[0063] Step 3: The quantized Sigmoid result is sent to the DSP to dequantize each data in the matrix.

[0064] Step 4: Filter the inverse quantized Sigmoid result. Take the 13*13*255 matrix as an example. The meaning of this matrix is: 13*13*255 can actually be decomposed into 13*13*3*85. That is, the image is divided into blocks of 13 rows and 13 columns. Each block predicts 3 results, each of which is 85 numbers. The first one represents whether the result has a target. The second to fifth are descriptions of the target area. The x and y coordinate values ​​of the fixed point in the upper left corner are length and width respectively. The last eighty numbers are the probability values ​​of each of the eighty candidates. The one with the highest probability is output as the prediction result.

[0065] Step 5: Non-maximum suppression, filter out candidate boxes with targets, and calculate the area of ​​the candidate boxes. Calculate the intersection-over-union ratio (the area ratio of the overlapping parts) of the candidate boxes one by one to determine whether they are repeated predictions of the same object. If they are the same object and the intersection-over-union ratio exceeds the threshold, merge them.

[0066] Step 6: Data standardization output.

[0067] It is worth noting that compared with the traditional sigmoid function mentioned in step 2, the general computing components of the neural network parallel acceleration unit are actually used for calculation. At the same time, DSP calculation can also be used. The performance of DSP is stronger than the general computing components of the neural network parallel acceleration unit, which can speed up the calculation without the need for quantization and dequantization again. In addition, the data copy is changed to a pointer. The copy in step 2 comes from the default process, and the default process uses the GPU for acceleration. The GPU has a separate video memory and a faster reading speed. When using the edge inference chip, all data is accessed through DDR, without copying, and can be accessed through pointers, which improves data access efficiency.

[0068] Specifically, taking yolov4-tiny as an example, the number of dequantization calculations originally required is 13*13*255+26*26*255 = 215475 dequantizations. If there are three objects after filtering without overlap, only 3*85+2 (2 refers to the values ​​of 0 and 1 before dequantization) dequantizations need to be calculated, which reduces the amount of calculation by 99% and greatly improves the calculation efficiency.

[0069] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0070] In one embodiment, Figure 5 As shown, a post-processing acceleration device for quantized neural network reasoning is provided, including: a neural network parallel acceleration unit 502, a DSP filtering module 504, a DSP inverse quantization module 506, a DSP Sigmoid processing module 508 and a non-maximum suppression processing module 510, wherein:

[0071] The DSP filtering module 504 is used to obtain the data inference result of the neural network parallel acceleration unit 502, and filter the data inference result to obtain the target data set;

[0072] The DSP dequantization module 506 is used to dequantize each data in the target data set to obtain a quantized target data set;

[0073] DSP Sigmoid processing module 508, used to perform probability conversion on the quantized target data set using a sigmoid function to obtain a target data probability matrix;

[0074] The non-maximum suppression processing module 510 is used to calculate the target candidate box area of ​​the target data probability matrix using non-maximum suppression and output target inference data.

[0075] In one embodiment, the non-maximum suppression processing module (NMS) 510 is also used to use non-maximum suppression to screen the target candidate frames that meet the preset conditions in the target data probability matrix, and to determine the size of the intersection and union ratio of any two target candidate frames and the threshold. If the intersection and union ratio is not less than the threshold, the area of ​​the two currently selected target candidate frames is the same object, and the two target candidate frames are merged to obtain the target candidate area, and the target reasoning data is output. Otherwise, the area of ​​the two currently selected target candidate frames is different objects, and one of the two currently selected target candidate frames is retained, and the next target candidate frame is matched to perform the intersection and union ratio calculation until all target candidate frames in the target data probability matrix have completed the intersection and union ratio calculation, and the target candidate area is obtained, and the target reasoning data is output.

[0076] For the specific definition of a post-processing acceleration device for quantized neural network reasoning, please refer to the definition of a post-processing acceleration method for quantized neural network reasoning mentioned above, which will not be repeated here. Each module in the above-mentioned post-processing acceleration device for quantized neural network reasoning can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0077] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a post-processing acceleration method for quantitative neural network reasoning is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0078] Those skilled in the art will understand that Figure 1 , Figure 4-Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0079] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0080] The data inference results of the neural network parallel acceleration unit are obtained through the digital signal processing unit DSP, and the data inference results are filtered to obtain the target data set.

[0081] Dequantize each data in the target data set to obtain a quantized target data set.

[0082] The sigmoid function is used to perform probability transformation on the quantized target data set to obtain the target data probability matrix.

[0083] The target candidate box area of ​​the target data probability matrix is ​​calculated using non-maximum suppression, and the target inference data is output.

[0084] In one embodiment, when the processor executes the computer program, the following steps are further implemented: obtaining data inference results from the DDR module of the neural network parallel acceleration unit through the pointer of the digital signal processing unit DSP. Dividing the data inference results into square arrays, and disassembling the data blocks of each square array to generate a prediction probability result sequence for each candidate area, and taking the prediction result corresponding to the maximum probability in the prediction probability result sequence to form a target data set.

[0085] In one embodiment, when the processor executes the computer program, the following steps are further implemented: dequantizing each data in the target data set to obtain a quantized target data set:

[0086] ;

[0087] in, To quantize the target dataset, For the target data, is the offset value of 0 after the coordinate conversion, is the scaling factor.

[0088] In one embodiment, when the processor executes the computer program, the following steps are further implemented: a sigmoid function is used to perform probability conversion on the quantized target data set to obtain a target data probability matrix:

[0089] ;

[0090] in, is the sigmoid function of the target data, For the target data, is the natural base of the objective function of Power.

[0091] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0092] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A post-processing acceleration method for quantized neural network inference, characterized in that: Applied to edge inference chips, the edge inference chips include: a neural network parallel acceleration unit and a digital signal processing unit DSP; the neural network parallel acceleration unit is composed of operator acceleration hardware and general computing components; The method comprises: Obtaining the data inference result of the neural network parallel acceleration unit through the digital signal processing unit DSP, and filtering the data inference result to obtain a target data set; Dequantizing each data in the target data set to obtain a quantized target data set; Using a sigmoid function to perform probability conversion on the quantized target data set to obtain a target data probability matrix; Non-maximum suppression is used to calculate the target candidate box area of ​​the target data probability matrix, and target inference data is output.

2. The method according to claim 1, characterized in that Obtaining the data inference result of the neural network parallel acceleration unit through the digital signal processing unit DSP, filtering the data inference result to obtain a target data set, including: Obtaining data inference results from the DDR module of the neural network parallel acceleration unit through the pointer of the digital signal processing unit DSP; The data inference results are divided into square arrays, and the data blocks of each square array are disassembled to generate a prediction probability result sequence for each candidate area, and the prediction result corresponding to the maximum probability in the prediction probability result sequence is taken to form the target data set.

3. The method according to claim 2, characterized in that Dequantizing each data in the target data set to obtain a quantized target data set includes: Dequantize each data in the target data set to obtain a quantized target data set: in, To quantify the target dataset, For the target data, is the offset value of 0 after the coordinate conversion, is the scaling factor.

4. The method according to claim 3, characterized in that The sigmoid function is used to perform probability conversion on the quantized target data set to obtain a target data probability matrix, including: The sigmoid function is used to perform probability conversion on the quantized target data set to obtain the target data probability matrix: in, is the sigmoid function of the target data, For the target data, is the natural base e of the objective function Power.

5. The method according to claim 4, characterized in that The target candidate box area of ​​the target data probability matrix is ​​calculated using non-maximum suppression, and the target inference data is output, including: Non-maximum suppression is used to screen the target candidate frames that meet the preset conditions in the target data probability matrix, and the size of the intersection-and-union ratio of any two target candidate frames and the threshold is judged. If the intersection-and-union ratio is not less than the threshold, the areas of the two currently selected target candidate frames are the same object, and the two target candidate frames are merged to obtain the target candidate area, and the target inference data is output; otherwise, the areas of the two currently selected target candidate frames are different objects, and one of the two currently selected target candidate frames is retained, and the next target candidate frame is matched for intersection-and-union ratio calculation until all target candidate frames in the target data probability matrix have completed the intersection-and-union ratio calculation, the target candidate area is obtained, and the target inference data is output.

6. A post-processing acceleration device for quantized neural network inference, characterized in that: The device comprises: A DSP filtering module is used to obtain data inference results from a neural network parallel acceleration unit and filter the data inference results to obtain a target data set; the neural network parallel acceleration unit is composed of operator acceleration hardware and general computing components; A DSP dequantization module is used to dequantize each data in the target data set to obtain a quantized target data set; DSPSigmoid processing module, used to use sigmoid function to perform probability conversion on the quantized target data set to obtain a target data probability matrix; The non-maximum suppression processing module is used to calculate the target candidate box area of ​​the target data probability matrix using non-maximum suppression and output target inference data.

7. The device according to claim 6, characterized in that The non-maximum suppression processing module is further used to use non-maximum suppression to screen target candidate frames that meet preset conditions in the target data probability matrix, determine the size of the intersection-and-union ratio of any two target candidate frames and a threshold, and if the intersection-and-union ratio is not less than the threshold, the areas within the two currently selected target candidate frames are the same object, merge the two target candidate frames to obtain a target candidate area, and output target inference data; Otherwise, if the areas of the two currently selected target candidate frames are different objects, one of the two currently selected target candidate frames is retained, and the next target candidate frame is matched to perform intersection-over-union calculation until all target candidate frames in the target data probability matrix have completed the intersection-over-union calculation, the target candidate area is obtained, and the target inference data is output.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.