Data processing device, data processing method, chip and electronic equipment
By executing AI acceleration unit operators at the CPU level, filtering data using pre-set confidence thresholds, and performing vectorized operations, the problem of invalid computation in post-processing operations of deep learning models for object detection is solved, thus improving computational performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING TSINGMICRO INTELLIGENT TECH CO LTD
- Filing Date
- 2026-01-04
- Publication Date
- 2026-05-19
AI Technical Summary
In the architecture that combines CPU and AI acceleration units, the post-processing operations of object detection deep learning models cannot be flexibly integrated, resulting in a large amount of invalid computation and low computational performance, especially under scalarization operations.
At the CPU level, AI acceleration unit operators are executed. Data is filtered by pre-set confidence thresholds, and vectorized dimension permutation, dequantization, and nonlinear activation calculations are performed using a single instruction multiple data instruction set, reducing redundant operations and improving computational efficiency.
By replacing scalar computation with vectorized operations, the processing performance of a single frame is significantly improved, redundant operations are reduced, and overall computational efficiency is increased.
Smart Images
Figure CN122065040A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, specifically to a data processing device, a data processing method, a chip, and an electronic device. Background Technology
[0002] In the architecture that combines a CPU (Central Processing Unit) and an AI Accelerator Unit, the CPU is responsible for general computing and system management, while the AI Accelerator Unit focuses on accelerating the execution of deep learning and AI algorithms.
[0003] Specifically, the CPU receives tasks and assigns AI (artificial intelligence) computation tasks to the AI acceleration unit as needed. For example, in an image recognition task, the CPU might handle image preprocessing while delegating the inference task of the deep learning model to the AI acceleration unit. Data is then transferred between the CPU and the AI acceleration unit via shared memory or a cache to reduce latency; the CPU transmits input data to the AI acceleration unit and retrieves the results after the AI acceleration unit completes its computation. Finally, after completing its computation, the AI acceleration unit returns the results to the CPU for further processing (such as displaying the results or making decisions).
[0004] This section is intended to provide background or context for the embodiments of this application set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section. Summary of the Invention
[0005] In order to solve at least one of the above-mentioned problems in the prior art, embodiments of this application provide a data processing apparatus, a data processing method, a chip, and an electronic device.
[0006] This application provides a data processing apparatus, including: an AI acceleration unit configured to perform an inference task of a target detection neural network model and generate inference results; and a processor coupled to the AI acceleration unit, the processor being configured to: filter data in the inference results based on a preset confidence threshold to obtain filtered data; and perform dimension permutation, dequantization, and nonlinear activation calculation operations on the filtered data in a vectorized manner using a single instruction multiple data instruction set.
[0007] In some embodiments, the processor filters the data in the inference result based on a preset confidence threshold to obtain filtered data, including: calculating the corresponding original score threshold by inverse nonlinear activation according to the preset confidence threshold; comparing the data in the inference result with the original score threshold to filter out data below the original score threshold, thereby obtaining filtered data.
[0008] In some embodiments, the inference result output by the AI acceleration unit is quantized data.
[0009] In some embodiments, the processor uses a single instruction multiple data instruction set to perform dimension permutation, dequantization, and nonlinear activation calculation operations on the filtered data in a vectorized manner, including: loading the filtered data in a vectorized manner, and expanding and converting the integer-type filtered data into floating-point type data; performing a vectorized dimension permutation operation on the floating-point type data; performing a vector multiplication operation on the dimension-permuted floating-point type data and the dequantization coefficients to obtain dequantized floating-point data; and performing a vectorized nonlinear activation calculation on the dequantized floating-point data.
[0010] In some embodiments, the nonlinear activation calculation is a Sigmoid function calculation.
[0011] In some embodiments, the apparatus further includes a shared storage module coupled to the AI acceleration unit and the processor, wherein the inference results output by the AI acceleration unit are stored in the shared storage module, and the processor reads the inference results from the shared storage module.
[0012] In some embodiments, the AI acceleration unit includes a neural network processing unit.
[0013] In some embodiments, the processor is further configured to: perform nonmaximum suppression operation on the obtained data after completing the dimension permutation, inverse quantization, and nonlinear activation calculation; and generate and output target detection information based on the result of the nonmaximum suppression operation.
[0014] This application also provides a data processing device applied to a processor. The method includes: filtering data in the inference results output by an AI acceleration unit based on a preset confidence threshold to obtain filtered data; and using a single instruction multiple data instruction set to perform dimension permutation, dequantization, and nonlinear activation calculation operations on the filtered data in a vectorized manner.
[0015] In some embodiments, the step of filtering the data in the inference result based on a preset confidence threshold to obtain filtered data includes: calculating the corresponding original score threshold by inverse nonlinear activation according to the preset confidence threshold; comparing the data in the inference result with the original score threshold to filter out data below the original score threshold, thereby obtaining filtered data.
[0016] In some embodiments, the step of using a single instruction multiple data instruction set to perform dimension permutation, dequantization, and nonlinear activation calculation operations on the filtered data in a vectorized manner includes: loading the filtered data in a vectorized manner, and expanding and converting the integer-type filtered data into floating-point type data; performing a vectorized dimension permutation operation on the floating-point type data; performing a vector multiplication operation on the dimension-permuted floating-point type data and the dequantization coefficients to obtain dequantized floating-point data; and performing a vectorized nonlinear activation calculation on the dequantized floating-point data.
[0017] In some embodiments, the method further includes: after completing the dimension permutation, inverse quantization, and nonlinear activation calculation, performing a nonmaximum suppression operation on the obtained data; and generating and outputting target detection information based on the result of the nonmaximum suppression operation.
[0018] This application also provides a chip, which includes the data processing device described in any of the above embodiments.
[0019] This application also provides an electronic device, which includes the chip described in any of the above embodiments.
[0020] The data processing apparatus, data processing method, chip, and electronic device proposed in this application break down the concept of layers (post-processing layer, NMS layer). First, graph optimization is performed, placing the AI acceleration unit operators—dimensional permutation and nonlinear activation calculation—at the processor level. Then, AI acceleration unit inference is performed to obtain the output result. In the post-processing process, the data in the inference result is first filtered based on a preset confidence threshold—which avoids all data from undergoing subsequent operations, thereby greatly reducing redundant operations. Then, the remaining data is processed using single instruction multiple data instructions in a vectorized manner to perform dimensional permutation, dequantization, and nonlinear activation calculation operations. In this way, using single instruction multiple data instructions and vector calculation to replace scalar calculation allows for batch processing of data, significantly improving the single-frame processing performance. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:
[0022] Figure 1 This is a schematic diagram of the current architecture that combines CPU and AI acceleration unit.
[0023] Figure 2This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application.
[0024] Figure 3 This is a schematic diagram of the structure of a data processing device provided in another embodiment of this application.
[0025] Figure 4 This is a flowchart illustrating a data processing method provided in an embodiment of this application.
[0026] Figure 5 This is a partial flowchart of a data processing method provided in an embodiment of this application.
[0027] Figure 6 This is a partial flowchart of a data processing method provided in an embodiment of this application.
[0028] Figure 7 This is a partial flowchart of a data processing method provided in an embodiment of this application.
[0029] Figure 8 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0030] Reference will now be made in detail to exemplary embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same component reference numerals are used in the drawings and description to denote the same or similar parts.
[0031] The term "coupled (or connected)" as used throughout this specification (including the claims) may refer to any direct or indirect means of connection. For example, if the text describes a first device coupled (or connected) to a second device, it should be interpreted as the first device being directly connected to the second device, or the first device being indirectly connected to the second device through other devices or some means of connection. The terms "first," "second," etc., used throughout this specification (including the claims) are used to name components and are not intended to limit the upper or lower limit of the number of components, nor to limit the order of components. Furthermore, wherever possible, components / components / steps using the same reference numerals in the drawings and embodiments represent the same or similar parts. Components / components / steps using the same reference numerals or the same terms in different embodiments may be referred to mutually in the relevant descriptions.
[0032] To better understand this application, the research background of this application will be explained in detail below.
[0033] Figure 1 This is a schematic diagram illustrating the workflow of the current architecture combining CPU and AI acceleration units. For example... Figure 1As shown, after inference in deep learning models for object detection such as YOLO series and FAST-RCNN series, NMS (Non-maximum Suppression) is usually required to remove overlapping bounding boxes or filter redundant detection boxes. However, the shape of the deep learning model after processing by the AI acceleration unit is usually inconsistent with that required by NMS, requiring operations such as reshape and permute. These operations are not very efficient in data transfer on the AI acceleration unit, increasing the burden on the AI acceleration unit. Furthermore, the output of the AI acceleration unit is directly fed to the subsequent post-processing layer and NMS layer, and its performance on the CPU is also low due to conventional algorithm processing. The combination of these factors results in low overall single-frame performance.
[0034] In other words, the post-processing operations of current deep learning models for object detection cannot break the concept between algorithm layers and cannot be flexibly integrated, resulting in a large amount of invalid computation. At the same time, most of the computation uses scalarization operations, which have low performance.
[0035] To address the aforementioned technical problems, embodiments of this application provide a data processing apparatus, a data processing method, a chip, and an electronic device, which can solve at least one of the aforementioned problems existing in the prior art.
[0036] Figure 2 This application provides a schematic diagram of the structure of a data processing device, as shown below. Figure 2 As shown, this application provides a data processing apparatus 100, including:
[0037] AI acceleration unit 1 is configured to perform the inference task of the object detection neural network model and generate inference results;
[0038] Processor 2, coupled to AI acceleration unit 1, is configured to: filter the data in the inference results based on a preset confidence threshold to obtain filtered data; and perform dimension permutation, dequantization, and nonlinear activation calculation operations on the filtered data in a vectorized manner using a single instruction multiple data instruction set.
[0039] Specifically, in the combined architecture of CPU and AI acceleration unit (AI acceleration unit), after the AI acceleration unit completes the model inference task, post-processing algorithms are generally performed first, and then the results are input into algorithms such as NMS for bounding box recognition. Usually, the post-processing algorithm layer and NMS are executed on the CPU (processor) platform. This application embodiment breaks the concept between layers (post-processing layer, NMS layer). First, graph optimization is performed, and the AI acceleration unit operators—dimensional permutation and nonlinear activation calculation—are executed at the CPU level. Then, AI acceleration unit inference is performed to obtain the output result. In the post-processing process, the data in the inference result is first filtered based on a preset confidence threshold—which can avoid all data from being processed in subsequent operations, thereby greatly reducing redundant operations. Then, the remaining data is processed by single instruction multiple data instructions in a vectorized manner to perform dimensional permutation, dequantization, and nonlinear activation calculation operations. In this way, using single instruction multiple data instructions, vector calculation replaces scalar calculation, batch processing of data, and a significant improvement in single-frame processing performance.
[0040] The target detection neural network models include, but are not limited to, YOLO series, FAST-R-CNN series, or models that require NMS post-processing algorithms.
[0041] The data processing device 100 provided in this application embodiment can be used to efficiently perform artificial intelligence (AI) and machine learning (ML) tasks, such as common neural network tasks like the YOLO series and Fast-R-CNN series, and is particularly suitable for applications in mobile devices, edge computing, and embedded systems.
[0042] In some embodiments, the processor 2 filters the data in the inference result based on a preset confidence threshold to obtain filtered data, including: calculating the corresponding original score threshold by inverse nonlinear activation according to the preset confidence threshold; comparing the data in the inference result with the original score threshold to filter out data below the original score threshold, thereby obtaining filtered data.
[0043] Specifically, the preset confidence threshold can be the confidence threshold of the target detection neural network model. This confidence threshold can be a score threshold (e.g., 0.5 or 0.7) pre-set by the user or the system to determine whether the detection result is valid. The data in the inference result refers to the raw output data generated by the AI acceleration unit after performing model inference, without post-processing. It is usually in quantized integer format and can contain a large amount of raw scores, coordinates, and other information of candidate boxes.
[0044] The inverse nonlinear activation calculation refers to performing a reverse calculation on the preset confidence threshold to obtain its corresponding value before the activation function calculation. Since the raw inference results (scores) directly output from the AI acceleration unit are usually quantized data that is not activated (or in a state before the activation function), they cannot be directly compared with the activated confidence threshold. Through inverse nonlinear activation calculation, the confidence threshold is converted to the same dimension and numerical space as the original data, allowing direct comparison with the raw score data output by the AI acceleration unit. This completes the filtering in the early data processing stage, significantly reducing redundant operations: before vectorization processing (dimensionality replacement, inverse quantization, nonlinear activation calculation), most low-quality data is eliminated in advance, so that subsequent vectorization processing only needs to be performed on a small amount of data, thereby greatly improving the overall post-processing efficiency.
[0045] In some embodiments, the processor uses a single instruction multiple data instruction set to perform dimension permutation, dequantization, and nonlinear activation calculation operations on the filtered data in a vectorized manner, including: loading the filtered data in a vectorized manner, and expanding and converting the integer-type filtered data into floating-point type data; performing a vectorized dimension permutation operation on the floating-point type data; performing a vector multiplication operation on the dimension-permuted floating-point type data and the dequantization coefficients to obtain dequantized floating-point data; and performing a vectorized nonlinear activation calculation on the dequantized floating-point data.
[0046] Specifically, loading the filtered data in a vectorized manner refers to using a single instruction multiple data instruction to read multiple consecutive data elements from memory into a vector register at once. This merges multiple independent memory access requests into one, greatly reducing memory access latency and instruction overhead.
[0047] The step of expanding the filtered integer data and converting it to floating-point data includes: widening the short-width integer in the vector register to a higher-width integer, and converting the expanded integer value into a floating-point format.
[0048] The vectorized dimension permute operation on the floating-point data refers to the parallel transformation of the logical dimension structure of multiple floating-point data elements within a vector register using vector permute and rearrangement instructions. This differs from the inefficient data movement performed on the CPU using scalar loops or on AI acceleration units. Vectorized dimension permute leverages SIMD (Single Instruction Multiple Data) hardware characteristics to complete the rearrangement within a data block (such as the transpose of a 4×4 matrix) in one operation, transforming the original memory-access-intensive operation into efficient in-register computation.
[0049] The vector multiplication operation performed on the floating-point data with dequantized coefficients after dimension permutation refers to using a vector multiplication instruction to perform element-wise parallel multiplication between a vector register containing multiple floating-point numbers and another vector register containing the same number of dequantized coefficients, thus completing the dequantization. In this way, a single instruction completes the dequantization of multiple data points, resulting in high computational density and full utilization of the processor's arithmetic logic unit. Furthermore, in this embodiment, the permute layer and the dequantization layer are integrated, reducing computational load and improving computational performance.
[0050] The aforementioned performing vectorized nonlinear activation calculation on the dequantized floating-point data refers to using vectorized arithmetic instructions to perform nonlinear activation function (such as Sigmoid) calculations in parallel on a vector register containing multiple floating-point numbers.
[0051] like Figure 3 As shown, in some embodiments, the device 100 further includes a shared storage module 3, which is coupled to the AI acceleration unit 1 and the processor 2. The inference results output by the AI acceleration unit 1 are stored in the shared storage module 3, and the processor 2 reads the inference results from the shared storage module 3. Specifically, the shared storage module 3 may be shared memory or a cache.
[0052] In some embodiments, the AI acceleration unit 1 includes a neural network processing unit (NPU).
[0053] In some embodiments, the processor 2 is further configured to: perform nonmaximum suppression operation on the obtained data after completing the dimension permutation, inverse quantization and nonlinear activation calculation; and generate and output target detection information based on the result of the nonmaximum suppression operation.
[0054] Specifically, non-maximum suppression refers to retaining only the candidate box with the highest confidence score among multiple overlapping candidate boxes of the same category, while suppressing (deleting) all other boxes whose overlap with it exceeds a certain threshold. After non-maximum suppression, scaling operations (such as restoring normalized coordinates to the original image coordinates) are performed to generate and output object detection information.
[0055] For example, in an image detection task, after completing the dimension permutation, inverse quantization, and nonlinear activation calculation, the obtained data is then converted into structured detection data containing bounding box coordinates, confidence scores, and category identifiers. The structured detection data is then subjected to nonmaximum suppression processing to remove redundant detection boxes. The bounding box coordinates after nonmaximum suppression processing are scaled to fit the original size of the input image. Based on the scaled coordinates and the corresponding category identifiers, target detection boxes are generated on the input image.
[0056] To better understand this application, the following detailed description of the operation method of the data processing apparatus provided in the embodiment of this application is provided through a specific embodiment.
[0057] In a post-processing scenario of a YOLOv5s model, after the actual inference is completed on the AI acceleration unit board, the three output shapes are [255, 80, 80], [255, 40, 40], and [255, 20, 20]. This data needs to be permute, dequantized, and processed using the sigmoid algorithm. The traditional approach to perform permute and dequantization on the CPU is scalar loop processing, resulting in a total of 255 * 8400 data entries. The following is the method flow using Neon acceleration in this embodiment:
[0058] Step 1: Initialization: Prepare input / output pointers, total number of elements, and dequantization coefficients.
[0059] Step 2: Perform reverse calculation on the confidence threshold of the YOLOv5 model, convert it into a threshold after inverse Sigmoid, and filter out score data below this threshold.
[0060] Step 3: Prefetch data: Use the __builtin_prefetch command to prefetch subsequent data blocks to improve access speed.
[0061] Step 4: Repeat the following operations:
[0062] ① Load 16 bytes (128 bits) of data twice, processing 32 uint8 t data each time;
[0063] ② Expand the data to uint32 t, convert it to float32 t type, and perform the permute operation;
[0064] ③ Multiply the converted data by the inverse quantization coefficient, store it in the specified output location, and perform a Sigmoid operation;
[0065] ④ Process the remaining elements.
[0066] Step 5: After permute, dequantization, and sigmoid are completed, perform coordinate transformation to obtain data that meets the confidence threshold [x1, y1, x2, y2, score, class id]; perform NMS processing on this data, and complete coordinate resizing and bounding box operations.
[0067] As can be seen, this example breaks the concept of algorithm layers between hardware computing and CPU software computing in the whole image. Through flexible fusion, multi-stage filtering and other algorithm optimizations, a large number of invalid calculations are eliminated. At the same time, most of the calculations are vectorized, which improves the single-frame processing performance.
[0068] Based on the same inventive concept, embodiments of this application also provide a data processing method applied to a processor.
[0069] Figure 4 This application also provides a flowchart illustrating a data processing method. For example... Figure 4 As shown, the method includes:
[0070] S1. Based on a preset confidence threshold, the data in the inference results output by the AI acceleration unit is filtered to obtain the filtered data;
[0071] S2. Using a single instruction multiple data instruction set, perform dimension permutation, dequantization, and nonlinear activation calculation operations on the filtered data in a vectorized manner.
[0072] The data processing method provided in this application breaks down the concept of layers (post-processing layer, NMS layer). First, graph optimization is performed, moving the AI acceleration unit operators—dimensional permutation and nonlinear activation calculation—to the CPU level. In the post-processing flow, the data in the inference results output by the AI acceleration unit is first filtered based on a preset confidence threshold, which avoids all data from undergoing subsequent operations, thereby greatly reducing redundant operations. Then, the remaining data is processed using single-instruction multiple-data instructions in a vectorized manner for dimensional permutation, dequantization, and nonlinear activation calculation. In this way, by using single-instruction multiple-data instructions and vector calculation to replace scalar calculation, data is processed in batches, and the computational performance is greatly improved.
[0073] like Figure 5 As shown, in some embodiments, the filtering of data in the inference results based on a preset confidence threshold to obtain filtered data includes:
[0074] S11. Based on the preset confidence threshold, the corresponding original score threshold is obtained by inverse nonlinear activation calculation;
[0075] S12. Compare the data in the reasoning result with the original score threshold to filter out data that is lower than the original score threshold, and obtain the filtered data.
[0076] like Figure 6 As shown, in some embodiments, the use of a single instruction multiple data instruction set to perform dimension permutation, dequantization, and nonlinear activation calculation operations on the filtered data in a vectorized manner includes:
[0077] S21. Load the filtered data in a vectorized manner, and expand the filtered data of integer type and convert it into floating-point type data;
[0078] S22. Perform a vectorization dimension permutation operation on the floating-point data;
[0079] S23. Perform vector multiplication on the floating-point data that has undergone dimension permutation and the dequantization coefficients to obtain the dequantized floating-point data.
[0080] S24. Perform vectorized nonlinear activation calculation on the dequantized floating-point data.
[0081] like Figure 7 As shown, in some embodiments, the method further includes:
[0082] S3. After completing the dimension permutation, dequantization, and nonlinear activation calculation, perform nonmaximum suppression operation on the obtained data.
[0083] S4. Based on the result of the non-maximum suppression operation, generate and output target detection information.
[0084] The embodiments of the method provided in this application are specifically the data processing flow of the data processing device in the above-described device embodiments. The specific implementation method will not be repeated here, but can be referred to the detailed description of the above-described device embodiments.
[0085] This application also provides a chip that includes the data processing device described in any of the above embodiments. For example, the chip may be a many-core artificial intelligence (AI) chip based on the RISC-V vector extended instruction architecture.
[0086] The chip provided in this application embodiment includes the data processing device, and therefore can achieve the same technical effect as the data processing device, which will not be described in detail here.
[0087] This application also provides an electronic device, which includes the chip described in any of the above embodiments.
[0088] The electronic device provided in this application embodiment includes the data processing device, and therefore can achieve the same technical effect as the data processing device, which will not be described in detail here.
[0089] Figure 8 This is a block diagram of an electronic device provided in an embodiment of this application. For example, the electronic device 200 may be an AI server, an artificial intelligence server cluster, a training and promotion integrated machine, etc.
[0090] Reference Figure 8The electronic device 200 may include one or more of the following components: an AI-accelerated computing module, a CPU module, a power supply module, a hard drive module, and a fan module. Each module works in conjunction with the bus system through a standardized hardware interface, with the specific architecture as follows:
[0091] The AI-accelerated computing module comprises multiple AI accelerator cards deployed in parallel. Each AI accelerator card integrates at least one AI accelerator chip (or AI accelerator unit, such as an RPU chip, GPU chip, or CGRA chip). The AI accelerator cards communicate with each other via a high-speed card-to-card (C2C) interconnect structure, supporting low-latency, high-bandwidth horizontal scaling. The AI accelerator chip is dedicated to performing AI computing tasks such as high-density matrix operations, neural network model training, and / or inference, providing the main computing power support.
[0092] The CPU module includes at least one CPU board, which houses a central processing unit (CPU) chip and associated CPU memory (such as DDR4 / DDR5, RAM). The CPU chip serves as the system control center, responsible for task scheduling, resource allocation, I / O management, and coordinating the parallel computing of the AI acceleration computing module, while also handling non-accelerated general-purpose computing tasks.
[0093] The power module is equipped with redundant power supply units to provide stable power distribution and management for the AI acceleration computing module, CPU module and other modules.
[0094] The hard drive module integrates a high-speed solid-state drive (SSD) and / or a large-capacity hard disk drive (HDD), connected to the system bus via a backplane. The hard drive stores the operating system, AI training datasets, model parameters, and computation results, providing high-throughput data read / write channels and supporting data preprocessing and persistence.
[0095] The fan module uses a multi-zone independent speed-controlled fan array, which is configured in key heat source areas (such as AI accelerator cards and CPU heat dissipation areas) to achieve system heat dissipation through forced air cooling and ensure the stable operation of high-efficiency computing components.
[0096] The CPU module is connected to the AI acceleration computing module via the PCIe bus to enable task distribution, result collection, and memory coordination.
[0097] The CPU module manages the data access of the hard drive module through SATA / SAS / NVMe interfaces.
[0098] The power module provides tiered power to all functional modules through the power distribution backplane.
[0099] The fan module adjusts the fan speed based on temperature monitoring signals from the CPU board and AI accelerator card.
[0100] This embodiment of the application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can perform the methods provided in the above-described method embodiments.
[0101] This embodiment provides a computer-readable storage medium storing a computer program that causes the computer to perform the methods provided in the above-described method embodiments.
[0102] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0103] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0104] In the description of this specification, the references to terms such as "an embodiment," "a specific embodiment," "some embodiments," "for example," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0105] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A data processing apparatus, characterized in that, include: The AI acceleration unit is configured to perform inference tasks of the object detection neural network model and generate inference results; A processor, coupled to the AI acceleration unit, is configured to: The data in the inference results are filtered based on a preset confidence threshold to obtain the filtered data; Using a single instruction multiple data instruction set, dimension permutation, dequantization, and nonlinear activation calculation operations are performed on the filtered data in a vectorized manner.
2. The data processing apparatus according to claim 1, characterized in that, The processor filters the data in the inference results based on a preset confidence threshold to obtain the filtered data, including: Based on the preset confidence threshold, the corresponding original score threshold is obtained through inverse nonlinear activation calculation; The data in the reasoning result is compared with the original score threshold to filter out data that is lower than the original score threshold, thus obtaining the filtered data.
3. The data processing apparatus according to claim 1, characterized in that, The inference results output by the AI acceleration unit are quantified data.
4. The data processing apparatus according to claim 3, characterized in that, The processor uses a single-instruction multiple-data instruction set to perform dimension permutation, dequantization, and nonlinear activation calculation operations on the filtered data in a vectorized manner, including: The filtered data is loaded in a vectorized manner, and the filtered data of integer type is expanded and converted into floating-point type data; Perform a vectorization dimension permutation operation on the floating-point data; Perform vector multiplication on the floating-point data that has undergone dimension permutation and the dequantization coefficients to obtain the dequantized floating-point data; A vectorized nonlinear activation calculation is performed on the dequantized floating-point data.
5. The data processing apparatus according to claim 4, characterized in that, The nonlinear activation calculation is a Sigmoid function calculation.
6. The data processing apparatus according to claim 1, characterized in that, The device further includes a shared storage module, which is coupled to the AI acceleration unit and the processor respectively. The inference results output by the AI acceleration unit are stored in the shared storage module, and the processor reads the inference results from the shared storage module.
7. The data processing apparatus according to claim 1, characterized in that, The AI acceleration unit includes a neural network processing unit.
8. The data processing apparatus according to claim 1, characterized in that, The processor is also configured to: After completing the dimension permutation, dequantization, and nonlinear activation calculations, the obtained data is subjected to nonmaximum suppression. Based on the result of the nonmaximum suppression operation, target detection information is generated and output.
9. A data processing method, characterized in that, Applied to a processor, the method includes: The data in the inference results output by the AI acceleration unit is filtered based on a preset confidence threshold to obtain the filtered data. Using a single instruction multiple data instruction set, dimension permutation, dequantization, and nonlinear activation calculation operations are performed on the filtered data in a vectorized manner.
10. The method according to claim 9, characterized in that, The process of filtering the data in the inference results based on a preset confidence threshold to obtain filtered data includes: Based on the preset confidence threshold, the corresponding original score threshold is obtained through inverse nonlinear activation calculation; The data in the reasoning result is compared with the original score threshold to filter out data that is lower than the original score threshold, thus obtaining the filtered data.
11. The method according to claim 9, characterized in that, The step of using a single instruction multiple data instruction set to perform dimension permutation, inverse quantization, and nonlinear activation calculation operations on the filtered data in a vectorized manner includes: The filtered data is loaded in a vectorized manner, and the filtered data of integer type is expanded and converted into floating-point type data; Perform a vectorization dimension permutation operation on the floating-point data; Perform vector multiplication on the dimension-transformed floating-point data and the dequantization coefficients to obtain the dequantized floating-point data; A vectorized nonlinear activation calculation is performed on the dequantized floating-point data.
12. The method according to claim 9, characterized in that, The method further includes: After completing the dimension permutation, dequantization, and nonlinear activation calculations, the obtained data is subjected to nonmaximum suppression. Based on the result of the nonmaximum suppression operation, target detection information is generated and output.
13. A chip, characterized in that, Includes the data processing apparatus according to any one of claims 1 to 8.
14. An electronic device, characterized in that, Includes the chip described in claim 13 above.