Model precision adjusting and optimizing method under heterogeneous computing architecture and electronic equipment
By analyzing and accuracy compensation of the layer output data of the neural network model, the calculation accuracy loss problem caused by quantitative compilation is solved, high-precision calculation on NPU is realized, and the deployment and application efficiency of the neural network model is improved.
Patent Information
- Application Number
- CN202410977124.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-22
AI Technical Summary
In heterogeneous computing architecture, the quantitative compilation process of neural network models leads to computational accuracy loss, resulting in large differences in the operation results of the same model on CPU/GPU and NPU, limiting the deployment and application of neural network models on NPU.
By analyzing the output data of each layer of the neural network model, determining whether there is a missing data distribution and performing accuracy compensation until the target accuracy range is reached, ensuring high-precision calculation of the model on the NPU.
It realizes high-precision model inference calculation while maintaining low bit and high-performance computing, so that the neural network accelerator (NPU) can be fully utilized and improves the deployment and application efficiency of the model on NPU.
Smart Images

Figure CN119005270A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a model accuracy tuning method and electronic device under a heterogeneous computing architecture. Background Art
[0002] With the rapid development of artificial intelligence, the automotive system on chip (SoC) generally integrates a dedicated neural network processor (NPU), a central processing unit (CPU), a graphics processing unit (GPU) and a digital signal processor (DSP) to carry out the calculation of the neural network model. These computing units (processors) together form a heterogeneous computing platform for various intelligent analysis applications such as document management system (DMS) and advanced driving assistance system (ADAS).
[0003] In order to improve computing efficiency and effectiveness, dedicated neural network processors generally use low-bit integers to store algorithm model parameters and activations, and perform integer calculations to replace the floating-point calculations of the original neural network model. Therefore, it is necessary to quantize the original neural network model into a fixed-point low-bit integer model for accelerated calculations, and finally generate a binary model file that drives the NPU for calculations. This process is called quantization compilation. The quantization compilation process generally leads to a loss of precision, so that the same model runs on the CPU or GPU in its floating-point calculation form and the results obtained when it is quantized and runs on the NPU in an integer calculation form are quite different. When the error is too large to affect the final performance of the model, the user has to still use other less efficient computing devices, which limits the deployment and application of the neural network model on the NPU. Therefore, it is necessary to provide a model accuracy tuning method under a heterogeneous computing architecture in order to solve the above problems. Summary of the invention
[0004] In view of at least one of the above technical problems existing in the prior art, this application is proposed. According to one aspect of this application, a model accuracy tuning method under a heterogeneous computing architecture is provided, and the method is applied to a system on a chip with multi-core heterogeneous characteristics, and the hardware part of the system on a chip includes at least a central processing unit and a graphics processing unit; the method includes:
[0005] Inputting the data in the first data verification set into the original neural network model to obtain the data output by each neural network layer of the original neural network model; wherein the original neural network model includes a plurality of neural network layers;
[0006] Analyze the data output by each neural network layer layer by layer to determine whether there is a data distribution missing, and the data distribution missing exceeds the first neural network layer of at least one order of magnitude;
[0007] When there is a data distribution missing, and the data distribution missing exceeds at least one order of magnitude of the first neural network layer, performing accuracy compensation on the first neural network layer to obtain a first neural network model after accuracy compensation;
[0008] Verifying whether the accuracy of the first neural network model reaches the target accuracy range;
[0009] When the accuracy of the first neural network model does not reach the target accuracy range, continue to adjust the accuracy of the first neural network model until it reaches the target accuracy range.
[0010] In some embodiments, the method further comprises:
[0011] The calculation target accuracy of the original neural network model is determined according to the data samples with true values in the first data verification set, and the accuracy loss coefficient is determined, so as to determine the target accuracy range according to the target accuracy and the accuracy loss coefficient.
[0012] In some embodiments, the data output by each neural network layer is analyzed layer by layer to determine whether there is a data distribution missing, and the data distribution missing exceeds the first neural network layer of at least one order of magnitude, including:
[0013] Get the data output by any neural network layer;
[0014] Determine whether there is a data break zone in the data output by any one of the neural network layers;
[0015] If a data break zone exists, determining whether the data represented by the data break zone is greater than or equal to at least one order of magnitude;
[0016] If it is greater than or equal to at least one order of magnitude, it is determined that there is a data distribution missing, and the data distribution missing exceeds the first neural network layer of at least one order of magnitude.
[0017] In some embodiments, when there is a data distribution missing, and the data distribution missing exceeds at least one order of magnitude of the neural network layer, the first neural network layer is compensated for accuracy, including:
[0018] At least one data segment of the first neural network layer is enlarged or reduced based on a preset step size so that the orders of magnitude of the various data segments remain consistent and the at least one data segment maintains an original proportional relationship.
[0019] In some embodiments, verifying whether the accuracy of the first neural network model reaches a target accuracy range includes:
[0020] Verify on the central processing unit or the graphics processing unit whether the accuracy of the first neural network model meets the target accuracy range; if not, the accuracy needs to be adjusted again.
[0021] In some embodiments, verifying on the central processing unit or the graphics processing unit whether the accuracy of the first neural network model meets the target accuracy range includes:
[0022] Compiling the first neural network model into a floating-point neural network model on the central processing unit or the graphics processing unit, so that the central processing unit or the graphics processing unit verifies the accuracy of the floating-point neural network model;
[0023] Wherein, the central processing unit or the graphics processing unit is used for:
[0024] Inputting the data in the first data validation set into the floating-point neural network model;
[0025] Obtaining the output result of each neural network layer of the floating-point neural network model;
[0026] Determining whether the output result is within a target accuracy range;
[0027] When the output result is within the target accuracy range, it is determined that the first neural network model has reached the target accuracy range; when the output result is not within the target accuracy range, it is determined that the first neural network model has not reached the target accuracy range.
[0028] In some embodiments, when the accuracy of the first neural network model does not reach the target accuracy range, continuing to adjust the accuracy of the first neural network model includes:
[0029] Verifying again whether the accuracy of the first neural network model reaches the target accuracy range;
[0030] When the data output by a certain neural network layer of the first neural network model still cannot reach the target accuracy, the accuracy of the certain neural network layer is adjusted again.
[0031] In some embodiments, verifying again whether the accuracy of the first neural network model reaches a target accuracy range includes:
[0032] Using a second data validation set, using the first neural network model to perform reasoning and calculate evaluation;
[0033] Determining whether the data output by the first neural network contains abnormal data;
[0034] When abnormal data exists, updating the second data verification set;
[0035] The updated second data verification set is used to verify again whether the accuracy of the first neural network model reaches the target accuracy range.
[0036] In some embodiments, when the data output by a certain neural network layer of the first neural network model still cannot reach the target accuracy, the accuracy of the certain neural network layer is adjusted again, and the method further includes:
[0037] The certain neural network layer of the first neural network model is cut out from the remaining neural network layers of the first neural network model, so that the certain neural network layer performs inference on the central processing unit or the graphics processing unit, and sends the inference result back to the first neural network model.
[0038] In some embodiments, the method further comprises:
[0039] Transplanting the first neural network model to an evaluation plug-in of a quantization compilation tool, so that the quantization compilation tool selects quantization parameters that meet preset conditions;
[0040] According to the quantization parameters, a first neural network model after quantization compilation is obtained.
[0041] In some embodiments, when there are multiple types of data in the validation set, when inputting data into the neural network model, a preset number of data is selected for each type of data so that the input data is balanced in type.
[0042] Another aspect of the present application embodiment provides a model accuracy tuning device under a heterogeneous computing architecture, the device is applied to a system on chip with multi-core heterogeneous characteristics, the hardware part of the system on chip includes at least a central processing unit and a graphics processing unit; the device includes:
[0043] An input module is used to input the data in the first data verification set into the original neural network model to obtain data output by each neural network layer of the original neural network model; wherein the original neural network model includes multiple neural network layers;
[0044] An analysis module is used to analyze the data output by each neural network layer layer by layer to determine whether there is a data distribution missing, and the data distribution missing exceeds the first neural network layer of at least one order of magnitude;
[0045] A compensation module, for performing precision compensation on the first neural network layer when there is a data distribution deficiency and the data distribution deficiency exceeds at least one order of magnitude, so as to obtain a first neural network model after precision compensation;
[0046] A verification module, used to verify whether the accuracy of the first neural network model reaches a target accuracy range;
[0047] The adjustment module is used to continue to adjust the accuracy of the first neural network model until the accuracy of the first neural network model reaches the target accuracy range when the accuracy of the first neural network model does not reach the target accuracy range.
[0048] According to another aspect of the present application, an electronic device is provided, the electronic device comprising:
[0049] A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the processor executes the model accuracy tuning method under the heterogeneous computing architecture as described above.
[0050] On the other hand, an embodiment of the present application provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes the model accuracy tuning method under the heterogeneous computing architecture as described above.
[0051] The model accuracy tuning method under the heterogeneous computing architecture of the embodiment of the present application inputs the data in the first data verification set into the original neural network model, determines whether there is a first neural network layer with data distribution missing and the data distribution missing exceeds at least one order of magnitude according to the data output by each neural network layer, and when there is a first neural network layer with data distribution missing and the data distribution missing exceeds at least one order of magnitude, performs accuracy compensation on the first neural network layer to obtain a first neural network model after accuracy compensation, and when the accuracy of the first neural network model does not reach the target accuracy range, continues to adjust the accuracy of the first neural network model until it reaches the target accuracy range. The embodiment of the present application can realize the neural network accelerator (Neural Processing Unit, NPU) while maintaining low-bit, high-performance computing, and perform high-precision model inference calculations, so that the neural network accelerator (Neural Processing Unit, NPU) in the system on chip (SoC) can be fully and effectively utilized. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0053] Figure 1 A schematic flow chart of a model accuracy tuning method 100 under a heterogeneous computing architecture according to an embodiment of the present application is shown;
[0054] Figure 2 A schematic flowchart of step S102 according to an embodiment of the present application is shown;
[0055] Figure 3 A histogram of feature tensor values output by a neural network layer according to an embodiment of the present application is shown;
[0056] Figure 4 A schematic flowchart of step S103 according to an embodiment of the present application is shown;
[0057] Figure 5 A schematic flow chart showing a method of determining a target accuracy range according to an embodiment of the present application is shown;
[0058] Figure 6 A schematic flow chart showing the accuracy of a first neural network model on a central processing unit according to an embodiment of the present application;
[0059] Figure 7 A schematic flowchart of step S601 according to an embodiment of the present application is shown;
[0060] Figure 8 A schematic flowchart of step S105 according to an embodiment of the present application is shown;
[0061] Fig. 9 A schematic flowchart of step S801 according to an embodiment of the present application is shown;
[0062] Fig.10 A schematic flowchart showing when a neural network layer cannot achieve a target accuracy according to an embodiment of the present application, the neural network layer is cut out from the first neural network model and reasoning is performed on a central processing unit;
[0063] Fig.11 A schematic flow chart showing a method of obtaining a first neural network model after quantization compilation according to a quantization parameter according to an embodiment of the present application;
[0064] Fig.12A schematic flow chart showing a model accuracy tuning method 1200 under a heterogeneous computing architecture according to another embodiment of the present application;
[0065] Fig.13 A schematic block diagram of a model accuracy tuning device under a heterogeneous computing architecture according to an embodiment of the present application is shown;
[0066] Fig.14 A schematic block diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0067] In order to enable those skilled in the art to better understand the technical solutions of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0068] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.
[0069] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, systems or data, and are not used to limit the order or interdependence of the functions performed by these devices, systems or data.
[0070] It should be noted that the modifications of "one" and "plurality" mentioned in this application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more". "Plurality" should be understood as two or more.
[0071] In traditional on-board system on chip (SoC), neural network accelerator (Neural Processing Unit, NPU), central processing unit (Central Processing Unit, CPU), graphics processing unit (Graphics Processing Unit, GPU) and digital signal processor (Digital Signal Processor, DSP) are used to carry out the calculation of neural network models. In order to improve computing efficiency and effectiveness, dedicated neural network accelerator (Neural Processing Unit, NPU) generally uses low-bit integer to store algorithm model parameters and activations, and performs integer calculations to replace the floating-point calculations of the original model. Therefore, it is necessary to quantize the original model into a fixed-point low-bit integer model for accelerated calculation, and finally generate a binary model file that drives the neural network accelerator (Neural Processing Unit, NPU) to perform calculations. This process is called quantization compilation. This process generally leads to a loss of precision, resulting in a significant difference in the results of the same model when it is run on a central processing unit (CPU) or graphics processing unit (GPU) in floating-point form and when it is quantized and run on a neural network accelerator (NPU) in integer form. Although the compilation tool chain of the neural network accelerator (NPU) is equipped with some tools during the quantization compilation process to reduce the output error of the quantized model, there is still no effective quantization tuning method to match it.
[0072] When the error is too large and affects the final performance of the model, users have to continue to use other less efficient computing devices, which limits the deployment and application of neural networks on neural network accelerators (Neural Processing Units, NPUs). The same model has different computational accuracy on heterogeneous computing hardware, and high-efficiency and high-performance low-precision computing hardware is prone to precision loss. How to use as many high-efficiency and high-performance neural network accelerators (NeuralProcessing Units, NPUs) as possible without excessive loss of precision to complete the deployment of the model is a challenge.
[0073] Based on at least one of the aforementioned technical problems, the present application provides a model accuracy tuning method under a heterogeneous computing architecture, the method being applied to a system on a chip with multi-core heterogeneous characteristics, the hardware part of the system on a chip comprising at least a central processing unit and a graphics processing unit; the method comprising: inputting data in a first data verification set into an original neural network model to obtain data output by each neural network layer of the original neural network model; wherein the original neural network model comprises multiple neural network layers; analyzing the data output by each neural network layer layer by layer to determine whether there is a first neural network layer with data distribution missing, and the data distribution missing exceeds at least one order of magnitude; when there is a first neural network layer with data distribution missing, and the data distribution missing exceeds at least one order of magnitude, performing accuracy compensation on the first neural network layer to obtain a first neural network model after accuracy compensation; verifying whether the accuracy of the first neural network model reaches the target accuracy range; if the accuracy of the first neural network model does not reach the target accuracy range, continuing to adjust the accuracy of the first neural network model until the target accuracy range is reached. The embodiments of the present application can enable a neural network accelerator (Neural Processing Unit, NPU) to perform high-precision model inference calculations while maintaining low-bit, high-performance operations, so that the neural network accelerator (Neural Processing Unit, NPU) in a system on a chip (SoC) can be fully and effectively utilized.
[0074] The model accuracy tuning method under the heterogeneous computing architecture of the embodiment of the present application is applied to a system-on-chip with multi-core heterogeneous characteristics. The hardware part of the system-on-chip includes at least a central processing unit (CPU), and may also include a dedicated neural network processor (NPU), a graphics processing unit (GPU) and a digital signal processor (DSP).
[0075] Figure 1 A schematic flow chart of a method for model accuracy tuning under a heterogeneous computing architecture according to an embodiment of the present application is shown; Figure 1 As shown, the model accuracy tuning method 100 under the heterogeneous computing architecture according to the embodiment of the present application may include the following steps S101, S102, S103, S104 and S105:
[0076] In step S101, the data in the first data verification set is input into the original neural network model to obtain the data output by each neural network layer of the original neural network model.
[0077] Wherein, the original neural network model includes multiple neural network layers.
[0078] In an embodiment of the present application, the data in the first data verification set can be input into the original neural network model, inference is performed by the original neural network model, and then the eigenvalues are output layer by layer, and statistical analysis is performed on these eigenvalues to determine whether there is any missing data distribution.
[0079] It is worth noting that when collecting sample data to build the first data validation set, if the input value of the model involves multi-category problems, the category of the sample data finally selected needs to be confirmed to ensure that there are enough and relatively balanced samples in the first data validation set. In all subsequent steps involving the validation set and selecting sample data, the above principles must be followed to ensure the diversity of the model input data and the accuracy of the output data.
[0080] In step S102, the data output by each neural network layer is analyzed layer by layer to determine whether there is a data distribution missing, and the data distribution missing exceeds the first neural network layer of at least one order of magnitude.
[0081] In an embodiment of the present application, the eigenvalues output by the original neural network model are analyzed. When the eigenvalues output by these layers are broken, and the corresponding tensor values of the eigenvalues have a distinct broken band shape in their corresponding histogram distribution, and the broken bands exceed at least one order of magnitude, it indicates that there is a missing data distribution and the accuracy of the model needs to be adjusted.
[0082] In some embodiments of the present application, Figure 2 As shown, in step S102, the data output by each neural network layer is analyzed layer by layer to determine whether there is a data distribution missing, and the data distribution missing exceeds the first neural network layer of at least one order of magnitude, including steps S201, S202, S203 and S204:
[0083] In step S201, data outputted by any neural network layer is obtained;
[0084] In step S202, it is determined whether the data output by any neural network layer has a fault zone;
[0085] In step S203, if there is a data fracture band, it is determined whether the data represented by the data fracture band is greater than or equal to at least one order of magnitude;
[0086] In step S204, if it is greater than or equal to at least one order of magnitude, it is determined that there is a data distribution missing, and the data distribution missing exceeds the first neural network layer of at least one order of magnitude.
[0087] like Figure 3 As shown in the figure, it is a histogram of the characteristic tensor values output by the neural network layer. It can be seen that there are obvious fracture bands in the figure. Please note that Figure 3 There are two distribution bands in FIG. 4 , which are only used as an example for explanation. In other examples, more than two band distributions may appear.
[0088] Combination Figure 3 , the data of some channels of a certain output feature tensor (tensor) are concentrated in the data segment [0.1,1.3], while the data of another channel are concentrated in the data segment [213,498]. There is an obvious data distribution missing in the middle, and the span is large, exceeding one data order of magnitude. In this way, when quantizing at low bits, one data segment will cover another data segment, which will cause serious loss of data precision in the other data segment. Taking 8-bit data as an example, if the entire data segment [0.1,498] combined by all channels is represented by 2^8 orders of magnitude, then scale = (498-0.1) / 255 = 1.95255, then the data of the channels concentrated in the data segment [0.1,1.3] will eventually be quantized to 0, which may completely lose its original meaning, resulting in a large deviation in the final integer calculation result and affecting the final accuracy. Therefore, the accuracy of the model needs to be adjusted.
[0089] In step S103, when there is data distribution missing, and the data distribution missing exceeds at least one order of magnitude of the first neural network layer, the first neural network layer is precision compensated to obtain a first neural network model after precision compensation.
[0090] In one example, if Figure 4 As shown, in step S103, when there is a data distribution missing, and the data distribution missing exceeds at least one order of magnitude of the neural network layer, the first neural network layer is compensated for accuracy, including step S401:
[0091] In step S401, at least one data segment of the first neural network layer is enlarged or reduced based on a preset step size so that the orders of magnitude of the various data segments remain consistent and the at least one data segment maintains an original proportional relationship.
[0092] For example, in Figure 3In the example shown, the channels of the data segment [0.1, 1.3] can be multiplied by a suitable value as a whole, for example, the data obtained after multiplication can be selected to be close to another data segment. In this example, multiplication by 256 is selected to obtain the adjusted data segment [25.6, 332.8], so that the relative proportional relationship between the data of these channels after quantization is still maintained, and the precision loss can be controlled. Alternatively, the channels of the data segment [213, 498] can be divided by a suitable value as a whole, for example, by 256, to obtain the adjusted data segment [0.83, 1.95]. For the case of two data segments, only one data segment can be selected for operation. For the case of multiple data segments, the principle is the same, that is, the order of magnitude of each data segment can be made as close as possible. Please note that no matter which data segment is operated, in the subsequent data processing, the data segment needs to be compensated by the inverse operation. For example, the operation of multiplying the data segment [0.1, 1.3] by 256 will be compensated by the equivalent operation of [25.6, 332.8] divided by 256. This compensation operation can occur in one network layer, or it can be completed by multiple network layers, or even by the post-processing of the model, to ensure the accuracy of the data.
[0093] In step S104, it is verified whether the accuracy of the first neural network model reaches the target accuracy range.
[0094] In the embodiment of the present application, the target accuracy range can be determined in advance, for example, based on data samples with true values. Figure 5 As shown, the method further includes step S501,
[0095] In step S501, the computational target accuracy of the original neural network model is determined based on data samples with true values in the first data verification set, and the accuracy loss coefficient is determined, so as to determine the target accuracy range based on the target accuracy and the accuracy loss coefficient.
[0096] In the first data validation set, two parts of data samples may be included, the first part is a data sample val with a true value, and the second part is a data sample calib used for inference. In step S101, the data sample calib may be input into the original neural network model to obtain the data output by each neural network layer of the original neural network model to determine whether there is data missing. In step S501, the data sample val with a true value may be used to determine the target accuracy p of the model.
[0097] For example, input the original neural network model, collect sample data to construct the first data verification set with true values, and perform inference on the original framework of the model to obtain the accuracy of the first data verification set. This accuracy is the target accuracy p of the original neural network model after quantization. Assuming that the final accuracy loss needs to be controlled above a*p (where a is the accuracy loss coefficient), for example, a=0.99, the accuracy loss is 1%, and the target accuracy range is p~0.99p.
[0098] In certain embodiments of the present application, Figure 6 As shown, the method further includes step S601:
[0099] In step S601, it is verified on the central processing unit or the graphics processing unit whether the accuracy of the first neural network model meets the target accuracy range; if not, the accuracy needs to be adjusted again.
[0100] In some embodiments of the present application, Figure 7 As shown, in the step S601, the accuracy of the first neural network model is verified on the central processing unit or the graphics processing unit, including steps S701, S702 and S703:
[0101] In step S701, the first neural network model is compiled into a floating-point neural network model on the central processing unit or the graphics processing unit, so that the central processing unit or the graphics processing unit verifies the accuracy of the floating-point neural network model;
[0102] Wherein, the central processing unit or the graphics processing unit is used for:
[0103] Inputting the data in the first data validation set into the floating-point neural network model;
[0104] Obtaining the output result of each neural network layer of the floating-point neural network model;
[0105] Determining whether the output result is within a target accuracy range;
[0106] In step S702, when the output result is within the target accuracy range, it is determined that the first neural network model has reached the target accuracy range;
[0107] In step S703, when the output result is not within the target accuracy range, it is determined that the first neural network model does not reach the target accuracy range.
[0108] Here, the roles played by the CPU and the GPU are the same, and they are both used to verify whether the accuracy of the first neural network model meets the target accuracy range. The following is an example of using the CPU for verification.
[0109] In an embodiment of the present application, after the neural network model is precision tuned, it is necessary to evaluate the tuned neural network model. Since the calculation accuracy of the central processing unit (CPU) is relatively high, the central processing unit (CPU) can be used for verification. The central processing unit can perform floating-point operations. At this time, the neural network model (that is, the original neural network model obtained after tuning) is a floating-point neural network model. The floating-point neural network model is made to perform floating-point inference calculations on the central processing unit (CPU) of the system on chip (SoC), and the output of each model is saved. The outputs of these models are used to evaluate the accuracy to ensure that the model pre-processing and post-processing are modified correctly, with the goal of making the final accuracy value the same as p.
[0110] It is worth noting that when there are data distribution missing in each neural network layer of the original neural network model, and the data distribution missing exceeds at least one order of magnitude of the first neural network layer, the original neural network model needs to be edited to obtain a new floating-point neural network model, and then the central processing unit or the graphics processing unit is used to verify the accuracy of the edited floating-point neural network model. If there is no data distribution missing in each neural network layer of the original neural network model, there is no need to edit it, but the original neural network model is used to verify the accuracy of the original floating-point neural network model.
[0111] In step S105, when the accuracy of the first neural network model does not reach the target accuracy range, the accuracy of the first neural network model continues to be adjusted until the target accuracy range is reached.
[0112] In some embodiments, Figure 8 As shown, in step S105, when the accuracy of the first neural network model does not reach the target accuracy range, the accuracy of the first neural network model continues to be adjusted, including step S801 and step S802:
[0113] In step S801, verify again whether the accuracy of the first neural network model reaches the target accuracy range;
[0114] In step S802, when the data output by a certain neural network layer of the first neural network model still cannot reach the target accuracy, the accuracy of the certain neural network layer is adjusted again.
[0115] In some embodiments, Fig. 9As shown, in step S801, it is verified again whether the accuracy of the first neural network model reaches the target accuracy range, including steps S901, S902, S903 and S904:
[0116] In step S901, using a second data validation set, the first neural network model is used to perform reasoning and calculate evaluation;
[0117] In step S902, determining whether the data output by the first neural network contains abnormal data;
[0118] In step S903, when there is abnormal data, the second data verification set is updated;
[0119] In step S904, the updated second data verification set is used to verify again whether the accuracy of the first neural network model reaches the target accuracy range.
[0120] When the accuracy of a neural network layer is tuned in the embodiment of the present application, it is necessary to first verify the accuracy of the model. When verifying the accuracy, a recreated or updated second data verification set for quantization compilation can be used to ensure that the number and types of verification samples are sufficient, so that more accurate accuracy can be obtained after reasoning and calculation evaluation using the second neural network model. When the accuracy of the second neural network model reaches the target accuracy range, there is no need to perform accuracy tuning on the model; when the accuracy of the second neural network model does not reach the target accuracy range, the model needs to be accuracy tuned.
[0121] In some embodiments, Fig.10 As shown, in step S802, when the data output by a certain neural network layer of the first neural network model still cannot reach the target accuracy, the accuracy of the certain neural network layer is adjusted again, including step S1001:
[0122] In step S1001, the certain neural network layer of the first neural network model is separated from the remaining neural network layers of the first neural network model, so that the certain neural network layer performs inference on the central processing unit or the graphics processing unit, and the inference result is sent back to the first neural network model.
[0123] The first neural network model performs reasoning on the first data validation set on the neural network processor (Neural Processing Unit, NPU) of the system on chip (System on Chip, SoC), calculates the evaluation (metric), and collects the abnormal sample set B. If the abnormal sample set B exists and has obvious bias, it means that the data samples in the validation set used to calculate the accuracy are biased, which will affect the selection of the quantization parameters. It is necessary to return to step S903 to update the validation set and perform subsequent steps; otherwise, if the accuracy loss reaches the target accuracy range, the accuracy adjustment ends. Otherwise, if there are still modification strategies to try, jump to step S102 and repeat the subsequent process. If the accuracy loss does not reach the expected range and all modification strategies have been tried, it is necessary to perform segmentation on the neural network layer so that the neural network layer with the accuracy loss reaching the expected range performs quantized integer calculations and finally runs on the neural network processor (Neural Processing Unit, NPU); while the neural network layer with the accuracy loss not reaching the expected range performs floating-point calculations and runs on the central processing unit (Central Processing Unit, CPU) or graphics processing unit (Graphics Processing Unit, GPU).
[0124] In some embodiments, Fig.11 As shown, the method further includes step S1101 and step S1102:
[0125] In step S1101, the first neural network model is transplanted to an evaluation plug-in of a quantization compilation tool, so that the quantization compilation tool selects quantization parameters that meet preset conditions;
[0126] In step S1102, a first neural network model after quantization compilation is obtained according to the quantization parameter.
[0127] In the embodiment of the present application, the accuracy evaluation code of step S101 is transplanted into the evaluation (metric) plug-in of the quantization compilation tool, so that the quantization compilation tool can evaluate the quantization effect of the data, thereby obtaining the preferred quantization parameters. Then, the first neural network model is obtained according to the preferred quantization parameters.
[0128] When there are multiple types of data in the verification set, when inputting data into the neural network model, a preset number of data is selected for each type of data so that the input data is balanced in type.
[0129] like Fig.12, which is a schematic flow chart of a model accuracy tuning method under a heterogeneous computing architecture according to some other embodiments of the present application. The model accuracy tuning method 1200 under a heterogeneous computing architecture according to an embodiment of the present application may include the following steps S1201, S1202, S1203, S1204, S1205, S1206, and S1207;
[0130] In step S1201, the original neural network model is input, a first data verification set with true values is collected, and the first data verification set is used to perform inference on the original neural network model to obtain a target accuracy corresponding to the first data verification set.
[0131] In the first data validation set, two parts of data samples may be included, the first part is a data sample val with a true value for determining the target accuracy, and the second part is a data sample for inference. In step S101, the data sample calib may be input into the original neural network model fm0 to obtain the data output by each neural network layer of the original neural network model fm0 to determine whether there is data missing. In step S501, the data sample val with a true value may be used to determine the target accuracy p of the model.
[0132] For example, the target accuracy is the accuracy target p of the original neural network model after quantization, and it is assumed that the final accuracy loss needs to be controlled above a*p, where a is the accuracy loss coefficient. For example, a=0.99, that is, the accuracy loss is 1%.
[0133] It is worth noting that if the samples of the original neural network model involve multiple category problems, when constructing the first data verification set, it is necessary to confirm the categories of the selected samples to ensure that there are enough categories of samples in the first data verification set and the number of samples of each category is relatively balanced.
[0134] In addition, during reasoning, it is generally divided into several stages: preprocessing, reasoning, postprocessing and evaluation (metric). In step S1201, it is necessary to go through several stages: preprocessing preprocess0, reasoning, postprocess0 and accuracy evaluation (metric).
[0135] In step S1202, statistical analysis is performed on the characteristic values output layer by layer during inference of the original neural network model to determine whether there is data distribution missing, and the data distribution missing exceeds at least one order of magnitude; if there is data distribution missing, and the data distribution missing exceeds at least one order of magnitude, the original neural network model is compensated for accuracy.
[0136] For example, for an original neural network model, it is necessary to analyze the output of each neural network layer of the original neural network model fm0 layer by layer, and focus on analyzing the output of the concat layer. When the distribution of the feature tensor value histograms of the outputs of these layers shows an obvious fractured band shape, refer to Figure 2 , which can present 2 or even more banded distributions. The data of some channels of a certain output feature tensor are concentrated in the data segment [0.1,1.3], while the data of other channels are concentrated in the data segment [213,498]. There is an obvious data distribution missing in the middle and the span is larger than one data magnitude. This makes it difficult to cover one data segment during low-bit quantization, which causes serious data loss in the other data segment. Taking the 8-bit data channel as an example, if the entire data segment [0.1,498] combined by all channels is represented by 2^8 magnitudes, then the magnitude parameter scale=(498-0.1) / 255=1.95255, then the data of the channels concentrated in the data segment [0.1,1.3] will eventually be quantized to 0, resulting in a large deviation in the final integer calculation result and affecting the final accuracy. This is because data in different numerical ranges are combined. In this case, the model needs to be edited. If the model editing affects the range of the final output data value, the model post-processing needs to be compensated and modified.
[0137] In the embodiment of the present application, the data distribution statistics S are analyzed layer by layer, and the neural network layer with the output in the form of a broken band is found, and the original neural network model (which is a floating-point neural network model) is edited to obtain a new neural network floating-point neural network model fml. It is worth noting that if there is no neural network layer in the form of a broken band, no editing is required, that is, fml=fm0. Post-processing needs to be modified synchronously to obtain postprocess1; if no synchronous modification is required, postprocess1=postproc ess0.
[0138] For example, in the above example, the channels of the [0.1, 1.3] data segment can be multiplied by a suitable value such as 256 to obtain [25.6, 332.8], so that the relative proportional relationship between the data of these channels is still basically maintained after quantization, and the loss of precision can be controlled. Of course, in subsequent processing, it is necessary to use an equivalent division by 256 to be compensated. This compensation operation can occur in one network layer, or it can be completed by multiple network layers, or even by the post-processing of the model.
[0139] In step S1203, the original neural network model is compiled into a floating-point neural network model so that the floating-point neural network model can perform floating-point inference calculations on the central processing unit (CPU) of the system on a chip (SoC), and the output of each floating-point neural network model is saved; the outputs of these floating-point neural network models are subjected to the same accuracy evaluation as in the above step S1201.
[0140] The above steps can also be run on a graphics processor, and are all used to verify whether the accuracy of the original neural network model is the same as the target accuracy. This embodiment is introduced by taking a central processing unit as an example.
[0141] Here, the floating point inference calculation is run on the central processing unit (CPU) because the calculation of the central processing unit (CPU) is more accurate. The same accuracy evaluation as the above step S1201 is performed, that is, verification is performed on the same first data verification set to ensure that the model pre-processing and post-processing are modified correctly, and the goal is to make the final accuracy after tuning the same as the target accuracy p.
[0142] Use the compilation tool to compile the floating-point model so that it can run on the central processing unit (CPU) or graphics processing unit (GPU) of the system on chip (SoC). In order to make the model run more efficiently, it is sometimes necessary to migrate some preprocessing operations to the model for execution. At this time, the neural network model fm2 is generated. If such migration is not required, fm2 = fml. After the corresponding part of the original proprocess operations are migrated to the floating-point model, they need to be modified to obtain a new simplified preprocessl. If there is no such migration, preprocessl = preprocess0. Compile the neural network model fm2 and run it on the central processing unit (CPU) or graphics processing unit (GPU) of the system on chip (SoC), perform inference on the data verification set val and calculate the final accuracy pl.
[0143] After verification by the central processing unit (CPU), the actual model accuracy p1 is obtained. When the actual model accuracy p1 is the same as the target accuracy p, step S1204 is continued. When the actual model accuracy p1 is different from the target accuracy p, the original neural network model needs to be corrected and calculated, and the original neural network model is returned to step S1202 for re-analysis.
[0144] In step S1204, the accuracy evaluation code of step S1201 is transplanted into the evaluation (metric) plug-in of the quantization compilation tool, so that the quantization compilation tool evaluates the quantization effect of the data, thereby obtaining the optimal quantization parameters.
[0145] In the embodiment of the present application, the evaluation (metric) code is extended as a plug-in as an optional evaluation type of the compilation quantization tool, and the configuration enables the evaluation (metric) so that the quantization compilation tool can calculate the inference and evaluation of the val data in the quantization parameter optimization stage. If the evaluation (metric) is not configured or the quantization compilation tool does not support the extended evaluation (metric), its default evaluation (metric), such as cosine_similarity, is used.
[0146] The quantization parameters here may include, for example, magnitude parameters scale, precision loss coefficients, and the like.
[0147] In step S1205, a second data verification set for quantization compilation is constructed or updated so that the original neural network model can still achieve the target accuracy on the second data verification set.
[0148] The embodiment of the present application can re-collect samples to construct a second data verification set, or can update part of the data on the basis of the first data verification set to form a second data verification set.
[0149] Similar to the principle of constructing the first data validation set, if the samples of the original neural network model involve multiple category problems, when constructing the second data validation set, it is necessary to confirm the categories of the selected samples to ensure that there are enough categories of samples in the second data validation set and the number of samples of each category is relatively balanced.
[0150] In an embodiment of the present application, if the second data verification set (val1) has been constructed in advance, the construction step is skipped, otherwise a very small amount of verification set data is randomly selected to construct the second data verification set (val1). If the second data verification set (val1) involves a multi-category problem, it is necessary to confirm the category of the data samples finally selected to ensure that the second data verification set (val1) has enough multi-category and relatively balanced data samples. It is worth noting that during the entire execution of this method, once it is necessary to rebuild the data set and regenerate the data distribution statistics S, the selection of data samples needs to refer to this logic.
[0151] In step S1206, quantization compilation of the tuned original neural network model is performed, quantization parameters are optimized and the best quantization model is obtained.
[0152] In an embodiment of the present application, a quantization algorithm is applied to the layer-by-layer data distribution S to realize the quantization of the model, and the quantization model is applied to infer the data samples of the second data verification set (val1), and statistical evaluation (metric) is performed to find the quantization parameter corresponding to the optimal evaluation (metric) value, thereby compiling and constructing the optimal quantization model qm.
[0153] In step S1207, the quantization model is inferred on the first data verification set and the metric is calculated to determine whether there are abnormal samples; if there are abnormal samples and the precision loss has not reached the predetermined range, return to step S1205 for re-verification; if there are abnormal samples and the precision loss has not reached the predetermined range, and there is an adjustment plan that can be tried to modify, return to step S1202 to perform precision compensation again.
[0154] In an embodiment of the present application, the optimal quantization model qm performs reasoning of a complete first data verification set (val) on a system-on-chip (SoC) neural network processor (Neural Processing Unit, NPU), and calculates a metric to determine whether the metric ultimately satisfies an acceptable loss of accuracy and to identify data samples B with abnormal performance.
[0155] Specifically, determine whether data sample B has obvious bias. If so, return to step S1205; otherwise, determine whether the accuracy loss is acceptable; if the accuracy loss is acceptable, quantize and compile the optimal quantization model qm on the neural processing unit (NPU); if the accuracy loss is unacceptable, determine whether there are still model editing attempts; if there are model editing attempts, return to step S1202; if there are no model editing attempts, execute step S1208.
[0156] In step S1208, when the target accuracy cannot be achieved after precision compensation is performed again, the neural network layer that cannot achieve the target accuracy is split out of the quantization model, the calculation is performed on the central processing unit, and then the calculation result is returned to the original neural network model.
[0157] The first neural network model performs reasoning on the first data validation set on the neural network processor (Neural Processing Unit, NPU) of the system on chip (System on Chip, SoC), and calculates the evaluation (metric) to determine whether there is an abnormal sample set. If there is an abnormal sample set and there is obvious bias, it means that the validation set of the calculation accuracy is biased, and it is necessary to return to step S1205 to update the validation set and perform subsequent steps; otherwise, if the accuracy loss reaches the target accuracy range, the tuning ends. Otherwise, if there are still modification strategies to try, jump to step S1202 and repeat the subsequent process. If the accuracy loss does not reach the expected range and all modification strategies have been tried, it is necessary to perform segmentation on the neural network layer so that the neural network layer with the accuracy loss reaching the expected range performs quantized integer calculations and finally runs on the neural network processor (Neural Processing Unit, NPU); while the neural network layer with the accuracy loss not reaching the expected range performs floating-point calculations and runs on the central processing unit (Central Processing Unit, CPU) or graphics processing unit (Graphics Processing Unit, GPU).
[0158] When there is no model editing attempt, starting from the neural network layer with precision loss, the floating-point neural network model fm2 is split, and the split neural network layer is compiled and executed in the central processing unit (CPU) or graphics processing unit (GPU) in the form of floating-point operations, and the unsplit neural network layer is quantized and compiled in the neural network processor (NPU), thereby realizing the process of executing the neural network model on a system-on-chip with multi-core heterogeneous characteristics.
[0159] The above steps are not necessarily executed in the above order when being executed, and the execution order can be adjusted as needed during the implementation process.
[0160] The embodiments of the present application can enable a neural network accelerator (Neural Processing Unit, NPU) to perform high-precision model inference calculations while maintaining low-bit, high-performance operations, so that the neural network accelerator (Neural Processing Unit, NPU) in a system on a chip (SoC) can be fully and effectively utilized.
[0161] Combine the following Fig.13 The model precision tuning device under the heterogeneous computing architecture of the present application is described. The model precision tuning device under the heterogeneous computing architecture is applied to a system on chip with multi-core heterogeneous characteristics, and the hardware part of the system on chip includes at least a neural network processor and a central processing unit.
[0162] Fig.13 A schematic block diagram of a model accuracy tuning device 1300 under a heterogeneous computing architecture according to an embodiment of the present application is shown. The model accuracy tuning device under the heterogeneous computing architecture is applied to a system on chip with multi-core heterogeneous characteristics, and the hardware part of the system on chip includes at least a neural network processor and a central processing unit.
[0163] like Fig.13 As shown, the model accuracy tuning device 1300 under the heterogeneous computing architecture includes:
[0164] An input module 1301 is used to input the data in the first data verification set into the original neural network model to obtain data output by each neural network layer of the original neural network model; wherein the original neural network model includes multiple neural network layers;
[0165] An analysis module 1302 is used to analyze the data output by each neural network layer layer by layer to determine whether there is a data distribution missing, and the data distribution missing exceeds the first neural network layer of at least one order of magnitude;
[0166] A compensation module 1303 is used to perform precision compensation on the first neural network layer when there is a data distribution deficiency and the data distribution deficiency exceeds at least one order of magnitude to obtain a first neural network model after precision compensation;
[0167] Verification module 1304, used to verify whether the accuracy of the first neural network model reaches the target accuracy range;
[0168] The adjustment module 1305 is used to continue to adjust the accuracy of the first neural network model until the accuracy reaches the target accuracy range when the accuracy of the first neural network model does not reach the target accuracy range.
[0169] Combine the following Fig.14 The electronic device of the present application is described, wherein: Fig.14 A schematic block diagram of an electronic device according to an embodiment of the present application is shown.
[0170] like Fig.14 As shown, the electronic device 1400 includes: one or more memories 1401 and one or more processors 1402, and the memory 1401 stores a computer program executed by the processor 1402. When the computer program is executed by the processor 1402, the processor 1402 executes the model accuracy tuning method under the heterogeneous computing architecture described above.
[0171] The electronic device 1400 may be part or all of a computer device that can implement a model accuracy tuning method under a heterogeneous computing architecture through software, hardware, or a combination of software and hardware.
[0172] like Fig.14 As shown, the electronic device 1400 includes one or more memories 1401, one or more processors 1402, a display (not shown), and a communication interface, etc. These components are interconnected through a bus system and / or other forms of connection mechanisms (not shown). It should be noted that Fig.14 The components and structures of the electronic device 1400 shown are merely exemplary and non-limiting. The electronic device 1400 may also have other components and structures as required.
[0173] The memory 1401 is used to store various data and executable program instructions generated during the operation of the related method, such as for storing various application programs or algorithms for implementing various specific functions. It can include one or more computer program products, and the computer program product can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory can include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory can include, for example, read-only memory (ROM), hard disk, flash memory, etc.
[0174] Processor 1402 may be a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other form of processing unit having data processing capabilities and / or instruction execution capabilities, and may be other components in electronic device 1400 to perform desired functions.
[0175] In one example, the electronic device 1400 also includes an output device that can output various information (such as images or sounds) to the outside (such as a user), and can include one or more of a display device, a speaker, and the like.
[0176] The communication interface can be an interface of any currently known communication protocol, such as a wired interface or a wireless interface, wherein the communication interface may include one or more serial ports, USB interfaces, Ethernet ports, WiFi, wired networks, DVI interfaces, device integrated interconnect modules or other suitable ports, interfaces, or connections.
[0177] In addition, according to an embodiment of the present application, a storage medium is also provided, on which program instructions are stored, and when the program instructions are run by a computer or a processor, the corresponding steps of the model accuracy tuning method under the heterogeneous computing architecture of the embodiment of the present application are used. The storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media.
[0178] The model accuracy tuning device, electronic device and storage medium under the heterogeneous computing architecture of the embodiments of the present application have the same advantages as the model accuracy tuning method under the aforementioned heterogeneous computing architecture because they can implement the model accuracy tuning method under the aforementioned heterogeneous computing architecture.
[0179] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present application to this. Those of ordinary skill in the art may make various changes and modifications therein without departing from the scope and spirit of the present application. All these changes and modifications are intended to be included within the scope of the present application as required by the appended claims.
[0180] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0181] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.
[0182] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.
[0183] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present application should not be interpreted as reflecting the following intention: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with features less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby explicitly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.
[0184] It will be understood by those skilled in the art that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.
[0185] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.
[0186] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some modules according to the embodiments of the present application. The application can also be implemented as a device program (e.g., computer program and computer program product) for executing a part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.
[0187] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present application may be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.
[0188] The above is only a specific implementation or description of a specific implementation of the present application, and the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application shall be based on the protection scope of the claims.
[0189] It should be understood that, although the steps in the flowcharts of the accompanying drawings of the specification are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.
Claims
1. A model accuracy tuning method under a heterogeneous computing architecture, characterized in that: The method is applied to a system on chip with multi-core heterogeneous characteristics, wherein the hardware part of the system on chip includes at least a central processing unit and a graphics processing unit; the method includes: Inputting the data in the first data verification set into the original neural network model to obtain the data output by each neural network layer of the original neural network model; wherein the original neural network model includes a plurality of neural network layers; Analyze the data output by each neural network layer layer by layer to determine whether there is a data distribution missing, and the data distribution missing exceeds the first neural network layer of at least one order of magnitude; When there is a data distribution missing, and the data distribution missing exceeds at least one order of magnitude of the first neural network layer, performing accuracy compensation on the first neural network layer to obtain a first neural network model after accuracy compensation; Verifying whether the accuracy of the first neural network model reaches the target accuracy range; When the accuracy of the first neural network model does not reach the target accuracy range, continue to adjust the accuracy of the first neural network model until it reaches the target accuracy range.
2. The method according to claim 1, characterized in that The method further comprises: The calculation target accuracy of the original neural network model is determined according to the data samples with true values in the first data verification set, and the accuracy loss coefficient is determined, so as to determine the target accuracy range according to the target accuracy and the accuracy loss coefficient.
3. The method according to claim 1, characterized in that: The data output by each neural network layer is analyzed layer by layer to determine whether there is a data distribution missing, and the data distribution missing exceeds the first neural network layer of at least one order of magnitude, including: Get the data output by any neural network layer; Determine whether there is a data break zone in the data output by any one of the neural network layers; If a data break zone exists, determining whether the data represented by the data break zone is greater than or equal to at least one order of magnitude; If it is greater than or equal to at least one order of magnitude, it is determined that there is a data distribution missing, and the data distribution missing exceeds the first neural network layer of at least one order of magnitude.
4. The method according to claim 3, characterized in that When there is a data distribution missing, and the data distribution missing exceeds at least one order of magnitude of the neural network layer, the first neural network layer is precision compensated, including: At least one data segment of the first neural network layer is enlarged or reduced based on a preset step size so that the orders of magnitude of the various data segments remain consistent and the at least one data segment maintains an original proportional relationship.
5. The method according to claim 1, characterized in that Verifying whether the accuracy of the first neural network model reaches the target accuracy range includes: Verifying on the central processing unit or the graphics processing unit whether the accuracy of the first neural network model meets the target accuracy range; if not, the accuracy needs to be adjusted again; Wherein, verifying on the central processing unit or the graphics processing unit whether the accuracy of the first neural network model meets the target accuracy range includes: Compiling the first neural network model into a floating-point neural network model on the central processing unit or the graphics processing unit, so that the central processing unit or the graphics processing unit verifies the accuracy of the floating-point neural network model; Wherein, the central processing unit or the graphics processing unit is used for: Inputting the data in the first data validation set into the floating-point neural network model; Obtaining the output result of each neural network layer of the floating-point neural network model; Determining whether the output result is within a target accuracy range; When the output result is within the target accuracy range, it is determined that the first neural network model has reached the target accuracy range; when the output result is not within the target accuracy range, it is determined that the first neural network model has not reached the target accuracy range.
6. The method according to claim 1, characterized in that When the accuracy of the first neural network model does not reach the target accuracy range, continuing to adjust the accuracy of the first neural network model includes: Verifying again whether the accuracy of the first neural network model reaches the target accuracy range; When the data output by a certain neural network layer of the first neural network model still cannot reach the target accuracy, adjusting the accuracy of the certain neural network layer again; Among them, verifying again whether the accuracy of the first neural network model reaches the target accuracy range includes: Using a second data validation set, using the first neural network model to perform reasoning and calculate evaluation; Determining whether the data output by the first neural network contains abnormal data; When abnormal data exists, updating the second data verification set; The updated second data verification set is used to verify again whether the accuracy of the first neural network model reaches the target accuracy range.
7. The method according to claim 6, characterized in that When the data output by a certain neural network layer of the first neural network model still cannot reach the target accuracy, the accuracy of the certain neural network layer is adjusted again, and the method further includes: The certain neural network layer of the first neural network model is cut out from the remaining neural network layers of the first neural network model, so that the certain neural network layer performs inference on the central processing unit or the graphics processing unit, and sends the inference result back to the first neural network model.
8. The method according to claim 1, characterized in that The method further comprises: Transplanting the first neural network model to an evaluation plug-in of a quantization compilation tool, so that the quantization compilation tool selects quantization parameters that meet preset conditions; According to the quantization parameter, a first neural network model after quantization compilation is obtained; Among them, when there are multiple types of data in the verification set, when inputting data into the neural network model, a preset number of data is selected for each type of data to achieve a balance in the type of the input data.
9. An electronic device, characterized in that: The electronic device comprises: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the processor executes the model accuracy tuning method under the heterogeneous computing architecture as described in any one of claims 1 to 8.
10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, enables the processor to execute the model accuracy tuning method under a heterogeneous computing architecture as described in any one of claims 1 to 8.
