A method and apparatus for adjusting calculation accuracy

Abstract

This application discloses a method and apparatus for adjusting computational precision. The method includes: obtaining network layers included in a memory access bottleneck set; determining whether the network layer can undergo half-precision memory conversion; if the network layer can undergo half-precision memory conversion, then performing computation on the network layer in a half-precision manner to obtain a first output result; determining whether the first output result is less than a preset first threshold; if the first output result is less than the first threshold, then adjusting the computational method of the network layer to half-precision. The method disclosed in this application can consider computational precision requirements when adjusting network layers with memory access bottlenecks, thus avoiding a decrease in the computational efficiency of the network layer.

Description

Technical Field

[0001] This application relates to the field of neural network training, and in particular to a method and apparatus for adjusting computational accuracy. Background Technology

[0002] In neural network applications, due to relatively excessive computing power and low data bandwidth, data access bottlenecks can occur. Furthermore, during high-density computing, device-level computing bottlenecks may also emerge.

[0003] Current AI acceleration hardware can provide various computational precisions, such as single-precision FP32 and integer INT32 represented by 32 bits, half-precision FP16 and integer INT16 represented by 16 bits, and FP8 and INT8 represented by 8 bits. With fewer bits representing data types, the hardware computing unit can exert greater computing power.

[0004] Currently, it's possible to improve training efficiency by adjusting the precision of certain network layers during the computation process. However, improper precision adjustment can decrease the computational efficiency of network layers, leading to a decline in training effectiveness. Summary of the Invention

[0005] In view of this, this application provides a method and apparatus for adjusting computational accuracy, aiming to solve the problem of decreased training effect caused by the network layer not adapting to the adjusted accuracy when adjusting the network layer.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] The first aspect of this application provides a method for adjusting computational precision, the method comprising:

[0008] Obtain the network layers included in the memory access bottleneck set;

[0009] Determine whether the network layer is capable of performing half-precision storage conversion;

[0010] If the network layer can perform half-precision storage conversion, then the network layer is calculated in a half-precision manner to obtain the first output result;

[0011] Determine whether the first output result is less than a preset first threshold;

[0012] If the first output result is less than the first threshold, the calculation method of the network layer is adjusted to half precision.

[0013] Optionally, the method further includes:

[0014] If the first output result is greater than or equal to a preset first threshold, the network layer is stored in a half-precision storage method and calculated in a high-precision calculation method to obtain a second output result.

[0015] Determine whether the second output result is less than a preset second threshold;

[0016] If the second output result is less than the preset second threshold, the storage method of the network layer is adjusted to half precision, and the calculation method of the network layer is adjusted to high precision.

[0017] Optionally, the method further includes:

[0018] If the second output result is greater than or equal to the preset second threshold, the calculation method of the network layer is adjusted to the original state.

[0019] Optionally, the network layers included in the process of obtaining the memory access bottleneck set include:

[0020] Obtain the first compute-to-memory access ratio of the computing device;

[0021] Obtain the second compute-to-memory ratio of the network layer to be added;

[0022] Determine whether the second calculated memory access ratio is less than the first calculated memory access ratio;

[0023] If the second calculated memory access ratio is less than the first calculated memory access ratio, then the network layer to be added is added to the memory access bottleneck set;

[0024] The network layer is obtained from the set of memory access bottlenecks.

[0025] Optionally, obtaining the first compute-memory access ratio of the computing device includes:

[0026] Obtain the computing power value of the computing device;

[0027] Obtain the transmission bandwidth value of the computing device;

[0028] Divide the computing power value by the transmission bandwidth value to obtain the first computing memory access ratio.

[0029] Another aspect of this application provides a device for adjusting calculation accuracy, the device comprising: an acquisition module, a judgment module, and a processing module;

[0030] The acquisition module is used to acquire the network layers included in the memory access bottleneck set;

[0031] The judgment module is used to determine whether the network layer can perform half-precision storage conversion;

[0032] If the network layer can be converted to half-precision storage, then the processing module is used to perform calculations on the network layer using half-precision storage and half-precision calculation to obtain a first output result.

[0033] The judgment module is also used to determine whether the first output result is less than a preset first threshold.

[0034] If the first output result is less than the first threshold, the processing module is further configured to adjust the calculation method of the network layer to half precision.

[0035] Optionally, the device is also used for:

[0036] If the first output result is greater than or equal to a preset first threshold, the processing module is further configured to store the network layer in a half-precision storage method and perform calculation in a high-precision calculation method to obtain a second output result;

[0037] The judgment module is also used to determine whether the second output result is less than a preset second threshold.

[0038] If the second output result is less than the preset second threshold, the processing module is further configured to adjust the storage method of the network layer to half precision and the calculation method of the network layer to high precision.

[0039] Optionally, the device is also used for:

[0040] If the second output result is greater than or equal to a preset second threshold, the processing module is further configured to adjust the calculation method of the network layer back to its original state.

[0041] Optionally, the device is specifically used for:

[0042] The acquisition module is also used to acquire the first computing memory access ratio of the computing device;

[0043] The acquisition module is also used to acquire the second computation-to-memory ratio of the network layer to be added;

[0044] The judgment module is further configured to determine whether the second calculated memory access ratio is less than the first calculated memory access ratio;

[0045] If the second calculated memory access ratio is less than the first calculated memory access ratio, the processing module is further configured to add the network layer to be added to the memory access bottleneck set.

[0046] The acquisition module is also used to acquire the network layer from the memory access bottleneck set.

[0047] Optionally, the device further includes: a computing module;

[0048] The acquisition module is also used to acquire the computing power value of the computing device;

[0049] The acquisition module is also used to acquire the transmission bandwidth value of the computing device;

[0050] The calculation module is used to divide the calculation capability value by the transmission bandwidth value to obtain the first calculation-to-memory ratio.

[0051] This application discloses a method and apparatus for adjusting computational precision. The method includes: obtaining network layers included in a memory access bottleneck set; determining whether the network layer can undergo half-precision memory conversion; if the network layer can undergo half-precision memory conversion, then performing computation on the network layer in a half-precision manner to obtain a first output result; determining whether the first output result is less than a preset first threshold; if the first output result is less than the preset first threshold, then adjusting the computational method of the network layer to half-precision. The method disclosed in this application can consider computational precision requirements when adjusting network layers with memory access bottlenecks, thus avoiding a decrease in the computational efficiency of the network layer. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart illustrating a calculation precision adjustment method provided in an embodiment of this application;

[0054] Figure 2 A flowchart illustrating another method for adjusting computational precision provided in this application embodiment;

[0055] Figure 3 This is a schematic diagram of a calculation accuracy adjustment device provided in an embodiment of this application;

[0056] Figure 4 A flowchart illustrating an application scenario of a calculation precision adjustment method provided in this application embodiment;

[0057] Figure 5 This is a flowchart illustrating an application scenario of a calculation accuracy adjustment method provided in an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0059] The following is combined with Figure 1 This application introduces a method for adjusting calculation accuracy, which can be implemented through the following steps S101-S105.

[0060] S101: Obtain the network layers included in the memory access bottleneck set.

[0061] Specifically, it can be derived from the network layers contained in the memory access bottleneck set, which is used to store network layers that have memory access bottlenecks.

[0062] S102: Determine whether the network layer can perform half-precision storage conversion.

[0063] Specifically, the process involves determining whether network layers identified from the memory access bottleneck set can undergo half-precision memory conversion. Taking current AI acceleration hardware, including 32-bit single-precision FP32 and integer INT32 representations, as examples, some hardware cannot perform half-precision memory conversion. Performing half-precision memory conversion on such hardware would reduce its computational power, thus affecting the computational efficiency of the network layers. This step helps identify network layers that cannot undergo half-precision memory conversion.

[0064] S103: Calculate the network layer using half-precision calculation to obtain the first output result.

[0065] Specifically, the storage and output of the network layer capable of half-precision storage conversion are calculated using half-precision FP16, and the calculation result is recorded as the first output result.

[0066] S104: Determine whether the first output result is less than the preset first threshold.

[0067] Specifically, the calculated first output result is compared with the baseline output result. If the difference is less than a preset first threshold, the network layer is considered to meet the half-precision storage and computation requirements.

[0068] Specifically, the acceleration hardware used in the baseline is single-precision FP32. It can be understood that the calculation results of the baseline can be used as standard results to determine whether the computing power of the hardware has decreased after half-precision storage conversion.

[0069] S105: Adjust the calculation method of the network layer to half precision.

[0070] Specifically, after determining that the acceleration hardware of the network layer meets the requirements for half-precision storage and computation, the computation method of the network layer is adjusted to half-precision storage and computation.

[0071] The method provided in this application embodiment can take into account the computational accuracy requirements when adjusting network layers with memory access bottlenecks, thereby avoiding a decrease in the computational efficiency of the network layers.

[0072] The following is combined with Figure 2 This application introduces another method for adjusting the calculation accuracy, which can be implemented through the following steps S201-S208.

[0073] This method takes the training task of speech recognition on Cambricon's MLU370 accelerator card as an example. Because the memory bandwidth of the MLU370 hardware is relatively lower than that of NVIDIA chips, the computation process of some network layers in the network model is limited by the memory bandwidth, resulting in low training efficiency. The computing device is the MLU370 hardware.

[0074] S201: Obtain the first compute-memory access ratio of the computing device.

[0075] Specifically, it can obtain the first compute-to-memory ratio of the MLU370 hardware.

[0076] Specifically, by obtaining the computing power value C and transmission bandwidth value M of the MLU370 hardware, the first computing-to-memory ratio is calculated using the following formula: I = C / M, where I is the first computing-to-memory ratio.

[0077] S202: Determine whether the computational accuracy of each network layer of the computing device is determined.

[0078] Specifically, it determines whether the network layers of the current computing device have undergone computational accuracy assessment.

[0079] If the computational accuracy of each network layer of the current computing device has been determined, the computational accuracy will be adjusted according to the previous computational accuracy determination results, and the process will end.

[0080] If the network layers of the current computing device have not undergone calculation accuracy determination, proceed to step S203.

[0081] S203: The second computational memory access ratio and absolute data memory access of the statistical network layer.

[0082] Specifically, a forward iteration is performed on the network layers contained in the MLU370 hardware to obtain the computational and memory access data of all network layers. Then, based on the computational and memory access data, the second computation-to-memory ratio and the absolute amount of data accessed for all network layers are calculated.

[0083] S204: Filter out the set of memory access bottlenecks.

[0084] Specifically, based on the second computation-to-memory ratio and the first computation-to-memory ratio corresponding to all network layers, it is determined whether the network layer has a memory access bottleneck. If the network layer has a memory access bottleneck, then the network layer is added to the memory access bottleneck set.

[0085] Understandably, it is possible to determine the second computation-to-memory ratio and the first computation-to-memory ratio of the network layer based on the RoofLine rule.

[0086] If the first computational memory access ratio is greater than the second computational memory access ratio, then there is a memory access bottleneck in this network layer.

[0087] If the first computation-to-memory ratio is less than the second computation-to-memory ratio, then the network layer has a computational bottleneck.

[0088] If the first computation-to-memory ratio is equal to the second computation-to-memory ratio, then there is no memory access bottleneck or computation bottleneck in this network layer.

[0089] Understandably, unfiltered network layers are referred to as network layers to be added.

[0090] Specifically, for the network layers in the memory access bottleneck set, a two-level sorting process is performed. First, they are sorted from smallest to largest by calculated memory access ratio. For network layers with the same calculated memory access ratio, they are then sorted from largest to smallest by absolute memory access volume, resulting in an ordered sequence of network layers. This allows the replacement order of network layers to be controlled by calculating the memory access ratio and absolute memory access volume, prioritizing the replacement of network layers that have a greater impact on computation speed.

[0091] S205: For all network layers in the network layer sequence, analyze each layer to determine whether half-precision storage conversion can be performed.

[0092] Specifically, the network layers in the network layer sequence are analyzed layer by layer to determine whether the network layer can be converted to half-precision storage.

[0093] If the network layer can perform half-precision storage conversion, proceed to step S207.

[0094] If the network layer cannot perform half-precision storage conversion, then continue analyzing the next network layer.

[0095] S206: Determine whether the network layer meets the half-precision requirement.

[0096] Specifically, the storage and output of the network layer capable of half-precision storage conversion are calculated using half-precision FP16, and the calculation result is recorded as the first output result.

[0097] The calculated first output result is compared with the baseline output result.

[0098] If the difference is less than the preset first threshold, the network layer is considered to meet the half-precision storage and calculation requirements. The first flag is set, and the process returns to step S205 to continue analyzing the next network layer.

[0099] It is understandable that by setting the first identifier for this network layer, the calculation method of this network layer can be adjusted to half precision based on the identifier. Furthermore, if the calculation precision needs to be adjusted later, it can be adjusted based on the identifier without the need for further judgment.

[0100] If the difference is greater than or equal to the preset first threshold, the network layer is considered not to meet the half-precision storage and calculation requirements, and the process proceeds to step S207.

[0101] S207: Determine whether the network layer meets the requirements for half-precision storage and high-precision computation.

[0102] Specifically, the network layer that can be converted to half-precision storage is stored in half-precision storage mode, and the calculation is performed in high-precision calculation mode to obtain the calculation result, which is recorded as the second output result.

[0103] The calculated second output result is compared with the baseline output result.

[0104] If the difference is less than the preset second threshold, the network layer is considered to meet the requirements of half-precision storage and high-precision calculation. The second flag is set, and the process returns to step S205 to continue analyzing the next network layer.

[0105] It is understandable that by setting a second identifier for the network layer, the storage method of the network layer can be adjusted to half precision and the calculation method of the network layer can be adjusted to high precision. Furthermore, if the calculation precision needs to be adjusted later, it can be adjusted simply by using this identifier without needing to make another judgment.

[0106] If the difference is greater than or equal to the preset second threshold, the network layer is considered not to meet the requirements of half-precision storage and high-precision calculation. The calculation method of the network layer is adjusted to the original state, and the process returns to step S205 to continue analyzing the next network layer.

[0107] Understandably, this network layer is not suitable for precision adjustment at this point. If precision adjustment is still performed on this network layer, the computational efficiency of MLU370 will be low, and the training effect will be reduced.

[0108] S208: Repeat the above steps until the calculation accuracy is stable.

[0109] Specifically, the above S201 - S207 complete the setting of the accuracy flags for each network layer in an iteration process. To avoid having to re - judge the progress every time training is performed, the steps of S201 - S207 are executed iteratively multiple times. If, after iterating a preset number of times, the flag settings of each network layer are consistent, it is considered that the accuracy of the current network layer is stable, the calculation accuracy is determined, and subsequent iterations do not need to continue with dynamic accuracy adjustment.

[0110] Through the method provided by the embodiments of this application, it is possible to adjust the calculation accuracy for the memory access bottleneck layer in the forward iteration process, and combine whether the network layer itself is suitable for calculating the accuracy requirements. While ensuring the accuracy of stable calculation results, it reduces the impact of the memory access bandwidth limit on the training efficiency, effectively accelerates the training process, and realizes the improvement of the training efficiency.

[0111] The following combines Figure 4 and Figure 5 , and introduces an application scenario example of a method for adjusting calculation accuracy provided by this application.

[0112] In the training of neural network models, currently the mainstream calculation uses a 32 - bit high - precision (FP32) data type to represent the weight parameters and training input data in the model. Assuming that the FP32 calculation ability of the computing hardware is C fp32FLOPS and the internal transfer bandwidth of the computing hardware is M GB / s, then according to the RoofLine model theory, the upper limit of the calculation intensity of this device is I = C / M. That is, for a certain calculation task, if its calculation - to - memory access ratio X > I, it means that the current task is a compute - intensive task and the bottleneck of the task is stuck in the computing process of the device; on the contrary, if the calculation - to - memory access ratio X < I, it means that the bottleneck of the current task is stuck in the memory access process of the device. For a given neural network model, assuming that the baseline of the training task is to use single - precision FP32 for model training, then the calculation accuracy of each layer of the network is adjusted according to the following scheme.

[0113] Step 1: Calculate the theoretical calculation - to - memory access ratio I of the computing device based on the theoretical data of the hardware.

[0114] Specifically, obtain the theoretical data of the computing hardware and combine it with the formula I = C / M to obtain the theoretical calculation - to - memory access ratio of the computing device. The theoretical data includes: the computing ability and internal transfer bandwidth of the computing hardware.

[0115] Step 2: Determine whether the calculation accuracy of each network layer in the current computing device has been determined.

[0116] If the calculation accuracy of each network layer has been determined, the neural network model is trained according to the set calculation accuracy and proceeds to step 8.

[0117] If the computational precision of each network layer is not determined, proceed to step 3.

[0118] Step 3: After one forward iteration, calculate the computation-to-memory ratio and the absolute memory access volume of the network for each layer.

[0119] After completing the data statistics for all network layers, based on the RoofLine rule and the theoretical memory access ratio I obtained in step 1, network layers with memory access bottlenecks are selected and added to the memory access bottleneck layer set U of the network model.

[0120] Step 4: For each network layer in set U, sort them in two levels. First, sort them in ascending order by the calculated memory access ratio X. For network layers with the same calculated memory access ratio, sort them in descending order by the absolute memory access amount to obtain an ordered network layer sequence M.

[0121] Step 5: Analyze the characteristics of each network layer according to the order in sequence M, and determine whether the network layer can be converted to low precision. If the network layer can be converted to low precision, then proceed to step 6; otherwise, continue to the next network layer.

[0122] Step 6: Try to store and compute the network layer using half-precision FP16. Calculate the error between the half-precision output value and the baseline output value. If the error is within 5%, the network layer is considered to meet the requirements for half-precision storage and computation. Set the representation flag of the network layer to I. Otherwise, proceed to step 7.

[0123] Understandably, the error range is not limited to 5%, and the specific data can be adjusted according to the actual situation.

[0124] Step 7: For the network layer, try to constrain the data in the network layer to the range of half-precision representation for storage, and use high-precision calculation for the network layer. Similarly, follow Step 6 to determine whether the calculation precision meets the requirements. If it meets the requirements, set the representation type of the network layer to half-precision storage, the calculation process to high precision, and set the flag to II. If it does not meet the requirements, proceed to Step 8.

[0125] Step 8: This network layer does not meet the half-precision representation requirement. Revert to the original precision representation and re-enter Step 5 to retrieve the next network layer for analysis and judgment.

[0126] Step 9: The above steps have completed the setting of the accuracy flags for each network layer in one iteration process. Continue the network data iteration calculation process. In order to accelerate the training calculation process and avoid the need to perform the above accuracy analysis for each calculation, which would affect the training efficiency, if the flag settings of each layer of the network are consistent for 1000 consecutive iterations, it is considered that the accuracy of the current network is stable and the calculation accuracy is determined. Subsequent iterations do not need to continue to perform dynamic accuracy adjustments.

[0127] Through the application scenarios provided in this application, the accuracy of the computation and memory access requirements of a forward iteration process can be dynamically adjusted and adapted. During the process, attention is paid to the computational performance of a single network layer, and a computation scheme that combines half-precision and high-precision storage is selected for each network layer. For the backward computation and gradient update processes, due to the high accuracy requirements, high-precision representation is maintained for computation to avoid affecting the final result.

[0128] The following is combined with Figure 3 This application introduces a device for adjusting calculation accuracy, which includes: an acquisition module 301, a judgment module 302, and a processing module 303.

[0129] The acquisition module 301 is used to acquire the network layers included in the memory access bottleneck set.

[0130] The judgment module 302 is used to determine whether the network layer can perform half-precision storage conversion.

[0131] If the network layer can be converted to half-precision storage, then the processing module 303 is used to perform calculations on the network layer using half-precision storage and half-precision calculation to obtain the first output result.

[0132] The judgment module 302 is also used to determine whether the first output result is less than a preset first threshold.

[0133] If the first output result is less than the preset first threshold, the processing module 303 is also used to adjust the calculation method of the network layer to half precision.

[0134] If the first output result is greater than or equal to the preset first threshold, the processing module 303 is further used to store the network layer in a half-precision storage method and to perform calculation in a high-precision calculation method to obtain the second output result.

[0135] The judgment module 302 is also used to determine whether the second output result is less than the preset second threshold.

[0136] If the second output result is less than the preset second threshold, the processing module 303 is further used to adjust the storage method of the network layer to half precision and the calculation method of the network layer to high precision.

[0137] If the second output result is greater than or equal to the preset second threshold, the processing module 303 is also used to adjust the calculation method of the network layer back to the original state.

[0138] The acquisition module 301 is also used to acquire the first computing memory access ratio of the computing device.

[0139] The acquisition module 301 is also used to acquire the second computation-to-memory ratio of the network layer to be added.

[0140] The judgment module 302 is also used to determine whether the second computation-to-memory ratio is less than the first computation-to-memory ratio.

[0141] If the second computed memory access ratio is less than the first computed memory access ratio, then the processing module 303 is also used to add the network layer to be added to the memory access bottleneck set.

[0142] The acquisition module 301 is also used to acquire the network layer from the memory access bottleneck set.

[0143] Optionally, the device may also include a computing module 304.

[0144] The acquisition module 301 is also used to acquire the computing power value of the computing device.

[0145] The acquisition module 301 is also used to acquire the transmission bandwidth value of the computing device.

[0146] The calculation module 304 is used to divide the calculation capability value by the transmission bandwidth value to obtain the first calculation-to-memory ratio.

[0147] The apparatus provided in this application embodiment can take into account the computational accuracy requirements when adjusting a network layer with a memory access bottleneck, thereby avoiding a decrease in the computational efficiency of the network layer.

[0148] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0149] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0150] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical business division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] Furthermore, the various business units in the embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software business unit.

[0153] If the integrated unit is implemented as a software business unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0154] Those skilled in the art will recognize that, in one or more of the examples above, the services described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these services can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. Storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0155] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention.

[0156] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for adjusting calculation accuracy, characterized in that, The method is applied to a neural network model for speech recognition, including: Obtain the network layers included in the memory access bottleneck set; Determine whether the network layer is capable of performing half-precision storage conversion; If the network layer can perform half-precision storage conversion, then the network layer is calculated in a half-precision manner to obtain the first output result; Determine whether the first output result is less than a preset first threshold; If the first output result is less than the first threshold, the calculation method of the network layer is adjusted to half precision; The network layers included in the set of memory access bottlenecks are: Obtain the first compute-to-memory access ratio of the computing device; Obtain the computational and memory access data for the network layer to be added; Based on the calculated data and the memory access data, the second calculation-to-memory ratio and the absolute memory access volume of the network layer to be added are calculated. Determine whether the second calculated memory access ratio is less than the first calculated memory access ratio based on the contour line rule; If the second calculated memory access ratio is less than the first calculated memory access ratio, then the network layer to be added is added to the memory access bottleneck set; The network layers are obtained from the memory access bottleneck set. The network layers in the memory access bottleneck set are sorted in ascending order of calculated memory access ratio. For network layers with the same calculated memory access ratio, they are sorted in descending order of absolute data access volume.

2. The method according to claim 1, characterized in that, The method further includes: If the first output result is greater than or equal to a preset first threshold, the network layer is stored in a half-precision storage method and calculated in a high-precision calculation method to obtain a second output result. Determine whether the second output result is less than a preset second threshold; If the second output result is less than the preset second threshold, the storage method of the network layer is adjusted to half precision, and the calculation method of the network layer is adjusted to high precision.

3. The method according to claim 2, characterized in that, The method further includes: If the second output result is greater than or equal to the preset second threshold, the calculation method of the network layer is adjusted to the original state.

4. The method according to claim 1, characterized in that, The acquisition of the first compute-memory access ratio of the computing device includes: Obtain the computing power value of the computing device; Obtain the transmission bandwidth value of the computing device; Divide the computing power value by the transmission bandwidth value to obtain the first computing memory access ratio.

5. A device for adjusting calculation accuracy, characterized in that, The device is applied to a neural network model for speech recognition and includes: an acquisition module, a judgment module, and a processing module; The acquisition module is used to acquire the network layers included in the memory access bottleneck set; The judgment module is used to determine whether the network layer can perform half-precision storage conversion; If the network layer can be converted to half-precision storage, then the processing module is used to perform calculations on the network layer using half-precision storage and half-precision calculation to obtain a first output result. The judgment module is also used to determine whether the first output result is less than a preset first threshold. If the first output result is less than the first threshold, the processing module is further configured to adjust the calculation method of the network layer to half precision. The device is specifically used for: The acquisition module is also used to acquire the first computing memory access ratio of the computing device; The acquisition module is further configured to acquire the computation data and memory access data of the network layer to be added; and to calculate the second computation-to-memory ratio and absolute memory access amount of the network layer to be added based on the computation data and the memory access data. The judgment module is also used to determine whether the second calculated memory access ratio is less than the first calculated memory access ratio based on the contour line rules; If the second calculated memory access ratio is less than the first calculated memory access ratio, the processing module is further configured to add the network layer to be added to the memory access bottleneck set. The acquisition module is further configured to acquire the network layer from the memory access bottleneck set, wherein the network layers in the memory access bottleneck set are sorted in ascending order of calculated memory access ratio, and for network layers with the same calculated memory access ratio, they are sorted in descending order of absolute data access volume.

6. The apparatus according to claim 5, characterized in that, The device is also used for: If the first output result is greater than or equal to a preset first threshold, the processing module is further configured to store the network layer in a half-precision storage method and perform calculation in a high-precision calculation method to obtain a second output result; The judgment module is also used to determine whether the second output result is less than a preset second threshold. If the second output result is less than the preset second threshold, the processing module is further configured to adjust the storage method of the network layer to half precision and the calculation method of the network layer to high precision.

7. The apparatus according to claim 6, characterized in that, The device is also used for: If the second output result is greater than or equal to a preset second threshold, the processing module is further configured to adjust the calculation method of the network layer back to its original state.

8. The apparatus according to claim 5, characterized in that, The device further includes: a computing module; The acquisition module is also used to acquire the computing power value of the computing device; The acquisition module is also used to acquire the transmission bandwidth value of the computing device; The calculation module is used to divide the calculation capability value by the transmission bandwidth value to obtain the first calculation-to-memory ratio.

Images