Model migration accuracy loss measurement method and system
By calculating the similarity of data distribution and the fitting relationship before and after model transfer, this method solves the problems of high deployment cost and inaccurate results in neural network model transfer, and provides a more accurate, safe and low-cost method for measuring the accuracy loss of model transfer.
Patent Information
- Application Number
- CN202411041588.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2026-02-03
AI Technical Summary
Existing neural network model transfer techniques suffer from high deployment costs and inaccurate results, especially when transferring between different smart chips, where it is difficult to accurately predict the loss of accuracy.
By calculating the data distribution similarity of parameters at each layer before and after model transfer, and using weighted average and fitting formulas, the accuracy loss of model transfer is predicted. JS divergence is used as a measure of data distribution similarity, and pseudo-quantization and dequantization are performed at different precision levels.
It enables accurate and secure prediction of model migration accuracy loss at different precision levels, reduces deployment costs, helps select appropriate migration precision and chips, and ensures model performance and security.
Smart Images
Figure CN121456493A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of neural network technology, and in particular to a method, system, storage medium, and computer device for measuring the accuracy loss of model transfer. Background Technology
[0002] Existing model transfer techniques for neural networks (NNs) primarily involve directly deploying the model onto a new smart chip or using a simulator to perform forward inference and predict the accuracy loss during model transfer. Direct deployment is simple and straightforward, allowing the model to directly perform forward inference to assess the accuracy loss. Simulator-based methods involve developing a simulator with functionality identical to the smart chip to be deployed, and then performing forward inference on the simulator to evaluate the accuracy loss.
[0003] Direct deployment of model transfer involves high deployment costs, including the development and optimization of supporting software such as high-performance deep learning operator libraries and compilers. Furthermore, differences in hardware characteristics among different intelligent chips, such as computational precision, order, and data truncation, directly affect the computational results after transfer, thus impacting the model's accuracy and security. Therefore, arbitrarily transferring models to other intelligent chips is impractical. Simulation methods suffer from high simulator development costs and inaccurate output results; moreover, simulators generally do not output model accuracy, making them unsuitable for predicting model transfer accuracy. Therefore, there is a need to find a lower-cost, more accurate technique to predict the accuracy loss during model transfer.
[0004] In conclusion, the existing technology obviously has inconveniences and defects in practical use, so it is necessary to improve it. Summary of the Invention
[0005] To address the aforementioned shortcomings, the present invention aims to provide a method, system, storage medium, and computer device for measuring accuracy loss in model transfer, which can predict accuracy loss more accurately and securely, and at a lower cost.
[0006] To solve the above-mentioned technical problems, the present invention is implemented as follows:
[0007] In a first aspect, embodiments of the present invention provide a method for measuring the accuracy loss of model transfer, comprising:
[0008] Calculate the single-layer data distribution similarity of parameters of each layer before and after the transfer of multiple sets of models with different accuracies;
[0009] Based on the weights corresponding to each layer in the model, the weighted average of the single-layer data distribution similarity of each layer is calculated to obtain multiple sets of first overall data distribution similarity as a measure of first accuracy loss;
[0010] The similarity of the first overall data distribution and the first accuracy loss of multiple groups are fitted together to obtain the fitting relationship between the two.
[0011] Based on the fitting relationship, the second accuracy loss corresponding to the second overall data distribution similarity at the current accuracy is calculated.
[0012] According to the model transfer accuracy loss measurement method of the present invention, the step of calculating the single-layer data distribution similarity of the parameters of each layer before and after the transfer of multiple sets of models with different accuracies includes:
[0013] Calculate the JS divergence of each layer parameter before and after migration for multiple sets of models with different accuracies, and use the JS divergence as the similarity of the single-layer data distribution.
[0014] According to the model transfer accuracy loss measurement method of the present invention, the step of calculating the single-layer data distribution similarity of the parameters of each layer before and after the transfer of multiple sets of models with different accuracies includes:
[0015] Train a full-precision model to obtain the parameters of each layer before model transfer;
[0016] Simulate the parameters of each layer after model transfer at different precision levels;
[0017] Based on the parameters of each layer before and after the model migration, calculate the single-layer data distribution similarity of the parameters of each layer before and after the model migration for multiple sets of models with different accuracies.
[0018] According to the model transfer accuracy loss measurement method of the present invention, the step of simulating the parameters after model transfer at different accuracies includes:
[0019] The model is pseudo-quantized at different precisions to obtain the parameters of each layer after the pseudo-quantized model is transferred.
[0020] The pseudo-quantized parameters of each layer of the model after transfer are dequantized to obtain the corresponding dequantized parameters of each layer of the model after transfer.
[0021] The step of calculating the single-layer data distribution similarity of multiple sets of model parameters before and after migration at different accuracies, based on the parameters of each layer before and after model migration, includes:
[0022] Based on the parameters of each layer before model migration and the parameters of each layer before model migration after dequantization, calculate the single-layer data distribution similarity of multiple sets of model parameters before and after migration at different accuracies.
[0023] According to the model transfer accuracy loss measurement method of the present invention, the step of dequantizing the pseudo-quantized parameters of each layer of the transferred model includes:
[0024] The parameters of each layer of the pseudo-quantized model after migration are dequantized using a predetermined dequantization formula, which is as follows:
[0025] r = S(qZ)
[0026]
[0027] Where r represents a floating-point real number, q represents a quantized fixed-point integer, and q max q min These are the maximum and minimum values of q, and r. max r min Similarly, S represents the ratio between floating-point numbers and fixed-point integers, and Z represents the integer corresponding to 0 in a floating-point real number after quantization.
[0028] According to the model transfer accuracy loss measurement method of the present invention, the step of fitting multiple sets of the first overall data distribution similarity and the first accuracy loss to obtain the fitting relationship between the two includes:
[0029] By fitting the similarity of the first overall data distribution across multiple groups with the first accuracy loss, the fitting relationship between the two is obtained as follows:
[0030]
[0031] In the formula, y represents the model accuracy loss, x represents the data distribution similarity, and p and q are constants. The fitting relationship described is a formulaic form found to best fit the relationship between the model accuracy loss and the data distribution similarity, used for curve fitting between the two.
[0032] According to the model transfer accuracy loss measurement method of the present invention, the step of calculating the second accuracy loss corresponding to the second overall data distribution similarity at the current accuracy based on the fitting relationship includes:
[0033] Calculate the single-layer data distribution similarity of parameters of each layer before and after the transfer of multiple models with the current accuracy;
[0034] Based on the weights corresponding to each layer in the model, the weighted average of the single-layer data distribution similarity of each layer is calculated to obtain the multiple sets of second overall data distribution similarity.
[0035] Based on the fitting relationship, the second accuracy loss corresponding to the second overall data distribution similarity at the current accuracy is calculated.
[0036] According to the model transfer accuracy loss measurement method of the present invention, the weights corresponding to each layer in the model are the proportions of the computational cost of each layer in the model to the total computational cost.
[0037] Secondly, embodiments of the present invention provide a model transfer accuracy loss measurement system, comprising:
[0038] The similarity calculation module is used to calculate the single-layer data distribution similarity of parameters of each layer before and after the migration of multiple sets of models with different precisions.
[0039] The weighted calculation module is used to calculate the weighted average of the single-layer data distribution similarity of each layer according to the weights corresponding to each layer in the model, and obtain multiple sets of first overall data distribution similarity as a measure of first accuracy loss;
[0040] The mathematical fitting module is used to fit the similarity of the distribution of the first overall data and the first accuracy loss of multiple groups to obtain the fitting relationship between the two.
[0041] The loss calculation module is used to calculate the second accuracy loss corresponding to the second overall data distribution similarity at the current accuracy, based on the fitting relationship.
[0042] Thirdly, embodiments of the present invention provide a storage medium for storing a computer program for performing the accuracy loss measurement method for model transfer as described in any of the preceding claims.
[0043] Fourthly, embodiments of the present invention provide an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement the accuracy loss measurement method for model transfer as described above.
[0044] This invention provides a technique for measuring the accuracy loss of model transfer. It calculates the similarity of single-layer data distributions of parameters at each layer before and after model transfer for multiple sets of models with different accuracies, and performs a weighted average of these single-layer data distribution similarities to obtain a first overall data distribution similarity as a measure of the first accuracy loss. The first overall data distribution similarity and the first accuracy loss are then fitted together to obtain a fitting relationship. A second overall data distribution similarity at the current accuracy is obtained, and based on the fitting relationship, a second accuracy loss corresponding to the second overall data distribution similarity is calculated. This invention employs a simulation prediction method that uses data distribution similarity as a measure of accuracy loss, which, compared to existing direct deployment solutions, eliminates the need for expensive deployment costs. Furthermore, this invention can accurately and safely reflect the accuracy loss of model transfer at various accuracies, helping model users find suitable transfer accuracies and select more accurate transfer models and intelligent chips. Therefore, this invention can predict the accuracy loss of model transfer more accurately and safely, and at a lower cost. Attached Figure Description
[0045] Figure 1 This is a flowchart illustrating the accuracy loss measurement method for model transfer in Embodiment 1 of the present invention.
[0046] Figure 2 This is a flowchart illustrating the accuracy loss measurement method for model transfer in Embodiment 2 of the present invention.
[0047] Figure 3 This is a schematic diagram of the accuracy loss measurement system for model transfer in Embodiment 1 of the present invention;
[0048] Figure 4 This is a schematic diagram of the accuracy loss measurement system for model transfer in Embodiment 2 of the present invention;
[0049] Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation
[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0051] It should be noted that references to "an embodiment," "embodiment," "example embodiment," etc., in this specification refer to the described embodiment including specific features, structures, or characteristics, but not every embodiment must include these specific features, structures, or characteristics. Furthermore, such expressions do not refer to the same embodiment. Moreover, when describing specific features, structures, or characteristics in conjunction with embodiments, whether or not explicitly described, it is indicated that incorporating such features, structures, or characteristics into other embodiments is within the knowledge of those skilled in the art.
[0052] Furthermore, certain terms are used in the specification and subsequent claims to refer to specific components or parts. Those skilled in the art will understand that manufacturers may use different names or terms to refer to the same component or part. This specification and subsequent claims do not distinguish components or parts by differences in name, but rather by differences in function. The terms "comprising" and "including" used throughout the specification and subsequent claims are open-ended and should be interpreted as "including but not limited to." Additionally, the term "connection" here includes any direct and indirect electrical connection means. Indirect electrical connection means include connections made through other means.
[0053] The following description, in conjunction with the accompanying drawings, details the method for measuring the accuracy loss of model transfer provided by the embodiments of the present invention through specific implementations and application scenarios.
[0054] In implementing an accuracy prediction method for neural network model transfer, this invention observes and analyzes the parameters before and after model transfer, ultimately arriving at two conclusions: First, the data distribution of parameters in each layer of the deep learning model is regular, approximating a normal distribution. Second, after migrating the model to different hardware, the data distribution of model parameters changes to varying degrees due to differences in hardware characteristics; the lower the similarity of the data distribution of parameters before and after model transfer, the greater the accuracy loss. Based on these two observations, this invention conceives a method for measuring the accuracy loss of model transfer, specifically based on a pre-deployment prediction method. This involves simulating the parameters after model transfer, statistically analyzing the similarity of the data distribution of parameters in each layer before and after model transfer, and obtaining a weighted average of the similarity of the data distribution in each layer to obtain a predicted measure of the overall accuracy loss of the model.
[0055] Figure 1 This is a flowchart illustrating the accuracy loss measurement method for model transfer in Embodiment 1 of the present invention, the method comprising:
[0056] Step S101: Calculate the single-layer data distribution similarity of parameters of each layer before and after the migration of multiple sets of models with different accuracies.
[0057] Preferably, the JS (Jensen Shannon) divergence of each layer parameter before and after migration is calculated for multiple sets of models with different accuracies, and the JS divergence is used as the similarity of single-layer data distribution.
[0058] Step S102: Based on the weights of each layer in the model, perform a weighted average calculation on the single-layer data distribution similarity of each layer to obtain multiple sets of first overall data distribution similarities as a measure of the first accuracy loss.
[0059] Preferably, the weights corresponding to each layer in the model are the proportions of the computational cost of each layer in the model to the total computational cost.
[0060] Step S103: Perform data fitting on the similarity of the first overall data distribution and the first accuracy loss of multiple groups to obtain the fitting relationship between the two.
[0061] Step S104: Calculate the second accuracy loss corresponding to the second overall data distribution similarity at the current accuracy level, based on the fitted relationship.
[0062] To address the high cost of evaluating accuracy loss after model transfer in practical neural network deployments, this invention proposes a method for measuring model transfer accuracy loss. This method employs a weighted average of the data distribution similarity of inter-layer parameters before and after model transfer to predict accuracy loss in advance. This invention uses a simulation-based prediction method that measures accuracy loss using data distribution similarity, eliminating the need for expensive deployment compared to existing direct deployment solutions. Furthermore, this invention accurately and reliably reflects the accuracy loss of model transfer at various precision levels, helping model users find suitable transfer precision, select more accurate transfer models and intelligent chips, and ensure model performance and security. Therefore, this invention can predict model transfer accuracy loss more accurately and safely at a lower cost.
[0063] Figure 2 This is a flowchart illustrating the accuracy loss measurement method for model transfer in Embodiment 2 of the present invention. The method includes:
[0064] Step S201: Train the full-precision model and obtain the parameters of each layer before model transfer.
[0065] Step S202: Simulate the parameters of each layer after model transfer at different precision levels.
[0066] Preferably, the model is pseudo-quantized at different precision levels to obtain the parameters of each layer after the pseudo-quantized model is transferred.
[0067] The parameters of each layer after the pseudo-quantized model transfer are dequantized to obtain the corresponding dequantized parameters of each layer after the model transfer. Preferably, the dequantization formula is:
[0068] r = S(qZ)
[0069]
[0070] Where r represents a floating-point real number, q represents a quantized fixed-point integer, and q max q min These are the maximum and minimum values of q, and r. max r min Similarly, S represents the ratio between floating-point numbers and fixed-point integers, and Z represents the integer corresponding to 0 in a floating-point real number after quantization.
[0071] Step S203: Based on the parameters of each layer before and after model migration, calculate the single-layer data distribution similarity of the parameters of each layer before and after model migration for multiple sets of models with different accuracies.
[0072] Preferably, the JS divergence of each layer parameter before and after migration is calculated for multiple sets of models with different accuracies, and the JS divergence is used as the similarity of single-layer data distribution.
[0073] Preferably, based on the parameters of each layer before model migration and the parameters of each layer before model migration after dequantization, the single-layer data distribution similarity of multiple sets of model parameters before and after migration at different accuracies is calculated.
[0074] Specifically, to obtain the data distribution similarity before and after model migration, it is necessary to obtain the parameters of each layer of the model before and after migration. The model before migration is a full-precision model. Since the model cannot be practically deployed, the parameters of each layer of the model after migration need to be obtained through a pseudo-quantization algorithm. After obtaining the parameters of each layer of the dequantized model before migration, it is necessary to calculate the data distribution similarity of the corresponding model parameters. In order to ensure that the parameter values of the pseudo-quantized model and the full-precision model are within the same range, the parameters of each layer of the dequantized model before migration need to be dequantized.
[0075] After obtaining the parameters of each layer of the model before the model transfer after dequantization, the similarity of their data distributions is calculated with that of the parameters of each layer before the model transfer, preferably by calculating the JS divergence between the two.
[0076] Step S204: Based on the weights of each layer in the model, calculate the weighted average of the single-layer data distribution similarity of each layer to obtain multiple sets of first overall data distribution similarities as a measure of the first accuracy loss.
[0077] Preferably, the weights corresponding to each layer in the model are the proportions of the computational cost of each layer to the total computational cost. After obtaining the data distribution similarity of each single layer, the weights are used to sum the data distribution similarities of each single layer to obtain the overall data distribution similarity. This data is then used as a measure of the accuracy loss before and after model transfer. The model accuracy loss needs to be calculated under the pseudo-quantized precision.
[0078] Step S205: Perform data fitting on the similarity of the first overall data distribution and the first accuracy loss of multiple groups to obtain the fitting relationship between the two.
[0079] Preferably, the similarity of the first overall data distribution and the first accuracy loss of multiple groups are fitted together to obtain the fitting relationship between the two as follows:
[0080]
[0081] In the formula, y represents the model accuracy loss, x represents the data distribution similarity, and p and q are constants. The fitting relationship described is a formulaic form found to best fit the relationship between the model accuracy loss and the data distribution similarity, used for curve fitting between the two.
[0082] Since multiple sets of data are needed when fitting the model data distribution similarity and model accuracy loss, the above operations of calculating the model data distribution similarity and determining the corresponding model accuracy loss need to be repeated at multiple different pseudo-quantization nodes to obtain multiple sets of relational data. Then, mathematical fitting needs to be performed on the numerical values of model data distribution similarity and model accuracy loss.
[0083] After determining the rational function model to be fitted and multiple sets of corresponding data, the fit function of Matlab (Matrix Laboratory) can be used to perform mathematical fitting to obtain the fitting relationship between the similarity of the model data distribution and the model accuracy loss.
[0084] Step S206: Calculate the single-layer data distribution similarity of parameters of each layer before and after the migration of multiple sets of models with the current accuracy.
[0085] Step S207: Based on the weights corresponding to each layer in the model, perform a weighted average calculation of the single-layer data distribution similarity of each layer to obtain multiple sets of second overall data distribution similarities.
[0086] Step S208: Calculate the second accuracy loss corresponding to the second overall data distribution similarity at the current accuracy level, based on the fitted relationship.
[0087] After obtaining the fitting relationship, the accuracy loss of other quantization nodes of the model can be predicted. The method is to calculate the overall data distribution similarity of the model as described above, and then find the corresponding model accuracy loss value in the fitting image after obtaining the data distribution similarity value, or calculate the model accuracy loss value according to the fitting relationship.
[0088] The technical solution of Embodiment 2 provided by the present invention includes the following key points:
[0089] Key Point 1: The similarity of data distributions before and after model transfer is used as a measure of accuracy loss, and JS divergence is used as a measure of data distribution similarity. When choosing a measure of data distribution similarity, JS (Jensen Shannon) divergence, derived from KL (Kullback Leibler) divergence, was ultimately chosen as the measure. This measure has the following advantages:
[0090] 1. Symmetry: JS divergence can measure the similarity between two data distributions simultaneously, regardless of their order.
[0091] 2. Normalization: The value of JS divergence ranges from 0 to 1, which makes its results easy to interpret and compare. The closer the value is to 0, the higher the similarity between the two distributions.
[0092] 3. Smoothness: JS divergence is calculated by normalizing the average value of KL divergence. This averaging method helps to reduce errors caused by noise or uncertainty in the data distribution.
[0093] Using the similarity of data distribution before and after model transfer can effectively measure the accuracy loss of a single layer after model transfer. Using JS divergence as a measure of data distribution similarity also has advantages such as symmetry, normalization, and smoothness.
[0094] Key Point 2: Regarding the overall data distribution similarity of the model, this invention proposes a weighted average method for the data distribution similarity of a single layer based on the importance of each layer, thus obtaining the overall data distribution similarity before and after model transfer. Furthermore, the proportion of computational cost within a layer to the total computational cost is used as a weight to measure the importance of each layer, rather than selecting the number of parameters in each layer as the metric.
[0095] The overall data distribution similarity before and after model transfer, obtained by weighting the computational cost, is a good measure of the accuracy loss during model transfer. The tool in this invention has good prediction performance in models such as AlexNet and ResNet using the CIFAR100 dataset.
[0096] Key Point 3: In the process of fitting the data distribution similarity-accuracy loss, it is necessary to determine a good fitting relationship. Among the enumeration of various common relationships, this invention selects the best-performing fitting relationship:
[0097]
[0098] The above fitting formula has the best fitting effect. After fitting the AlexNet and VGG_19 models with the CIFAR100 dataset, the R2 values are 0.9732 and 0.9818, respectively, which are very close to 1, indicating a good fitting effect.
[0099] In general, the method for predicting model accuracy loss in Embodiment 2 of this invention is as follows: The model is pseudo-quantized at different accuracies, and the inverse quantization values of the parameters in each layer of the model after pseudo-quantization are calculated. Then, the data distribution similarity with the parameters of the full-accuracy model is calculated. After obtaining the single-layer data distribution similarity of each layer, the single-layer data distribution similarities of each layer are weighted and summed according to the proportion of computational cost of each layer to obtain the overall data distribution similarity. Then, the accuracy loss of the model at the current accuracies is calculated. Multiple sets of corresponding overall data distribution similarities and model accuracy losses are mathematically fitted to derive their mathematical relationship. When predicting the model accuracy loss, simply substitute the calculated overall data distribution similarity into the mathematical relationship to calculate the corresponding model accuracy loss.
[0100] It should be noted that the accuracy loss measurement method for model transfer provided in this embodiment of the invention can be executed by an electronic device, a model transfer accuracy loss measurement system, or a control module within the model transfer accuracy loss measurement system for executing the model transfer accuracy loss measurement method. This embodiment of the invention uses the execution of the model transfer accuracy loss measurement method by the model transfer accuracy loss measurement system as an example to illustrate the model transfer accuracy loss measurement system provided in this embodiment of the invention.
[0101] Figure 3 This is a schematic diagram of the model transfer accuracy loss measurement system 100 according to Embodiment 1 of the present invention. The model transfer accuracy loss measurement system 100 includes at least a similarity calculation module 10, a weighted calculation module 20, a mathematical fitting module 30, and a loss calculation module 40, wherein:
[0102] The similarity calculation module 10 is used to calculate the single-layer data distribution similarity of parameters of each layer before and after migration for multiple sets of models with different accuracies. Preferably, the JS divergence of parameters of each layer before and after migration for multiple sets of models with different accuracies is calculated, and the JS divergence is used as the single-layer data distribution similarity.
[0103] The weighted calculation module 20 is used to calculate the weighted average of the single-layer data distribution similarity of each layer according to the weights corresponding to each layer in the model, and obtain multiple sets of first overall data distribution similarities as a measure of the first accuracy loss. Preferably, the weights corresponding to each layer in the model are the proportions of the computational amount of each layer in the model to the total computational amount.
[0104] The mathematical fitting module 30 is used to fit the similarity of the distribution of multiple sets of first overall data and the first accuracy loss to obtain the fitting relationship between the two.
[0105] The loss calculation module 40 is used to calculate the second accuracy loss corresponding to the second overall data distribution similarity of the current accuracy based on the fitting relationship.
[0106] Figure 4 This is a schematic diagram of the model transfer accuracy loss measurement system 100 in Embodiment 2 of the present invention. The model transfer accuracy loss measurement system 100 includes at least a similarity calculation module 10, a weighted calculation module 20, a mathematical fitting module 30, and a loss calculation module 40, wherein:
[0107] The similarity calculation module 10 is used to calculate the single-layer data distribution similarity of parameters at each layer before and after the migration of multiple sets of models with different accuracies. Preferably, it calculates the JS divergence of parameters at each layer before and after the migration of multiple sets of models with different accuracies, and uses the JS divergence as the single-layer data distribution similarity. Preferably, the similarity calculation module 10 further includes:
[0108] Training submodule 11 is used to train the full-precision model and obtain the parameters of each layer before model transfer.
[0109] The simulation submodule 12 is used to simulate the parameters of each layer after model transfer at different accuracies. Preferably, the model is pseudo-quantized at different accuracies to obtain the pseudo-quantized parameters of each layer after model transfer; and the pseudo-quantized parameters of each layer after model transfer are dequantized to obtain the corresponding dequantized parameters of each layer after model transfer. Preferably, the dequantization formula is:
[0110] r = S(qZ)
[0111]
[0112] Where r represents a floating-point real number, q represents a quantized fixed-point integer, and q max q min These are the maximum and minimum values of q, and r. max r min Similarly, S represents the ratio between floating-point numbers and fixed-point integers, and Z represents the integer corresponding to 0 in a floating-point real number after quantization.
[0113] The first calculation submodule 13 is used to calculate the single-layer data distribution similarity of multiple sets of model parameters before and after model migration at different accuracies, based on the parameters of each layer before model migration and the parameters of each layer after model migration. Preferably, the single-layer data distribution similarity of multiple sets of model parameters before and after model migration at different accuracies is calculated based on the parameters of each layer before model migration and the parameters of each layer before model migration after dequantization.
[0114] The weighted calculation module 20 is used to calculate the weighted average of the single-layer data distribution similarity of each layer according to the weights corresponding to each layer in the model, and obtain multiple sets of first overall data distribution similarities as a measure of the first accuracy loss. Preferably, the weights corresponding to each layer in the model are the proportions of the computational amount of each layer in the model to the total computational amount.
[0115] The mathematical fitting module 30 is used to fit the similarity of the first overall data distribution and the first accuracy loss of multiple sets of data to obtain a fitting relationship between the two. Preferably, the fitting relationship is:
[0116]
[0117] In the formula, y represents the model accuracy loss, x represents the data distribution similarity, and p and q are constants. The fitting relationship described is a formulaic form found to best fit the relationship between the model accuracy loss and the data distribution similarity, used for curve fitting between the two.
[0118] The loss calculation module 40 is used to calculate the second accuracy loss corresponding to the second overall data distribution similarity at the current accuracy, based on the fitted relationship. Preferably, the loss calculation module 40 includes:
[0119] The second calculation submodule 41 is used to calculate the single-layer data distribution similarity of parameters of each layer before and after the migration of multiple sets of models with the current accuracy.
[0120] The weighted submodule 42 is used to calculate the weighted average of the single-layer data distribution similarity of each layer according to the weights corresponding to each layer in the model, so as to obtain multiple sets of second overall data distribution similarities.
[0121] The third calculation submodule 43 calculates the second accuracy loss corresponding to the second overall data distribution similarity at the current accuracy based on the fitting relationship.
[0122] The model transfer accuracy loss measurement system provided in this embodiment of the invention can achieve Figures 1-2 The various processes implemented in the example of the accuracy loss measurement method for model transfer shown will not be repeated here to avoid duplication.
[0123] The model transfer accuracy loss measurement system provided in this invention calculates the single-layer data distribution similarity of parameters at each layer before and after model transfer for multiple sets of models with different accuracies. It then performs a weighted average of these single-layer data distribution similarities to obtain multiple sets of first overall data distribution similarities as a measure of first accuracy loss. The system then fits these first overall data distribution similarities and the first accuracy loss to obtain a fitting relationship. Finally, it obtains a second overall data distribution similarity for the current accuracy and calculates the second accuracy loss corresponding to this second overall data distribution similarity based on the fitting relationship. This invention employs a simulation prediction method that uses data distribution similarity as a measure of accuracy loss, which, compared to existing direct deployment schemes, eliminates the need for expensive deployment costs. Furthermore, this invention can accurately and safely reflect the accuracy loss of model transfer at various accuracies, helping model users find suitable transfer accuracies and select more accurate transfer models and intelligent chips. Therefore, this invention can predict the accuracy loss of model transfer more accurately and safely, and at a lower cost.
[0124] The present invention also provides a storage medium for storing, for example, Figures 1-2 A computer program for measuring the accuracy loss of any model transfer method. For example, computer program instructions, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the present invention through the operation of the computer, achieving the same technical effect; to avoid repetition, these will not be elaborated further here. The program instructions for invoking the methods of the present invention may be stored in a fixed or removable storage medium, and / or transmitted via data streams in broadcast or other signal carrying media, and / or stored in the storage medium of a computer device operating according to the program instructions.
[0125] According to one embodiment of the present invention, the present invention also provides such a Figure 5The illustrated electronic device 400 may optionally include a storage medium 200 for storing a computer program and a processor 300 for executing the computer program. When the computer program is executed by the processor 300, it implements any of the aforementioned model transfer accuracy loss measurement methods, triggering the electronic device 400 to execute methods and / or technical solutions based on the foregoing embodiments, achieving the same technical effect. To avoid repetition, these methods will not be elaborated upon here. It should be noted that the electronic devices in this embodiment include mobile electronic devices and non-mobile electronic devices. For example, mobile electronic devices may be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, super mobile personal computers, netbooks, or personal digital assistants, etc., while non-mobile electronic devices may be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This embodiment does not specifically limit the scope of the invention.
[0126] It should be noted that the present invention can be implemented in software and / or a combination of software and hardware, for example, using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In one embodiment, the software program of the present invention can be executed by a processor to implement the steps or functions described above. Similarly, the software program of the present invention (including associated data structures) can be stored in a computer-readable recording medium, such as RAM memory, a magnetic or optical drive, a floppy disk, or similar devices. Furthermore, some steps or functions of the present invention can be implemented in hardware, for example, as circuitry that works with a processor to perform the various steps or functions.
[0127] This invention can be implemented on a computer as a computer-based method, or in dedicated hardware, or a combination of both. Executable code or portions thereof for the method according to the invention can be stored on a computer program product. Examples of computer program products include memory devices, optical storage devices, integrated circuits, servers, online software, etc.
Claims
1. A method for measuring the accuracy loss of model transfer, characterized in that, include: Calculate the single-layer data distribution similarity of parameters of each layer before and after the transfer of multiple sets of models with different accuracies; Based on the weights corresponding to each layer in the model, the weighted average of the single-layer data distribution similarity of each layer is calculated to obtain multiple sets of first overall data distribution similarity as a measure of first accuracy loss; The similarity of the first overall data distribution and the first accuracy loss of multiple groups are fitted together to obtain the fitting relationship between the two. Based on the fitting relationship, the second accuracy loss corresponding to the second overall data distribution similarity at the current accuracy is calculated.
2. The method for measuring the accuracy loss of model transfer according to claim 1, characterized in that, The steps for calculating the single-layer data distribution similarity of parameters of each layer before and after the migration of multiple sets of models with different accuracies include: Calculate the JS divergence of each layer parameter before and after migration for multiple sets of models with different accuracies, and use the JS divergence as the similarity of the single-layer data distribution.
3. The method for measuring the accuracy loss of model transfer according to claim 1, characterized in that, The steps for calculating the single-layer data distribution similarity of parameters of each layer before and after the migration of multiple sets of models with different accuracies include: Train a full-precision model to obtain the parameters of each layer before model transfer; Simulate the parameters of each layer after model transfer at different precision levels; Based on the parameters of each layer before and after the model migration, calculate the single-layer data distribution similarity of the parameters of each layer before and after the model migration for multiple sets of models with different accuracies.
4. The method for measuring the accuracy loss of model transfer according to claim 3, characterized in that, The steps for simulating the parameters after model transfer at different accuracies include: The model is pseudo-quantized at different precisions to obtain the parameters of each layer after the pseudo-quantized model is transferred. The pseudo-quantized parameters of each layer of the model after transfer are dequantized to obtain the corresponding dequantized parameters of each layer of the model after transfer. The step of calculating the single-layer data distribution similarity of multiple sets of model parameters before and after migration at different accuracies, based on the parameters of each layer before and after model migration, includes: Based on the parameters of each layer before model migration and the parameters of each layer before model migration after dequantization, calculate the single-layer data distribution similarity of multiple sets of model parameters before and after migration at different accuracies.
5. The method for measuring the accuracy loss of model transfer according to claim 4, characterized in that, The step of dequantizing the parameters of each layer of the model after pseudo-quantization includes: The parameters of each layer of the pseudo-quantized model after migration are dequantized using a predetermined dequantization formula, which is as follows: r = S(qZ) Where r represents a floating-point real number, q represents a quantized fixed-point integer, and q max q min These are the maximum and minimum values of q, and r. max r min Similarly, S represents the ratio between floating-point numbers and fixed-point integers, and Z represents the integer corresponding to 0 in a floating-point real number after quantization.
6. The method for measuring the accuracy loss of model transfer according to claim 1, characterized in that, The step of fitting the similarity of the distribution of the first overall data across multiple groups and the first accuracy loss to obtain the fitting relationship between the two includes: By fitting the similarity of the first overall data distribution across multiple groups with the first accuracy loss, the fitting relationship between the two is obtained as follows: In the formula, y is the model accuracy loss, x is the data distribution similarity, and p and q are constants in the formula.
7. The method for measuring the accuracy loss of model transfer according to claim 1, characterized in that, The step of calculating the second accuracy loss corresponding to the second overall data distribution similarity at the current accuracy based on the fitting relationship includes: Calculate the single-layer data distribution similarity of parameters of each layer before and after the transfer of multiple models with the current accuracy; Based on the weights corresponding to each layer in the model, the weighted average of the single-layer data distribution similarity of each layer is calculated to obtain the multiple sets of second overall data distribution similarity. Based on the fitting relationship, the second accuracy loss corresponding to the second overall data distribution similarity at the current accuracy is calculated.
8. The method for measuring the accuracy loss of model transfer according to claim 1, characterized in that, The weights corresponding to each layer in the model represent the proportion of the computational cost of each layer in the total computational cost.
9. A model transfer accuracy loss measurement system constructed based on the method described in any one of claims 1 to 8, characterized in that, include: The similarity calculation module is used to calculate the single-layer data distribution similarity of parameters of each layer before and after the migration of multiple sets of models with different precisions. The weighted calculation module is used to calculate the weighted average of the single-layer data distribution similarity of each layer according to the weights corresponding to each layer in the model, and obtain multiple sets of first overall data distribution similarity as a measure of first accuracy loss; The mathematical fitting module is used to fit the similarity of the distribution of the first overall data and the first accuracy loss of multiple groups to obtain the fitting relationship between the two. The loss calculation module is used to calculate the second accuracy loss corresponding to the second overall data distribution similarity at the current accuracy, based on the fitting relationship.
10. A storage medium, characterized in that, A computer program for storing a method for measuring the accuracy loss of model transfer as described in any one of claims 1 to 8.
11. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the accuracy loss measurement method for model transfer as described in any one of claims 1 to 8.