Data processing method and device, computer equipment, storage medium and computer program product

Abstract

The invention relates to a data processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a to-be-deployed original neural network model and a corresponding calibration data set; inputting data in the calibration data set into the original neural network model, and obtaining an activation value corresponding to each model parameter in the original neural network; based on the size of the activation value corresponding to each model parameter, performing quantization processing on the model parameters above the initial quantization retention ratio in the original neural network to obtain a quantized neural network model; and based on the performance of the quantized neural network, performing iterative adjustment on the initial quantization retention ratio until a preset requirement is met, and obtaining a target quantization retention ratio. According to the method, the model storage amount or calculation amount can be effectively reduced under the condition that the model performance is ensured.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a data processing method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] In the field of deep learning, large models have become a hot topic of research due to their excellent learning capabilities. However, these models have a huge number of parameters, usually hundreds of millions, billions or even hundreds of billions of parameters, which leads to huge computing and storage challenges in the deployment and inference process. In order to solve these problems, quantization technology came into being. This technology reduces the numerical precision of model parameters to achieve model compression in order to reduce hardware requirements and accelerate inference.

[0003] Mixed precision quantization technology refers to the process of selecting a certain proportion of parameters with higher weights and not quantizing them during the model quantization process. Parameters with higher weights refer to parameters that have a greater impact on the performance of the model, in the hope that this can both ensure model performance and reduce the size of the model. However, the commonly used quantization technology currently relies on empiricism to determine the proportion during the screening of important high-weight parameters, which cannot ensure the balance between data compression and model performance. Summary of the invention

[0004] Based on this, it is necessary to provide a data processing method, device, computer equipment, computer-readable storage medium and computer program product that can reduce the model storage or calculation amount while ensuring the model performance in order to address the above technical problems.

[0005] In a first aspect, the present application provides a data processing method. The method comprises:

[0006] Obtain the original neural network model to be deployed and the corresponding calibration dataset;

[0007] Inputting the data in the calibration data set into the original neural network model to obtain activation values ​​corresponding to each model parameter in the original neural network;

[0008] Based on the size of the activation value corresponding to each model parameter, the model parameters above the initial quantization retention ratio in the original neural network are quantized to obtain a quantized neural network model;

[0009] Based on the performance of the quantized neural network, iteratively adjusting the initial quantization retention ratio until a preset requirement is met to obtain a target quantization retention ratio;

[0010] Based on the target quantization retention ratio, the original neural network model is quantized to obtain a target neural network model, and the target neural network model is deployed.

[0011] In one embodiment, the quantizing of the model parameters above the initial quantization retention ratio in the original neural network based on the size of the activation value corresponding to each model parameter includes:

[0012] Based on the size of the activation value corresponding to each model parameter, sort the model parameters in descending order;

[0013] The model parameters with the highest initial quantization retention ratio are not quantized, and the remaining model parameters are quantized.

[0014] In one embodiment, the iterative adjustment of the initial quantization retention ratio based on the performance of the quantized neural network until a preset requirement is met to obtain a target quantization retention ratio includes:

[0015] When the performance of the quantized neural network meets the preset requirements, the initial quantization retention ratio is reduced to half of the original ratio to obtain an updated quantization retention ratio;

[0016] Based on the updated quantization retention ratio, quantizing the original neural network model to obtain an intermediate neural network model;

[0017] When the performance of the intermediate neural network model meets the preset requirements, the updated quantization retention ratio is reduced to half of the original value to obtain a further updated quantization retention ratio, until the updated retention parameter amount is lower than the preset threshold value to obtain the target quantization retention ratio.

[0018] In one of the embodiments, after the original neural network model is quantized based on the updated quantization retention ratio to obtain the intermediate neural network model, the method further includes:

[0019] When the performance of the intermediate neural network model does not meet the preset requirements, the updated quantization retention ratio is increased by half of the original one to obtain a further updated quantization retention ratio, until the updated retention parameter amount is lower than the preset threshold to obtain the target quantization retention ratio.

[0020] In one embodiment, when the performance of the quantized neural network meets the preset requirements, the initial quantization retention ratio is reduced to half of the original one, and before obtaining the updated quantization retention ratio, the method further includes:

[0021] When the performance of the quantized neural network cannot meet the preset requirements, the initial quantization retention ratio is doubled.

[0022] In one embodiment, the quantizing the original neural network model based on the target quantization retention ratio to obtain a target neural network model, and deploying the target neural network model includes:

[0023] Determining a target number of parameters to be retained without quantization based on the target quantization retention ratio and the total number of model parameters;

[0024] The target number of model parameters with the largest activation value in the original neural network model are not quantized, and the remaining model parameters are quantized to obtain a target neural network model, and the target neural network model is deployed.

[0025] In a second aspect, the present application further provides a data processing device, the device comprising:

[0026] A first acquisition module is used to acquire an original neural network model to be deployed and a corresponding calibration data set;

[0027] A second acquisition module is used to input the data in the calibration data set into the original neural network model to obtain the activation value corresponding to each model parameter in the original neural network;

[0028] An initial quantization module, used to quantize the model parameters above the initial quantization retention ratio in the original neural network based on the size of the activation value corresponding to each model parameter, to obtain a quantized neural network model;

[0029] An adjustment module, configured to iteratively adjust the initial quantization retention ratio based on the performance of the quantized neural network until a preset requirement is met to obtain a target quantization retention ratio;

[0030] A deployment module is used to quantize the original neural network model based on the target quantization retention ratio to obtain a target neural network model and deploy the target neural network model.

[0031] In one embodiment, the initial quantization module is further used for:

[0032] Based on the size of the activation value corresponding to each model parameter, sort the model parameters in descending order;

[0033] The model parameters with the highest initial quantization retention ratio are not quantized, and the remaining model parameters are quantized.

[0034] In one embodiment, the adjustment module is further used for:

[0035] When the performance of the quantized neural network meets the preset requirements, the initial quantization retention ratio is reduced to half of the original ratio to obtain an updated quantization retention ratio;

[0036] Based on the updated quantization retention ratio, quantizing the original neural network model to obtain an intermediate neural network model;

[0037] When the performance of the intermediate neural network model meets the preset requirements, the updated quantization retention ratio is reduced to half of the original value to obtain a further updated quantization retention ratio, until the updated retention parameter amount is lower than the preset threshold value to obtain the target quantization retention ratio.

[0038] In one embodiment, the adjustment module is further used for:

[0039] When the performance of the intermediate neural network model does not meet the preset requirements, the updated quantization retention ratio is increased by half of the original one to obtain a further updated quantization retention ratio, until the updated retention parameter amount is lower than the preset threshold to obtain the target quantization retention ratio.

[0040] In one embodiment, the adjustment module is further used for:

[0041] When the performance of the quantized neural network cannot meet the preset requirements, the initial quantization retention ratio is doubled.

[0042] In one embodiment, the deployment module is further used to:

[0043] Determining a target number of parameters to be retained without quantization based on the target quantization retention ratio and the total number of model parameters;

[0044] The target number of model parameters with the largest activation value in the original neural network model are not quantized, and the remaining model parameters are quantized to obtain a target neural network model, and the target neural network model is deployed.

[0045] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any one of the embodiments of the present disclosure is implemented.

[0046] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method described in any one of the embodiments of the present disclosure is implemented.

[0047] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the method described in any one of the embodiments of the present disclosure is implemented.

[0048] The above-mentioned data processing method, apparatus, computer equipment, storage medium and computer program product iteratively adjust the initial quantization retention ratio based on the performance of the quantized neural network to obtain the target quantization retention. The target quantization retention ratio can filter out the number of model parameters that cannot be quantified to ensure the performance of the model. On the other hand, the target quantization retention ratio can filter out the number of model parameters that need to be quantized to improve the compression efficiency of the model. Therefore, the embodiments of the present disclosure can balance the model performance and data compression rate, which not only improves the practicality and applicability of the model, but also provides strong technical support for the application of large models on resource-constrained devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of a data processing method in one embodiment;

[0050] Figure 2 is a flow chart of a data processing method in another embodiment;

[0051] Figure 3 is a flow chart of a data processing method in another embodiment;

[0052] Figure 4 is a structural block diagram of a data processing device in one embodiment;

[0053] Figure 5 is an internal structure diagram of a computer device in one embodiment;

[0054] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] In one embodiment, Figure 1 As shown, a data processing method is provided. This embodiment takes the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0057] Step S101, obtaining the original neural network model to be deployed and the corresponding calibration data set.

[0058] The original neural network model may include neural network models for various application scenarios, such as a question-answering model, a text-graph model, and a language model. Optionally, the number of model parameters in the original neural network model satisfies a preset threshold.

[0059] The calibration data set matches the original neural network model. Taking the question-answering model as an example, the calibration data set may include question text, answer text, etc. Optionally, the calibration data set can reflect the data type and distribution that the model will process in actual applications, and is used to characterize the performance of the model.

[0060] Step S103, inputting the data in the calibration data set into the original neural network model to obtain the activation value corresponding to each model parameter in the original neural network.

[0061] In the disclosed embodiment, the activation value is the output after the input and weight calculation and the activation function, which determines whether a neuron will be activated and pass information to the next layer. Therefore, when the activation value is large, it means that the neuron has a stronger influence on the subsequent layers. The model parameters act on the input features and determine the influence of these features on the final output of the model. Model parameters with larger weights indicate that the feature contributes more to the output of the neuron and is therefore more likely to affect the activation value of the subsequent layer.

[0062] Step S105, based on the size of the activation value corresponding to each model parameter, the model parameters above the initial quantization retention ratio in the original neural network are quantized to obtain a quantized neural network model.

[0063] The initial quantization retention ratio is an initially set ratio, and in an exemplary embodiment, it can be set to 5%.

[0064] Taking the initial quantization retention ratio of 5% as an example, quantizing the model parameters above the initial quantization retention ratio in the original neural network may include: quantizing the model parameters greater than 5%, and not quantizing the model parameters less than or equal to 5%. For example, the total number of model parameters is m, and 5%×m model parameters with larger weights are not quantized, and 95%×m model parameters with smaller weights are quantized.

[0065] Step S107, based on the performance of the quantized neural network, iteratively adjust the initial quantization retention ratio until it meets the preset requirements to obtain a target quantization retention ratio.

[0066] The performance of the quantized neural network can be determined based on the performance of the original neural network model, for example, retaining at least 95% of the performance of the original model. The model performance can be measured by accuracy, precision, recall, F1 score, ROC curve, and AUC (area under the curve).

[0067] In an exemplary embodiment, when the performance of the quantized neural network is higher than the target performance, the initial quantization retention ratio is reduced, and the original neural network model is requantized. The specific method of quantization can refer to the above embodiment. In an exemplary embodiment, when the performance of the quantized neural network is lower than the target performance, the initial quantization retention ratio is increased, and the original neural network model is requantized again. Until the number of reduced or increased parameters is lower than a preset threshold, the processing of the quantization retention ratio is stopped, and the quantization retention ratio corresponding to the number of parameters lower than the preset threshold is used as the target quantization retention ratio.

[0068] Step S109, based on the target quantization retention ratio, quantize the original neural network model to obtain a target neural network model, and deploy the target neural network model.

[0069] Specifically, based on the target quantization retention ratio and the model parameters of the original neural network model, the target number of model parameters that are not quantized is determined. The target number of model parameters with larger activation values ​​in the original neural network model are not quantized, and the remaining model parameters are quantized to obtain the target neural network model, which can be deployed to the server.

[0070] In the above-mentioned embodiment, the embodiment of the present disclosure iteratively adjusts the initial quantization retention ratio based on the performance of the quantized neural network to obtain the target quantization retention. The target quantization retention ratio can filter out the number of model parameters that cannot be quantized to ensure the performance of the model. On the other hand, the target quantization retention ratio can filter out the number of model parameters that need to be quantized to improve the compression efficiency of the model. Therefore, the embodiment of the present disclosure can balance the model performance and data compression rate, which not only improves the practicality and applicability of the model, but also provides strong technical support for the application of large models on resource-constrained devices.

[0071] In one embodiment, the quantizing of the model parameters above the initial quantization retention ratio in the original neural network based on the size of the activation value corresponding to each model parameter includes:

[0072] Based on the size of the activation value corresponding to each model parameter, sort the model parameters in descending order;

[0073] The model parameters with the highest initial quantization retention ratio are not quantized, and the remaining model parameters are quantized.

[0074] Specifically, the activation value of the model input layer can be the input feature value itself. The activation value of the model hidden layer can be the result of the neurons in this layer multiplying the input value with the corresponding weight, adding the bias, and then applying the activation function. The activation value of the model output layer can be the final output of the model, which can be the probability of the classification result, the predicted value of the regression problem, etc.

[0075] In an exemplary embodiment, the model parameters are sorted in descending order based on the activation values ​​of the model parameters. Optionally, if the initial quantization retention ratio is 5%, based on the total number of model parameters m, it is determined that the top 5%×m model parameters are not to be quantized, and the remaining 95% of the model parameters are to be quantized.

[0076] In the above embodiment, by sorting the model parameters in descending order based on the size of the activation values ​​corresponding to the model parameters, the model parameters that need to be quantized and the model parameters that do not need to be quantized can be quickly determined, thereby improving efficiency.

[0077] In one embodiment, reference Figure 2 As shown, the initial quantization retention ratio is iteratively adjusted based on the performance of the quantized neural network until the preset requirements are met to obtain the target quantization retention ratio, including:

[0078] Step S201, when the performance of the quantized neural network meets the preset requirements, the initial quantization retention ratio is reduced to half of the original one to obtain an updated quantization retention ratio.

[0079] In an exemplary embodiment, if the performance of the quantized neural network is 96% of the performance of the original neural network, which is higher than the target value (95% of the performance of the original neural network), the initial quantization retention ratio is reduced to half of the original value. For example, originally 9% of the model parameters were not quantized, but now it is updated to 4.5% of the model parameters are not quantized.

[0080] Step S203, based on the updated quantization retention ratio, quantize the original neural network model to obtain an intermediate neural network model.

[0081] Specifically, for example, in the above embodiment, 4.5% of the model parameters are not quantized, and the remaining 95.5% of the model parameters are quantized to obtain an intermediate neural network model.

[0082] Step S205, when the performance of the intermediate neural network model meets the preset requirements, the updated quantization retention ratio is reduced to half of the original one to obtain a further updated quantization retention ratio, until the updated retention parameter amount is lower than the preset threshold, and the target quantization retention ratio is obtained.

[0083] In an exemplary embodiment, if the model performance of the intermediate neural network is 95.5% of the performance of the original neural network, which is higher than the target value (95% of the performance of the original neural network), the updated quantization retention ratio is reduced to half of the original value. For example, originally 4.5% of the model parameters were not quantized, but now it is updated to 2.25% of the model parameters are not quantized.

[0084] In another exemplary embodiment, when the performance of the intermediate neural network model does not meet the preset requirements, the updated quantization retention ratio is increased by half of the original one to obtain a further updated quantization retention ratio, until the updated retention parameter amount is lower than the preset threshold to obtain the target quantization retention ratio.

[0085] Specifically, for example, in the above example, if the model performance of the intermediate neural network is 94% of the original neural network performance, which is lower than the target value (95% of the original neural network performance), the updated quantization retention ratio is increased to half of the original value, for example, 4.5% of the model parameters were not quantized, and now it is updated to 6.75% (4.5% + 2.25%). This cycle is repeated until the updated retained parameter amount is lower than the preset threshold, for example, the increased retained parameter amount may be lower than the preset threshold, or the reduced retained parameter amount may be the first preset threshold.

[0086] In the above embodiment, when the performance of the quantized neural network meets the preset requirements, the initial quantization retention ratio is reduced to half of the original ratio to obtain an updated quantization retention ratio; based on the updated quantization retention ratio, the original neural network model is quantized to obtain an intermediate neural network model; when the performance of the intermediate neural network model meets the preset requirements, the updated quantization retention ratio is reduced to half of the original ratio to obtain an updated quantization retention ratio, until the updated retention parameter amount is lower than the preset threshold value to obtain the target quantization retention ratio. The above method can improve iteration efficiency and determine the target quantization retention ratio more quickly.

[0087] In one embodiment, when the performance of the quantized neural network meets the preset requirements, the initial quantization retention ratio is reduced to half of the original one, and before obtaining the updated quantization retention ratio, the method further includes:

[0088] When the performance of the quantized neural network cannot meet the preset requirements, the initial quantization retention ratio is doubled.

[0089] For example, the initial quantization retention ratio has 9% of the model parameters not being quantized, and now it is updated to 18% of the model parameters not being quantized.

[0090] In the disclosed embodiment, for the first iteration, the model parameters within the initial quantization retention ratio of 9% are quantized. If the performance of the quantized neural network cannot meet the preset requirements, it is necessary to increase the proportion of the retained parameters to ensure the performance of the model. Optionally, the initial quantization retention ratio can be doubled to lock the target quantization retention ratio more quickly.

[0091] In one embodiment, the quantizing the original neural network model based on the target quantization retention ratio to obtain a target neural network model, and deploying the target neural network model includes:

[0092] Determining a target number of parameters to be retained without quantization based on the target quantization retention ratio and the total number of model parameters;

[0093] The target number of model parameters with the largest activation value in the original neural network model are not quantized, and the remaining model parameters are quantized to obtain a target neural network model, and the target neural network model is deployed.

[0094] In the specific implementation process, for example, the target quantization retention ratio is 6%, and the total number of model parameters is m. Then 6%×m parameters are retained without quantization. The m model parameters are sorted from large to small according to the activation value, the first 6%×m model parameters are not quantized, and the remaining 94%×m model parameters are quantized. The target neural network model is obtained, and optionally, the target neural network model can be deployed to the server.

[0095] In a specific embodiment, referring to Figure 3As shown, the embodiment of the present disclosure includes, step 1: selecting the original large model. Select an original large neural network model that has proven its high performance on a specific task as the original model for quantization. Step 2: Determine a calibration data set. Determine a calibration data set that can truly reflect the data type and distribution that the model will process in actual applications. Step 3: Set the quantization retention ratio. Set an initial quantization retention ratio, such as 5%, which means that at the beginning of quantization, 5% of the weight parameters will be retained and not quantized. Step 4: The parameter importance judgment method uses a method based on the activation value distribution to sort the model parameters by importance, and selects the weight with a larger activation value as an important parameter. Step 5: Clarify the performance requirements. Define the performance requirements of the quantized model to ensure that the quantized model at least retains the performance of the original model. The performance requirement refers to the minimum allowable error between the quantized model and the original model on the calibration set. If it is 5%, it means that the performance requirement is 95%. Step 6: Iterate the quantization process. Use the calibration set and the parameter importance judgment method to evaluate the parameter importance of the original model, and start the quantization iteration with the initial reserved parameter ratio. The specific iterative process is to evaluate the performance of the two models after each quantization. If the performance requirements are met, the binary method is used to reduce half of the retained parameters in the retention interval, and quantify and compare the performance again; if the performance requirements are not met in the first iteration, the initial retained parameter ratio is doubled; if the performance requirements are not met in the non-first iteration, half of the parameters outside the retention interval are increased according to the binary method and then iterated. In order to improve the efficiency of iterative solution, if the proportion of the retained parameters updated each time is less than 0.01%, it is also regarded as the optimal solution and the iteration loop is jumped out.

[0096] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0097] Based on the same inventive concept, the embodiment of the present application also provides a data processing device for implementing the data processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more data processing device embodiments provided below can refer to the limitations on the data processing method above, and will not be repeated here.

[0098] In one embodiment, Figure 4 As shown, a data processing device is provided, the device comprising:

[0099] A first acquisition module 401 is used to acquire an original neural network model to be deployed and a corresponding calibration data set;

[0100] A second acquisition module 403 is used to input the data in the calibration data set into the original neural network model to obtain the activation value corresponding to each model parameter in the original neural network;

[0101] An initial quantization module 405 is used to quantize the model parameters above the initial quantization retention ratio in the original neural network based on the size of the activation value corresponding to each model parameter, so as to obtain a quantized neural network model;

[0102] An adjustment module 407, configured to iteratively adjust the initial quantization retention ratio based on the performance of the quantized neural network until a preset requirement is met to obtain a target quantization retention ratio;

[0103] The deployment module 409 is used to quantize the original neural network model based on the target quantization retention ratio to obtain a target neural network model and deploy the target neural network model.

[0104] In one embodiment, the initial quantization module is further used for:

[0105] Based on the size of the activation value corresponding to each model parameter, sort the model parameters in descending order;

[0106] The model parameters with the highest initial quantization retention ratio are not quantized, and the remaining model parameters are quantized.

[0107] In one embodiment, the adjustment module is further used for:

[0108] When the performance of the quantized neural network meets the preset requirements, the initial quantization retention ratio is reduced to half of the original ratio to obtain an updated quantization retention ratio;

[0109] Based on the updated quantization retention ratio, quantizing the original neural network model to obtain an intermediate neural network model;

[0110] When the performance of the intermediate neural network model meets the preset requirements, the updated quantization retention ratio is reduced to half of the original value to obtain a further updated quantization retention ratio, until the updated retention parameter amount is lower than the preset threshold value to obtain the target quantization retention ratio.

[0111] In one embodiment, the adjustment module is further used for:

[0112] When the performance of the intermediate neural network model does not meet the preset requirements, the updated quantization retention ratio is increased by half of the original one to obtain a further updated quantization retention ratio, until the updated retention parameter amount is lower than the preset threshold to obtain the target quantization retention ratio.

[0113] In one embodiment, the adjustment module is further used for:

[0114] When the performance of the quantized neural network cannot meet the preset requirements, the initial quantization retention ratio is doubled.

[0115] In one embodiment, the deployment module is further used to:

[0116] Determining a target number of parameters to be retained without quantization based on the target quantization retention ratio and the total number of model parameters;

[0117] The target number of model parameters with the largest activation value in the original neural network model are not quantized, and the remaining model parameters are quantized to obtain a target neural network model, and the target neural network model is deployed.

[0118] Each module in the above data processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0119] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data of the data processing method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data processing method is implemented.

[0120] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a data processing method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0121] Those skilled in the art will understand that Figure 6The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0122] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0123] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0124] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data processing method, characterized in that: The method comprises: Obtain the original neural network model to be deployed and the corresponding calibration dataset; Inputting the data in the calibration data set into the original neural network model to obtain activation values ​​corresponding to each model parameter in the original neural network; Based on the size of the activation value corresponding to each model parameter, the model parameters above the initial quantization retention ratio in the original neural network are quantized to obtain a quantized neural network model; Based on the performance of the quantized neural network, iteratively adjusting the initial quantization retention ratio until a preset requirement is met to obtain a target quantization retention ratio; Based on the target quantization retention ratio, the original neural network model is quantized to obtain a target neural network model, and the target neural network model is deployed.

2. The method according to claim 1, characterized in that The step of quantizing the model parameters above the initial quantization retention ratio in the original neural network based on the size of the activation value corresponding to each model parameter comprises: Based on the size of the activation value corresponding to each model parameter, sort the model parameters in descending order; The model parameters with the highest initial quantization retention ratio are not quantized, and the remaining model parameters are quantized.

3. The method according to claim 1, characterized in that The iterative adjustment of the initial quantization retention ratio based on the performance of the quantized neural network until a preset requirement is met to obtain a target quantization retention ratio includes: When the performance of the quantized neural network meets the preset requirements, the initial quantization retention ratio is reduced to half of the original ratio to obtain an updated quantization retention ratio; Based on the updated quantization retention ratio, quantizing the original neural network model to obtain an intermediate neural network model; When the performance of the intermediate neural network model meets the preset requirements, the updated quantization retention ratio is reduced to half of the original value to obtain a further updated quantization retention ratio, until the updated retention parameter amount is lower than the preset threshold value to obtain the target quantization retention ratio.

4. The method according to claim 3, characterized in that After the original neural network model is quantized based on the updated quantization retention ratio to obtain an intermediate neural network model, the method further includes: When the performance of the intermediate neural network model does not meet the preset requirements, the updated quantization retention ratio is increased by half of the original one to obtain a further updated quantization retention ratio, until the updated retention parameter amount is lower than the preset threshold to obtain the target quantization retention ratio.

5. The method according to claim 3, characterized in that: When the performance of the quantized neural network meets the preset requirements, the initial quantization retention ratio is reduced to half of the original one, and before obtaining the updated quantization retention ratio, the method further includes: When the performance of the quantized neural network cannot meet the preset requirements, the initial quantization retention ratio is doubled.

6. The method according to claim 1, characterized in that The method of quantizing the original neural network model based on the target quantization retention ratio to obtain a target neural network model and deploying the target neural network model includes: Determining a target number of parameters to be retained without quantization based on the target quantization retention ratio and the total number of model parameters; The target number of model parameters with the largest activation value in the original neural network model are not quantized, and the remaining model parameters are quantized to obtain a target neural network model, and the target neural network model is deployed.

7. A data processing device, characterized in that: The device comprises: A first acquisition module is used to acquire an original neural network model to be deployed and a corresponding calibration data set; A second acquisition module is used to input the data in the calibration data set into the original neural network model to obtain the activation value corresponding to each model parameter in the original neural network; An initial quantization module, used to quantize the model parameters above the initial quantization retention ratio in the original neural network based on the size of the activation value corresponding to each model parameter, to obtain a quantized neural network model; An adjustment module, configured to iteratively adjust the initial quantization retention ratio based on the performance of the quantized neural network until a preset requirement is met to obtain a target quantization retention ratio; A deployment module is used to quantize the original neural network model based on the target quantization retention ratio to obtain a target neural network model and deploy the target neural network model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images