Model deployment method and device of terminal equipment, computer equipment and storage medium
By adjusting the bit width and structural information of the model parameters and mapping it to a numerical type suitable for the terminal device, the accuracy loss problem during model deployment is solved, and efficient and accurate model deployment on the terminal device is achieved.
Patent Information
- Application Number
- CN202411347426.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-25
- Publication Date
- 2025-01-14
AI Technical Summary
When deploying a model in terminal equipment, parameter quantization leads to loss of model information, reducing model accuracy. How to reduce this accuracy loss has become an urgent problem.
By obtaining the parameter bit width of the model to be trained, determine the numerical structure information, including the exponential bit width and the mantissa bit width, adjust the model parameters until the target model is obtained and mapped to a numerical type suitable for the terminal device to reduce accuracy loss.
While reducing the model size, the accuracy of the model is ensured, the accuracy loss when mapping from floating point numbers to integers is reduced, and the operation efficiency of the model on terminal devices is improved.
Smart Images
Figure CN119312848A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence, and in particular to a model deployment method, device, computer equipment and storage medium for a terminal device. Background Art
[0002] To ensure the accuracy of the model, the model usually uses high-precision floating-point numbers, such as 32-bit floating-point format to calculate and store the model's weights, biases, and activations, etc., to reduce rounding errors and numerical overflow problems during training, more accurately represent the model's gradient values, and help the model learn and converge better. Due to the limited computing power of terminal devices, model parameters are usually quantized to integers during deployment to reduce the storage space required for model parameters and improve computing efficiency. However, parameter quantization will lead to the loss of model information and reduce the accuracy of the model. How to deploy the model to reduce the information lost during parameter quantization has become an urgent problem to be solved. Summary of the invention
[0003] The main purpose of this application is to provide a model deployment method, apparatus, device and computer storage medium for a terminal device, aiming to reduce the accuracy loss during model deployment.
[0004] In a first aspect, the present application provides a model deployment method for a terminal device, the model deployment method for the terminal device comprising the following steps:
[0005] Acquire a first parameter bit width of a model to be trained, wherein the model parameter of the model to be trained belongs to a first numerical type;
[0006] Determining numerical structure information of the model to be trained according to the first parameter bit width, wherein the numerical structure information at least includes an exponent bit width and a mantissa bit width of the model parameter;
[0007] Adjusting the numerical structure information and model parameters of the model to be trained until a target model is obtained;
[0008] The model parameters of the target model are mapped to a second numerical type to obtain a model to be deployed, and the model to be deployed is deployed to the terminal device.
[0009] In a second aspect, the present application further provides a model deployment device for a terminal device, the model deployment device for the terminal device comprising:
[0010] A bit width acquisition module, used to acquire a first parameter bit width of a model to be trained, wherein the model parameter of the model to be trained belongs to a first numerical type;
[0011] A numerical structure determination module, used to determine the numerical structure information of the model to be trained according to the first parameter bit width, wherein the numerical structure information at least includes the exponent bit width and the mantissa bit width of the model parameter;
[0012] A model training module is used to adjust the numerical structure information and model parameters of the model to be trained until a target model is obtained;
[0013] The model deployment module is used to map the model parameters of the target model to a second numerical type, obtain the model to be deployed, and deploy the model to be deployed to the terminal device.
[0014] In a third aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the model deployment method for the terminal device as described above is implemented.
[0015] In a fourth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the model deployment method for the terminal device as described above is implemented.
[0016] The present application provides a model deployment method, apparatus, device and computer storage medium for a terminal device, wherein the present application obtains the first parameter bit width of a model to be trained, and the model parameters of the model to be trained belong to a first numerical type; determines the numerical structure information of the model to be trained according to the first parameter bit width, and the numerical structure information at least includes the exponent bit width and the mantissa bit width of the model parameters; adjusts the numerical structure information and model parameters of the model to be trained until a target model is obtained; maps the model parameters of the target model to a second numerical type, obtains a model to be deployed, and deploys the model to be deployed to the terminal device. Since the model is trained based on the first parameter bit width corresponding to the second numerical type, the precision loss when the model is mapped from the first numerical type to the second numerical type is reduced, and the accuracy of the model is ensured while reducing the size of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0018] Figure 1 A schematic diagram of a process flow of a model deployment method for a terminal device provided in one embodiment of the present application;
[0019] Figure 2 A usage scenario diagram of a model deployment method for a terminal device provided in one embodiment of the present application;
[0020] Figure 3 Sub-steps of the target model training process provided in one embodiment of the present application;
[0021] Figure 4 A schematic block diagram of a model deployment device for a terminal device provided in one embodiment of the present application;
[0022] Figure 5 The present invention is a block diagram showing the structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION
[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0024] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.
[0025] For example, the artificial intelligence model is deployed on the edge device of the terminal side, such as electronic products such as smartphones, tablets and smart home devices. The edge device does not need to upload the collected data to the cloud server through the network, which is conducive to the model to process the data in the first time, reducing the pressure of network transmission and server, improving data processing efficiency, and being able to clean and protect some sensitive data locally, improving data privacy. However, due to the hardware resource limitations of the edge device, the computing power of the edge device itself has certain limitations. Before deploying the model to the edge device, it is usually necessary to quantize the model to reduce the computing pressure of the model running on the edge device. For example, the model is usually trained with parameters in float32 or even float64 format. When deploying, considering the insufficient computing power of the edge device, the parameters of the model are usually quantized to int8 format. From float32 to int8, the mapping range differs by 10 to the 36th power, which will cause the model deployed on the edge device to have a large quantization error and precision loss.
[0026] In order to avoid the above situation, the embodiments of the present application provide a model deployment method, apparatus, computer equipment and computer-readable storage medium for a terminal device.
[0027] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0028] Please refer to Figure 1 , Figure 1 A flow chart of a model deployment method for a terminal device provided in an embodiment of the present application. The model deployment method for the terminal device can be used in a terminal or a server to implement training and deployment of the model, reducing the accuracy loss when the model is deployed to the terminal device. Among them, the terminal can be an electronic device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, and a wearable device; the server can be an independent server, or a server cluster, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.
[0029] Please refer to Figure 2 , Figure 2 This is a usage scenario diagram provided by an embodiment of the present application. Figure 2 As shown, the target model is obtained by adjusting the parameters of the first numerical type based on the first parameter bit width in the server, and mapped to the model to be deployed of the second numerical type, wherein the second parameter bit width of the second numerical type is the same as the first parameter bit width, and the model to be deployed is deployed to the terminal device. Of course, it is not limited to this, and the training device and deployment device of the model are not limited here.
[0030] like Figure 1 As shown, the model deployment method of the terminal device includes steps S101 to S104.
[0031] Step S101: Obtain a first parameter bit width of a model to be trained, where the model parameter of the model to be trained belongs to a first numerical type.
[0032] Exemplarily, the first numerical type is the numerical type of the parameter used when training the model, for example, it can be a floating point number (float), and the first parameter bit width is the bit width of the parameter used when training the model. For example, the first parameter bit width can be 8, 16, etc. In this way, the specification of the parameter used when training the model can be float8, float16, etc., of course, it is not limited to this, it can also be float6, float10, etc., which is not limited here.
[0033] Exemplarily, the model to be trained provided in the embodiment of the present application can be a single convolutional neural network, such as a convolutional neural network with a kernel size of 33, an output channel number of 64, and a step size of 2. Of course, it is not limited to this and is not limited here.
[0034] Exemplarily, the training data set of the training model may be preprocessed in advance. For example, if the data in the training data set is an image, at least one of brightness adjustment, contrast adjustment, and pixel normalization may be performed on the image.
[0035] In some implementations, obtaining the first parameter bit width of the model to be trained includes:
[0036] The first parameter bit width is determined according to a second parameter bit width corresponding to the second value type of the terminal device.
[0037] Exemplarily, the first parameter bit width should be determined according to the second parameter bit width used by the terminal device to which the target model is to be deployed. Specifically, the first parameter bit width is equal to the second parameter bit width. For example, if the parameter of the model in the terminal device is int8, the target model is trained using float8 parameters; if the parameter of the model in the terminal device is int10, the target model is trained using float10 parameters. Of course, it is not limited to this, and the difference between the second parameter bit width and the first parameter bit width can also be less than the preset difference, which is not limited here.
[0038] Exemplarily, since the first parameter bit width is determined according to the second parameter bit width corresponding to the second numerical type of the terminal device, for example, the first parameter bit width and the second parameter bit width are float8 and int8 respectively, the gap before and after parameter mapping is reduced, thereby reducing the precision loss during parameter mapping. By analogy, if the parameter used by the terminal device is of int6 specification, the target model is trained based on float6; if the parameter used by the terminal device is of int10 specification, the target model is trained based on float10.
[0039] Step S102: determine the numerical structure information of the model to be trained according to the first parameter bit width, wherein the numerical structure information at least includes the exponent bit width and the mantissa bit width of the model parameter.
[0040] For example, there is no unified standard for some bit width numeric types in the related technology. For example, according to the provisions of the relevant standard IEEE754, float32 includes 1 sign bit, 8 binary exponent bits and 23 mantissa bits. Since the exponent is called Exponent (abbreviated as E) and the mantissa is called Mantissa (abbreviated as M), the specification of float32 can be abbreviated as "E8M23". However, the specification of float8 is not specified in the relevant standards. Therefore, in the process of training the model, it is also necessary to optimize the exponent bit width and mantissa bit width of float8, that is, to optimize the exponent bit width and mantissa bit width in the first parameter bit width.
[0041] In some implementations, determining the numerical structure information of the model to be trained according to the first parameter bit width includes:
[0042] The sum of the exponent bit width and the mantissa bit width is determined according to the first parameter bit width, wherein the exponent bit width and the mantissa bit width are positive integers.
[0043] Exemplarily, in machine learning and deep learning models, parameters are usually represented by signed floating-point numbers because signed numbers can represent positive numbers, negative numbers, and zero. This is necessary in most cases because the parameters of the model can take positive or negative values, which reflects the weights or preferences of the model on different features. However, not all parameters must have a sign bit. In some specific cases, unsigned numbers can be used. For example, when the parameter range is known and is always positive, or the sign of the parameter is not important, the first parameter bit width can only include the exponent bit width and the mantissa bit width.
[0044] It can be understood that the exponent bit width and the mantissa bit width are both positive integers, and the sum of the exponent bit width and the mantissa bit width is the first parameter bit width. Of course, it is not limited to this. The first parameter bit width may also include sign bits, implicit bits and other digits, which are not limited here.
[0045] In some implementations, determining the sum of the exponent bit width and the mantissa bit width according to the first parameter bit width includes:
[0046] The sum of the exponent bit width, the mantissa bit width and a preset sign bit width is equal to the first parameter bit width.
[0047] For example, in most cases, the parameters of the model are signed because they may be updated to negative values during training, which makes sense in many algorithms. For example, the gradient descent algorithm may update the weights according to the gradient of the loss function, which may be positive or negative, so the weight parameters need to be able to represent positive and negative values, and the first parameter bit width needs to include a sign bit. Specifically, the preset sign bit width is usually 1 bit, that is, the sign of the parameter is represented by a 1-bit binary number.
[0048] Take float8 as an example, that is, the first value type is a floating point number, the first parameter bit width is 8, and when the sign bit width is 1, the sum of the exponent bit width and the mantissa bit width is equal to 7. Assuming that a represents the exponent bit width and b represents the mantissa bit width, the value range of the value structure information (a, b) is (1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1).
[0049] Step S103: Adjust the numerical structure information and model parameters of the model to be trained until a target model is obtained.
[0050] Illustratively, the model deployment method for terminal devices provided in this embodiment, when training the model, in addition to optimizing the model parameters, also needs to optimize the exponent bit width and the mantissa bit width, so that the model parameters constructed based on the exponent bit width and the mantissa bit width can make the model's loss function, accuracy and other indicators reach the best.
[0051] Please refer to Figure 3 , Figure 3 Sub-steps of the target model training process provided in one embodiment of the present application.
[0052] like Figure 3 As shown, in some implementations, step S103 adjusts the numerical structure information and model parameters of the model to be trained until a target model is obtained, including:
[0053] Step S1031, adjusting the model parameters of the model to be trained until the loss function of the model to be trained meets a first preset condition; and
[0054] Step S1032, adjusting the exponent bit width and the mantissa bit width of the model to be trained until the accuracy of the model to be trained meets the second preset condition;
[0055] Step S1033: When the model to be trained meets the first preset condition and the second preset condition, the target model is obtained.
[0056] Exemplarily, the parameters of the model and the numerical structure information of the parameters are evaluated according to the first preset condition and the second preset condition, respectively, wherein the first preset condition may be, for example, a loss function, and the second preset condition may be, for example, an accuracy rate, but of course it is not limited thereto and is not limited here.
[0057] It can be understood that step S1031 and step S1032 are performed alternately, and the order of step S1031 and step S1032 is not limited here. Step S1031 may be performed first and then step S1032, or step S1032 may be performed first and then step S1031.
[0058] Specifically, assume that the model is f(x; w, θ), where x represents the input, w represents the model parameter, and θ represents the object (a, b) of the automatic hyperparameter search, that is, the exponent bit width and mantissa bit width of the model parameter. Step S1031 is calculated by formula w * =argminL(f(x;w,θ * ),D train ) determine the optimal model parameters, where w * is the optimal model parameter, and L is used to train the model
[0059] loss function, D train is the training set of data; Step S1032 is then performed by θ=argmaxM(f(x; w * ,θ),D valid ) Search for the best exponent bit width and mantissa bit width, where,
[0060] θ is the optimal hyperparameter (a * ,b * ), M is the measured model accuracy, D valid is the validation set of data. * ,θ * ) does not meet the second preset condition, then return to step S1031.
[0061] Exemplarily, the training parameters of the model in step S1031 and step S1032 can be set according to actual needs. The optimizer used can be, for example, Adam, the learning rate is 0.001, the batch size is set according to the hardware of the training device, which is 64 here, and the iteration period epoch is set to 100.
[0062] In some implementations, adjusting the model parameters of the model to be trained until the loss function of the model to be trained satisfies a first preset condition includes:
[0063] Acquire preset numerical structure information, wherein the preset numerical structure information includes a preset exponent bit width and a preset mantissa bit width;
[0064] Based on the preset exponent bit width and the preset mantissa bit width, the model parameters of the model to be trained are adjusted until the loss function of the model to be trained meets a first preset condition.
[0065] Exemplarily, since the value range of the numerical structure information is relatively limited, a preset exponent bit width and a preset mantissa bit width can be set in advance, and step S1031 is executed to optimize the model parameters based on the preset exponent bit width and the preset mantissa bit width. After the model parameters are determined, step S1031 is entered to adjust the numerical structure information, thereby improving the training efficiency of the target model.
[0066] Step S104: Map the model parameters of the target model to a second numerical type to obtain a model to be deployed, and deploy the model to be deployed to the terminal device.
[0067] Exemplarily, in order to facilitate the deployment of the model to the edge terminal device, the parameter is mapped from the first numerical type of the first parameter bit width to the second numerical type of the second parameter bit width. Since the first parameter bit width is determined based on the second parameter bit width, the loss of parameter information can be ensured to be small during the mapping process, thereby reducing the loss of model accuracy.
[0068] In some implementations, mapping the model parameters of the target model to a second numerical type to obtain a model to be deployed includes:
[0069] The model parameters of the target model are mapped from floating point numbers to integer numbers.
[0070] For example, since integer operations are usually faster than floating-point operations, and floating-point operations have higher accuracy, floating-point numbers are used to train the model, and when the model is deployed, the model parameters are mapped to integers (int). For example, the parameters based on float8 are mapped to int8.
[0071] Specifically, the first numerical type can be quantized into the second numerical type by the following formula:
[0072]
[0073] Among them, Q represents the quantized value, R is the value to be quantized; S represents the scaling factor, which can be calculated according to R max and R min Respectively represent the maximum and minimum values of R, Q max and Q minThey represent the maximum and minimum values of Q respectively; Z is the zero point value, which is used to adjust the offset of Q. In symmetric quantization, the zero point is usually set to 0 so that the zero points before and after quantization correspond.
[0074] In some implementations, after mapping the model parameters of the target model to a second numerical type to obtain a model to be deployed, the method further includes:
[0075] Based on a preset test data set, the model accuracy of the model to be deployed is tested;
[0076] When the model accuracy meets a preset accuracy threshold, the model to be deployed is deployed to the terminal device.
[0077] Exemplarily, the mapping of parameters may produce certain errors. After the model to be deployed is mapped, the accuracy of the model is tested through a test data set. If the accuracy meets the accuracy threshold, the model is deployed to the terminal device. Otherwise, return to step S103 to adjust the model to ensure the accurate operation of the model on the terminal device.
[0078] The model deployment method for the terminal device provided in the above embodiment obtains the first parameter bit width of the model to be trained, and the model parameters of the model to be trained belong to the first numerical type; determines the numerical structure information of the model to be trained according to the first parameter bit width, and the numerical structure information at least includes the exponent bit width and the mantissa bit width of the model parameters; adjusts the numerical structure information and model parameters of the model to be trained until the target model is obtained; maps the model parameters of the target model to the second numerical type to obtain the model to be deployed, and deploys the model to be deployed to the terminal device. Since the model is trained based on the first parameter bit width corresponding to the second numerical type, the precision loss when mapping the model from the first numerical type to the second numerical type is reduced, thereby reducing the model size while ensuring the accuracy of the model.
[0079] See also Figure 4 , Figure 4 It is a schematic diagram of a model deployment device for a terminal device provided in an embodiment of the present application. The model deployment device for the terminal device can be configured in a server or a terminal to execute the aforementioned model deployment method for the terminal device.
[0080] like Figure 4 As shown, the model deployment device of the terminal device includes: a bit width acquisition module 110, a numerical structure determination module 120, a model training module 130, and a model deployment module 140.
[0081] A bit width acquisition module 110, used to acquire a first parameter bit width of a model to be trained, wherein the model parameter of the model to be trained belongs to a first numerical type;
[0082] A numerical structure determination module 120, configured to determine numerical structure information of the model to be trained according to the first parameter bit width, wherein the numerical structure information at least includes an exponent bit width and a mantissa bit width of the model parameter;
[0083] A model training module 130 is used to adjust the numerical structure information and model parameters of the model to be trained until a target model is obtained;
[0084] The model deployment module 140 is used to map the model parameters of the target model into a second numerical type, obtain the model to be deployed, and deploy the model to be deployed to the terminal device.
[0085] In some implementations, the bit width acquisition module 110, in the process of acquiring the first parameter bit width of the model to be trained, is used to implement:
[0086] The first parameter bit width is determined according to a second parameter bit width corresponding to the second value type of the terminal device.
[0087] In some implementations, the numerical structure determination module 120, in the process of determining the numerical structure information of the model to be trained according to the first parameter bit width, is used to implement:
[0088] The sum of the exponent bit width and the mantissa bit width is determined according to the first parameter bit width, wherein the exponent bit width and the mantissa bit width are positive integers.
[0089] In some implementations, the numerical structure determination module 120, in the process of determining the sum of the exponent bit width and the mantissa bit width according to the first parameter bit width, is configured to implement:
[0090] The sum of the exponent bit width, the mantissa bit width and a preset sign bit width is equal to the first parameter bit width.
[0091] In some implementations, the model training module 130, in the process of adjusting the numerical structure information and model parameters of the model to be trained until the target model is obtained, is used to implement:
[0092] Adjusting the model parameters of the model to be trained until the loss function of the model to be trained meets a first preset condition; and
[0093] Adjusting the exponent bit width and the mantissa bit width of the model to be trained until the accuracy of the model to be trained meets the second preset condition;
[0094] When the model to be trained meets the first preset condition and the second preset condition, the target model is obtained.
[0095] In some implementations, the model training module 130, in the process of adjusting the model parameters of the model to be trained until the loss function of the model to be trained satisfies the first preset condition, is used to implement:
[0096] Acquire preset numerical structure information, wherein the preset numerical structure information includes a preset exponent bit width and a preset mantissa bit width;
[0097] Based on the preset exponent bit width and the preset mantissa bit width, the model parameters of the model to be trained are adjusted until the loss function of the model to be trained meets a first preset condition.
[0098] In some implementations, the model deployment module 140, in the process of mapping the model parameters of the target model to the second numerical type to obtain the model to be deployed, is used to implement:
[0099] The model parameters of the target model are mapped from floating point numbers to integer numbers.
[0100] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0101] The method and apparatus of the present application can be used in many general or special computing system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
[0102] Exemplarily, the above method and apparatus may be implemented in the form of a computer program. The computer program may be implemented in Figure 5 Runs on the computer device shown.
[0103] See also Figure 5 , Figure 5 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. The computer device may be a server or a terminal.
[0104] like Figure 5 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a storage medium and an internal memory.
[0105] The storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any model deployment method of the terminal device.
[0106] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.
[0107] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute the model deployment method of any terminal device.
[0108] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0109] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0110] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:
[0111] Acquire a first parameter bit width of a model to be trained, wherein the model parameter of the model to be trained belongs to a first numerical type;
[0112] Determining numerical structure information of the model to be trained according to the first parameter bit width, wherein the numerical structure information at least includes an exponent bit width and a mantissa bit width of the model parameter;
[0113] Adjusting the numerical structure information and model parameters of the model to be trained until a target model is obtained;
[0114] The model parameters of the target model are mapped to a second numerical type to obtain a model to be deployed, and the model to be deployed is deployed to the terminal device.
[0115] In some implementations, the processor, in the process of executing the step of obtaining the first parameter bit width of the model to be trained, is configured to implement:
[0116] The first parameter bit width is determined according to a second parameter bit width corresponding to the second value type of the terminal device.
[0117] In some implementations, the processor, in the process of determining the numerical structure information of the model to be trained according to the first parameter bit width, is used to implement:
[0118] The sum of the exponent bit width and the mantissa bit width is determined according to the first parameter bit width, wherein the exponent bit width and the mantissa bit width are positive integers.
[0119] In some implementations, the processor, during the process of determining the sum of the exponent bit width and the mantissa bit width according to the first parameter bit width, is configured to implement:
[0120] The sum of the exponent bit width, the mantissa bit width and a preset sign bit width is equal to the first parameter bit width.
[0121] In some implementations, the processor, in the process of adjusting the numerical structure information and model parameters of the model to be trained until the target model is obtained, is used to implement:
[0122] Adjusting the model parameters of the model to be trained until the loss function of the model to be trained meets a first preset condition; and
[0123] Adjusting the exponent bit width and the mantissa bit width of the model to be trained until the accuracy of the model to be trained meets the second preset condition;
[0124] When the model to be trained meets the first preset condition and the second preset condition, the target model is obtained.
[0125] In some implementations, the processor, in the process of adjusting the model parameters of the model to be trained until the loss function of the model to be trained satisfies the first preset condition, is used to implement:
[0126] Acquire preset numerical structure information, wherein the preset numerical structure information includes a preset exponent bit width and a preset mantissa bit width;
[0127] Based on the preset exponent bit width and the preset mantissa bit width, the model parameters of the model to be trained are adjusted until the loss function of the model to be trained meets a first preset condition.
[0128] In some implementations, the processor, in executing the process of mapping the model parameters of the target model to the second numerical type to obtain the model to be deployed, is used to implement:
[0129] The model parameters of the target model are mapped from floating point numbers to integer numbers.
[0130] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the model deployment of the terminal device described above can refer to the corresponding process in the aforementioned model deployment control method embodiment of the terminal device, and will not be repeated here.
[0131] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the model deployment method of the terminal device of the present application.
[0132] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.
[0133] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0134] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.
[0135] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A model deployment method for a terminal device, characterized in that: The method comprises: Acquire a first parameter bit width of a model to be trained, wherein the model parameter of the model to be trained belongs to a first numerical type; Determining numerical structure information of the model to be trained according to the first parameter bit width, wherein the numerical structure information at least includes an exponent bit width and a mantissa bit width of the model parameter; Adjusting the numerical structure information and model parameters of the model to be trained until a target model is obtained; The model parameters of the target model are mapped to a second numerical type to obtain a model to be deployed, and the model to be deployed is deployed to the terminal device.
2. The model deployment method of a terminal device according to claim 1, characterized in that: The obtaining the first parameter bit width of the model to be trained includes: The first parameter bit width is determined according to a second parameter bit width corresponding to the second value type of the terminal device.
3. The model deployment method of a terminal device according to claim 1, characterized in that: The determining the numerical structure information of the model to be trained according to the first parameter bit width includes: The sum of the exponent bit width and the mantissa bit width is determined according to the first parameter bit width, wherein the exponent bit width and the mantissa bit width are positive integers.
4. The model deployment method of the terminal device according to claim 3, characterized in that: The determining the sum of the exponent bit width and the mantissa bit width according to the first parameter bit width comprises: The sum of the exponent bit width, the mantissa bit width and a preset sign bit width is equal to the first parameter bit width.
5. The model deployment method of a terminal device according to claim 1, characterized in that: The adjusting of the numerical structure information and model parameters of the model to be trained until a target model is obtained includes: Adjusting the model parameters of the model to be trained until the loss function of the model to be trained meets a first preset condition; and Adjusting the exponent bit width and the mantissa bit width of the model to be trained until the accuracy of the model to be trained meets the second preset condition; When the model to be trained meets the first preset condition and the second preset condition, the target model is obtained.
6. The model deployment method of a terminal device according to claim 5, characterized in that: The adjusting the model parameters of the model to be trained until the loss function of the model to be trained meets a first preset condition includes: Acquire preset numerical structure information, wherein the preset numerical structure information includes a preset exponent bit width and a preset mantissa bit width; Based on the preset exponent bit width and the preset mantissa bit width, the model parameters of the model to be trained are adjusted until the loss function of the model to be trained meets a first preset condition.
7. The model deployment method for a terminal device according to any one of claims 1 to 6, characterized in that: Mapping the model parameters of the target model to a second numerical type to obtain a model to be deployed includes: The model parameters of the target model are mapped from floating point numbers to integer numbers.
8. A model deployment device for a terminal device, characterized in that: The model deployment device of the terminal device includes: A bit width acquisition module, used to acquire a first parameter bit width of a model to be trained, wherein the model parameter of the model to be trained belongs to a first numerical type; A numerical structure determination module, used to determine the numerical structure information of the model to be trained according to the first parameter bit width, wherein the numerical structure information at least includes the exponent bit width and the mantissa bit width of the model parameter; A model training module is used to adjust the numerical structure information and model parameters of the model to be trained until a target model is obtained; The model deployment module is used to map the model parameters of the target model to a second numerical type, obtain the model to be deployed, and deploy the model to be deployed to the terminal device.
9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the model deployment method for the terminal device as described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the model deployment method for a terminal device according to any one of claims 1 to 7 are implemented.