Model optimization method, system, equipment and medium

Abstract

The invention provides a model optimization method, equipment and a medium. The method comprises the following steps: determining a target model architecture based on a preset search algorithm; constructing an original model based on the target model architecture, and training the original model to generate an undetermined model; and optimizing the to-be-determined model based on a preset memory optimization rule to generate a target model. According to the method, a memory is added as a constraint condition during model design, so that a memory-friendly target model architecture is obtained, and the model is further optimized through a memory optimization method after the target model architecture is determined, so that the energy consumption is remarkably reduced, the maximum optimization of the model under limited memory resources is realized, and the optimization efficiency of the model is improved. Therefore, the response speed and performance of the model are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model optimization method, system, device and medium. Background Art

[0002] With the continuous development of computer technology, the demand for memory and computing resources of computer equipment continues to increase, while high memory usage and frequent memory access will lead to higher resource waste and equipment costs.

[0003] With the development of model technology, many application scenarios require low latency and fast response, so the requirements for model memory management are getting higher and higher. Therefore, a model optimization method is urgently needed to solve the above technical problems. Summary of the invention

[0004] Based on this, it is necessary to provide a model optimization method, equipment and medium to address the above technical problems.

[0005] In a first aspect, the present application provides a model optimization method, the method comprising:

[0006] Determine the target model architecture based on a preset search algorithm;

[0007] Building an original model based on the target model architecture and training the original model to generate a pending model;

[0008] The pending model is optimized based on preset memory optimization rules to generate a target model.

[0009] In some embodiments, determining the target model architecture based on a preset search algorithm includes:

[0010] Identify multiple candidate architectures that match current business mission requirements;

[0011] Based on the evaluation index formula, calculate the evaluation scores of multiple candidate architectures;

[0012] According to the evaluation score, a target model architecture matching the current business task requirement is determined from the multiple candidate architectures.

[0013] In some embodiments, the evaluation index formula is:

[0014] Eval(a)=ωAcc(a)-λmax(0,Mem(α)-Mmax);

[0015] Among them, a represents the candidate architecture, Eval(a) represents the evaluation score, ω represents the weight coefficient of accuracy, λ represents the penalty coefficient for exceeding the memory limit, Mem(a) represents the memory usage, Acc(a) represents the accuracy of the candidate architecture, and Mmax represents the maximum memory limit.

[0016] In some embodiments, the method of confirming multiple candidate architectures that match the current business task requirements includes:

[0017] Determine the basic model architecture according to the current business task requirements;

[0018] Defining a search space according to the structure of the basic model architecture;

[0019] Searching for a model architecture to be processed in the search space according to a preset search strategy;

[0020] Obtaining an estimated memory usage of the model architecture to be processed, and comparing the estimated memory usage with the maximum memory limit;

[0021] In response to the estimated memory usage being less than the maximum memory limit, determining the to-be-processed model architecture as a candidate architecture;

[0022] In response to the estimated memory usage being greater than or equal to the maximum memory limit, determining that the model architecture to be processed is a non-candidate architecture.

[0023] In some embodiments, the process of constructing an original model based on the target model architecture and training the original model includes:

[0024] Insert gradient checkpoints to reduce memory consumption during backpropagation;

[0025] Configure the mixed precision training environment and select the number of operations and operation precision that match the operation;

[0026] Selecting a computing path according to data characteristics of input data of the network layer, wherein the computing path includes a parallel path and a low resource path;

[0027] The memory usage is monitored and when the memory usage reaches a preset condition, the input data is transferred to a designated storage medium.

[0028] In some embodiments, optimizing the model to be determined based on a preset memory optimization rule to generate a target model includes:

[0029] Obtaining data characteristics of input data of a network layer of a pending model, and determining a scaling factor according to the data characteristics;

[0030] determining a quantization parameter according to the scaling factor and the data characteristic;

[0031] The input data in the network layer is quantized according to the quantization parameter.

[0032] In some embodiments, determining a quantization parameter calculation rule according to the data characteristics includes:

[0033] In response to detecting that the data characteristic is a wide dynamic range and uniform distribution, calculating a scaling factor according to a first formula, wherein the first formula is: scale(x)=max(x)-min(x), scale(x) represents a quantization factor, max(x) represents a maximum value in the input data, and min(x) represents a minimum value in the input data;

[0034] In response to detecting that the data characteristic is a near-normal distribution, calculating a scaling factor according to a second formula, wherein the second formula is: scale(x)=2*σ(x), σ(x) represents a standard deviation of the input data;

[0035] In response to detecting that the data characteristic is the presence of a large number of outliers or a severely skewed distribution, a scaling factor is calculated according to a third formula, wherein the third formula is: scale(x)=Q3-Q1, Q3 represents the third quartile, and Q1 represents the first quartile.

[0036] In some embodiments, determining the scaling factor according to the data characteristics includes:

[0037] In response to detecting that the data characteristic is a wide dynamic range and a uniform distribution, a quantization parameter is calculated according to a fourth formula, wherein the fourth formula is: Quantized(x) represents the quantization parameter, x represents the input data, and y represents the quantization level;

[0038] In response to detecting that the data characteristic is a near-normal distribution, the quantization parameter is calculated according to a fifth formula, wherein the fifth formula is μ(x) represents the mean of the input data;

[0039] In response to detecting that the data characteristic has a large number of outliers or a serious skewed distribution, the quantization parameter is calculated according to a sixth formula, wherein the sixth formula is

[0040] In a second aspect, the present application provides a model optimization system, the system comprising:

[0041] A design module for determining a target model architecture based on a preset search algorithm;

[0042] A training module, used for constructing an original model based on the target model architecture and training the original model to generate a pending model;

[0043] The optimization module is used to optimize the model to be determined based on preset memory optimization rules to generate a target model.

[0044] In a third aspect, the present application provides a computer program product, which implements the steps of the following method when the computer program is executed by a processor:

[0045] Determine the target model architecture based on a preset search algorithm;

[0046] Building an original model based on the target model architecture and training the original model to generate a pending model;

[0047] The pending model is optimized based on preset memory optimization rules to generate a target model.

[0048] In a fourth aspect, the present application provides an electronic device, the electronic device comprising: one or more processors;

[0049] and a memory associated with one or more processors, the memory being used to store program instructions, which, when read and executed by the one or more processors, perform the following operations:

[0050] Determine the target model architecture based on a preset search algorithm;

[0051] Building an original model based on the target model architecture and training the original model to generate a pending model;

[0052] The pending model is optimized based on preset memory optimization rules to generate a target model.

[0053] In a fifth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program enables a computer to perform the following operations:

[0054] Determine the target model architecture based on a preset search algorithm;

[0055] Building an original model based on the target model architecture and training the original model to generate a pending model;

[0056] The pending model is optimized based on preset memory optimization rules to generate a target model.

[0057] The beneficial effects achieved by this application are:

[0058] The present application provides a model optimization method, including determining a target model architecture based on a preset search algorithm; constructing an original model based on the target model architecture and training the original model to generate a pending model; optimizing the pending model based on preset memory optimization rules to generate a target model. At the same time, the probability of a successful read is greatly increased, while ensuring stability and reducing the impact on read performance. In the model design, memory is added as a constraint condition to obtain a memory-friendly target model architecture, and after the target model architecture is determined, the model is further optimized through a memory optimization method, which significantly reduces energy consumption and achieves maximum optimization of the model under limited memory resources, thereby improving the response speed and performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0060] Figure 1 is a flow chart of a model optimization method provided in an embodiment of the present application;

[0061] Figure 2 It is a schematic diagram of a method for confirming a target model architecture provided in an embodiment of the present application;

[0062] Figure 3 This is a model optimization system architecture diagram provided in an embodiment of the present application;

[0063] Figure 4 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0065] It should be understood that in the description of the present application, unless the context clearly requires otherwise, words such as "include", "comprises", and the like in the entire specification and claims should be interpreted as inclusive rather than exclusive or exhaustive; that is, the meaning of "including but not limited to".

[0066] It should also be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more.

[0067] It should be noted that the terms "S1", "S2", etc. are only used for the purpose of describing the steps, and do not specifically refer to the order or sequence, nor are they used to limit the present application. They are only for the convenience of describing the method of the present application, and cannot be understood as indicating the order of the steps. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0068] It is worth noting that the model optimization method disclosed in this application can be applied to the design and training of any model to achieve memory optimization of the model.

[0069] Embodiment 1

[0070] The present application embodiment provides a model optimization method, specifically, Figure 1 As shown, the method disclosed in the embodiment of the present application is used to optimize the model design and training process, including the following contents:

[0071] S100, determining a target model architecture based on a preset search algorithm.

[0072] In order to reduce the memory usage of the final generated model, this application proposes to add memory usage constraints in the model architecture search process to optimize the model structure. Specifically, the above-mentioned target model architecture is determined based on a preset search algorithm, such as Figure 2 As shown, including the following:

[0073] S101. Confirm multiple candidate architectures that match current business task requirements.

[0074] Specifically, first, determine the basic model architecture according to the current business task requirements, that is, select a basic model architecture that suits the business needs, such as using recurrent neural networks to handle tasks such as natural language processing and time series prediction; using transformers (sequence models based on attention mechanisms) to process long sequence data, such as natural language processing and time series analysis.

[0075] Secondly, the search space is defined according to the structure of the basic model architecture; by analyzing the structure and characteristics of the basic model architecture, the modules and parameters that can be optimized are determined, thereby ensuring that the staff defines the search space based on the optimizable modules and parameters obtained above. This application does not limit the specific definition method. For example, in order to respond to image classification tasks, a convolutional neural network is selected as the basic model architecture. The model architecture usually includes: input layer: 224*224*3; convolution layer 1: 64 3*3 convolution kernels, stride 1, padding 1; pooling layer 1: 2*2 maximum pooling, stride 2; convolution layer 2: 128 3*3 convolution kernels, stride 1, padding 1; pooling layer 2: 2*2 maximum pooling, stride 2; fully connected layer 1: 256 nodes; fully connected layer 2: 128 nodes; output layer: 10 nodes (corresponding to 10 categories). At this time, the search space can be defined as the convolution layer: convolution kernel size: {3,5}, stride: {1,2}, padding: {1,2}, number of channels: {32,6 4,128,256}; Pooling layer: pooling kernel size: {2,3}, stride: {2,3}; Fully connected layer: number of nodes: {128,256,512}; Number of layers: number of convolutional layers: {3,4,5}, number of fully connected layers: {1,2}; Activation function: {ReLU,Swish}; Dropout ratio: {0.1,0.2,0.3,0.4,0.5}; Optimizer: {Adam,SGD}; Learning rate: {1e-4,1e-3,1e-2}; Learning rate schedule: {cosine annealing, learning rate decay}; Loss function: {cross entropy}; Data augmentation: random cropping: {224*224}, horizontal flip, color jitter.

[0076] Then, the model architecture to be processed is searched in the search space according to the preset search strategy. Specifically, a grid search method can be used to exhaust all possible combinations; or a random search method can be used to randomly select hyperparameter combinations and other methods to exhaust all possible combinations contained in the search space.

[0077] Finally, the estimated memory usage of the model architecture to be processed is obtained, and the estimated memory usage is compared with the maximum memory limit; in response to the estimated memory usage being less than the maximum memory limit, the model architecture to be processed is determined to be a candidate architecture; in response to the estimated memory usage being greater than or equal to the maximum memory limit, the model architecture to be processed is determined to be a non-candidate architecture. That is, in the process of architecture generation, an estimation of the memory usage of each candidate architecture is added; specifically, an estimation model can be established through the memory usage data of some existing architectures or components to estimate the memory of the newly generated model architecture to be processed, and further realize preliminary screening of the model architecture to be processed to obtain the candidate architecture.

[0078] The present application also proposes that the candidate model architecture can be determined directly using the neural network architecture search algorithm. Specifically, the candidate model architecture is determined by optimizing parameters based on the following formula: θ represents the parameters of the network architecture, including the choice of layers and the way of connection, T represents the set of tasks, P(T) is the probability distribution of the tasks, Represents the training data set for the current business task T In the above example, we use the loss function of the architecture parameter θ, λ is the coefficient of the regularization term, R(θ) represents the complexity penalty of the model, such as the number of parameters, which is used to avoid overly complex architectures, θ opt is the optimization target, that is, the architecture parameter corresponding to the expected value of the minimum loss function plus the regularization term under the current business task distribution. By adjusting the coefficients of the loss function and the regularization term, multiple θ opt , which corresponds to multiple candidate model architectures.

[0079] Through the above content, the present application realizes the preliminary screening of the processing space that may correspond to all combinations contained in the search space, reduces the subsequent computational complexity of determining the target model architecture among multiple candidate architectures, and further improves the speed of the entire simulation design and training.

[0080] S102. Calculate the evaluation scores of multiple candidate architectures based on the evaluation index formula. The evaluation index formula is: Eval(a) = ωAcc(a)-λmax(0,Mem(a)-Mmax); where a represents the candidate architecture, Eval(a) represents the evaluation score, ω represents the weight coefficient of the accuracy, λ represents the penalty coefficient for exceeding the memory limit, Mem(a) represents the memory usage, Acc(a) represents the accuracy of the candidate architecture, and Mmax represents the maximum memory limit. The above evaluation scores are usually obtained by verifying and evaluating.

[0081] Among them, in order to ensure that the model architecture runs within the memory limit, it is necessary to penalize the architecture that exceeds the memory usage. By setting a penalty term, it is automatically activated when the memory usage of the model architecture exceeds the limit. The design of the penalty term can be linear or exponential, depending on the strength of the penalty; it can be understood that the function design of the penalty term can be designed as p(a) = λ(Mem(a)-Mmax)k; where k>=1, which is used to control the nonlinear program of the penalty; further, the strictness of the memory constraint in the evaluation index formula can be adjusted by adjusting the parameter λ, so as to adapt to the search strategy at different times. For example, in the early stage of the search, a relatively loose standard may be allowed to obtain a variety of candidate architectures for basic evaluation, and in the later stage, the constraints on the memory limit are gradually strengthened to improve the memory friendliness of the model architecture finally selected. It is worth noting that the above function design is only an example and has no restrictive effect.

[0082] S103: According to the evaluation scores, determine a target model architecture that matches the current business task requirements from multiple candidate architectures, that is, use the architecture with the highest evaluation score among the multiple candidate architectures as the target model architecture.

[0083] Based on the model architecture search in a predefined search space, this application adds memory usage as a constraint, adjusts the generation and selection strategy of candidate architectures, and ensures that the target model architecture finally generated is an architecture that meets the memory constraints.

[0084] In addition, in order to further improve the memory friendliness of the model, the present application can design the model as multiple independent models when designing the model, and dynamically load and unload modules as needed to achieve memory optimization; reconfigurable architecture can also be used to dynamically adjust the allocation of computing resources by utilizing reconfigurable computing resources to optimize memory and computing efficiency.

[0085] S200, constructing an original model based on the target model architecture and training the original model to generate a pending model.

[0086] In a specific implementation, in the above process of training the original model to generate a pending model, the embodiment of the present application proposes the following optimization method:

[0087] S201. Insert gradient checkpoints to reduce memory consumption during back propagation. Specifically, by integrating the gradient checkpoint function in the training framework, for example, in the PyTorch (dynamic graph computing mode) training framework, import the torch.utils.checkpoint module to implement gradient checkpoints. Specifically, after defining the original model, use the checkpoint function in the forward method to insert gradient checkpoints. The checkpoint function accepts a function and input tensors as parameters, and only saves the input tensors during forward propagation, and recalculates the intermediate results during back propagation; then initialize the model and optimizer, and perform training. Or in the TensorFlow (static graph computing mode) training framework, import tensorflow.keras.layers and tensorflow.keras.models modules. After defining the original model, you can use tf.GradientTape to implement gradient checkpoints; tf.GradientTape allows us to recalculate intermediate results when calculating gradients; then initialize the model and optimizer, and perform training.

[0088] S202, configure the mixed precision training environment, select the operands and operation precision that match the operation. Use floating point numbers of different precisions to reduce memory usage and increase speed. Specifically, first configure the mixed precision training environment, ensure that the hardware supports floating point 16-bit operations, and then modify the training process, that is, convert the appropriate operations in the training process into floating point 16-bit format, through the formula: To implement, specifically, for each operation i (from 1 to N), select an operand operand i , select a precision i To perform the operation, perform the operation on the operand and the selected precision. i (such as addition, multiplication, etc.), and then multiply the result of the operation by a coefficient α i (This coefficient may be 1 or other values, which is used to adjust the impact of the operation on the final result). All adjusted operation results are added together to get the final mixed precision result mixed_result. Finally, the training speed and model accuracy loss are balanced by testing and adjusting the mixed precision.

[0089] S203, select a computing path according to the data characteristics of the input data of the network layer of the original model, and the computing path includes a parallel path and a low-resource path. Specifically, by analyzing the size, complexity and type of the input data of the input model, it is determined whether a high-performance parallel path or a simplified low-resource path is required, such as selecting a parallel path for a large data set, selecting a low-resource path for a small data set, or automatically selecting a parallel path when the data size exceeds a certain threshold, which is set by a person skilled in the art according to the actual scenario, or selecting a low-resource path when the complexity is low.

[0090] S204, monitor memory usage and transfer input data to a designated storage medium when memory usage reaches a preset condition. Specifically, real-time monitoring of the load of the CPU, GPU, FPGA, etc. is performed to determine memory usage, and when memory usage reaches a preset condition, the input data is transferred. The preset condition can be set to memory usage being 80% busy, etc., which is set by a person skilled in the art according to actual conditions, and this application does not limit this.

[0091] It is understandable that the above steps S201-S204 have no sequential execution order and can be executed simultaneously or in a certain order, and this application does not impose any restrictions on this. Through the above method, this application improves the training speed and saves memory to a certain extent during the training process of the model.

[0092] S300: Optimize the model to be determined based on preset memory optimization rules to generate a target model.

[0093] In a specific implementation scenario, the target model is generated by optimizing the model to be determined based on the preset memory optimization rules, including: obtaining the data characteristics of the input data of the network layer of the model to be determined, and determining the scaling factor according to the data characteristics; determining the quantization parameter according to the scaling factor and the data characteristics; and quantizing the input data in the network layer according to the quantization parameter. The compression effect is evaluated by the accuracy loss, inference speed and model size of the quantized model. If the accuracy loss is within an acceptable range after quantization, and the speed and storage are significantly improved, the compression is considered successful.

[0094] Wherein, the above-mentioned determination of the scaling factor according to the data characteristics includes: in response to the detection data characteristics being a wide dynamic range and uniform distribution (such as the pixel value in the figure), calculating the scaling factor according to the first formula, wherein the first formula is: scale(x)=max(x)-min(x), scale(x) represents the quantization factor, max(x) represents the maximum value in the input data, and min(x) represents the minimum value in the input data; on this basis, the above-mentioned determination of the quantization parameter according to the scaling factor and the data characteristics includes: in response to the detection data characteristics being a wide dynamic range and uniform distribution, calculating the quantization parameter according to the fourth formula, wherein the fourth formula is: Quantized(x) represents the quantization parameter, x represents the input data, and y represents the quantization level. The above quantization method based on maximum and minimum values ​​is direct and simple, and is suitable for stable and evenly distributed data.

[0095] Wherein, the above-mentioned determination of the scaling factor according to the data characteristics includes: in response to detecting that the data characteristics are nearly normally distributed, calculating the scaling factor according to the second formula, wherein the second formula is: scale(x)=2*σ(x), σ(x) represents the standard deviation of the input data; on this basis, the above-mentioned determination of the quantization parameter according to the scaling factor and the data characteristics includes: in response to detecting that the data characteristics are nearly normally distributed, calculating the quantization parameter according to the fifth formula, wherein the fifth formula is μ(x) represents the mean of the input data. The above quantification method based on mean and standard deviation is suitable for data with normal distribution or close to normal distribution. It has good practicality for mean and standard deviation, especially when the data distribution is centralized.

[0096] Wherein, the above-mentioned determination of the scaling factor according to the data characteristics includes: in response to detecting that the data characteristics are that there are a large number of outliers or the distribution is severely skewed, calculating the scaling factor according to the third formula, wherein the third formula is: scale(x)=Q3-Q1, Q3 represents the third quartile, and Q1 represents the first quartile. On this basis, the above-mentioned determination of the quantization parameter according to the scaling factor and the data characteristics includes: in response to detecting that the data characteristics are that there are a large number of outliers or the distribution is severely skewed, calculating the quantization parameter according to the sixth formula, wherein the sixth formula is

[0097] This application dynamically selects an appropriate quantization method and calculates the corresponding scaling factor based on the data characteristics of the input data of the network layer of the model to be determined; it realizes that the input data of each input network layer can get the scaling factor that best suits its characteristics, further determines the most appropriate quantization formula, realizes the adaptive quantization of the model, and reduces the quantization error, especially in the case of uneven data distribution. It is understandable that trigger conditions can also be set to update the scaling factor to achieve continuous optimization of the quantization effect; and through indicators such as the accuracy of the model after quantization and the value of the loss function, the effect and necessity of quantization are evaluated to determine whether the quantization parameters need to be tuned. The above trigger conditions can be set by technicians in this field according to the actual scenario, and this application does not limit them here.

[0098] On the basis of the above-disclosed adaptive quantization, the present application also discloses that after executing the above-mentioned quantization method to compress the model, if the performance of the quantized model still does not meet the requirements, the model can be further compressed in combination with pruning or low-rank decomposition. Specifically, in response to detecting that the performance of the quantized model does not meet the requirements, wherein the above-mentioned requirements are a standard set by technicians in the field based on experience, a pruning operation is triggered, such as weight-based threshold pruning or gradient-based pruning; in response to the end of the pruning operation, it is determined whether the model is compressed successfully; if the model is compressed successfully, the compression is stopped. If the model compression fails, the accuracy loss and acceleration effect of the model are obtained; in response to detecting that the accuracy loss is greater than the first preset value, a recovery operation is triggered and low-rank decomposition is triggered after the recovery operation is executed; in response to detecting that the model acceleration effect is improved and the accuracy loss is less than the second preset value, low-rank decomposition is directly triggered, wherein the first preset value and the second preset value are set by technicians in the field according to the actual scenario, and this application does not limit this. Determine whether the model is compressed successfully. If the model is compressed successfully, stop compression. If the model fails to compress, perform low-rank decomposition operation here and compare the performance of the model after secondary decomposition. If the performance of the model after secondary decomposition does not improve, the compression is considered successful. The success of model compression is determined by evaluating the performance, memory usage and task accuracy of the model on the hardware. If the accuracy of the model is maintained within an acceptable range while achieving the performance and storage goals, and increasing compression no longer brings significant benefits, the compression is considered complete.

[0099] This application adds memory as a constraint during model design to obtain a memory-friendly target model architecture, and further optimizes the model through memory optimization methods after determining the target model architecture, significantly reducing energy consumption and achieving maximum optimization of the model under limited memory resources, thereby improving the response speed and performance of the model.

[0100] Embodiment 2

[0101] Corresponding to the above-mentioned embodiment 1, as Figure 3 As shown, the present application provides a model optimization system, comprising:

[0102] A design module 310, for determining a target model architecture based on a preset search algorithm;

[0103] A training module 320 is used to construct an original model based on the target model architecture and train the original model to generate a pending model;

[0104] The optimization module 330 is used to optimize the pending model based on preset memory optimization rules to generate a target model.

[0105] In some implementation scenarios, the design module 310 is further used to confirm multiple candidate architectures that match the current business task requirements; calculate the evaluation scores of the multiple candidate architectures based on the evaluation index formula; and determine the target model architecture that matches the current business task requirements from the multiple candidate architectures according to the evaluation scores. The evaluation index formula is:

[0106] Eval(a)=ωAcc(a)-λmax(0,Mem(a)-Mmax);

[0107] Among them, a represents the candidate architecture, Eval(a) represents the evaluation score, ω represents the weight coefficient of accuracy, λ represents the penalty coefficient for exceeding the memory limit, Mem(a) represents the memory usage, Acc(a) represents the accuracy of the candidate architecture, and Mmax represents the maximum memory limit.

[0108] In some implementation scenarios, the design module 310 is also used to determine the basic model architecture according to the current business task requirements; define a search space according to the structure of the basic model architecture; search for the model architecture to be processed in the search space according to a preset search strategy; obtain the expected memory usage of the model architecture to be processed, and compare the expected memory usage with the maximum memory limit; in response to the expected memory usage being less than the maximum memory limit, determine that the model architecture to be processed is a candidate architecture; in response to the expected memory usage being greater than or equal to the maximum memory limit, determine that the model architecture to be processed is a non-candidate architecture.

[0109] In some implementation scenarios, the training module 320 is also used to insert gradient checkpoints to reduce memory consumption during back propagation; configure a mixed precision training environment, select operands and operation precision that match the operation; select a calculation path based on the data characteristics of the input data of the network layer, the calculation path including a parallel path and a low-resource path; monitor memory usage and transfer the input data to a designated storage medium when the memory usage reaches a preset condition.

[0110] In some implementation scenarios, the optimization module 330 is also used to obtain data characteristics of the input data of the network layer of the model to be determined, and determine a scaling factor based on the data characteristics; determine a quantization parameter based on the scaling factor and the data characteristics; and quantize the input data in the network layer based on the quantization parameter.

[0111] In some implementation scenarios, the optimization module 330 is also used to, in response to detecting that the data characteristic is a wide dynamic range and uniform distribution, calculate a scaling factor according to a first formula, wherein the first formula is: scale(x)=max(x)-min(x), scale(x) represents the quantization factor, max(x) represents the maximum value in the input data, and min(x) represents the minimum value in the input data; in response to detecting that the data characteristic is a near-normal distribution, calculate the scaling factor according to a second formula, wherein the second formula is: scale(x)=2*σ(x), σ(x) represents the standard deviation of the input data; in response to detecting that the data characteristic is the presence of a large number of outliers or a severely skewed distribution, calculate the scaling factor according to a third formula, wherein the third formula is: scale(x)=Q3-Q1, Q3 represents the third quartile, and Q1 represents the first quartile.

[0112] In some implementation scenarios, the optimization module 330 is further configured to calculate the quantization parameter according to a fourth formula in response to detecting that the data characteristic has a wide dynamic range and is evenly distributed, wherein the fourth formula is: Quantized(x) represents a quantization parameter, x represents input data, and y represents a quantization level; in response to detecting that the data characteristic is a near-normal distribution, the quantization parameter is calculated according to the fifth formula, wherein the fifth formula is μ(x) represents the mean of the input data; in response to detecting that the data characteristic is that there are a large number of outliers or the distribution is severely skewed, the quantization parameter is calculated according to the sixth formula, wherein the sixth formula is

[0113] Embodiment 3

[0114] Corresponding to all the above embodiments, the present application provides a computer program product, which implements the steps of the following method when the computer program is executed by a processor:

[0115] Determine the target model architecture based on a preset search algorithm;

[0116] Building an original model based on the target model architecture and training the original model to generate a pending model;

[0117] The model to be determined is optimized based on preset memory optimization rules to generate a target model.

[0118] In some implementation scenarios, when the computer program is executed by a processor, the steps of the following method are also implemented:

[0119] Identify multiple candidate architectures that match current business mission requirements;

[0120] Based on the evaluation index formula, calculate the evaluation scores of multiple candidate architectures;

[0121] According to the evaluation score, a target model architecture matching the current business task requirement is determined from the multiple candidate architectures.

[0122] The evaluation index formula is:

[0123] Eval(a)=ωAcc(a)-λmax(0,Mem(a)-Mmax);

[0124] Among them, a represents the candidate architecture, Eval(a) represents the evaluation score, ω represents the weight coefficient of accuracy, λ represents the penalty coefficient for exceeding the memory limit, Mem(a) represents the memory usage, Acc(a) represents the accuracy of the candidate architecture, and Mmax represents the maximum memory limit.

[0125] In some implementation scenarios, when the computer program is executed by a processor, the steps of the following method are also implemented:

[0126] Determine the basic model architecture according to the current business task requirements;

[0127] Defining a search space according to the structure of the basic model architecture;

[0128] Searching for a model architecture to be processed in the search space according to a preset search strategy;

[0129] Obtaining an estimated memory usage of the model architecture to be processed, and comparing the estimated memory usage with the maximum memory limit;

[0130] In response to the estimated memory usage being less than the maximum memory limit, determining the to-be-processed model architecture as a candidate architecture;

[0131] In response to the estimated memory usage being greater than or equal to the maximum memory limit, determining that the model architecture to be processed is a non-candidate architecture.

[0132] In some implementation scenarios, in the process of constructing the original model based on the target model architecture and training the original model, the computer program further implements the following method steps when executed by the processor:

[0133] Insert gradient checkpoints to reduce memory consumption during backpropagation;

[0134] Configure the mixed precision training environment and select the number of operations and operation precision that match the operation;

[0135] Selecting a computing path according to data characteristics of input data of the network layer, wherein the computing path includes a parallel path and a low resource path;

[0136] The memory usage is monitored and when the memory usage reaches a preset condition, the input data is transferred to a designated storage medium.

[0137] In some implementation scenarios, when the computer program is executed by a processor, the steps of the following method are also implemented:

[0138] Obtaining data characteristics of input data of a network layer of a pending model, and determining a scaling factor according to the data characteristics;

[0139] determining a quantization parameter according to the scaling factor and the data characteristic;

[0140] The input data in the network layer is quantized according to the quantization parameter.

[0141] In some implementation scenarios, when the computer program is executed by a processor, the steps of the following method are also implemented:

[0142] In response to detecting that the data characteristic is a wide dynamic range and uniform distribution, calculating a scaling factor according to a first formula, wherein the first formula is: scale(x)=max(x)-min(x), scale(x) represents a quantization factor, max(x) represents a maximum value in the input data, and min(x) represents a minimum value in the input data;

[0143] In response to detecting that the data characteristic is a near-normal distribution, calculating a scaling factor according to a second formula, wherein the second formula is: scale(x)=2*σ(x), σ(x) represents a standard deviation of the input data;

[0144] In response to detecting that the data characteristic is the presence of a large number of outliers or a severely skewed distribution, a scaling factor is calculated according to a third formula, wherein the third formula is: scale(x)=Q3-Q1, Q3 represents the third quartile, and Q1 represents the first quartile.

[0145] In some implementation scenarios, when the computer program is executed by a processor, the steps of the following method are also implemented:

[0146] In response to detecting that the data characteristic is a wide dynamic range and a uniform distribution, a quantization parameter is calculated according to a fourth formula, wherein the fourth formula is: Quantized(x) represents the quantization parameter, x represents the input data, and y represents the quantization level;

[0147] In response to detecting that the data characteristic is a near-normal distribution, the quantization parameter is calculated according to a fifth formula, wherein the fifth formula is μ(x) represents the mean of the input data;

[0148] In response to detecting that the data characteristic has a large number of outliers or a serious skewed distribution, the quantization parameter is calculated according to a sixth formula, wherein the sixth formula is

[0149] Embodiment 4

[0150] Corresponding to all the above embodiments, an embodiment of the present application provides an electronic device, including: one or more processors; and a memory associated with the one or more processors, the memory is used to store program instructions, and when the program instructions are read and executed by the one or more processors, the following operations are performed:

[0151] Determine the target model architecture based on a preset search algorithm;

[0152] Building an original model based on the target model architecture and training the original model to generate a pending model;

[0153] The pending model is optimized based on preset memory optimization rules to generate a target model.

[0154] in, Figure 4 The architecture of the electronic device is exemplarily shown, which may include a processor 410, a video display adapter 411, a disk drive 412, an input / output interface 413, a network interface 414, and a memory 420. The processor 410, the video display adapter 411, the disk drive 412, the input / output interface 413, the network interface 414, and the memory 420 may be communicatively connected via a bus 430.

[0155] Among them, the processor 410 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in this application.

[0156] The memory 420 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 420 can store an operating system 421 for controlling the execution of the electronic device 400, and a basic input and output system (BIOS) 422 for controlling the low-level operations of the electronic device 400. In addition, a web browser 423, a data storage management system 424, and an icon font processing system 425, etc. can also be stored. The above-mentioned icon font processing system 425 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided in the present application is implemented by software or firmware, the relevant program code is stored in the memory 420 and is called and executed by the processor 410.

[0157] The input / output interface 413 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0158] The network interface 414 is used to connect to a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0159] The bus 430 comprises a pathway for transmitting information between the various components of the device (eg, the processor 410, the video display adapter 411, the disk drive 412, the input / output interface 413, the network interface 414, and the memory 420).

[0160] In addition, the electronic device 400 can also obtain information on specific collection conditions from the virtual resource object collection condition information database for use in condition judgment, etc.

[0161] It should be noted that, although the above device only shows a processor 410, a video display adapter 411, a disk drive 412, an input / output interface 413, a network interface 414, a memory 420, a bus 430, etc., in the specific implementation process, the device may also include other components necessary for normal execution. In addition, it can be understood by those skilled in the art that the above device may also only include components necessary for implementing the solution of the present application, and does not necessarily include all the components shown in the figure.

[0162] It can be seen from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for a computer device (which can be a personal computer, a cloud service end, or a network device, etc.) to execute the methods of various embodiments of the present application or certain parts of the embodiments.

[0163] Embodiment 5

[0164] Corresponding to all the above embodiments, the embodiments of the present application further provide a computer-readable storage medium storing a computer program, which enables a computer to perform the following operations:

[0165] Determine the target model architecture based on a preset search algorithm;

[0166] Building an original model based on the target model architecture and training the original model to generate a pending model;

[0167] The pending model is optimized based on preset memory optimization rules to generate a target model.

[0168] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0169] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A model optimization method, characterized in that: The method comprises: Determine the target model architecture based on a preset search algorithm; Building an original model based on the target model architecture and training the original model to generate a pending model; The pending model is optimized based on preset memory optimization rules to generate a target model.

2. The method according to claim 1, characterized in that: The determining of the target model architecture based on a preset search algorithm includes: Identify multiple candidate architectures that match current business mission requirements; Based on the evaluation index formula, calculate the evaluation scores of multiple candidate architectures; According to the evaluation score, a target model architecture matching the current business task requirement is determined from the multiple candidate architectures.

3. The method according to claim 2, characterized in that The evaluation index formula is: Eval(a)=ωAcc(a)-λmax(0,Mem(a)-Mmax); Among them, a represents the candidate architecture, Eval(a) represents the evaluation score, ω represents the weight coefficient of accuracy, λ represents the penalty coefficient for exceeding the memory limit, Mem(a) represents the memory usage, Acc(a) represents the accuracy of the candidate architecture, and Mmax represents the maximum memory limit.

4. The method according to claim 3, characterized in that: The method of confirming a plurality of candidate architectures matching the current business task requirements includes: Determine the basic model architecture according to the current business task requirements; Defining a search space according to the structure of the basic model architecture; Searching for a model architecture to be processed in the search space according to a preset search strategy; Obtaining an estimated memory usage of the model architecture to be processed, and comparing the estimated memory usage with the maximum memory limit; In response to the estimated memory usage being less than the maximum memory limit, determining the to-be-processed model architecture as a candidate architecture; In response to the estimated memory usage being greater than or equal to the maximum memory limit, determining that the model architecture to be processed is a non-candidate architecture.

5. The method according to claim 1, characterized in that The process of constructing the original model based on the target model architecture and training the original model includes: Insert gradient checkpoints to reduce memory consumption during backpropagation; Configure the mixed precision training environment and select the number of operations and operation precision that match the operation; Selecting a computing path according to data characteristics of input data of the network layer, wherein the computing path includes a parallel path and a low resource path; The memory usage is monitored and when the memory usage reaches a preset condition, the input data is transferred to a designated storage medium.

6. The method according to claim 3, characterized in that: The step of optimizing the pending model based on a preset memory optimization rule to generate a target model includes: Acquire data characteristics of input data of the network layer of the pending model, and determine a scaling factor according to the data characteristics; determining a quantization parameter according to the scaling factor and the data characteristic; The input data in the network layer is quantized according to the quantization parameter.

7. The method according to claim 6, characterized in that The determining of a quantization parameter calculation rule according to the data characteristics includes: In response to detecting that the data characteristic is a wide dynamic range and uniform distribution, calculating a scaling factor according to a first formula, wherein the first formula is: scale(x)=max(x)-min(x), scale(x) represents a quantization factor, max(x) represents a maximum value in the input data, and min(x) represents a minimum value in the input data; In response to detecting that the data characteristic is a near-normal distribution, calculating a scaling factor according to a second formula, wherein the second formula is: scale(x)=2*′(x), where ′(x) represents a standard deviation of the input data; In response to detecting that the data characteristic is the presence of a large number of outliers or a severely skewed distribution, a scaling factor is calculated according to a third formula, wherein the third formula is: scale(x)=Q3-Q1, Q3 represents the third quartile, and Q1 represents the first quartile.

8. The method according to claim 6, characterized in that The step of determining the scaling factor according to the data characteristics comprises: In response to detecting that the data characteristic is a wide dynamic range and a uniform distribution, a quantization parameter is calculated according to a fourth formula, wherein the fourth formula is: Quantized(x) represents the quantization parameter, x represents the input data, and y represents the quantization level; In response to detecting that the data characteristic is a near-normal distribution, the quantization parameter is calculated according to a fifth formula, wherein the fifth formula is μ(x) represents the mean of the input data; In response to detecting that the data characteristic has a large number of outliers or a serious skewed distribution, the quantization parameter is calculated according to a sixth formula, wherein the sixth formula is 9. An electronic device, characterized in that: The electronic device comprises: one or more processors; And a memory associated with the one or more processors, the memory is used to store program instructions, and when the program instructions are read and executed by the one or more processors, the method according to any one of claims 1-7 is executed.

10. A computer-readable storage medium, characterized in that: The computer program stores a computer program, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 7.

Images