A model deployment method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a model deployment method and device, electronic equipment and storage medium. The electronic equipment obtains a service sample corresponding to a target platform; according to a preset pruning rate and a preset alignment parameter, prunes a channel of a to-be-pruned model; obtains a time consumption and an accuracy of the pruned model running on the target platform; determines a pruning loss based on the time consumption and the accuracy level; updates the preset pruning rate and the preset alignment parameter according to the pruning loss; returns the channel of the to-be-pruned model until the preset pruning rate before and after the update is the same; prunes the to-be-pruned model based on the target alignment parameter and the target pruning rate; and deploys the target model to the target platform. Since the pruning loss is determined according to the time consumption difference and the accuracy difference, the change of the time consumption and the accuracy before and after the model pruning is reflected, the preset pruning rate and the preset alignment parameter are updated according to the pruning loss, the time consumption of the model running on the target platform is reduced on the premise of ensuring the model accuracy, and the running speed and the accuracy of the model after being deployed to the platform are improved.

Description

Technical Field

[0001] This application relates to the field of model processing technology, and in particular to a model deployment method, apparatus, electronic device and storage medium. Background Technology

[0002] When a model is deployed to an embedded device on the platform, the model's weight coefficients become redundant for specific business data due to the specific nature of the business data processed by the platform and the often large depth of the model. Therefore, before deployment, redundancy can be eliminated through pruning, resulting in a more streamlined model and improved running speed.

[0003] Currently, related technologies typically involve cropping the model while keeping the model platform alignment parameters fixed. Since the model platform alignment parameters are constant, the cropping process is identical across all platforms, resulting in low compatibility between the cropped model and the platform. Consequently, the model's speed and accuracy are both low after deployment. Summary of the Invention

[0004] The purpose of this application is to provide a model deployment method, apparatus, electronic device, and storage medium to improve the running speed and accuracy of models after deployment to the platform. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide a model deployment method, the method comprising:

[0006] Obtain business samples corresponding to the target platform, wherein the target platform is the platform where the model to be tailored is to be deployed;

[0007] Based on the preset clipping rate and preset alignment parameters, the channels of the model to be clipped are clipped to obtain the clipped model.

[0008] The first time required to process the business sample and the first accuracy required to process the business sample are obtained when the cropped model is run on the target platform.

[0009] Based on the difference between the first time consumption and the second time consumption, and the difference between the first precision and the second precision, the pruning loss is determined, wherein the second time consumption is the time required for the model to be pruned to process the business sample when running on the target platform, and the second precision is the precision of the model to be pruned in processing the business sample when running on the target platform;

[0010] Update the preset cropping rate and the preset alignment parameters based on the cropping loss;

[0011] Return to the step of cropping the channels of the model to be cropped according to the preset cropping rate and the preset alignment parameter, until the updated preset cropping rate is the same as the original preset cropping rate, and use the updated preset cropping rate as the target cropping rate and the updated preset alignment parameter as the target alignment parameter;

[0012] Based on the target alignment parameters, the model to be clipped is clipped according to the target clipping rate to obtain the target model, wherein the target alignment parameters reflect the degree of adaptation between the target platform and the target model;

[0013] The target model is deployed to the target platform so that the target platform runs the target model and processes the business data of the target platform through the target model.

[0014] Optionally, the cutting loss characterizes the degree of reduction in the first time consumption relative to the second time consumption under the constraint of the difference between the first precision and the second precision;

[0015] The step of updating the preset cropping rate and the preset alignment parameter based on the cropping loss includes:

[0016] With the reduction of the cutting loss as the optimization direction, the preset cutting rate and the preset alignment parameters are updated.

[0017] Optionally, the step of determining the cutting loss based on the difference between the first time consumption and the second time consumption, and the difference between the first accuracy and the second accuracy, includes:

[0018] Based on the difference between the first and second time consumption, and the difference between the first and second precision, the clipping loss F(X, δ) is calculated according to the following loss function:

[0019] F(X, δ)=f(X)+δmax{e(X)-ε, 0} 2

[0020] X=[PR]=[pr1, pr2,…,pr n ,align]

[0021] Where δ is a preset penalty coefficient, δ is greater than a preset positive number, X is a vector composed of the preset clipping rate and the preset alignment parameter, f(X) is the ratio of the first time consumption to the second time consumption, e(X) is the difference between the first precision and the second precision, ε is the precision error threshold, and pr i The preset clipping rate is the i-th convolutional group of the model to be clipped, where i = 1, 2, ..., n, n is the number of convolutional groups of the model to be clipped, and align is the preset alignment parameter.

[0022] Optionally, the step of updating the preset cut rate and the preset alignment parameters with the reduction of the cut loss as the optimization direction includes:

[0023] If the difference between the first precision and the second precision is not greater than the precision error threshold, a target X is determined that minimizes F(X, δ).

[0024] Optionally, the step of determining the target X that minimizes F(X, δ) when the difference between the first precision and the second precision is not greater than the precision error threshold includes:

[0025] For each particle, the particle with the smallest value of F(X, δ) in each update is taken as the individual optimal particle, where the position vector of the particle is X, and the elements in the position vector represent the preset clipping rate and the preset alignment parameter.

[0026] The individual optimal particle with the smallest corresponding F(X, δ) value is determined as the global optimal particle;

[0027] For each particle, the position vector and velocity of the particle are updated based on the inertia weight, the position vector of the globally optimal particle, the position vector of the individual optimal particle corresponding to the particle, the position vector and velocity of the particle.

[0028] Optionally, the step of updating the position vector and velocity of the particle based on the inertia weight, the position vector of the globally optimal particle, the position vector of the individual optimal particle corresponding to the particle, and the position vector and velocity of the particle includes:

[0029] The velocity of the particle is updated based on the position vector of the individual optimal particle corresponding to the particle, the position vector and velocity of the particle, the position vector of the global optimal particle, and the inertia weight.

[0030] If the updated velocity of the particle is greater than the preset velocity, the updated velocity will be adjusted to a value no greater than the preset velocity.

[0031] The particle's position vector is updated based on the updated velocity and the particle's position vector.

[0032] If the updated position vector contains elements that exceed the boundary, the direction of the updated velocity is reversed, and the particle's position vector is updated based on the reversed velocity and the particle's position vector.

[0033] Optionally, the initial value of the element indicating the preset clipping rate in the particle's position vector is less than a preset threshold, and the inertia weight is less than the inertia weight before the update.

[0034] Secondly, embodiments of this application provide a model deployment apparatus, the apparatus comprising:

[0035] The sample acquisition module is used to acquire business samples corresponding to the target platform, wherein the target platform is the platform where the model to be tailored is to be deployed;

[0036] The first trimming module is used to trim the channels of the model to be trimmed according to a preset trimming rate and preset alignment parameters to obtain the trimmed model.

[0037] The parameter acquisition module is used to acquire the first time required for the cropped model to process the business sample and the first accuracy required to process the business sample when the model is running on the target platform.

[0038] The loss determination module is used to determine the pruning loss based on the difference between the first time consumption and the second time consumption, and the difference between the first precision and the second precision, wherein the second time consumption is the time required for the model to be pruned to process the business sample when the model to be pruned is running on the target platform, and the second precision is the precision of the model to be pruned in processing the business sample when the model to be pruned is running on the target platform.

[0039] The parameter update module is used to update the preset cropping rate and the preset alignment parameters according to the cropping loss;

[0040] The target parameter determination module is used to return the step of cropping the channels of the model to be cropped according to the preset cropping rate and the preset alignment parameter, until the updated preset cropping rate is the same as the original preset cropping rate, the updated preset cropping rate is used as the target cropping rate, and the updated preset alignment parameter is used as the target alignment parameter.

[0041] The second trimming module is used to trim the model to be trimmed according to the target trimming rate based on the target alignment parameter to obtain the target model, wherein the target alignment parameter reflects the degree of adaptation between the target platform and the target model;

[0042] The model deployment module is used to deploy the target model to the target platform so that the target platform runs the target model and processes the business data of the target platform through the target model.

[0043] Thirdly, embodiments of this application provide an electronic device, including:

[0044] Memory, used to store computer programs;

[0045] When a processor executes a program stored in memory, it implements any of the methods described in the first aspect above.

[0046] Fourthly, a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the methods described in the first aspect.

[0047] Beneficial effects of the embodiments in this application:

[0048] In the solution provided in this application embodiment, the electronic device can obtain business samples corresponding to the target platform, wherein the target platform is the platform where the model to be pruned is to be deployed; according to a preset pruning rate and preset alignment parameters, the channels of the model to be pruned are pruned to obtain the pruned model; the first time and the first precision required for the pruned model to process the business samples when running on the target platform are obtained; based on the difference between the first time and the second time, and the difference between the first precision and the second precision, the pruning loss is determined, wherein the second time is the time required for the model to be pruned to process the business samples when running on the target platform, and the second precision is the time required for the model to be pruned to process the business samples when running on the target platform. This process involves: refining the processing accuracy; updating the preset clipping rate and preset alignment parameters based on the clipping loss; returning to the steps of clipping the channels of the model to be clipped according to the preset clipping rate and preset alignment parameters until the updated preset clipping rate is the same as the original preset clipping rate, using the updated preset clipping rate as the target clipping rate, and using the updated preset alignment parameters as the target alignment parameters; clipping the model to be clipped according to the target clipping rate based on the target alignment parameters to obtain the target model, where the target alignment parameters reflect the degree of adaptation between the target platform and the target model; deploying the target model to the target platform so that the target platform can run the target model and process the business data of the target platform through the target model. Since the clipping loss is determined based on the differences in time consumption and accuracy, the clipping loss can reflect the changes in time consumption and accuracy before and after model clipping. Therefore, updating the preset clipping rate and preset alignment parameters according to the clipping loss can reduce the time consumption of the model running on the target platform while ensuring model accuracy. When the updated preset clipping rate is the same as the original preset clipping rate, the model takes the shortest time to run on the target platform while meeting accuracy constraints. Therefore, in this case, the updated preset clipping rate and preset alignment parameters can be used as the target clipping rate and target alignment parameters, respectively. Deploying the target model, obtained by clipping the channels of the model to be clipped according to the target alignment parameters and the target clipping rate, to the target platform can improve the running speed and accuracy of the model after deployment.

[0049] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0051] Figure 1 A flowchart illustrating a model deployment method provided in an embodiment of this application;

[0052] Figure 2 Based on Figure 1 A flowchart illustrating a method for updating the velocity and position vectors of particles in the illustrated embodiment;

[0053] Figure 3 Based on Figure 2 Another flowchart illustrating the updating method of the particle's velocity and position vectors in the illustrated embodiment;

[0054] Figure 4 Based on Figure 1 The illustrated embodiment is a flowchart of determining the optimal clipping ratio and optimal alignment parameters, and clipping the original model according to the optimal clipping ratio and optimal alignment parameters.

[0055] Figure 5 Based on Figure 1 A flowchart illustrating the determination of the optimal cropping model in the embodiment shown;

[0056] Figure 6 This is a schematic diagram of the structure of a model deployment device provided in an embodiment of this application;

[0057] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0059] To improve the running speed and accuracy of models after deployment to a platform, embodiments of this application provide a model deployment method, apparatus, electronic device, computer-readable storage medium, and computer program product. The model deployment method provided in this application embodiment is described below.

[0060] The model deployment method provided in this application can be applied to any electronic device that needs to deploy models, such as a model deployment system or model deployment device, etc., without specific limitations. For clarity, it will be referred to as an electronic device herein.

[0061] like Figure 1 As shown, a model deployment method includes:

[0062] S101, Obtain the business sample corresponding to the target platform;

[0063] The target platform is the platform on which the model to be pruned is to be deployed.

[0064] S102, according to the preset clipping rate and preset alignment parameters, the channels of the model to be clipped are clipped to obtain the clipped model;

[0065] S103, obtain the first time required to process the business sample and the first accuracy required to process the business sample when the trimmed model is running on the target platform;

[0066] S104, Based on the difference between the first time consumption and the second time consumption, and the difference between the first precision and the second precision, determine the trimming loss;

[0067] Wherein, the second time consumption is the time required for the model to be pruned to process the business sample when running on the target platform, and the second accuracy is the accuracy of the model to be pruned in processing the business sample when running on the target platform.

[0068] S105, Update the preset cutting rate and the preset alignment parameters according to the cutting loss;

[0069] S106, return to the step of cropping the channels of the model to be cropped according to the preset cropping rate and the preset alignment parameter, until the updated preset cropping rate is the same as the original preset cropping rate, and use the updated preset cropping rate as the target cropping rate and the updated preset alignment parameter as the target alignment parameter.

[0070] S107, Based on the target alignment parameters, the model to be clipped is clipped according to the target clipping rate to obtain the target model;

[0071] The target alignment parameter reflects the degree of compatibility between the target platform and the target model.

[0072] S108, the target model is deployed to the target platform so that the target platform runs the target model and processes the business data of the target platform through the target model.

[0073] As can be seen, in this embodiment, the electronic device can obtain business samples corresponding to the target platform, where the target platform is the platform where the model to be pruned is to be deployed; according to a preset pruning rate and preset alignment parameters, the channels of the model to be pruned are pruned to obtain the pruned model; the first time required for the pruned model to process the business samples and the first precision required for processing the business samples when running on the target platform are obtained; based on the difference between the first time and the second time, and the difference between the first precision and the second precision, the pruning loss is determined, where the second time is the time required for the model to be pruned to process the business samples when running on the target platform, and the second precision is the time required for the model to be pruned to process the business samples when running on the target platform. The process involves: refining the processing accuracy; updating the preset clipping rate and preset alignment parameters based on the clipping loss; returning the steps of clipping the channels of the model to be clipped according to the preset clipping rate and preset alignment parameters until the updated preset clipping rate is the same as the original preset clipping rate; using the updated preset clipping rate as the target clipping rate and the updated preset alignment parameters as the target alignment parameters; clipping the model to be clipped according to the target clipping rate based on the target alignment parameters to obtain the target model, where the target alignment parameters reflect the degree of compatibility between the target platform and the target model; and deploying the target model to the target platform so that the target platform can run the target model and process the business data of the target platform through the target model. Since the clipping loss is determined based on the differences in time consumption and accuracy, it can reflect the changes in time consumption and accuracy before and after model clipping. Therefore, updating the preset clipping rate and preset alignment parameters based on the clipping loss can reduce the time consumption of the model running on the target platform while ensuring model accuracy. When the updated preset clipping rate is the same as the original preset clipping rate, the model takes the shortest time to run on the target platform while meeting accuracy constraints. Therefore, in this case, the updated preset clipping rate and preset alignment parameters can be used as the target clipping rate and target alignment parameters, respectively. Deploying the target model, obtained by clipping the channels of the model to be clipped according to the target alignment parameters and the target clipping rate, to the target platform can improve the running speed and accuracy of the model after deployment.

[0074] Before deploying the model to be trimmed to the target platform, the electronic device can obtain the corresponding business samples of the target platform, i.e., execute step S101. In this way, after trimming the channels of the model to be trimmed, the trimmed model can be run on the target platform, thereby obtaining the time consumed by the trimmed model on the target platform, the time required to process the business samples, and the accuracy required to process the business samples. Then, based on the obtained time and accuracy, the model is further trimmed, optimizing in the direction of minimizing time while ensuring accuracy, so that the trimmed model reduces the time consumed on the target platform while meeting accuracy constraints.

[0075] For example, the target platform can be a CPU (Central Processing Unit), GPU (Graphics Processing Unit), or NPU (Neural Processing Unit). If the application scenario is intelligent transportation, then the business samples can be road condition videos or images captured by cameras installed on the road, and the model to be cropped can be a relevant deep learning model used for license plate detection or lane detection.

[0076] To reduce the runtime of the model to be clipped under accuracy constraints, electronic devices can iteratively update the preset clipping rate and preset alignment parameters based on the runtime and accuracy of the clipped model on the target platform. When clipping the channels of the model to be clipped for the first time, the preset clipping rate and preset alignment parameters can be the initial clipping rate and initial alignment parameters. To speed up the search, the initial clipping rate is usually set to a small value. In subsequent clipping operations on the channels of the model to be clipped, the preset clipping rate and preset alignment parameters can be determined based on the runtime and accuracy of the clipped model on the target platform.

[0077] Then, the electronic device can trim the channels of the model to be trimmed according to the preset trimming rate and preset alignment parameters to obtain the trimmed model, i.e., execute step S102.

[0078] In one implementation, the electronic device can use L1NORM (L1 Norm filter pruner), L2NORM (L2 Norm filter pruner), or other methods that measure the channel weights of the model to be pruned to prune the model.

[0079] If the trimmed model, when running on the target platform, requires a short initial processing time and a high initial processing accuracy for the business samples, it indicates a good fit between the trimmed model and the target platform. Therefore, to determine the fit between the trimmed model and the target platform, the electronic device can obtain the initial processing time and the initial processing accuracy of the business samples when the trimmed model runs on the target platform, i.e., execute step S103.

[0080] If the difference between the first and second execution times is significant, it indicates that the execution time of the trimmed model on the target platform is reduced considerably compared to the execution time of the model to be trimmed on the target platform. If the difference between the first and second precision is small, it indicates that the precision reduction of the trimmed model in processing business samples is less compared to the precision of the model to be trimmed in processing business samples. Therefore, the electronic device can determine the trimming loss based on the difference between the first and second execution times, and the difference between the first and second precision, i.e., execute step S104.

[0081] Since the pruning loss reflects the reduction in running time and accuracy of the pruned model on the target platform compared to the model to be pruned, the electronic device can update the preset pruning rate and preset alignment parameters based on the pruning loss, i.e., execute step S105. This further shortens the running time of the model to be pruned on the target platform and further improves the accuracy of processing business samples after pruning based on the updated preset pruning rate and preset alignment parameters.

[0082] If the updated preset cropping rate is different from the original preset cropping rate, it means that the updated preset cropping rate is still decreasing compared to the original preset cropping rate. The preset cropping rate of the model to be cropped can be further reduced, so the electronic device can return to step S102.

[0083] If the updated preset clipping ratio is the same as the original preset clipping ratio, it means that the updated preset clipping ratio has not changed compared to the original preset clipping ratio. The preset clipping ratio of the model to be clipped has been reduced to the minimum under the accuracy constraint. Therefore, the electronic device can use the updated preset clipping ratio as the target clipping ratio and the updated preset alignment parameter as the target alignment parameter.

[0084] Under the accuracy constraint, after determining the target clipping ratio and target alignment parameters that minimize the running time on the target platform, the electronic device can clip the model to be clipped according to the target clipping ratio based on the target alignment parameters to obtain the target model, i.e., execute step S107.

[0085] After obtaining the target model, the electronic device can deploy the target model to the target platform so that the target platform can run the target model and process the business data of the target platform through the target model, that is, execute step S108.

[0086] As can be seen, in this embodiment, since the pruning loss is determined based on the differences in time consumption and accuracy, the pruning loss can reflect the changes in time consumption and accuracy before and after model pruning. Therefore, updating the preset pruning rate and preset alignment parameters according to the pruning loss can reduce the time consumed by the model running on the target platform while ensuring model accuracy. When the updated preset pruning rate is the same as the original preset pruning rate, the model has the shortest running time on the target platform while meeting accuracy constraints. Therefore, in the above case, the updated preset pruning rate and preset alignment parameters can be used as the target pruning rate and target alignment parameters, respectively. In this way, the target model obtained by pruning the model to be pruned according to the target pruning rate based on the target alignment parameters can be deployed to the target platform, which can improve the running speed and accuracy of the model after deployment to the platform.

[0087] As one embodiment of this application, the above-mentioned trimming loss characterizes the degree of reduction of the first time consumption relative to the second time consumption under the limitation of the difference between the first precision and the second precision;

[0088] The steps described above for updating the preset cropping rate and the preset alignment parameter based on the cropping loss may include:

[0089] With the reduction of the cutting loss as the optimization direction, the preset cutting rate and the preset alignment parameters are updated.

[0090] Since the trimming loss can characterize the degree of reduction in the first time consumption relative to the second time consumption under the constraint of the difference between the first precision and the second precision, in order to determine the preset trimming rate and preset alignment parameters that can reduce the second time consumption under the condition of satisfying the difference between the first precision and the second precision, the electronic device can update the preset trimming rate and preset alignment parameters with the reduction of trimming loss as the optimization direction.

[0091] As can be seen, in this embodiment, the electronic device can update the preset cut rate and preset alignment parameters with the reduction of cut loss as the optimization direction. Since cut loss characterizes the degree to which the first time consumption is reduced relative to the second time consumption under the constraint of the difference between the first and second precision, in order to determine the preset cut rate and preset alignment parameters that can reduce the second time consumption under the constraint of the difference between the first and second precision, the electronic device can update the preset cut rate and preset alignment parameters with the reduction of cut loss as the optimization direction. In this way, the preset cut rate and preset alignment parameters can be updated quickly and accurately based on the cut loss.

[0092] As one implementation of this application, the step of determining the trimming loss based on the difference between the first time consumption and the second time consumption, and the difference between the first accuracy and the second accuracy, may include:

[0093] Based on the difference between the first and second time consumption, and the difference between the first and second precision, the clipping loss F(X, δ) is calculated according to the following loss function:

[0094] F(X, δ)=f(X)+δmax{e(X)-ε, 0} 2

[0095] Wherein, δ is a preset penalty coefficient, δ is greater than a preset positive number, X is a vector composed of the preset clipping rate and the preset alignment parameter, f(X) is the ratio of the first time consumption to the second time consumption, e(X) is the difference between the first precision and the second precision, and ε is the precision error threshold.

[0096] As one implementation, the preset clipping ratio and preset alignment parameters can be represented as a vector X = [PR] = [pr1, pr2, ..., pr... n [align]. The model to be pruned includes n convolutional groups, pr i is the clipping rate corresponding to the i-th convolutional group of the model to be clipped, and align is the preset alignment parameter.

[0097] If the first time taken is time' and the second time taken is time, then f(X) can be expressed as: Since the first time is the time taken for the model to be pruned to run on the target platform, and the second time is the time taken for the pruned model to run on the target platform, and the pruned model takes less time to run on the target platform than the unpruned model, the first time is greater than the second time, and f(X) < 1.

[0098] If the first precision is acc' and the second precision is acc, then e(X) can be expressed as e(X) = acc' - acc. Since the first precision is the precision of the model to be pruned in processing business samples, and the second precision is the precision of the model to be pruned in processing business samples, and in general, the pruned model has lower precision in processing business samples than the uncrunted model, the first precision is less than the second precision.

[0099] As the preset clipping rate increases, the initial execution time of the clipped model on the target platform decreases, and the initial accuracy of the clipped model in processing business samples also decreases. Therefore, the electronic device can take the minimum value of f(X), which is the minimum initial execution time, while ensuring that the error between the first and second accuracy is no greater than the accuracy error threshold. This can be expressed as:

[0100] As one implementation method, constrained optimization methods can be used to solve the problem. Specifically, a loss function can be constructed as F(X, δ) = f(X) + δmax{e(X) - ε, 0} 2 To calculate the trimming loss, in the loss function, the precision error constraint e(X)≤ε is transformed into a penalty term δmax{e(X)-ε,0}. 2 If the difference between the first precision and the second precision is less than or equal to the precision error threshold, it means that the precision error limit is met.

[0101] If the precision error limit is not met, i.e., X is outside the feasible region, then e(X)-ε>0, and thus max{e(X)-ε,0} 2 =[e(X)-ε] 2 >0. Since the penalty term coefficient δ is a pre-set value greater than a preset positive number, F(X, δ) > f(X).

[0102] Under the condition that the accuracy error limit is met, that is, when X is within the feasible region, e(X)-ε≤0, and thus max{e(X)-ε,0} 2 =0, then F(X,δ) = f(X).

[0103] As can be seen, in this embodiment of the application, the electronic device can calculate the clipping loss F(X, δ) based on the difference between the first time consumption and the second time consumption, and the difference between the first precision and the second precision, according to the following loss function: F(X, δ)=f(X)+δmax{e(X)-ε,0} 2 Where δ is the preset penalty coefficient, which is greater than a preset positive number; X is a vector composed of the preset clipping rate and preset alignment parameters; f(X) is the ratio of the first time consumption to the second time consumption; e(X) is the difference between the first precision and the second precision; and ε is the precision error threshold. Since the loss function can accurately reflect the precision and time consumption of the clipped model, updating the preset clipping rate and preset alignment parameters based on the loss function can quickly and accurately determine the target clipping rate and target alignment parameters.

[0104] As one embodiment of this application, the above-described step of updating the preset cut rate and the preset alignment parameter with the reduction of the cut loss as the optimization direction may include:

[0105] If the difference between the first precision and the second precision is not greater than the precision error threshold, a target X is determined that minimizes F(X, δ).

[0106] Since the penalty term is a large positive number when X is outside the feasible region, and zero when X is inside the feasible region, F(X, δ) for X outside the feasible region is greater than F(X, δ) for X inside the feasible region. Furthermore, the shorter the initial execution time, the smaller the value of f(X). Therefore, when F(X, δ) is minimized, the difference between the first and second precision is no greater than the precision error threshold, and the shortest initial execution time corresponds to the highest degree of adaptation between the trimmed model and the target platform.

[0107] Therefore, the solution can be obtained. By converting the problem into solving for minF(X,δ), the electronic device can determine the target X that minimizes F(X,δ) when the difference between the first precision and the second precision is no greater than the precision error threshold.

[0108] As can be seen, in this embodiment, the electronic device can determine the target X that minimizes F(X, δ) when the difference between the first precision and the second precision is no greater than the precision error threshold. Since F(X, δ) for X outside the feasible region is greater than F(X, δ) for X within the feasible region, and the shorter the first time consumption, the smaller the value of f(X), the difference between the first precision and the second precision is no greater than the precision error threshold when F(X, δ) is minimized, and the first time consumption is the shortest, meaning the corresponding trimmed model has the highest degree of adaptation to the target platform. Thus, the electronic device can quickly and accurately determine the target X.

[0109] As one implementation method of this application, such as Figure 2 As shown, the step of determining the target X that minimizes F(X, δ) when the difference between the first precision and the second precision is not greater than the precision error threshold may include:

[0110] S201, For each particle, the particle with the smallest value of F(X, δ) in each update is taken as the individual optimal particle.

[0111] Wherein, the position vector of the particle is X, and the elements in the position vector represent the preset clipping rate and the preset alignment parameter.

[0112] Since a formulaically expressible functional relationship cannot be established between F(X, δ) and the independent variable X, the PSO (Particle Swarm Optimization) algorithm can be used to solve for the objective X that minimizes F(X, δ). It should be noted that the algorithm used to solve for the objective X can also be a genetic algorithm or a reinforcement learning algorithm; no specific limitation is made here.

[0113] The loss function in the PSO algorithm is F(X, δ) = f(X) + δmax{e(X) - ε, 0} 2 The search space has the same dimension as X, containing the same number of elements, and each particle's position vector is X. For example, if X = [pr1, pr2, pr3, pr4, align], then the search space has 5 dimensions, and the spatial positions of the particles in the 5 dimensions are pr1, pr2, pr3, pr4, and align, respectively.

[0114] There are usually multiple particles, and each particle corresponds to a position vector X. For example, if there are 50 particles, then the position vectors corresponding to particles 1 to 50 are position vectors 1 to 50, respectively.

[0115] The electronic device can update the velocity and position of each particle in the search space iteratively. In this way, the electronic device can use the preset clipping rate and preset alignment parameters represented by the elements of the updated particle's position vector as the updated preset clipping rate and updated preset alignment parameters.

[0116] Specifically, the electronic device can calculate the velocity of the i-th particle in the d-th dimension during the (t+1)-th iteration using the following formula:

[0117] Wherein, ω is the inertia weight, and the value of ω is usually between 0.4 and 0.9. Let be the velocity of the i-th particle in the d-th dimension during the t-th iteration, c1 be the individual learning factor, c2 be the social learning factor, and r1 and r2 be random numbers taking values ​​in the range [0, 1]. pbest i，d Let gbest be the position of the i-th particle in the d-th dimension. The optimal particle for the i-th particle in the d-th dimension is the particle that minimizes the loss function in each iteration of the d-th dimension. d Let d be the position of the globally optimal particle in the d-th dimension. The globally optimal particle in the d-th dimension is the particle that minimizes the loss function value among all particles in each iteration of the d-th dimension. Let be the position of the i-th particle in the d-th dimension during the t-th iteration.

[0118] To update the velocity and position of particles, the electronic device can select the particle with the smallest F(X, δ) value in each update as the individual optimal particle. For example, if the loss functions of the particles in the first to fifth iterations are 56, 54, 53, 50, and 51 respectively, the electronic device can select the particle in the fourth iteration as the individual optimal particle.

[0119] S202, the individual optimal particle with the smallest corresponding F(X,δ) value is determined as the global optimal particle;

[0120] Since the globally optimal particle is always the individually optimal particle, the electronic device can determine the individually optimal particle with the smallest corresponding F(X, δ) value as the globally optimal particle. For example, if there are 5 particles, and the F(X, δ) values ​​corresponding to the individually optimal particles of particles 1 to 5 are 43, 42, 43, 41, and 43 respectively, then the electronic device can determine the individually optimal particle corresponding to particle 4 as the globally optimal particle.

[0121] S203, for each particle, update the particle's position vector and velocity based on the inertia weight, the position vector of the globally optimal particle, the position vector of the individual optimal particle corresponding to the particle, the particle's position vector and velocity.

[0122] After determining the individual optimal particle and the global optimal particle, for each particle, the electronic device can update the position vector and velocity of the particle based on the inertia weight, the position vector of the global optimal particle, the position vector of the corresponding individual optimal particle, and the position vector and velocity of the particle.

[0123] In one implementation, the electronic device can calculate the velocity of the i-th particle in the d-th dimension during the (t+1)-th iteration according to the following formula: The formula for calculating the position vector of the i-th particle in the d-th dimension during the (t+1)-th iteration is: in, Let be the position vector of the i-th particle in the d-th dimension during the t-th iteration.

[0124] In this way, the electronic device can use the preset clipping rate and preset alignment parameters represented by the elements in the position vector corresponding to the updated particle as the updated preset clipping rate and updated preset alignment parameters.

[0125] As can be seen, in this embodiment, the electronic device can, for each particle, identify the particle with the smallest F(X, δ) value in each update as the individual optimal particle, where the particle's position vector is X, and the elements in the position vector represent a preset clipping rate and a preset alignment parameter; the individual optimal particle with the smallest corresponding F(X, δ) value is determined as the global optimal particle; for each particle, the position vector and velocity of the particle are updated based on the inertia weight, the position vector of the global optimal particle, the position vector of the corresponding individual optimal particle, the particle's position vector, and its velocity. Since the electronic device can use the PSO algorithm to iteratively determine the target X that minimizes F(X, δ), the electronic device can use the preset clipping rate and preset alignment parameter represented by the elements in the updated particle's position vector as the updated preset clipping rate and updated preset alignment parameter, thus quickly and accurately determining the target X that minimizes F(X, δ).

[0126] As one implementation method of this application, such as Figure 3 As shown, the steps described above for updating the position vector and velocity of a particle based on the inertia weight, the position vector of the globally optimal particle, the position vector of the individual optimal particle corresponding to that particle, and the particle's position vector and velocity may include:

[0127] S301, Update the velocity of the particle based on the position vector of the individual optimal particle corresponding to the particle, the position vector and velocity of the particle, the position vector of the global optimal particle, and the inertia weight;

[0128] Electronic devices can be based on the formula Calculate the velocity of the i-th particle in the d-th dimension during the (t+1)-th iteration.

[0129] S302, if the updated velocity of the particle is greater than the preset velocity, adjust the updated velocity to a value not greater than the preset velocity;

[0130] To prevent excessively high particle velocities that could cause significant jumps in the updated position vector, the electronic device can determine if the updated particle velocity exceeds a preset velocity. If so, the device can adjust the velocity to a value no greater than the preset velocity. For example, if the updated velocity is 65 and the preset velocity is 60, the device can adjust the velocity to 58.

[0131] In one implementation, if the updated speed is greater than a preset speed, the electronic device can adjust the updated speed to the preset speed. For example, if the updated speed is 65 and the preset speed is 60, then the electronic device can adjust the updated speed to 60.

[0132] S303, Update the particle's position vector based on the updated velocity and the particle's position vector;

[0133] Electronic devices can be based on the formula Calculate the position vector of the i-th particle in the d-th dimension during the (t+1)-th iteration.

[0134] S304, if there are elements in the updated position vector that exceed the boundary, reverse the direction of the updated velocity, and update the position vector of the particle according to the reversed velocity and the position vector of the particle.

[0135] If an element in the updated position vector exceeds the boundary, it means that the element will converge to the boundary value, and the elements in the position vector cannot be updated effectively. Therefore, if there are elements in the updated position vector that exceed the boundary, the electronic device can reverse the direction of the updated velocity to make the particle move away from the boundary. Based on the reversed velocity and the particle's position vector, the particle's position vector is updated. Typically, the preset clipping rate ranges from [0,1), and the preset alignment parameter ranges from [1,256].

[0136] As can be seen, in this embodiment, the electronic device can update the particle's velocity based on the position vector of the individual optimal particle, the particle's position vector and velocity, the position vector of the globally optimal particle, and the inertia weight. If the updated velocity is greater than a preset velocity, the updated velocity is adjusted to a value no greater than the preset velocity. The particle's position vector is then updated based on the updated velocity and the particle's position vector. If the updated position vector contains elements that exceed the boundary, the direction of the updated velocity is reversed, and the particle's position vector is updated based on the reversed velocity and the particle's position vector. Since the particle's position vector will jump significantly when its velocity exceeds the preset velocity, the electronic device can set the updated velocity to a value less than the preset velocity if the updated velocity is greater than the preset velocity. If an element in the updated position vector exceeds the boundary, it means that the element will converge to the boundary value, and the element in the position vector cannot be effectively updated further. Therefore, if there is an element exceeding the boundary in the updated position vector, the electronic device can reverse the direction of the updated velocity to make the particle move away from the boundary. This ensures that the updated velocity and position vector values ​​are reasonable.

[0137] As one embodiment of this application, the initial value of the element indicating the preset clipping rate in the position vector of the particle is less than a preset threshold, and the inertial weight is less than the inertial weight before the update.

[0138] Since the first precision and the second precision are equal without loss of accuracy, it means that the preset clipping ratios for each convolutional layer in the model to be clipped are all relatively small. Therefore, to quickly determine the target clipping ratio and target alignment parameters, the electronic device can set the initial value of the element indicating the preset clipping ratio in the particle vector to be less than a preset threshold. In this way, the target clipping ratio and target alignment parameters can be determined with fewer iterations.

[0139] To improve the accuracy of determining the target clipping rate and target alignment parameters in subsequent iterations, the electronic device can control the updated inertia weight to be smaller than the original inertia weight. This reduces the impact of the inertia weight on the speed update.

[0140] As can be seen, in this embodiment, the initial values ​​of each element in the particle's position vector are less than a preset threshold, and the inertial weight is less than the inertial weight before the update. Since the preset clipping rate is relatively small without sacrificing accuracy, the electronic device can set the initial values ​​of the elements in the particle vector indicating the preset clipping rate to be less than the preset threshold in order to quickly determine the target clipping rate and target alignment parameters. To improve the accuracy of determining the target clipping rate and target alignment parameters in subsequent iterations, the electronic device can control the updated inertial weight to be less than the inertial weight before the update. This reduces the impact of the inertial weight on the velocity update.

[0141] As one embodiment of this application, a flowchart illustrating the process by which an electronic device determines the optimal cropping ratio and optimal alignment parameters and crops the original model according to the optimal cropping ratio and optimal alignment parameters can be shown as follows: Figure 4 As shown, the specific steps may include:

[0142] S401, obtain the accuracy and time consumption of the original model;

[0143] The electronic device can acquire the original model 421 and the corresponding business samples of the target platform. The original model 421 is run on the target platform to obtain its execution time. The business samples are then processed using the original model 421 to obtain its accuracy.

[0144] S402, Initialize particles;

[0145] The electronic device can employ the PSO algorithm to determine the clipping rate and alignment parameters iteratively. The elements of the particle's position vector in the search space represent the clipping rate and alignment parameters. During the first iteration, the electronic device can set the particle's position vector as the initial clipping rate and initial alignment parameters.

[0146] S403, trim the original model;

[0147] After determining the clipping rate and alignment parameters, the electronic device can clip the original model according to the determined alignment parameters and the clipping rate, using L1NORM, L2NORM, or other methods to measure the channel weights of the model to be clipped, thereby obtaining the clipped model.

[0148] S404, Do you want to make a fine adjustment? If yes, proceed to step S405; if no, proceed to step S406.

[0149] The electronic device can determine whether the trimmed model needs to be fine-tuned. If so, step S405 is executed; if not, step S406 is executed.

[0150] S405, fine-tuning;

[0151] Since cropping a model will reduce its accuracy, electronic devices can fine-tune the cropped model to compensate for this reduction in accuracy.

[0152] S406, obtain the accuracy and time taken for the cropped model;

[0153] The electronic device can run the trimmed model on the target platform, obtain the time taken by the trimmed model, process business samples using the trimmed model, and obtain the accuracy of the trimmed model.

[0154] S407, calculate the objective function for each particle;

[0155] After determining the accuracy of the trimmed model, the electronic device can use the following formula: F(X, δ)=f(X)+δmax{e(X)-ε,0} based on the accuracy of the trimmed model. 2 Calculate the objective function for each particle.

[0156] S408, determine the individual optimal particle and the global optimal particle;

[0157] To update the velocity and position of particles, the electronic device can determine the individual optimal particle and the global optimal particle. The individual optimal particle is the single particle that minimizes the objective function value in each iteration, while the global optimal particle is the individual optimal particle that minimizes the objective function value.

[0158] S409, updates particle velocity and position;

[0159] The electronic device can calculate the velocity of the i-th particle in the d-th dimension during the (t+1)-th iteration using the following formula: The position vector of the i-th particle in the d-th dimension during the (t+1)-th iteration is calculated using the following formula:

[0160] Wherein, ω is the inertia weight, and the value of ω is usually between 0.4 and 0.9. Let be the velocity of the i-th particle in the d-th dimension during the t-th iteration, c1 be the individual learning factor, c2 be the social learning factor, and r1 and r2 be random numbers taking values ​​in the range [0, 1]. pbest i，d Let gbest be the position of the i-th particle in the d-th dimension. The optimal particle for the i-th particle in the d-th dimension is the particle that minimizes the loss function in each iteration of the d-th dimension. d Let d be the position of the globally optimal particle in the d-th dimension. The globally optimal particle in the d-th dimension is the particle that minimizes the loss function value among all particles in each iteration of the d-th dimension. Let be the position of the i-th particle in the d-th dimension during the t-th iteration. Let be the position vector of the i-th particle in the d-th dimension during the t-th iteration.

[0161] S410, Has the termination condition been met? If yes, proceed to step S411; if no, return to step S403.

[0162] The termination condition is that the clipping rate corresponding to the globally optimal particle obtained in this iteration is the same as the clipping rate corresponding to the globally optimal particle obtained in the previous iteration. If the termination condition is met, it means that the clipping rate has reached its maximum under the precision limit. Therefore, the electronic device can execute step S411, that is, output the optimal clipping rate.

[0163] If the termination condition is not met, it means that the pruning rate has not reached its maximum. The electronic device can continue to iterate until the termination condition is met. Therefore, if the termination condition is not met, the electronic device can return to step S403, that is, prune the original model according to the updated speed and position using L1NORM, L2NORM or other methods to measure the channel weights of the model to be pruned.

[0164] S411 outputs the optimal cropping ratio;

[0165] If the termination condition is met, it means that the cutting rate has reached its maximum under the precision limit, so the electronic device can output the optimal cutting rate and the optimal alignment parameters.

[0166] S412, trim the original model according to the optimal trimming rate.

[0167] After obtaining the optimal clipping ratio and the optimal alignment parameters, the electronic device can clip the original model 421 according to the optimal alignment parameters and the optimal clipping ratio, using L1NORM, L2NORM, or other methods to measure the channel weights of the model to be clipped, and thus obtain the clipped model 422.

[0168] As one embodiment of this application, the flowchart illustrating the process of an electronic device determining the optimal cropping model can be as follows: Figure 5 As shown, the specific steps may include:

[0169] S501, Model Cutting;

[0170] The electronic device can trim the original model 511 to obtain the trimmed model 512.

[0171] S502, Platform Adaptation Loss Function under Accuracy Constraints;

[0172] To determine the clipping rate that minimizes the runtime of the clipped model on the target platform under accuracy constraints, the electronic device can solve for the platform adaptation loss function under accuracy constraints. Step S502 can be specifically implemented through steps S503-S506.

[0173] S503 uses the standard loss function;

[0174] The clipping ratio and alignment parameters can be represented as a vector [PR] = [pr1, pr2, ..., pr... n The original model 501 includes n convolutional groups, pr i This represents the clipping rate corresponding to the i-th convolutional group of the original model 501.

[0175] Electronic devices can be based on the standard loss function Determine the accuracy and time consumption levels of the trimmed model. Here, e(PR) can be expressed as e(PR) = acc' - acc, where the first accuracy is acc' and the second accuracy is acc; f(PR) can be expressed as... The first time taken is time', and the second time taken is time.

[0176] S504, adds platform compatibility information;

[0177] To improve the platform compatibility of the trimmed model, platform adaptation information can be added to the above vectors. Specifically, we can construct X = [PR, align] = [pr1, pr2, ..., pr...]. n [align], where align is platform adaptation information, i.e., alignment parameter.

[0178] S505, with added precision constraints;

[0179] By employing constrained optimization algorithms, the following can be achieved: The solution is transformed into solving F(X, δ) = f(X) + δmax{e(X) - ε, 0} 2 The penalty term is δmax{e(X)-ε, 0}. 2 It can represent precision constraints.

[0180] If the precision error limit is not met, i.e., X is outside the feasible region, then e(X)-ε>0, and thus max{e(X)-ε,0} 2 =[e(X)-ε] 2 >0. Since the penalty term coefficient δ is a pre-set value greater than a preset positive number, F(X, δ) > f(X).

[0181] Under the condition that the accuracy error limit is met, that is, when X is within the feasible region, e(X)-ε≤0, and thus max{e(X)-ε,0} 2 =0, then F(X,δ) = f(X).

[0182] S506, Unconstrained Problem Transformation;

[0183] Since the penalty term is a large positive number when X is outside the feasible region, and zero when X is inside the feasible region, F(X, δ) for X outside the feasible region is greater than F(X, δ) for X inside the feasible region. Furthermore, the shorter the runtime of the trimmed model on the target platform, the smaller the value of f(X). Therefore, minimizing F(X, δ) minimizes the runtime while satisfying accuracy constraints, resulting in the highest degree of fit between the trimmed model and the target platform.

[0184] Therefore, the solution can be obtained. By transforming the problem into solving for minF(X,δ), the electronic device can determine the target X that minimizes F(X,δ) under precision constraints, i.e., perform an unconstrained problem transformation.

[0185] S507, the optimization algorithm.

[0186] Electronic devices can employ the PSO algorithm to iteratively update the clipping ratio 513 and alignment parameters. The PSO algorithm is an optimization algorithm. Alternatively, electronic devices can also use genetic algorithms or reinforcement learning algorithms to update the clipping ratio 513 and alignment parameters; these are also optimization algorithms.

[0187] The termination condition is that the clipping rate corresponding to the globally optimal particle obtained in this iteration is the same as the clipping rate corresponding to the globally optimal particle obtained in the previous iteration. If the termination condition is reached, it means that the clipping rate has reached its maximum under the accuracy limit. Therefore, the electronic device can determine the clipped model 512 as the optimal clipped model 514.

[0188] The model deployment method provided in this embodiment of the invention will be illustrated below with an application scenario. For example, if the target platform for model deployment is an intelligent transportation platform, and the model to be pruned is a license plate detection model, then the electronic device can obtain the corresponding business samples of the intelligent transportation platform. The business samples can be license plate images captured by cameras installed on the road. The electronic device can obtain preset alignment parameters. The preset alignment parameters are the initial values ​​of the alignment parameters, which represent the degree of compatibility between the hardware architecture, software architecture, data format, and data storage method of the intelligent transportation platform's GPU and the license plate detection model. The electronic device can prune the channels of the license plate detection model according to the preset pruning rate and the preset alignment parameters to obtain the pruned model.

[0189] The electronic device can obtain the first time required for the cropped model to process the license plate image on the GPU, and the first accuracy of the license plate detection result obtained after processing the license plate image. Based on the difference between the first time and the second time, and the difference between the first accuracy and the second accuracy, the electronic device determines the cropping loss, where the second time is the time required for the uncropped license plate detection model to process the license plate image on the GPU, and the second accuracy is the accuracy of the license plate detection result obtained by the uncropped license plate detection model on the GPU.

[0190] Furthermore, the electronic device can employ an optimization algorithm, using elements in the particle's position vector to represent the clipping rate and alignment parameters corresponding to the license plate detection model. Based on the clipping loss, the preset clipping rate and alignment parameters are updated, and the particle corresponding to the clipping rate and alignment parameters that minimizes the clipping loss is determined as the globally optimal particle. The electronic device can iteratively update the position vectors of each particle until the updated globally optimal particle has the same clipping rate and alignment parameters as the previously updated globally optimal particle. The electronic device can then use the updated clipping rate as the target clipping rate and the updated alignment parameters as the target alignment parameters.

[0191] In this way, electronic devices can crop the license plate detection model according to the target cropping rate based on the target alignment parameters to obtain the target license plate detection model. This target license plate detection model is then deployed to the GPU of the intelligent transportation platform, allowing the GPU to run it and process the license plate images acquired in the intelligent transportation scene to obtain the license plate detection results. This approach ensures higher accuracy in license plate detection results while improving the compatibility between the target license plate detection model and the GPU of the intelligent transportation platform, thereby increasing the running speed of the target license plate detection model on the GPU.

[0192] Corresponding to the above model deployment method, this application embodiment also provides a model deployment device, which will be described below.

[0193] like Figure 6 As shown, a model deployment device includes:

[0194] The sample acquisition module 601 is used to acquire business samples corresponding to the target platform, wherein the target platform is the platform where the model to be tailored is to be deployed;

[0195] The first trimming module 602 is used to trim the channels of the model to be trimmed according to a preset trimming rate and preset alignment parameters to obtain the trimmed model.

[0196] The parameter acquisition module 603 is used to acquire the first time required to process the business sample and the first accuracy required to process the business sample when the cropped model is running on the target platform.

[0197] The loss determination module 604 is used to determine the pruning loss based on the difference between the first time consumption and the second time consumption, and the difference between the first precision and the second precision, wherein the second time consumption is the time required for the model to be pruned to process the business sample when the model to be pruned is running on the target platform, and the second precision is the precision of the model to be pruned in processing the business sample when the model to be pruned is running on the target platform.

[0198] The parameter update module 605 is used to update the preset cutting rate and the preset alignment parameter according to the cutting loss.

[0199] The target parameter determination module 606 is used to return the step of cropping the channel of the model to be cropped according to the preset cropping rate and the preset alignment parameter until the updated preset cropping rate is the same as the original preset cropping rate. The updated preset cropping rate is used as the target cropping rate, and the updated preset alignment parameter is used as the target alignment parameter.

[0200] The second trimming module 607 is used to trim the model to be trimmed according to the target trimming rate based on the target alignment parameter to obtain the target model, wherein the target alignment parameter reflects the degree of adaptation between the target platform and the target model;

[0201] The model deployment module 608 is used to deploy the target model to the target platform so that the target platform runs the target model and processes the business data of the target platform through the target model.

[0202] As can be seen, in this embodiment, the electronic device can obtain business samples corresponding to the target platform, where the target platform is the platform where the model to be pruned is to be deployed; according to a preset pruning rate and preset alignment parameters, the channels of the model to be pruned are pruned to obtain the pruned model; the first time required for the pruned model to process the business samples and the first precision required for processing the business samples when running on the target platform are obtained; based on the difference between the first time and the second time, and the difference between the first precision and the second precision, the pruning loss is determined, where the second time is the time required for the model to be pruned to process the business samples when running on the target platform, and the second precision is the time required for the model to be pruned to process the business samples when running on the target platform. The process involves: refining the processing accuracy; updating the preset clipping rate and preset alignment parameters based on the clipping loss; returning the steps of clipping the channels of the model to be clipped according to the preset clipping rate and preset alignment parameters until the updated preset clipping rate is the same as the original preset clipping rate; using the updated preset clipping rate as the target clipping rate and the updated preset alignment parameters as the target alignment parameters; clipping the model to be clipped according to the target clipping rate based on the target alignment parameters to obtain the target model, where the target alignment parameters reflect the degree of compatibility between the target platform and the target model; and deploying the target model to the target platform so that the target platform can run the target model and process the business data of the target platform through the target model. Since the clipping loss is determined based on the differences in time consumption and accuracy, it can reflect the changes in time consumption and accuracy before and after model clipping. Therefore, updating the preset clipping rate and preset alignment parameters based on the clipping loss can reduce the time consumption of the model running on the target platform while ensuring model accuracy. When the updated preset clipping rate is the same as the original preset clipping rate, the model takes the shortest time to run on the target platform while meeting accuracy constraints. Therefore, in this case, the updated preset clipping rate and preset alignment parameters can be used as the target clipping rate and target alignment parameters, respectively. Deploying the target model, obtained by clipping the channels of the model to be clipped according to the target alignment parameters and the target clipping rate, to the target platform can improve the running speed and accuracy of the model after deployment.

[0203] As one embodiment of this application, the above-mentioned trimming loss characterizes the degree of reduction of the first time consumption relative to the second time consumption under the limitation of the difference between the first precision and the second precision;

[0204] The parameter update module 605 mentioned above may include:

[0205] The parameter update submodule is used to update the preset cropping rate and the preset alignment parameters with the reduction of cropping loss as the optimization direction.

[0206] As one embodiment of this application, the loss determination module 604 described above may include:

[0207] The loss calculation submodule is used to calculate the clipping loss F(X, δ) based on the difference between the first time consumption and the second time consumption, and the difference between the first precision and the second precision, according to the following loss function:

[0208] F(X, δ)=f(X)+δmax{e(X)-ε, 0} 2

[0209] X=[PR]=[pr1, pr2,…,pr n ,align]

[0210] Where δ is a preset penalty coefficient, δ is greater than a preset positive number, X is a vector composed of the preset clipping rate and the preset alignment parameter, f(X) is the ratio of the first time consumption to the second time consumption, e(X) is the difference between the first precision and the second precision, ε is the precision error threshold, and pr i The preset clipping rate is the i-th convolutional group of the model to be clipped, where i = 1, 2, ..., n, n is the number of convolutional groups of the model to be clipped, and align is the preset alignment parameter.

[0211] As one implementation of this application, the above parameter update submodule may include:

[0212] The parameter determination unit is used to determine the target X that minimizes F(X, δ) when the difference between the first precision and the second precision is not greater than the precision error threshold.

[0213] As one embodiment of this application, the parameter determination unit may include:

[0214] The first particle determination subunit is used to determine the particle with the smallest F(X, δ) value in each update as the individual optimal particle, wherein the position vector of the particle is X, and the elements in the position vector represent the preset clipping rate and the preset alignment parameter.

[0215] The second particle determination subunit is used to determine the individual optimal particle with the smallest corresponding F(X,δ) value as the global optimal particle.

[0216] The parameter update subunit is used to update the position vector and velocity of each particle based on the inertia weight, the position vector of the globally optimal particle, the position vector of the individual optimal particle corresponding to the particle, and the position vector and velocity of the particle.

[0217] As one implementation of this application, the above parameter update subunit may include:

[0218] The velocity update subunit is used to update the velocity of the particle based on the position vector of the individual optimal particle corresponding to the particle, the position vector and velocity of the particle, the position vector of the global optimal particle, and the inertia weight.

[0219] The velocity setting subunit is used to adjust the updated velocity to a value not greater than the preset velocity if the updated velocity of the particle is greater than the preset velocity.

[0220] The first position vector update subunit is used to update the position vector of the particle according to the updated velocity and the position vector of the particle.

[0221] The second position vector update subunit is used to reverse the direction of the updated velocity if there are elements in the updated position vector that exceed the boundary, and update the position vector of the particle according to the reversed velocity and the position vector of the particle.

[0222] In one embodiment of this application, the initial value of the element indicating the preset clipping rate in the position vector of the particle is less than a preset threshold, and the inertial weight is less than the inertial weight before the update.

[0223] This application also provides an electronic device, such as... Figure 7 As shown, it includes:

[0224] Memory 701 is used to store computer programs;

[0225] When the processor 702 executes the program stored in the memory 701, it implements the model deployment method steps described in any of the above embodiments.

[0226] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 702, the communication interface, and the memory 701 communicating with each other via the communication bus.

[0227] As can be seen, in this embodiment, the electronic device can obtain business samples corresponding to the target platform, where the target platform is the platform where the model to be pruned is to be deployed; according to a preset pruning rate and preset alignment parameters, the channels of the model to be pruned are pruned to obtain the pruned model; the first time required for the pruned model to process the business samples and the first precision required for processing the business samples when running on the target platform are obtained; based on the difference between the first time and the second time, and the difference between the first precision and the second precision, the pruning loss is determined, where the second time is the time required for the model to be pruned to process the business samples when running on the target platform, and the second precision is the time required for the model to be pruned to process the business samples when running on the target platform. The process involves: refining the processing accuracy; updating the preset clipping rate and preset alignment parameters based on the clipping loss; returning the steps of clipping the channels of the model to be clipped according to the preset clipping rate and preset alignment parameters until the updated preset clipping rate is the same as the original preset clipping rate; using the updated preset clipping rate as the target clipping rate and the updated preset alignment parameters as the target alignment parameters; clipping the model to be clipped according to the target clipping rate based on the target alignment parameters to obtain the target model, where the target alignment parameters reflect the degree of compatibility between the target platform and the target model; and deploying the target model to the target platform so that the target platform can run the target model and process the business data of the target platform through the target model. Since the clipping loss is determined based on the differences in time consumption and accuracy, it can reflect the changes in time consumption and accuracy before and after model clipping. Therefore, updating the preset clipping rate and preset alignment parameters based on the clipping loss can reduce the time consumption of the model running on the target platform while ensuring model accuracy. When the updated preset clipping rate is the same as the original preset clipping rate, the model takes the shortest time to run on the target platform while meeting accuracy constraints. Therefore, in this case, the updated preset clipping rate and preset alignment parameters can be used as the target clipping rate and target alignment parameters, respectively. Deploying the target model, obtained by clipping the channels of the model to be clipped according to the target alignment parameters and the target clipping rate, to the target platform can improve the running speed and accuracy of the model after deployment.

[0228] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0229] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0230] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0231] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0232] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described model deployment methods.

[0233] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the model deployment methods described above.

[0234] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0235] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0236] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for devices, electronic devices, computer-readable storage media, and computer program products, since they are basically similar to the method embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions of the method embodiments.

[0237] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A model deployment method, characterized in that, The method includes: Obtain business samples corresponding to the target platform, wherein the target platform is the platform where the model to be tailored is to be deployed; Based on the preset clipping rate and preset alignment parameters, the channels of the model to be clipped are clipped to obtain the clipped model. The first time required to process the business sample and the first accuracy required to process the business sample are obtained when the cropped model is run on the target platform. Based on the difference between the first time consumption and the second time consumption, and the difference between the first precision and the second precision, the pruning loss is determined, wherein the second time consumption is the time required for the model to be pruned to process the business sample when running on the target platform, and the second precision is the precision of the model to be pruned in processing the business sample when running on the target platform; Update the preset cropping rate and the preset alignment parameters based on the cropping loss; Return to the step of cropping the channels of the model to be cropped according to the preset cropping rate and the preset alignment parameter, until the updated preset cropping rate is the same as the original preset cropping rate, and use the updated preset cropping rate as the target cropping rate and the updated preset alignment parameter as the target alignment parameter; Based on the target alignment parameters, the model to be clipped is clipped according to the target clipping rate to obtain the target model, wherein the target alignment parameters reflect the degree of adaptation between the target platform and the target model; The target model is deployed to the target platform so that the target platform runs the target model and processes the business data of the target platform through the target model.

2. The method according to claim 1, characterized in that, The cutting loss characterizes the degree of reduction in the first time consumption relative to the second time consumption under the constraint of the difference between the first precision and the second precision. The step of updating the preset cropping rate and the preset alignment parameter based on the cropping loss includes: With the reduction of the cutting loss as the optimization direction, the preset cutting rate and the preset alignment parameters are updated.

3. The method according to claim 2, characterized in that, The step of determining the trimming loss based on the difference between the first and second time consumption, and the difference between the first and second accuracy, includes: Based on the difference between the first and second time consumption, and the difference between the first and second precision, the clipping loss F(X,δ) is calculated according to the following loss function: F(X,δ)=f(X)+δmax{e(X)-,0} 2 X=[PR]=[pr1,pr2,…,pr n ,align] Where δ is a preset penalty coefficient, δ is greater than a preset positive number, X is a vector composed of the preset clipping rate and the preset alignment parameter, f(X) is the ratio of the first time consumption to the second time consumption, e(X) is the difference between the first precision and the second precision, ε is the precision error threshold, and pr i The preset clipping rate is the i-th convolutional group of the model to be clipped, where i = 1, 2, ..., n, n is the number of convolutional groups of the model to be clipped, and align is the preset alignment parameter.

4. The method according to claim 3, characterized in that, The step of updating the preset cut rate and the preset alignment parameters with the reduction of the cutting loss as the optimization direction includes: If the difference between the first precision and the second precision is not greater than the precision error threshold, a target X is determined that minimizes F(X,δ).

5. The method according to claim 4, characterized in that, The step of determining the target X that minimizes F(X,δ) when the difference between the first precision and the second precision is not greater than the precision error threshold includes: For each particle, the particle with the smallest value of F(X,δ) in each update is taken as the individual optimal particle, where the position vector of the particle is X, and the elements in the position vector represent the preset clipping rate and the preset alignment parameter. The individual optimal particle with the smallest corresponding F(X,δ) value is determined as the global optimal particle. For each particle, the position vector and velocity of the particle are updated based on the inertia weight, the position vector of the globally optimal particle, the position vector of the individual optimal particle corresponding to the particle, the position vector and velocity of the particle.

6. The method according to claim 5, characterized in that, The step of updating the position vector and velocity of the particle based on the inertia weight, the position vector of the globally optimal particle, the position vector of the individual optimal particle corresponding to the particle, and the position vector and velocity of the particle includes: The velocity of the particle is updated based on the position vector of the individual optimal particle corresponding to the particle, the position vector and velocity of the particle, the position vector of the global optimal particle, and the inertia weight. If the updated velocity of the particle is greater than the preset velocity, the updated velocity will be adjusted to a value no greater than the preset velocity. The particle's position vector is updated based on the updated velocity and the particle's position vector. If the updated position vector contains elements that exceed the boundary, the direction of the updated velocity is reversed, and the particle's position vector is updated based on the reversed velocity and the particle's position vector.

7. The method according to claim 5 or 6, characterized in that, The initial value of the element indicating the preset clipping rate in the particle's position vector is less than a preset threshold, and the inertia weight is less than the inertia weight before the update.

8. A model deployment device, characterized in that, The device includes: The sample acquisition module is used to acquire business samples corresponding to the target platform, wherein the target platform is the platform where the model to be tailored is to be deployed; The first trimming module is used to trim the channels of the model to be trimmed according to a preset trimming rate and preset alignment parameters to obtain the trimmed model. The parameter acquisition module is used to acquire the first time required for the cropped model to process the business sample and the first accuracy required to process the business sample when the model is running on the target platform. The loss determination module is used to determine the pruning loss based on the difference between the first time consumption and the second time consumption, and the difference between the first precision and the second precision, wherein the second time consumption is the time required for the model to be pruned to process the business sample when the model to be pruned is running on the target platform, and the second precision is the precision of the model to be pruned in processing the business sample when the model to be pruned is running on the target platform. The parameter update module is used to update the preset cropping rate and the preset alignment parameters according to the cropping loss; The target parameter determination module is used to return the step of cropping the channels of the model to be cropped according to the preset cropping rate and the preset alignment parameter, until the updated preset cropping rate is the same as the original preset cropping rate, the updated preset cropping rate is used as the target cropping rate, and the updated preset alignment parameter is used as the target alignment parameter. The second trimming module is used to trim the model to be trimmed according to the target trimming rate based on the target alignment parameter to obtain the target model, wherein the target alignment parameter reflects the degree of adaptation between the target platform and the target model; The model deployment module is used to deploy the target model to the target platform so that the target platform runs the target model and processes the business data of the target platform through the target model.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images