Method, device, equipment and medium for optimizing inference parameters of deep learning model

Abstract

The present invention relates to the field of computer technology, and discloses a method, device, equipment and medium for optimizing inference parameters of a deep learning model. The method comprises: in a current iteration round, obtaining a first influencing factor set, a second influencing factor set, an optimal position, a fitness value and a global optimal position at a previous moment; determining the fitness value of the first particle at the current moment according to the first influencing factor set, the second influencing factor set, a particle swarm parameter, a graphics card utilization rate, the optimal position, and the global optimal position; determining the optimal position at the current moment according to the fitness value corresponding to the optimal position at the previous moment and the fitness value at the current moment; determining the global optimal position at the current moment according to the optimal positions and corresponding fitness values ​​corresponding to all particles at the current moment; stopping the iteration when a stop iteration condition is met, and determining the final model parameters; or, when the stop iteration condition is not met, entering the next iteration round.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, equipment and medium for optimizing inference parameters of a deep learning model. Background Art

[0002] In recent years, research on applications such as image classification, speech recognition, and natural language processing based on deep learning has made significant progress. However, as the size and complexity of models increase, the requirements for algorithm performance and efficiency are also increasing. The Machine Learning Performance (MLPerf) benchmark is widely used to evaluate the inference performance of AI models on different hardware platforms.

[0003] In the actual MLPerf benchmark evaluation, the reasoning of deep neural networks needs to be evaluated under specific parameter conditions. The tuner obtains a series of performance data through these evaluation tests. If the performance does not meet the expected results, the debugger must rely on personal experience to continuously adjust the parameters and test repeatedly. This method is not only time-consuming and labor-intensive, but also has a low degree of automation, making it difficult to quickly and accurately find the best model parameter combination. Summary of the invention

[0004] In view of this, the present invention provides a deep learning model inference parameter optimization method, device, equipment and medium to solve the problem in related technologies that debugging personnel must rely on personal experience to continuously adjust parameters, which is time-consuming and labor-intensive, has a low degree of automation, and is difficult to quickly find the best model parameter combination.

[0005] In a first aspect, the present invention provides a method for optimizing inference parameters of a deep learning model, the method comprising:

[0006] In the current iteration round, the first influencing factor set corresponding to the position of each particle in the particle swarm at the previous moment, the second influencing factor set corresponding to the speed, the optimal position of each particle, the fitness value corresponding to the optimal position of each particle, and the global optimal position of the particle swarm are obtained;

[0007] Determine the fitness value of the first particle at the current moment according to a first influencing factor set corresponding to the first particle, a second influencing factor set corresponding to the first particle, pre-configured particle swarm parameters, a pre-acquired graphics card utilization rate, an optimal position of the first particle at the previous moment, and a global optimal position of the particle swarm at the previous moment, wherein the first particle is any particle in the particle swarm;

[0008] Determine the optimal position of the first particle at the current moment according to the fitness value corresponding to the optimal position of the first particle at the previous moment and the fitness value of the first particle at the current moment;

[0009] Determine the global optimal position of the particle swarm at the current moment according to the optimal positions of all particles in the particle swarm at the current moment and the fitness value of each particle at the optimal position;

[0010] When the current round meets the conditions for stopping iteration, the iteration is stopped, the optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, the target influence factor set includes an influence factor set corresponding to the position and an influence factor set corresponding to the speed;

[0011] Alternatively, when the current round does not meet the iteration stop condition, the next iteration round is entered.

[0012] The present invention provides a method for optimizing deep learning model inference parameters, which has the following advantages:

[0013] According to the first set of influencing factors corresponding to the first particle, the second set of influencing factors, the pre-configured particle swarm parameters, the optimal position at the previous moment, and the global optimal position of the particle swarm at the previous moment, the fourth set of influencing factors corresponding to the position of the first particle at the current moment is determined, which is actually the position of the first particle at the current moment. Then, according to the fitness value corresponding to the optimal position of the first particle at the previous moment and the fitness value of the first particle at the current moment, the optimal position of the first particle at the current moment is determined. By comprehensively considering these factors, the optimal position of the particle at the current moment can be more accurately inferred, which improves the accuracy and reliability of particle position optimization.

[0014] Then, according to the optimal positions of multiple particles at the current moment and the fitness value corresponding to each optimal position, the global optimal position is selected. This method reflects that by searching for the optimal solution through the collaborative search of multiple particles, a better solution can be searched in a shorter time, which improves the search efficiency. By continuously iteratively updating the position and speed of the particles and the global optimal position of the particle swarm, the algorithm can gradually approach the optimal solution. In each iteration, the particles adjust their positions and speeds according to their own historical optimal positions and the global optimal position of the particle swarm, thereby achieving more efficient search. The particle swarm optimization algorithm can effectively avoid falling into the local optimal solution through parallel search and information sharing of multiple particles.

[0015] When determining the fitness value of the first particle at the current moment, the multiple influencing factor sets of the first particle, the pre-configured particle swarm parameters, the pre-acquired graphics card utilization, the optimal position of the particle at the previous moment, and the global optimal position of the particle swarm at the previous moment are comprehensively considered. The comprehensive consideration of multiple factors can more comprehensively evaluate the performance of particles and improve the accuracy and reliability of fitness values. The introduction of pre-acquired graphics card utilization helps to better consider the utilization of computing resources during the particle swarm optimization process. By incorporating graphics card utilization into the calculation of fitness values, the algorithm can be prompted to pay more attention to the effective use of resources and improve computing efficiency in the process of searching for the optimal solution.

[0016] The whole process is an automatic iterative process that does not require too much human intervention. The algorithm automatically updates the particle position, calculates the fitness value, and determines the optimal position according to the preset conditions and rules, reducing the impact of human factors on the results and improving the objectivity and reliability of the results. Automated processing can save labor costs and improve work efficiency. Applying these optimization parameters to the deep learning model can improve the accuracy and efficiency of the model. Moreover, the algorithm can automatically adjust the parameters according to the actual situation to adapt to different models and task requirements. By continuously optimizing the model parameters, the generalization ability and robustness of the model can be improved, so that it can achieve better performance in different data sets and application scenarios.

[0017] In an optional embodiment, the first influencing factor set is composed of position data corresponding to particle dimension data respectively, and the particle dimension data includes: batch size, number of replication streams, number of inference streams, first identification information for indicating whether to run inference on the replication stream, and second identification information for indicating whether to use graph optimization, wherein the batch size, the number of replication streams, the number of inference streams, the first identification information and the second identification information are multiple influencing factors, and the second influencing factor set is composed of speed data corresponding to each of the influencing factors.

[0018] Specifically, batch size, number of replication streams, number of inference streams, whether to run inference on replication streams, and whether to use graph optimization are all main performance indicators used to measure the quality of models in deep learning models. These factors cover many aspects of model operation and can more comprehensively reflect the performance and characteristics of the model, thereby helping to more accurately optimize the model and evaluate performance. When the specific parameter values ​​of these parameters are tuned according to the method of this application, the prediction function of the model will be more accurate and intelligent. Speed ​​represents the direction and amplitude of particle movement, while position represents the current position of the particle in the search space. In this application, the position of the particle represents the parameter value, and the speed represents the amplitude of change of the parameter value. These influencing factors are specific to particle dimension data, and corresponding speed data is provided for each influencing factor, so that these parameters can be adjusted more finely during particle swarm optimization. This can improve the accuracy and flexibility of parameter adjustment and help find a better model configuration. Moreover, after clarifying these specific influencing factors, targeted optimization can be performed according to actual needs and hardware conditions. For example, if the graphics card utilization is low, you can improve the utilization by adjusting parameters such as batch size, number of replication streams, or number of inference streams; if you want to increase the inference speed, you can consider whether to run inference on the replication stream and whether to use strategies such as graph optimization. Compared with blindly trying different parameters, this method can save a lot of time and computing resources and improve optimization efficiency. It is precisely because a variety of influencing factors are taken into account that flexible adjustments can be made based on these factors to better adapt to different hardware platforms and application scenarios. Whether in an environment with limited resources or in a scenario with high performance requirements, the best effect can be achieved by adjusting these parameters.

[0019] In an optional implementation, determining the fitness value of each particle at the current moment according to the first influencing factor set, the second influencing factor set, and the pre-acquired graphics card utilization rate specifically includes:

[0020] Determine the model reasoning speed corresponding to each particle according to the first influencing factor set and the second influencing factor set corresponding to each particle;

[0021] The fitness value of each particle at the current moment is determined according to the model inference speed and the graphics card utilization rate respectively corresponding to each particle.

[0022] Specifically, by determining the model inference speed according to the first influencing factor set and the second influencing factor set of the particle, the influence of different parameter combinations on the model inference performance can be more accurately reflected. The evaluation method based on multiple factors avoids the limitations of single factor evaluation, making the evaluation of model performance more comprehensive and accurate. When determining the fitness value of the particle, the model inference speed and the graphics card utilization rate are combined. It can ensure that in the optimization process, not only the inference speed of the model is paid attention to, but also the utilization efficiency of the graphics card resources is fully considered. It helps to achieve reasonable allocation and utilization of resources while improving model performance and avoid waste of resources. Moreover, comprehensively considering the model inference speed and graphics card utilization to determine the fitness value can provide more targeted evaluation indicators for the particle swarm optimization algorithm. In the process of searching for the optimal solution, the algorithm can more effectively find a parameter combination that is relatively excellent in both model performance and resource utilization, thereby improving the optimization effect.

[0023] In an optional implementation, the fitness value of each particle at the current moment is determined according to the model inference speed and the graphics card utilization rate corresponding to each particle, which is specifically expressed by the following formula:

[0024]

[0025] in, is the fitness value of the particle at the current moment, S is the model reasoning speed, M is the graphics card utilization rate, is the adjustment coefficient.

[0026] In an optional embodiment, the first influencing factor set corresponding to the first particle, the second influencing factor set corresponding to the first particle, the preconfigured particle swarm parameters, the optimal position of the first particle at the previous moment, and the global optimal position of the particle swarm at the previous moment are input into the deep learning model to be tested to obtain the model inference speed, specifically including:

[0027] Determine a third set of influencing factors corresponding to the velocity of the first particle at the current moment according to a first set of influencing factors corresponding to the first particle, a second set of influencing factors corresponding to the first particle, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment;

[0028] Determining the fourth influencing factor set according to the influencing factors in the first influencing factor set corresponding to the first particle and the influencing factors in the third influencing factor set;

[0029] The model reasoning speed is determined according to the fourth influencing factor set and the third influencing factor set.

[0030] Specifically, the reasoning speed of the model is determined by comprehensively considering the multiple influencing factor sets of the first particle, the particle swarm parameters, and the optimal position of the previous moment. This method can adjust the parameters of the model more accurately, thereby improving the reasoning efficiency of the model. The speed and position of the particle are determined according to multiple factors, which in turn affects the reasoning speed of the model, finds a better model parameter configuration, and improves the search ability and convergence speed of the algorithm. This method can flexibly adjust the influencing factors and parameters according to different problems and scenarios, so that the model can better adapt to various situations, and improves the versatility and adaptability of the model. By accurately adjusting the model parameters, it is expected to improve the performance of the model, such as accuracy, generalization ability, etc., so as to provide better support for practical applications. By optimizing the model reasoning speed, the consumption of computing time and resources can be reduced to a certain extent, computing efficiency can be improved, and costs can be reduced.

[0031] In an optional embodiment, determining a third set of influencing factors corresponding to the speed of the first particle at the current moment according to a first set of influencing factors corresponding to the first particle, a second set of influencing factors corresponding to the first particle, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment includes:

[0032] Determine the velocity data corresponding to the first influencing factor at the current moment according to the position data corresponding to the first influencing factor included in the first influencing factor set corresponding to the first particle at the previous moment, the velocity data corresponding to the first influencing factor in the second influencing factor set, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment, wherein the first influencing factor is any one of the influencing factors in the first influencing factor set;

[0033] The data corresponding to each of the multiple influencing factors at the current moment together constitute the third influencing factor set.

[0034] In an optional embodiment, the speed data corresponding to the first influencing factor at the current moment is determined based on the position data corresponding to the first influencing factor included in the first influencing factor set corresponding to the first particle at the previous moment, the speed data corresponding to the first influencing factor in the second influencing factor set, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment, which is specifically expressed by the following formula:

[0035]

[0036] in, is the speed data corresponding to the first influencing factor at the current moment, is the inertia weight, is the speed data corresponding to the first influencing factor at the previous moment, c1 and c2 are the learning factors, r1 and r2 are random numbers randomly drawn from the uniform distribution U(0,1), and the value range is [0, 1], is the optimal position of the first particle at the previous moment, is the global optimal position of the particle swarm at the previous moment, is the position data corresponding to the first influencing factor at the previous moment.

[0037] In an optional implementation, the fourth influencing factor set is determined according to the influencing factors in the first influencing factor set and the influencing factors in the third influencing factor set corresponding to the first particle, as embodied by the following expression:

[0038]

[0039] in, is the speed data corresponding to the first influencing factor in the third influencing factor set at the current moment, is the position data corresponding to the first influencing factor in the first influencing factor at the previous moment, It is the position data corresponding to the first influencing factor in the fourth influencing factor set at the current moment, wherein the first influencing factor is any one of multiple influencing factors.

[0040] In an optional implementation, determining the global optimal position of the particle swarm at the current moment according to the optimal positions of all particles in the particle swarm at the current moment and the fitness value of each particle at the optimal position specifically includes:

[0041] The optimal position of the particle with the highest fitness value at the current moment is selected as the global optimal position of the particle swarm at the current moment.

[0042] Specifically, by selecting the optimal position of the particle with the highest fitness value as the global optimal position of the particle swarm, the algorithm can quickly search in the direction of the optimal solution. This method directly utilizes the information of the best performing particle in the particle swarm, reduces unnecessary search processes, and improves the efficiency of the algorithm. Moreover, after determining the global optimal position, it can provide a clear direction for the movement of other particles. Other particles can adjust their speed and position according to the global optimal position, so that the entire particle swarm can search more specifically and increase the possibility of finding a better solution. Since the determination of the global optimal position can guide the particle swarm to approach the optimal solution faster, the convergence speed of the algorithm can be improved in the future, reducing computing time and resource consumption.

[0043] In an optional embodiment, the preconfigured particle swarm parameters include the number of iterations;

[0044] When the current round meets the conditions for stopping iteration, the iteration is stopped, the optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, the target influence factor set includes an influence factor set corresponding to the position and an influence factor set corresponding to the speed, including:

[0045] When the current round reaches the number of iterations, determining that the current round meets the condition for stopping iteration, and stopping iteration;

[0046] The optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, and the target influence factor set includes an influence factor set corresponding to the position and an influence factor set corresponding to the speed.

[0047] In an optional embodiment, the preconfigured particle swarm parameters include the number of iterations; when the current round meets the condition for stopping iteration, the iteration is stopped, the optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, the target influence factor set includes an influence factor set corresponding to the position, and an influence factor set corresponding to the speed, including:

[0048] When the difference between the fitness value of the particle corresponding to the global optimal position of the particle swarm at the current moment and the fitness value of the particle corresponding to the global optimal position of the particle swarm at the previous moment is less than or equal to the preset threshold, it is determined that the current round meets the stop iteration condition, and the iteration is stopped;

[0049] The optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, and the target influence factor set includes an influence factor set corresponding to the position and an influence factor set corresponding to the speed.

[0050] In an optional embodiment, the method further includes: before the initial iteration round, initializing a first set of influencing factors corresponding to the position of each particle in the particle swarm, a second set of influencing factors corresponding to the speed, and preconfigured particle swarm parameters.

[0051] In a second aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the deep learning model inference parameter optimization method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0052] In a third aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the deep learning model inference parameter optimization method of the first aspect or any corresponding embodiment thereof.

[0053] In a fourth aspect, the present invention provides a computer program product, comprising computer instructions, which are used to enable a computer to execute the deep learning model inference parameter optimization method of the above-mentioned first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 It is a flowchart of a method for optimizing inference parameters of a deep learning model provided by an embodiment of the present invention;

[0056] Figure 2 It is a flowchart of another method for optimizing deep learning model inference parameters provided by an embodiment of the present invention;

[0057] Figure 3 It is a schematic diagram of the overall process of the deep learning model inference parameter optimization method provided by an embodiment of the present invention;

[0058] Figure 4 It is a structural block diagram of a deep learning model inference parameter optimization device provided by an embodiment of the present invention;

[0059] Figure 5 It is a schematic diagram of the hardware structure of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0061] In recent years, application research based on deep learning, such as image classification, speech recognition, and natural language processing, has made significant progress. However, with the increase in model size and complexity, the requirements for algorithm performance and efficiency are also getting higher and higher. How to apply the trained model to actual data such as prediction, classification, or generation, and how to accurately evaluate and optimize its reasoning performance have become important research topics in the field of deep learning. Among them, hardware performance evaluation is a key link, which can objectively measure and compare the performance of different hardware architectures, thereby guiding users to reasonably choose a suitable hardware platform. The MLPerf benchmark is widely used to evaluate the reasoning performance of AI models on different hardware platforms. However, the reasoning speed and resource usage of the model are closely related to the parameter settings.

[0062] In the actual MLPerf benchmark evaluation, the reasoning of deep neural networks needs to be evaluated under specific parameter conditions. The tuner obtains a series of performance data through these evaluation tests. If the performance does not meet the expected results, the debugger must rely on personal experience to continuously adjust the parameters and test repeatedly. This method is not only time-consuming and laborious, but also has a low degree of automation and it is difficult to quickly and accurately find the best model parameter combination. Therefore, how to quickly screen out the best parameter combination and optimize it in the MLPerf test is of great significance to improving the performance of deep learning algorithms.

[0063] To solve the above problems, an embodiment of the present invention provides an embodiment of deep learning model inference parameter optimization. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system (computer device) including a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0064] In this embodiment, a method for optimizing inference parameters of a deep learning model is provided, which can be used in the above-mentioned terminal devices, such as mobile phones, tablet computers, etc. Figure 1 is a flow chart of a method for optimizing inference parameters of a deep learning model provided by an embodiment of the present invention. Figure 1 As shown, the process includes the following steps.

[0065] Before executing the method steps, the initialization work that needs to be performed is first introduced.

[0066] Specifically, this method is applied to the optimization application scenario of deep learning model inference parameters based on particle swarm. In this method, the Retinanet model in the MLPerf benchmark is used as an example. The initialization work includes:

[0067] 1. Initialize the MLPerf parameter set.

[0068] In an optional example, the parameter set includes: particle swarm size, a first influencing factor set corresponding to the position of each particle in the particle swarm, a second influencing factor set corresponding to the speed, and particle swarm parameters. The particle swarm parameters include: inertia weight, learning factor, and in an optional example, the particle swarm parameters may also include the number of iterations. The particle swarm size determines the total number of particles involved in the optimization.

[0069] For example, setting the particle swarm size to 50 means that during the entire optimization process, 50 particles will participate in the parameter search simultaneously.

[0070] The number of iterations determines the maximum number of cycles the algorithm will run.

[0071] For example, setting the maximum number of iterations to 100 means that the algorithm will run a maximum of 100 cycles to find the optimal parameter combination.

[0072] The inertia weight indicates how much the particle's current velocity affects its next velocity.

[0073] For example, setting the inertia weight to 0.5 means that the current speed has a 50% influence on the next speed.

[0074] The learning factor represents the weight of the influence of the individual optimal position (p_best) and the global optimal position (g_best) on the next velocity of the particle.

[0075] For example, setting the two learning factors c1 and c2 to 1.5 means that the individual optimal position and the global optimal position each have 150% influence on the next speed.

[0076] In addition, it is also necessary to initialize the first influencing factor set and the second influencing factor set.

[0077] In an optional example, the first influencing factor set is composed of position data corresponding to particle dimension data respectively, and the particle dimension data includes: batch size, number of replication streams, number of inference streams, first identification information for indicating whether to run inference on the replication stream, and second identification information for indicating whether to use graph optimization, wherein the batch size, the number of replication streams, the number of inference streams, the first identification information and the second identification information are multiple influencing factors, and the second influencing factor set is composed of speed data corresponding to each of the influencing factors.

[0078] Specifically, each particle dimension data represents the number of parameters that need to be optimized. Each dimension corresponds to a specific parameter. As mentioned above, the particle dimension data includes:

[0079] Batch size

[0080] Number of copy streams

[0081] Number of infer streams

[0082] Whether to run infer on copy streams

[0083] Whether to use graph optimization (use graph).

[0084] During inference, the batch size refers to the number of samples that are input into the model for inference at one time. A larger batch size can speed up inference, but it may also increase memory consumption.

[0085] Copy streams: In a multi-GPU or distributed system, this refers to the number of streams that are being inferred at the same time. Each stream may correspond to a GPU or other computing resources. Increasing the number of copy streams can process more requests in parallel and improve inference efficiency.

[0086] Infer streams: Similar to the number of replication streams, this refers to the number of streams that process inference requests simultaneously. Increasing the number of infer streams can improve the system's concurrency and response speed in a high-concurrency environment.

[0087] Whether to run infer on copy streams: Indicates whether to perform infer operations on copied streams. This may be used in some scenarios that need to process multiple tasks simultaneously to make full use of the computing resources of multiple GPUs or distributed systems.

[0088] Whether to use graph optimization: refers to whether to use graph optimization technology to optimize the reasoning process. Graph optimization can reorganize and optimize the computational graph through the graph model to improve the execution efficiency and speed of the model on the hardware.

[0089] In a specific example, the setting parameter range of the parameter in the particle dimension (the position data in the first impact shadow set) is as follows:

[0090] batchsize range: [2,64];

[0091] copy_streams range: [1,8];

[0092] infer_streams range: [1,8];

[0093] The default values ​​of run_infer_on_copy_streams and use_graph are False, which is 0.

[0094] For example, a particle might have the following initial dimensional parameter combination:

[0095] gpu_batch_size=24, gpu_copy_streams=5, gpu_inference_streams=5, run_infer_on_copy_streams=0, use_graphs=0.

[0096] An initial velocity (the velocity data corresponding to each influencing factor in the second influencing factor set) is randomly assigned to each particle. The velocity represents the magnitude of the parameter value change. The velocity range is set according to the parameter value range, for example, as follows:

[0097] gpu_batch_size: speed range is [-1, 1]

[0098] gpu_copy_streams: speed range is [-0.5, 0.5]

[0099] gpu_inference_streams: speed range is [-0.5, 0.5]

[0100] run_infer_on_copy_streams and use_graphs: Since they are boolean parameters, the speed is set to 0 (i.e. it will not change on initialization).

[0101] Record initial position: Record the initial position of each particle as the current optimal position of the particle.

[0102] The specific steps are as follows:

[0103] For each particle, its initial position is the current parameter combination of the particle. The current parameter combination of each particle is recorded as its current optimal position p_best.

[0104] For example, for a particle, its initial parameter combination is:

[0105] gpu_batch_size=32;

[0106] gpu_copy_streams=4;

[0107] gpu_inference_streams=4;

[0108] run_infer_on_copy_streams=0;

[0109] use_graphs=0;

[0110] Then the p_best position of the particle is [32, 4, 4, 0, 0].

[0111] Initialize the global optimal position: Select the particle with the highest fitness in the initial particle swarm as the global optimal position.

[0112] The specific steps are as follows:

[0113] a) Calculate the fitness value of each particle.

[0114] b) Select the particle with the highest fitness value, and its position is the current global optimal position g_best.

[0115] For example, if the parameter combination of the particle with the highest fitness value is:

[0116] gpu_batch_size=28;

[0117] gpu_copy_streams=6;

[0118] gpu_inference_streams=5;

[0119] run_infer_on_copy_streams=0;

[0120] use_graphs=0;

[0121] Then the global optimal position g_best is [28, 6, 5, 0, 0].

[0122] Among them, in the initialization stage, the initial position of the particle is its p_best, because this is the only known position so far. As the algorithm iterates, p_best will be continuously updated to a new and better position.

[0123] After the aforementioned initialization work is completed, the specific implementation process of the deep learning model inference parameter optimization method of this application is performed as follows:

[0124] Step S101, in the current iteration round, obtain the first influencing factor set corresponding to the position of each particle in the particle swarm at the previous moment, the second influencing factor set corresponding to the speed, the optimal position of each particle, the fitness value corresponding to the optimal position of each particle, and the global optimal position of the particle swarm.

[0125] Specifically, if the current iteration round is the first iteration round, the data at the previous moment are all 0. If the current iteration round is the second iteration round, the corresponding data at the previous moment are the data of the initialization configuration. If the current iteration round is the nth iteration round, the data obtained is the data corresponding to the previous iteration round. Among them, n is a positive integer greater than 2.

[0126] Step S102, determining the fitness value of the first particle at the current moment according to the first influencing factor set corresponding to the first particle, the second influencing factor set corresponding to the first particle, the pre-configured particle swarm parameters, the pre-acquired graphics card utilization, the optimal position of the first particle at the previous moment, and the global optimal position of the particle swarm at the previous moment.

[0127] Specifically, the fitness value of a particle can evaluate the performance of each particle. The optimal parameter setting can be effectively located based on the fitness value later. Therefore, when determining the fitness value of a particle, it is necessary to combine the position-related data and speed-related data of the particle at the previous moment, including the combination of the first influencing factor, the set of second influencing factors corresponding to the first particle, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment. At the same time, it is also necessary to consider the impact of the pre-configured particle group parameters on the fitness value, such as the inertia weight and learning factor mentioned above. Calculating the fitness value through these parameters will be more accurate.

[0128] When determining the set of influencing factors of particle speed, the first set of influencing factors of the particle itself (such as batch size, number of replication streams, etc.), the second set of influencing factors (speed data corresponding to each influencing factor), the pre-configured particle swarm parameters (inertia weight, learning factor), the optimal position of the particle at the previous moment, and the global optimal position of the particle swarm at the previous moment are comprehensively considered. The comprehensive consideration of multiple factors can more comprehensively reflect the changes in particle speed. By introducing inertia weight and learning factor, the global search and local search capabilities of particles can be better balanced. Inertia weight can keep particles in a certain inertia to prevent particles from changing direction too frequently during the search process; learning factor can control the speed at which particles learn to their own optimal position and the global optimal position. It enables particles to find the optimal solution more efficiently during the search process.

[0129] Step S103, determining the optimal position of the first particle at the current moment according to the fitness value corresponding to the optimal position of the first particle at the previous moment and the fitness value of the first particle at the current moment.

[0130] Specifically, as described above, the fitness value evaluates the performance of each particle. Therefore, the higher the fitness value, the better the position of the particle. Therefore, the optimal position of the first particle at the current moment can be determined based on the fitness value corresponding to the optimal position of the first particle at the previous moment and the fitness value of the first particle at the current moment.

[0131] Step S104, determining the global optimal position of the particle swarm at the current moment according to the optimal positions corresponding to all particles in the particle swarm at the current moment and the fitness value corresponding to each particle at the optimal position.

[0132] Specifically, the one with the highest fitness value is selected from the optimal positions corresponding to all particles as the global optimal position.

[0133] Step S105, when the current round meets the conditions for stopping iteration, the iteration is stopped, the optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, and the target influence factor set includes the influence factor set corresponding to the position and the influence factor set corresponding to the speed.

[0134] or,

[0135] Step S106: When the current round does not meet the conditions for stopping iteration, enter the next iteration round.

[0136] Specifically, after multiple rounds of iterations, when it is finally determined that the iteration conditions are met, the iteration is stopped, the final target optimal position and the target global optimal position are determined, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter. The target influence factor set includes the influence factor set corresponding to the position and the influence factor set corresponding to the speed.

[0137] The deep learning model inference parameter optimization method provided in this embodiment determines the fourth influencing factor set corresponding to the position of the first particle at the current moment, which is actually the position of the first particle at the current moment, based on the first influencing factor set corresponding to the first particle, the second influencing factor set, the pre-configured particle swarm parameters, the optimal position at the previous moment, and the global optimal position of the particle swarm at the previous moment. Then, based on the fitness value corresponding to the optimal position of the first particle at the previous moment and the fitness value of the first particle at the current moment, the optimal position of the first particle at the current moment is determined. By comprehensively considering these factors, the optimal position of the particle at the current moment can be more accurately inferred, which improves the accuracy and reliability of particle position optimization.

[0138] Then, according to the optimal positions of multiple particles at the current moment and the fitness value corresponding to each optimal position, the global optimal position is selected. This method reflects that by searching for the optimal solution through the collaborative search of multiple particles, a better solution can be searched in a shorter time, which improves the search efficiency. By continuously iteratively updating the position and speed of the particles and the global optimal position of the particle swarm, the algorithm can gradually approach the optimal solution. In each iteration, the particles adjust their positions and speeds according to their own historical optimal positions and the global optimal position of the particle swarm, thereby achieving more efficient search. The particle swarm optimization algorithm can effectively avoid falling into the local optimal solution through parallel search and information sharing of multiple particles.

[0139] When determining the fitness value of the first particle at the current moment, the multiple influencing factor sets of the first particle, the pre-configured particle swarm parameters, the pre-acquired graphics card utilization, the optimal position of the particle at the previous moment, and the global optimal position of the particle swarm at the previous moment are comprehensively considered. The comprehensive consideration of multiple factors can more comprehensively evaluate the performance of particles and improve the accuracy and reliability of fitness values. The introduction of pre-acquired graphics card utilization helps to better consider the utilization of computing resources during the particle swarm optimization process. By incorporating graphics card utilization into the calculation of fitness values, the algorithm can be prompted to pay more attention to the effective use of resources and improve computing efficiency in the process of searching for the optimal solution.

[0140] The whole process is an automatic iterative process that does not require too much human intervention. The algorithm automatically updates the particle position, calculates the fitness value, and determines the optimal position according to the preset conditions and rules, reducing the impact of human factors on the results and improving the objectivity and reliability of the results. Automated processing can save labor costs and improve work efficiency. Applying these optimization parameters to the deep learning model can improve the accuracy and efficiency of the model. Moreover, the algorithm can automatically adjust the parameters according to the actual situation to adapt to different models and task requirements. By continuously optimizing the model parameters, the generalization ability and robustness of the model can be improved, so that it can achieve better performance in different data sets and application scenarios.

[0141] In this embodiment, a method for optimizing inference parameters of a deep learning model is provided, which can be used in the above-mentioned mobile terminals, such as mobile phones, tablet computers, etc. Figure 2 FIG. 1 is a flow chart of another method for optimizing deep learning model inference parameters provided by an embodiment of the present invention. Figure 2 shown.

[0142] According to the first influencing factor set, the second influencing factor set, and the pre-acquired graphics card utilization, the fitness value of each particle at the current moment is determined. The process includes the following steps:

[0143] Step S201, determining the model inference speed corresponding to the first particle according to the first influencing factor set corresponding to the first particle, the second influencing factor set corresponding to the first particle, the preconfigured particle swarm parameters, the optimal position of the first particle at the previous moment, and the global optimal position of the particle swarm at the previous moment.

[0144] In an optional example, the model inference speed refers to the inference speed of the model to be tested under certain configurations. The specific configuration here refers to the aforementioned first image factor set, the second influence factor set, the preconfigured particle swarm parameters, the optimal position of the first particle at the previous moment, and the global optimal position of the particle swarm at the previous moment.

[0145] Among them, it is first necessary to determine the third influencing factor set corresponding to the speed of the first particle at the current moment based on the first influencing factor set corresponding to the first particle, the second influencing factor set corresponding to the first particle, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment.

[0146] In an optional example, according to the first influencing factor set corresponding to the first particle, the second influencing factor set corresponding to the first particle, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment, determining the third influencing factor set corresponding to the speed of the first particle at the current moment includes the following method steps, specifically see as follows:

[0147] According to the position data corresponding to the first influencing factor included in the first influencing factor set corresponding to the first particle at the previous moment, the speed data corresponding to the first influencing factor in the second influencing factor set, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment, the speed data corresponding to the first influencing factor at the current moment is determined, wherein the first influencing factor is any one of the influencing factors in the first influencing factor set.

[0148] Specifically, the influencing factor mentioned here is any one of the batch size, the number of replication streams, the number of inference streams, the first identification information, and the second identification information mentioned above.

[0149] According to the position data corresponding to the first influencing factor included in the first influencing factor set corresponding to the first particle at the previous moment, the speed data corresponding to the first influencing factor in the second influencing factor set, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment, the speed data corresponding to the first influencing factor at the current moment is determined, which is specifically expressed by the following formula:

[0150] (Formula 1)

[0151] in, is the speed data corresponding to the first influencing factor at the current moment, is the inertia weight, is the speed data corresponding to the first influencing factor at the previous moment, c1 and c2 are learning factors, r1 and r2 are random numbers randomly drawn from the uniform distribution U(0,1), and the value range is [0, 1]. is the optimal position of the first particle at the previous moment, is the global optimal position of the particle swarm at the previous moment, is the position data corresponding to the first influencing factor at the previous moment.

[0152] The data corresponding to each of the multiple influencing factors at the current moment together constitute a third influencing factor set.

[0153] In a specific example, assuming the inertia weight , learning factors c1=1.5 and c2=1.5, random numbers r1 and r2 are 0.8 and 0.9 respectively. The local optimal position p_best and global optimal position g_best of the current particle are as follows:

[0154] p_best={gpu_batch_size=34;

[0155] gpu_copy_streams=3;

[0156] gpu_inference_streams=5;

[0157] run_infer_on_copy_streams=0;

[0158] use_graphs=0}

[0159] g_best={gpu_batch_size=36;

[0160] gpu_copy_streams=4;

[0161] gpu_inference_streams=6;

[0162] run_infer_on_copy_streams=0;

[0163] use_graphs=0}

[0164] The initial velocity of a particle can be randomly generated as follows:

[0165] The initial speed of gpu_batch_size is -1;

[0166] The initial speed of gpu_copy_streams is 0.5;

[0167] The initial speed of gpu_inference_streams is -0.5;

[0168] At the previous moment, the particle's position data is as follows:

[0169] gpu_batch_size=32;

[0170] gpu_copy_streams=4;

[0171] gpu_inference_streams=4;

[0172] run_infer_on_copy_streams=0;

[0173] use_graphs=0;

[0174] Then the initial position of the particle is [32, 4, 4, 0, 0].

[0175] Calculate the updated speed: (the initial value of each parameter is a value randomly selected within the specified speed range):

[0176] gpu_batch_size speed update:

[0177]

[0178] gpu_copy_streams speed update:

[0179]

[0180] gpu_inference_streams speed update:

[0181]

[0182] Then, a fourth influencing factor set is determined according to the influencing factors in the first influencing factor set and the influencing factors in the third influencing factor set corresponding to the first particle.

[0183] Specifically, according to the influence factors in the first influence factor set and the influence factors in the third influence factor set corresponding to the first particle, the fourth influence factor set is determined, which is reflected by the following expression:

[0184] (Formula 2)

[0185] in, is the speed data corresponding to the first influencing factor in the third influencing factor set at the current moment, is the position data corresponding to the first influencing factor in the previous moment, is the position data corresponding to the first influencing factor in the fourth influencing factor set at the current moment, wherein the first influencing factor is any one of the multiple influencing factors.

[0186] Still referring to the above example, calculate the updated position and round it:

[0187] gpu_batch_size position update:

[0188]

[0189] gpu_copy_streams position updated:

[0190]

[0191] gpu_inference_streams position update:

[0192]

[0193] The updated parameter combination is:

[0194] gpu_batch_size = 39

[0195] gpu_copy_streams = 3

[0196] gpu_inference_streams = 8

[0197] run_infer_on_copy_streams = 0

[0198] use_graphs = 0

[0199] Finally, the model reasoning speed is determined according to the fourth influencing factor set and the third influencing factor set.

[0200] Specifically, when the above data is run in the model to be tested, the model inference speed will be automatically displayed.

[0201] Step S202, determining the fitness value of each particle at the current moment according to the model inference speed and the graphics card utilization rate corresponding to each particle.

[0202] In a specific example, the fitness value of each particle at the current moment is determined according to the model inference speed and graphics card utilization rate corresponding to each particle, which is specifically expressed by the following formula:

[0203] (Formula 3)

[0204] in, is the fitness value of the particle at the current moment, S is the model inference speed, M is the graphics card utilization rate, is the adjustment coefficient.

[0205] The fitness function combines the model inference speed S and resource utilization efficiency M to form an indicator for comprehensively evaluating model performance. The MLPerf benchmark focuses on inference performance, so the fitness function directly uses the performance indicator of model inference (samples / s), while also considering resource consumption factors. Here, memory consumption or energy consumption is usually represented by video memory utilization. When video memory utilization M is high (close to 1), is very small, so its contribution to fitness is also small, and fitness is mainly determined by the inference speed S. When the memory utilization M is low (much less than 1), Larger, It will have a significant impact on fitness, thereby reducing the fitness value, suggesting that graphics card utilization needs to be further optimized.

[0206] In a specific example, assume that the inference speed S = 1000 samples / s, the video memory utilization M = 0.8, and α = 0.3.

[0207]

[0208] If the video memory utilization is low, assuming M=0.5:

[0209]

[0210] Obviously, the low utilization rate has an impact on our performance, and the fitness function design is relatively reasonable.

[0211] Specifically, by determining the model inference speed according to the first influencing factor set and the second influencing factor set of the particle, the influence of different parameter combinations on the model inference performance can be more accurately reflected. The evaluation method based on multiple factors avoids the limitations of single factor evaluation, making the evaluation of model performance more comprehensive and accurate. When determining the fitness value of the particle, the model inference speed and the graphics card utilization rate are combined. It can ensure that in the optimization process, not only the inference speed of the model is paid attention to, but also the utilization efficiency of the graphics card resources is fully considered. It helps to achieve reasonable allocation and utilization of resources while improving model performance and avoid waste of resources. Moreover, comprehensively considering the model inference speed and graphics card utilization to determine the fitness value can provide more targeted evaluation indicators for the particle swarm optimization algorithm. In the process of searching for the optimal solution, the algorithm can more effectively find a parameter combination that is relatively excellent in both model performance and resource utilization, thereby improving the optimization effect.

[0212] In an optional example, in order to improve the performance of the particle swarm algorithm, the ability of global search and local search is balanced. Based on the above embodiment, the method may also include:

[0213] The values ​​of inertia weight and learning factor are dynamically adjusted according to the particle's search situation.

[0214] Specifically, it can be implemented in the following ways:

[0215] For example, the inertia weight can be adjusted according to the fitness value of the particle or the distance from the global optimal position. If the fitness value of the particle is good (higher than a preset fitness threshold) or the position of the particle is close to the global optimal position (for example, the distance between the position of a preset proportion of examples and the global optimal position is less than a preset distance threshold), the inertia weight can be reduced to strengthen the local search; otherwise, the inertia weight can be increased to encourage the global search.

[0216] Specifically, the method of reducing or increasing the weight may include linear adjustment according to a certain ratio, or nonlinear adjustment may be adopted.

[0217] In parallel with the above method, in a feasible method, the inertia weight can also be configured to a large value at the time of initialization, and then gradually reduced in each iteration. This can encourage particles to conduct global search at the beginning, and then gradually turn to local search. Alternatively, a nonlinear decreasing method can be used in each iteration. For example, the value of the inertia weight can be adjusted using functions such as exponential decreasing or logarithmic decreasing. Better control of the exploration and utilization capabilities of particles during the search process.

[0218] Similarly, the learning factor can also be adjusted according to the particle's search situation.

[0219] For example, the value of the learning factor can be adjusted according to the fitness value of the particle or the distance from the global optimal position. If the fitness value of the particle is good (higher than the preset fitness threshold, which can be the same as or different from the preset fitness value set in the weight) or the position of the particle is close to the global optimal position (for example, the distance between the position of the preset proportion of examples and the global optimal position is less than the preset distance threshold, which can also be the same as or different from the preset fitness value set in the weight), the learning factor can be increased to strengthen the learning of the optimal position; otherwise, the learning factor can be reduced to avoid over-reliance on historical experience.

[0220] Of course, it is also possible to combine multiple adjustment strategies to further improve the performance of the particle swarm algorithm. For example, you can use linear decreasing inertia weights and adaptive learning factors at the same time, or use different adjustment strategies to switch at different iteration stages.

[0221] Furthermore, by conducting experiments and comparing different parameter settings, one can find the adjustment strategy for the inertia weight and learning factor that best suits a specific problem.

[0222] By trying different initial values, decrement methods, and adjustment ranges, observe their impact on algorithm performance and select the optimal parameter combination.

[0223] In an optional example, the preconfigured particle swarm parameters include the number of iterations;

[0224] When the current round meets the conditions for stopping iteration, the iteration is stopped, the optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter. The target influence factor set includes the influence factor set corresponding to the position and the influence factor set corresponding to the speed, including:

[0225] When the current round reaches the number of iterations, it is determined that the current round meets the conditions for stopping iterations, and the iteration is stopped;

[0226] The optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, and the target influence factor set includes the influence factor set corresponding to the position and the influence factor set corresponding to the speed.

[0227] In another optional example, the preconfigured particle swarm parameters include the number of iterations; when the current round meets the stop iteration condition, the iteration is stopped, the optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, the target influence factor set includes an influence factor set corresponding to the position, and an influence factor set corresponding to the speed, including:

[0228] When the difference between the fitness value of the particle corresponding to the global optimal position of the particle swarm at the current moment and the fitness value of the particle corresponding to the global optimal position of the particle swarm at the previous moment is less than or equal to the preset threshold, it is determined that the current round meets the stop iteration condition and the iteration is stopped;

[0229] The optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, and the target influence factor set includes the influence factor set corresponding to the position and the influence factor set corresponding to the speed.

[0230] Specifically, update the local optimal position (p_best): for each particle, if the fitness value corresponding to its current position x[t+1] is better than its historical optimal position p_best, update its p_best to the current position x[t+1].

[0231] For example, if the current position of particle i is [t+1]=[39,3,8,0,0], and its fitness value is better than the previous p_best, then update p_best to [39,3,8,0,0].

[0232] Update the global optimal position (g_best): In each iteration, check the current optimal position p_best of all particles, find the position with the highest fitness value, and update the global optimal position g_best.

[0233] For example, if there is a particle in the particle swarm whose fitness value corresponding to p_best is the highest and its position is [39,3,8,0,0], then update g_best to [39,3,8,0,0].

[0234] Setting Termination Conditions: To ensure that the algorithm completes in a reasonable amount of time and avoids falling into an infinite loop, the following termination conditions need to be set;

[0235] Maximum number of iterations: Set the maximum number of iterations allowed for the algorithm to run to 100. When the algorithm runs for 100 iterations, the algorithm will terminate regardless of whether the optimal solution is found.

[0236] Fitness change threshold: Set the fitness change threshold to 0.001. If the fitness value of the global optimal position changes by less than 0.001 in several consecutive iterations, the algorithm is considered to have converged and the iteration is stopped.

[0237] In this embodiment, a deep learning model inference parameter optimization device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0238] Figure 3 The overall flow chart of the above process is shown in FIG. Since the detailed method steps in the process have been described in detail in the previous text, they will not be repeated here.

[0239] This embodiment provides a deep learning model inference parameter optimization device, such as Figure 4As shown, it includes: an acquisition module 401, a fitness value determination module 402, a position determination module 403 and a processing module 404.

[0240] The acquisition module 401 is used to acquire, in the current iteration round, the first influencing factor set corresponding to the position of each particle in the particle swarm at the previous moment, the second influencing factor set corresponding to the speed, the optimal position of each particle, the fitness value corresponding to the optimal position of each particle, and the global optimal position of the particle swarm;

[0241] The fitness value determination module 402 determines the fitness value of the first particle at the current moment according to the first influencing factor set corresponding to the first particle, the second influencing factor set corresponding to the first particle, the pre-configured particle swarm parameters, the pre-acquired graphics card utilization rate, the optimal position of the first particle at the previous moment, and the global optimal position of the particle swarm at the previous moment, wherein the first particle is any particle in the particle swarm;

[0242] The position determination module 403 is used to determine the optimal position of the first particle at the current moment according to the fitness value corresponding to the optimal position of the first particle at the previous moment and the fitness value of the first particle at the current moment; determine the global optimal position of the particle group at the current moment according to the optimal positions corresponding to all particles in the particle group at the current moment and the fitness value corresponding to each particle at the optimal position;

[0243] The processing module 404 is used to stop the iteration when the current round meets the conditions for stopping the iteration, determine the optimal position of the first particle at the current moment as the target optimal position of the first particle, determine the global optimal position of the particle swarm at the current moment as the target global optimal position of the particle swarm, and use the target influence factor set corresponding to the target optimal position and the target global optimal position as the final model parameter, the target influence factor set includes the influence factor set corresponding to the position and the influence factor set corresponding to the speed; or, when the current round does not meet the conditions for stopping the iteration, enter the next iteration round.

[0244] In an optional example, the first influencing factor set is composed of position data corresponding to the particle dimension data respectively, and the particle dimension data includes: batch size, number of replication streams, number of inference streams, first identification information for indicating whether to run inference on the replication stream, and second identification information for indicating whether to use graph optimization, wherein the batch size, number of replication streams, number of inference streams, first identification information and second identification information are multiple influencing factors, and the second influencing factor set is composed of speed data corresponding to each influencing factor.

[0245] In an optional example, the fitness value determination module 402 is specifically configured to:

[0246] Determine the model inference speed corresponding to the first particle according to a first influencing factor set corresponding to the first particle, a second influencing factor set corresponding to the first particle, preconfigured particle swarm parameters, an optimal position of the first particle at a previous moment, and a global optimal position of the particle swarm at a previous moment;

[0247] According to the model inference speed and graphics card utilization rate corresponding to each particle, the fitness value of each particle at the current moment is determined.

[0248] In an optional example, the fitness value module determines the fitness value of each particle at the current moment according to the model inference speed and graphics card utilization rate corresponding to each particle, which is specifically expressed by the following formula:

[0249] (Formula 4)

[0250] in, is the fitness value of the particle at the current moment, S is the model inference speed, M is the graphics card utilization rate, is the adjustment coefficient.

[0251] In an optional example, the preconfigured particle swarm parameters include: inertia weight, learning factor;

[0252] The fitness value determination module 402 is specifically used to determine a third influencing factor set corresponding to the speed of the first particle at the current moment according to the first influencing factor set corresponding to the first particle, the second influencing factor set corresponding to the first particle, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment;

[0253] Determine a fourth influence factor set according to the influence factors in the first influence factor set and the influence factors in the third influence factor set corresponding to the first particle;

[0254] The model reasoning speed is determined according to the fourth influencing factor set and the third influencing factor set.

[0255] In an optional example, the fitness value determination module 402 is specifically configured to:

[0256] According to the position data corresponding to the first influencing factor included in the first influencing factor set corresponding to the first particle at the previous moment, the speed data corresponding to the first influencing factor in the second influencing factor set, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment, the speed data corresponding to the first influencing factor at the current moment is determined, wherein the first influencing factor is any one of the influencing factors in the first influencing factor set;

[0257] The data corresponding to each of the multiple influencing factors at the current moment together constitute a third influencing factor set.

[0258] In an optional example, the fitness value determination module 402 determines the speed data corresponding to the first influencing factor at the current moment according to the position data corresponding to the first influencing factor included in the first influencing factor set corresponding to the first particle at the previous moment, the speed data corresponding to the first influencing factor in the second influencing factor set, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment, which is specifically expressed by the following formula:

[0259] (Formula 5)

[0260] in, is the speed data corresponding to the first influencing factor at the current moment, is the inertia weight, is the speed data corresponding to the first influencing factor at the previous moment, c1 and c2 are learning factors, r1 and r2 are random numbers randomly drawn from the uniform distribution U(0,1), and the value range is [0, 1]. is the optimal position of the first particle at the previous moment, is the global optimal position of the particle swarm at the previous moment, is the position data corresponding to the first influencing factor at the previous moment.

[0261] In an optional example, the fitness value determination module 402 determines a fourth influencing factor set according to the influencing factors in the first influencing factor set and the influencing factors in the third influencing factor set corresponding to the first particle, which is embodied by the following expression:

[0262] (Formula 6)

[0263] in, is the speed data corresponding to the first influencing factor in the third influencing factor set at the current moment, is the position data corresponding to the first influencing factor in the previous moment, is the position data corresponding to the first influencing factor in the fourth influencing factor set at the current moment, wherein the first influencing factor is any one of the multiple influencing factors.

[0264] In an optional example, the location determination module 403 is specifically configured to:

[0265] The optimal position of the particle with the highest fitness value at the current moment is selected as the global optimal position of the particle swarm at the current moment.

[0266] In an optional example, the preconfigured particle swarm parameters include the number of iterations;

[0267] The processing module 404 is specifically used to: when the current round reaches the number of iterations, determine that the current round meets the condition for stopping iteration, and stop iteration;

[0268] The optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, and the target influence factor set includes the influence factor set corresponding to the position and the influence factor set corresponding to the speed.

[0269] In an optional example, the preconfigured particle swarm parameters include the number of iterations;

[0270] The processing module 404 is specifically used to: when the fitness value of the particle corresponding to the global optimal position of the particle swarm at the current moment and the fitness value of the particle corresponding to the global optimal position of the particle swarm at the previous moment are less than or equal to a preset threshold, determine that the current round meets the stop iteration condition, and stop iteration;

[0271] The optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, and the target influence factor set includes the influence factor set corresponding to the position and the influence factor set corresponding to the speed.

[0272] In an optional example, the device further includes: an initialization module 405;

[0273] The initialization module 405 is used to initialize the first influencing factor set corresponding to the position of each particle in the particle swarm, the second influencing factor set corresponding to the velocity, and the preconfigured particle swarm parameters before the initial iteration round.

[0274] The deep learning model inference parameter optimization device in this embodiment is presented in the form of a functional module, where the module refers to an application specific integrated circuit (ASIC), a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0275] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0276] An embodiment of the present invention provides a deep learning model inference parameter optimization device, which determines the fourth influencing factor set corresponding to the position of the first particle at the current moment, which is actually the position of the first particle at the current moment, based on the first influencing factor set, the second influencing factor set, the pre-configured particle swarm parameters, the optimal position at the previous moment, and the global optimal position of the particle swarm at the previous moment corresponding to the first particle. Then, based on the fitness value corresponding to the optimal position of the first particle at the previous moment and the fitness value of the first particle at the current moment, the optimal position of the first particle at the current moment is determined. By comprehensively considering these factors, the optimal position of the particle at the current moment can be more accurately inferred, which improves the accuracy and reliability of particle position optimization.

[0277] Then, according to the optimal positions of multiple particles at the current moment and the fitness value corresponding to each optimal position, the global optimal position is selected. This method reflects that by searching for the optimal solution through the collaborative search of multiple particles, a better solution can be searched in a shorter time, which improves the search efficiency. By continuously iteratively updating the position and speed of the particles and the global optimal position of the particle swarm, the algorithm can gradually approach the optimal solution. In each iteration, the particles adjust their positions and speeds according to their own historical optimal positions and the global optimal position of the particle swarm, thereby achieving more efficient search. The particle swarm optimization algorithm can effectively avoid falling into the local optimal solution through parallel search and information sharing of multiple particles.

[0278] When determining the fitness value of the first particle at the current moment, the multiple influencing factor sets of the first particle, the pre-configured particle swarm parameters, the pre-acquired graphics card utilization, the optimal position of the particle at the previous moment, and the global optimal position of the particle swarm at the previous moment are comprehensively considered. The comprehensive consideration of multiple factors can more comprehensively evaluate the performance of particles and improve the accuracy and reliability of fitness values. The introduction of pre-acquired graphics card utilization helps to better consider the utilization of computing resources during the particle swarm optimization process. By incorporating graphics card utilization into the calculation of fitness values, the algorithm can be prompted to pay more attention to the effective use of resources and improve computing efficiency in the process of searching for the optimal solution.

[0279] The whole process is an automatic iterative process that does not require too much human intervention. The algorithm automatically updates the particle position, calculates the fitness value, and determines the optimal position according to the preset conditions and rules, reducing the impact of human factors on the results and improving the objectivity and reliability of the results. Automated processing can save labor costs and improve work efficiency. Applying these optimization parameters to the deep learning model can improve the accuracy and efficiency of the model. Moreover, the algorithm can automatically adjust the parameters according to the actual situation to adapt to different models and task requirements. By continuously optimizing the model parameters, the generalization ability and robustness of the model can be improved, so that it can achieve better performance in different data sets and application scenarios.

[0280] The embodiment of the present invention also provides a computer device having the above Figure 4 The deep learning model inference parameter optimization device shown.

[0281] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0282] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include an integrated circuit. The integrated circuit may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0283] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0284] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0285] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0286] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0287] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device may be a touch screen.

[0288] The embodiment of the present invention also provides a computer-readable storage medium. The method provided in the above embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or is implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0289] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0290] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for optimizing inference parameters of a deep learning model, characterized in that: The method comprises: In the current iteration round, the first influencing factor set corresponding to the position of each particle in the particle swarm at the previous moment, the second influencing factor set corresponding to the speed, the optimal position of each particle, the fitness value corresponding to the optimal position of each particle, and the global optimal position of the particle swarm are obtained; Determine the fitness value of the first particle at the current moment according to a first influencing factor set corresponding to the first particle, a second influencing factor set corresponding to the first particle, pre-configured particle swarm parameters, a pre-acquired graphics card utilization rate, an optimal position of the first particle at the previous moment, and a global optimal position of the particle swarm at the previous moment, wherein the first particle is any particle in the particle swarm; Determine the optimal position of the first particle at the current moment according to the fitness value corresponding to the optimal position of the first particle at the previous moment and the fitness value of the first particle at the current moment; Determine the global optimal position of the particle swarm at the current moment according to the optimal positions of all particles in the particle swarm at the current moment and the fitness value of each particle at the optimal position; When the current round meets the conditions for stopping iteration, the iteration is stopped, the optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, the target influence factor set includes an influence factor set corresponding to the position and an influence factor set corresponding to the speed; Alternatively, when the current round does not meet the iteration stop condition, enter the next iteration round; The determining of the fitness value of the first particle at the current moment according to the first influencing factor set corresponding to the first particle, the second influencing factor set corresponding to the first particle, the pre-configured particle swarm parameters, the pre-acquired graphics card utilization, the optimal position of the first particle at the previous moment, and the global optimal position of the particle swarm at the previous moment specifically includes: Determine a model inference speed corresponding to the first particle according to a first influencing factor set corresponding to the first particle, a second influencing factor set corresponding to the first particle, preconfigured particle swarm parameters, an optimal position of the first particle at the previous moment, and a global optimal position of the particle swarm at the previous moment; Determine the fitness value of each particle at the current moment according to the model reasoning speed and the graphics card utilization rate respectively corresponding to each particle; The fitness value of each particle at the current moment is determined according to the model inference speed and the graphics card utilization rate corresponding to each particle, which is specifically expressed by the following formula: in, is the fitness value of the particle at the current moment, S is the model reasoning speed, M is the graphics card utilization rate, is the adjustment coefficient.

2. The method according to claim 1, characterized in that The first influencing factor set is composed of position data corresponding to particle dimension data respectively, and the particle dimension data includes: batch size, number of replication streams, number of inference streams, first identification information for indicating whether to run inference on the replication stream, and second identification information for indicating whether to use graph optimization, wherein the batch size, the number of replication streams, the number of inference streams, the first identification information and the second identification information are multiple influencing factors, and the second influencing factor set is composed of speed data corresponding to each of the influencing factors.

3. The method according to claim 1, characterized in that The preconfigured particle swarm parameters include: inertia weight, learning factor; The step of inputting a first influencing factor set corresponding to the first particle, a second influencing factor set corresponding to the first particle, preconfigured particle swarm parameters, the optimal position of the first particle at the previous moment, and the global optimal position of the particle swarm at the previous moment into the deep learning model to be tested to obtain the model inference speed specifically includes: Determine a third set of influencing factors corresponding to the velocity of the first particle at the current moment according to a first set of influencing factors corresponding to the first particle, a second set of influencing factors corresponding to the first particle, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment; Determine a fourth influence factor set according to the influence factors in the first influence factor set corresponding to the first particle and the influence factors in the third influence factor set; The model reasoning speed is determined according to the fourth influencing factor set and the third influencing factor set.

4. The method according to claim 3, characterized in that The determining, according to a first influencing factor set corresponding to the first particle, a second influencing factor set corresponding to the first particle, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment, a third influencing factor set corresponding to the speed of the first particle at the current moment includes: Determine the velocity data corresponding to the first influencing factor at the current moment according to the position data corresponding to the first influencing factor included in the first influencing factor set corresponding to the first particle at the previous moment, the velocity data corresponding to the first influencing factor in the second influencing factor set, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment, wherein the first influencing factor is any one of the influencing factors in the first influencing factor set; The data corresponding to each of the multiple influencing factors at the current moment together constitute the third influencing factor set.

5. The method according to claim 4, characterized in that The method of determining the speed data corresponding to the first influencing factor at the current moment according to the position data corresponding to the first influencing factor included in the first influencing factor set corresponding to the first particle at the previous moment, the speed data corresponding to the first influencing factor in the second influencing factor set, the inertia weight, the learning factor, the optimal position of the first particle at the previous moment, and the global optimal position of the particle group at the previous moment is specifically expressed by the following formula: in, is the speed data corresponding to the first influencing factor at the current moment, is the inertia weight, is the speed data corresponding to the first influencing factor at the previous moment, c1 and c2 are the learning factors, r1 and r2 are random numbers randomly drawn from the uniform distribution U(0,1), and the value range is [0, 1], is the optimal position of the first particle at the previous moment, is the global optimal position of the particle swarm at the previous moment, is the position data corresponding to the first influencing factor at the previous moment.

6. The method according to claim 3, characterized in that The fourth influencing factor set is determined according to the influencing factors in the first influencing factor set and the influencing factors in the third influencing factor set corresponding to the first particle, which is reflected by the following expression: in, is the speed data corresponding to the first influencing factor in the third influencing factor set at the current moment, is the position data corresponding to the first influencing factor in the first influencing factor at the previous moment, It is the position data corresponding to the first influencing factor in the fourth influencing factor set at the current moment, wherein the first influencing factor is any one of multiple influencing factors.

7. The method according to any one of claims 1 to 6, characterized in that: Determining the global optimal position of the particle swarm at the current moment according to the optimal positions of all particles in the particle swarm at the current moment and the fitness value of each particle at the optimal position, specifically includes: The optimal position of the particle with the highest fitness value at the current moment is selected as the global optimal position of the particle swarm at the current moment.

8. The method according to any one of claims 1 to 6, characterized in that: The preconfigured particle swarm parameters include the number of iterations; When the current round meets the conditions for stopping iteration, the iteration is stopped, the optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, the target influence factor set includes an influence factor set corresponding to the position and an influence factor set corresponding to the speed, including: When the current round reaches the number of iterations, determining that the current round meets the condition for stopping iteration, and stopping iteration; The optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, and the target influence factor set includes an influence factor set corresponding to the position and an influence factor set corresponding to the speed.

9. The method according to any one of claims 1 to 6, characterized in that: The preconfigured particle swarm parameters include the number of iterations; when the current round meets the conditions for stopping iteration, the iteration is stopped, the optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, the target influence factor set includes an influence factor set corresponding to the position, and an influence factor set corresponding to the speed, including: When the difference between the fitness value of the particle corresponding to the global optimal position of the particle swarm at the current moment and the fitness value of the particle corresponding to the global optimal position of the particle swarm at the previous moment is less than or equal to a preset threshold, it is determined that the current round meets the stop iteration condition and the iteration is stopped; The optimal position of the first particle at the current moment is determined as the target optimal position of the first particle, the global optimal position of the particle swarm at the current moment is determined as the target global optimal position of the particle swarm, and the target influence factor set corresponding to the target optimal position and the target global optimal position is used as the final model parameter, and the target influence factor set includes an influence factor set corresponding to the position and an influence factor set corresponding to the speed.

10. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Before the initial iteration round, a first set of influencing factors corresponding to the position of each particle in the particle swarm, a second set of influencing factors corresponding to the velocity, and preconfigured particle swarm parameters are initialized.

11. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the deep learning model inference parameter optimization method according to any one of claims 1 to 10 by executing the computer instructions.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the deep learning model inference parameter optimization method described in any one of claims 1 to 10.

13. A computer program product, characterized in that It includes computer instructions, which are used to enable a computer to execute the deep learning model inference parameter optimization method described in any one of claims 1 to 10.

Images