A model pruning method, device and storage medium for heterogeneous computing clusters

Abstract

The present invention discloses a model pruning method, device and storage medium of a heterogeneous computing cluster, which relates to the field of artificial intelligence technology, including: obtaining a neural network model to be pruned; testing the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster; calculating the total time corresponding to each heterogeneous computing power according to each execution time; obtaining the memory size requirement for running the neural network model to be pruned; determining the heterogeneous computing power that meets the memory size requirement and has the smallest total time consumption as the target heterogeneous computing power; using the target heterogeneous computing power to prune the neural network model to be pruned according to the preset accuracy requirements and preset time requirements that the neural network model after pruning needs to meet. The present invention can select a suitable heterogeneous computing power for the neural network model to be pruned, and improves the execution efficiency and accuracy of the compressed model.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a model pruning method, device, computer-readable storage medium and computer program product for a heterogeneous computing cluster. Background Art

[0002] In a multi-heterogeneous computing system, heterogeneous computing power with different computing performance will be connected to the same distributed computing system for different neural network calculations. In this system, heterogeneous computing power can be allocated to perform neural network calculations that meet its computing characteristics, thereby improving the performance of the calculation. Model pruning and compression technology is often used when deploying neural network models.

[0003] Current model pruning and compression technologies mostly target single computing power or a single neural network model. It is impossible to select appropriate heterogeneous computing power in a heterogeneous computing cluster for model pruning and compression. The execution efficiency and accuracy of the compressed model are both low.

[0004] In summary, how to effectively solve the problems of being unable to select appropriate heterogeneous computing power in a heterogeneous computing cluster for model pruning and compression, and the low execution efficiency and accuracy of the compressed model, is an issue that technical personnel in this field urgently need to solve. Summary of the invention

[0005] The purpose of the present invention is to provide a model pruning method for a heterogeneous computing cluster, which method can select appropriate heterogeneous computing power for the neural network model to be pruned, thereby improving the execution efficiency and accuracy of the compressed model; another purpose of the present invention is to provide a model pruning device, equipment and computer-readable storage medium for a heterogeneous computing cluster.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] A model pruning method for a heterogeneous computing cluster, comprising:

[0008] Obtain the neural network model to be pruned;

[0009] Test the execution time of each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster;

[0010] Calculate the total time consumption corresponding to each heterogeneous computing power based on the execution time consumption;

[0011] Obtaining the memory size requirement for running the neural network model to be pruned;

[0012] Determine the heterogeneous computing power that meets the memory size requirement and has the smallest total time consumption as the target heterogeneous computing power;

[0013] The target heterogeneous computing power is used to prune the neural network model to be pruned according to the preset accuracy requirements and preset time requirements that the neural network model after pruning needs to meet.

[0014] In a specific embodiment of the present invention, after pruning the neural network model to be pruned by using the target heterogeneous computing power according to the preset accuracy requirement and the preset time requirement that the neural network model after pruning needs to meet, it also includes:

[0015] Retraining the pruned neural network model obtained by pruning to obtain a trained neural network model;

[0016] The trained neural network model is sent to the heterogeneous computing cluster to deploy the trained neural network model using the heterogeneous computing cluster.

[0017] In a specific implementation of the present invention, the total time consumption corresponding to each heterogeneous computing power is calculated according to each execution time consumption, including:

[0018] The execution times corresponding to each heterogeneous computing power are summed up to obtain the total execution times corresponding to each heterogeneous computing power.

[0019] In a specific embodiment of the present invention, after testing the execution time corresponding to each neural layer in the to-be-pruned neural network model by each heterogeneous computing power in the heterogeneous computing cluster, the method further includes:

[0020] When it is detected that there is a software update and / or a hardware update in the heterogeneous computing cluster, the execution time corresponding to each neural layer in the neural network model to be pruned is retested by each heterogeneous computing power in the heterogeneous computing cluster.

[0021] In a specific implementation of the present invention, before determining the heterogeneous computing power that meets the memory size requirement and has the minimum total time consumption as the target heterogeneous computing power, the method further includes:

[0022] Collect the available memory size corresponding to each heterogeneous computing power;

[0023] Generates a list of available memory sizes based on the available memory sizes.

[0024] In a specific implementation of the present invention, determining the heterogeneous computing power that meets the memory size requirement and has the minimum total time consumption as the target heterogeneous computing power includes:

[0025] Sort the total time consumption by size to obtain the total time consumption sequence;

[0026] Select the current minimum total time consumption from the end with the smaller total time consumption in the total time consumption sequence;

[0027] Searching the available memory size list for the currently available memory of the heterogeneous computing power corresponding to the currently minimum total time consumption;

[0028] Determining whether the currently available memory meets the memory size requirement;

[0029] If yes, the heterogeneous computing power corresponding to the current minimum total time consumption is determined as the target heterogeneous computing power;

[0030] If not, the next total time consumption in the total time consumption sequence that is greater than the current minimum total time consumption is determined as the new current minimum total time consumption, and the step of searching the current available memory of the heterogeneous computing power corresponding to the current minimum total time consumption from the available memory size list is repeated.

[0031] In a specific embodiment of the present invention, the target heterogeneous computing power is used to prune the neural network model to be pruned according to the preset accuracy requirement and the preset time requirement that the neural network model after pruning needs to meet, including:

[0032] Get the preset number of iterations and the preset initial pruning ratio;

[0033] Determine the initial pruning ratio as the current pruning ratio of the prunable neural layer in each neural layer;

[0034] Prune each prunable neural layer according to the current pruning ratio to obtain a current pruned neural network model;

[0035] Determine whether the current number of iterations reaches the preset number of iterations;

[0036] If yes, the current pruning ratio is determined as the target pruning ratio, and the neural network model to be pruned is pruned according to the target pruning ratio;

[0037] If not, obtaining the target memory size occupied by the current pruned neural network model;

[0038] Adjust the current pruning ratio of each prunable neural layer to obtain a new current pruning ratio;

[0039] Prune each prunable neural layer according to the new current pruning ratio to obtain a new current pruned neural network model;

[0040] Obtaining the model accuracy and model time consumption of the current pruned neural network model;

[0041] When it is determined that the model accuracy meets the preset accuracy requirement and the model time meets the preset time requirement, calculating a probability value of receiving the new current pruning ratio;

[0042] The final current pruning ratio is determined from the new current pruning ratio and the current pruning ratio before adjustment according to the probability value, and the step of determining whether the current number of iterations reaches the preset number of iterations is repeated.

[0043] In a specific implementation of the present invention, calculating the probability value of receiving the new current pruning ratio includes:

[0044] Get the preset initial temperature value and the current number of iterations;

[0045] Calculate a temperature parameter according to the initial temperature value and the current number of iterations;

[0046] Obtain the preset model accuracy weight coefficient and model time consumption weight coefficient;

[0047] Calculate the energy value according to the model accuracy weight coefficient, the model time consumption weight coefficient, the model accuracy and the model time consumption;

[0048] A probability value of receiving the new current pruning ratio is calculated according to the energy value and the temperature parameter.

[0049] In a specific embodiment of the present invention, the energy value is calculated according to the model accuracy weight coefficient, the model time consumption weight coefficient, the model accuracy and the model time consumption, including:

[0050] The energy value is calculated according to the model accuracy weight coefficient, the model time consumption weight coefficient, the model accuracy and the model time consumption by the formula:

[0051] ;

[0052] ;

[0053] in, is the model accuracy weight coefficient, is the model time-consuming weight coefficient, is the model accuracy, The model takes time. is the intermediate objective function value, Is the energy value.

[0054] In a specific implementation of the present invention, pruning each prunable neural layer according to the current pruning ratio includes:

[0055] Obtain the norm of each sub-neural layer in each prunable neural layer respectively;

[0056] Sort the one-norms of each sub-neural layer contained in each prunable neural layer respectively to obtain one-norm sequences;

[0057] According to the current pruning ratio, each target one-norm is obtained by selecting from the end with the smaller one-norm in each one-norm sequence;

[0058] Prune the sub-neural layers corresponding to each target norm separately.

[0059] In a specific embodiment of the present invention, after testing the execution time corresponding to each neural layer in the to-be-pruned neural network model by each heterogeneous computing power in the heterogeneous computing cluster, the method further includes:

[0060] The execution time corresponding to each neural layer in the neural network model to be pruned is recorded for each heterogeneous computing power to obtain time record information.

[0061] In a specific embodiment of the present invention, adjusting the current pruning ratio of each prunable neural layer includes:

[0062] Find the execution time of each prunable neural layer by the target heterogeneous computing power from the time-consuming record information;

[0063] Sort the execution times found by size to get the execution time sequence;

[0064] Determine the first preset number of execution times of the end with the longest execution time in the execution time sequence as the first execution time;

[0065] Increase the current pruning ratio of the prunable neural layer corresponding to each first execution time by a first preset ratio;

[0066] Determine other execution times in the execution time sequence except the first execution times as second execution times;

[0067] Lowering the current pruning ratio of the prunable neural layer corresponding to each second execution time by a second preset ratio;

[0068] Among them, the memory size occupied by the model before the pruning ratio is adjusted is the same as the memory size occupied by the model after the pruning ratio is adjusted.

[0069] A model pruning device for a heterogeneous computing cluster, comprising:

[0070] Memory for storing computer programs;

[0071] A processor is used to implement the steps of the model pruning method for a heterogeneous computing cluster as described above when executing the computer program.

[0072] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the model pruning method for a heterogeneous computing cluster as described above.

[0073] A computer program product includes a computer program, which, when executed by a processor, implements the steps of the model pruning method for a heterogeneous computing cluster as described above.

[0074] The model pruning method of the heterogeneous computing cluster provided by the present invention obtains the neural network model to be pruned; tests the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster; calculates the total time corresponding to each heterogeneous computing power according to each execution time; obtains the memory size requirement for running the neural network model to be pruned; determines the heterogeneous computing power that meets the memory size requirement and has the smallest total time consumption as the target heterogeneous computing power; and uses the target heterogeneous computing power to prune the neural network model to be pruned according to the preset accuracy requirement and preset time consumption requirement that the neural network model after pruning needs to meet.

[0075] The beneficial effect of the present invention is that a model pruning framework for heterogeneous computing clusters is proposed, and the computing power of each heterogeneous computing power in the heterogeneous computing cluster is evaluated by testing the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster. According to the memory and computing power of each heterogeneous computing power, a suitable heterogeneous computing power can be selected for the neural network model to be pruned, and the model can be pruned. According to constraints such as model accuracy requirements and execution time requirements, the pruned model structure with the highest execution efficiency and accuracy is searched, which improves the execution efficiency and accuracy of the compressed model and alleviates the resource pressure of the neural network model during deployment.

[0076] Correspondingly, the present invention also provides a model pruning device for a heterogeneous computing cluster, a computer-readable storage medium, and a computer program product corresponding to the above-mentioned model pruning method for a heterogeneous computing cluster, which have the above-mentioned technical effects and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0078] FIG1 is a flowchart of an implementation of a model pruning method for a heterogeneous computing cluster in an embodiment of the present invention;

[0079] FIG2 is a block diagram of a heterogeneous computing system according to an embodiment of the present invention;

[0080] FIG3 is another implementation flow chart of the model pruning method for a heterogeneous computing cluster in an embodiment of the present invention;

[0081] FIG4 is an architecture diagram of a model pruning system for a heterogeneous computing cluster according to an embodiment of the present invention;

[0082] FIG5 is a structural block diagram of a model pruning device for a heterogeneous computing cluster according to an embodiment of the present invention;

[0083] FIG6 is a structural block diagram of a model pruning device of a heterogeneous computing cluster in an embodiment of the present invention;

[0084] FIG. 7 is a schematic diagram of the specific structure of a model pruning device for a heterogeneous computing cluster provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0085] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0086] Referring to FIG. 1 , FIG. 1 is a flowchart of an implementation of a model pruning method for a heterogeneous computing cluster in an embodiment of the present invention. The method may include the following steps:

[0087] S101: Obtain a neural network model to be pruned.

[0088] When deploying an inference model, since the inference model to be deployed is large, it needs to be pruned. Get the neural network model to be pruned.

[0089] S102: Testing the execution time of each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster.

[0090] See Figure 2, which is a structural block diagram of a heterogeneous computing system in an embodiment of the present invention. The heterogeneous computing system includes multiple heterogeneous computing nodes, such as various types of computing acceleration cards, field programmable gate arrays (FPGAs), etc., and also includes communication between them, such as intra-server communication and inter-server communication.

[0091] S103: Calculate the total time consumption corresponding to each heterogeneous computing power according to each execution time consumption.

[0092] After the execution time corresponding to each neural layer is obtained through testing, the total time corresponding to each heterogeneous computing power is calculated based on each execution time. For each heterogeneous computing power, the execution time of each neural layer of the heterogeneous computing power is summed up to obtain the total time corresponding to the heterogeneous computing power.

[0093] S104: Obtain the memory size requirement for running the neural network model to be pruned.

[0094] The memory size requirement for running the neural network model to be pruned is preset, and can be set based on the memory size occupied by the model parameters of the neural network model to be pruned and the amount of data generated during the model operation to obtain the memory size requirement for running the neural network model to be pruned.

[0095] S105: Determine the heterogeneous computing power that meets the memory size requirement and has the smallest total time consumption as the target heterogeneous computing power.

[0096] Obtain the available memory size corresponding to each heterogeneous computing power, and select the heterogeneous computing power that meets the memory size requirement from each heterogeneous computing power. Obtain the total time consumption corresponding to each heterogeneous computing power that meets the memory size requirement, and determine the heterogeneous computing power that meets the memory size requirement and has the smallest total time consumption as the target heterogeneous computing power, so as to select the appropriate heterogeneous computing power for the neural network model to be pruned.

[0097] S106: Prune the neural network model to be pruned according to the preset accuracy requirements and preset time requirements that the neural network model must meet after pruning by using the target heterogeneous computing power.

[0098] The preset accuracy requirements and preset time requirements that the pruned neural network model must meet are obtained. After the heterogeneous computing power that meets the memory size requirements and has the smallest total time consumption is determined as the target heterogeneous computing power, the target heterogeneous computing power is used to prune the neural network model to be pruned according to the preset accuracy requirements and preset time requirements. By pruning the model according to the preset accuracy requirements and preset time requirements that the pruned neural network model must meet, the execution efficiency and accuracy of the compressed model are improved.

[0099] From the above technical solution, it can be seen that a model pruning framework for heterogeneous computing clusters is proposed. By testing the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster, the computing power of each heterogeneous computing power in the heterogeneous computing cluster is evaluated. According to the memory and computing power of each heterogeneous computing power, the appropriate heterogeneous computing power can be selected for the neural network model to be pruned, and the model can be pruned. According to constraints such as model accuracy requirements and execution time requirements, the pruned model structure with the highest execution efficiency and accuracy is searched, which improves the execution efficiency and accuracy of the compressed model and alleviates the resource pressure of the neural network model during deployment.

[0100] It should be noted that, based on the above embodiment, the embodiment of the present invention also provides corresponding improved solutions. In the subsequent embodiments, the same steps or corresponding steps as those in the above embodiment can be referenced to each other, and the corresponding beneficial effects can also be referenced to each other, which will not be repeated one by one in the following improved embodiments.

[0101] Referring to FIG. 3 , FIG. 3 is another implementation flow chart of the model pruning method for a heterogeneous computing cluster in an embodiment of the present invention. The method may include the following steps:

[0102] S301: Obtain a neural network model to be pruned.

[0103] S302: Testing the execution time of each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster.

[0104] You can use commonly used AI benchmarks (AIbenchmarks) such as the Machine Learning Performance Benchmark (MLPerf) to test, and then collect and record performance. Ultimately, you can get the execution time of each heterogeneous computing power in the heterogeneous computing cluster for a single neural layer of different types and sizes, and thus get the computing power of each heterogeneous computing power for each neural layer.

[0105] S303: Sum up the execution times corresponding to the various heterogeneous computing powers to obtain the total execution times corresponding to the various heterogeneous computing powers.

[0106] After testing and obtaining the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster, the execution time corresponding to each heterogeneous computing power is summed up to obtain the total time corresponding to each heterogeneous computing power. In this way, the total time consumed by each heterogeneous computing power to execute each neural layer separately is obtained, which facilitates the selection of heterogeneous computing power based on the total time consumed by each heterogeneous computing power.

[0107] Given a neural network model to be pruned, its neural network layer And the corresponding quantity It can be expressed as According to the performance evaluation results of the neural network layer of heterogeneous computing power, a certain heterogeneous computing power j has a certain effect on the neural network layer. The execution time of , the heterogeneous computing power corresponding to the minimum value can be used for model deployment through the following calculation formula:

[0108] ;

[0109] If the currently available memory of the heterogeneous computing power corresponding to the minimum value does not meet the maximum memory requirement for deployment, the heterogeneous computing power corresponding to the next smallest value is used, and so on.

[0110] S304: Obtain the memory size requirement for running the neural network model to be pruned.

[0111] S305: Collect the available memory sizes corresponding to the heterogeneous computing powers.

[0112] After obtaining the memory size requirements for running the neural network model to be pruned, collect the available memory sizes corresponding to each heterogeneous computing power.

[0113] S306: Generate an available memory size list according to each available memory size.

[0114] After collecting the available memory sizes corresponding to each heterogeneous computing power, an available memory size list is generated according to each available memory size. By generating an available memory size list including the available memory sizes corresponding to each heterogeneous computing power, effective statistics of the available memory sizes of each heterogeneous computing power are achieved.

[0115] S307: Sort the total time consumptions by size to obtain a total time consumption sequence.

[0116] After obtaining the total time consumption corresponding to each heterogeneous computing power, the total time consumption is sorted by size, such as sorting from large to small, or sorting from small to large, to obtain a total time consumption sequence.

[0117] S308: Select the current minimum total time consumption from the end of the total time consumption sequence with a smaller total time consumption.

[0118] After sorting the total times to obtain the total time sequence, the current minimum total time is selected from the end of the total time sequence with the smallest total time. For example, when the total time sequence is a sequence obtained by sorting the total time from the smallest to the largest, the total time at the first position in the total time sequence is selected as the current minimum total time; when the total time sequence is a sequence obtained by sorting the total time from the largest to the smallest, the total time at the last position in the total time sequence is selected as the current minimum total time.

[0119] S309: Search the available memory of the heterogeneous computing power corresponding to the current minimum total time consumption from the available memory size list.

[0120] After selecting the current minimum total time consumption, the current available memory of the heterogeneous computing power corresponding to the current minimum total time consumption is searched from the available memory size list. For example, the available memory size list can store the identification information of each heterogeneous computing power and the total time consumption corresponding to each heterogeneous computing power. Thus, the current available memory of the heterogeneous computing power corresponding to the current minimum total time can be searched from the available memory size list through the identification information of the heterogeneous computing power corresponding to the current minimum total time.

[0121] S310: Determine whether the currently available memory meets the memory size requirement, if so, execute step S311, if not, execute step S312.

[0122] After finding the currently available memory of the heterogeneous computing power corresponding to the current minimum total time consumption, determine whether the current memory meets the memory size requirement. If so, it means that the heterogeneous computing power corresponding to the current minimum total time consumption is the most suitable heterogeneous computing power, and the heterogeneous computing power can be used for model pruning, and execute step S311. If not, it means that the heterogeneous computing power corresponding to the current minimum total time consumption does not meet the requirements, and further heterogeneous computing power search is required, and execute step S312.

[0123] S311: Determine the heterogeneous computing power corresponding to the current minimum total time consumption as the target heterogeneous computing power.

[0124] When it is determined that the current memory meets the memory size requirements, it means that the heterogeneous computing power corresponding to the current minimum total time consumption is the most suitable heterogeneous computing power. The heterogeneous computing power can be used for model pruning, and the heterogeneous computing power corresponding to the current minimum total time consumption can be determined as the target heterogeneous computing power.

[0125] S312: Determine the next total time consumption in the total time consumption sequence that is greater than the current minimum total time consumption as the new current minimum total time consumption, and return to execute step S309.

[0126] When it is determined that the current memory does not meet the memory size requirements, it means that the heterogeneous computing power corresponding to the current minimum total time consumption does not meet the requirements, and further heterogeneous computing power search is required. The next total time consumption in the total time consumption sequence that is larger than the current minimum total time consumption is determined as the new current minimum total time consumption, and the current available memory of the heterogeneous computing power corresponding to the current minimum total time consumption is searched from the available memory size list to further determine whether the available memory meets the requirements.

[0127] S313: Obtain a preset number of iterations and a preset initial pruning ratio.

[0128] The number of iterations and the initial pruning ratio are set in advance. After the target heterogeneous computing power is determined, the preset number of iterations and the preset initial pruning ratio are obtained.

[0129] It should be noted that the initial pruning ratio can be set and adjusted according to actual conditions, and the embodiment of the present invention does not limit this.

[0130] It should also be noted that the number of iterations can be set and adjusted according to actual conditions, and the embodiment of the present invention does not limit this.

[0131] S314: Determine the initial pruning ratio as the current pruning ratio of the prunable neural layer in each neural layer.

[0132] After the preset initial pruning ratio is obtained, the initial pruning ratio is determined as the current pruning ratio of the prunable neural layer in each neural layer.

[0133] S315: Prune each prunable neural layer according to the current pruning ratio to obtain the current pruned neural network model.

[0134] After determining the current pruning ratio, each prunable neural layer is pruned according to the current pruning ratio to obtain the current pruned neural network model.

[0135] S316: Determine whether the current number of iterations reaches the preset number of iterations, if so, execute step S317, if not, execute step S318.

[0136] The number of required iterations is pre-set. After obtaining the current pruned neural network model, it is determined whether the current number of iterations reaches the preset number of iterations. If so, it means that the preset upper limit of the number of iterations has been reached, and step S317 is executed. If not, it means that the preset upper limit of the number of iterations has not been reached, and step S318 is executed.

[0137] S317: Determine the current pruning ratio as the target pruning ratio, and prune the neural network model to be pruned according to the target pruning ratio.

[0138] When it is determined that the current number of iterations has reached the preset number of iterations, it means that the preset upper limit of the number of iterations has been reached, and the current pruning ratio is determined as the target pruning ratio. The pruned neural network model is pruned according to the target pruning ratio, thereby pruning the pruned neural network model according to the target pruning ratio obtained after multiple iterations, thereby improving the execution efficiency and accuracy of the compressed model.

[0139] In a specific embodiment of the present invention, pruning each prunable neural layer according to the current pruning ratio may include the following steps:

[0140] Step 1: Obtain the norm of each sub-neural layer in each prunable neural layer respectively;

[0141] Step 2: Sort the one-norms of each sub-neural layer contained in each prunable neural layer to obtain one-norm sequences;

[0142] Step 3: According to the current pruning ratio, select the target one-norm from the end with the smaller one-norm in each one-norm sequence;

[0143] Step 4: Prune the sub-neural layers corresponding to each target norm.

[0144] For the convenience of description, the above four steps can be combined for explanation.

[0145] When pruning each prunable neural layer, obtain the one-norm of each sub-neural layer in each prunable neural layer, sort the one-norms of each sub-neural layer contained in each prunable neural layer, obtain each one-norm sequence, select each target one-norm from the end with the smaller one-norm in each one-norm sequence according to the current pruning ratio, and prune the sub-neural layers corresponding to each target one-norm. In this way, the sub-neural layers with lower one-norms are pruned first, and the sub-neural layers with higher one-norms are retained, thereby improving the model accuracy of the pruned neural network model.

[0146] S318: Obtain the target memory size occupied by the current pruned neural network model.

[0147] When the current number of iterations does not reach the preset number of iterations, the target memory size occupied by the current pruned neural network model is obtained.

[0148] S319: Adjust the current pruning ratio of each prunable neural layer to obtain a new current pruning ratio.

[0149] The current pruning ratio of each prunable neural layer is adjusted to obtain a new current pruning ratio. For example, the current pruning ratio of some prunable neural layers may be increased, and the current pruning ratio of some prunable neural layers may be decreased.

[0150] S320: Prune each prunable neural layer according to the new current pruning ratio to obtain a new current pruned neural network model.

[0151] After adjusting to obtain a new current pruning ratio, each prunable neural layer is pruned according to the new current pruning ratio to obtain a new current pruned neural network model.

[0152] S321: Obtain the model accuracy and model time consumption of the current pruned neural network model.

[0153] After obtaining the new current pruned neural network model, the model accuracy and model time of the current pruned neural network model are obtained. The process of obtaining the model accuracy and model time of the current pruned neural network model may include training with the original training set, obtaining the model accuracy when the model converges, running the converged current pruned neural network model, and obtaining the model time.

[0154] S322: When it is determined that the model accuracy meets the preset accuracy requirement and the model time meets the preset time requirement, a probability value of receiving a new current pruning ratio is calculated.

[0155] After obtaining the model accuracy and model time consumption of the current pruned neural network model, determine whether the model accuracy and model time consumption meet their corresponding set requirements. When it is determined that the model accuracy meets the preset accuracy requirements and the model time consumption meets the preset time consumption requirements, calculate the probability value of receiving the new current pruning ratio.

[0156] In a specific implementation of the present invention, calculating the probability value of receiving a new current pruning ratio may include the following steps:

[0157] Step 1: Get the preset initial temperature value and the current number of iterations;

[0158] Step 2: Calculate the temperature parameter based on the initial temperature value and the current number of iterations;

[0159] Step 3: Obtain the preset model accuracy weight coefficient and model time consumption weight coefficient;

[0160] Step 4: Calculate the energy value according to the model accuracy weight coefficient, model time consumption weight coefficient, model accuracy and model time consumption;

[0161] Step 5: Calculate the probability value of receiving the new current pruning ratio based on the energy value and temperature parameters.

[0162] For the convenience of description, the above two steps can be combined for explanation.

[0163] When it is determined that the model accuracy meets the preset accuracy requirements and the model time meets the preset time requirements, the preset initial temperature value and the current number of iterations are obtained, the temperature parameters are calculated based on the initial temperature value and the current number of iterations, the preset model accuracy weight coefficient and the model time weight coefficient are obtained, the energy value is calculated based on the model accuracy weight coefficient, the model time weight coefficient, the model accuracy and the model time, and the probability value of receiving the new current pruning ratio is calculated based on the energy value and the temperature parameter. By setting the corresponding model accuracy weight coefficient and model time weight coefficient according to the accuracy requirements and time requirements in advance, the accuracy and execution time of the pruned neural network model are weighed according to the set search direction, so that the trained pruned neural network model is more in line with the actual needs of users and improves the user experience.

[0164] In a specific embodiment of the present invention, calculating the energy value according to the model accuracy weight coefficient, the model time consumption weight coefficient, the model accuracy and the model time consumption may include the following steps:

[0165] The energy value is calculated using the formula based on the model accuracy weight coefficient, model time weight coefficient, model accuracy, and model time consumption:

[0166] ;

[0167] ;

[0168] in, is the model accuracy weight coefficient, is the model time-consuming weight coefficient, and are all non-negative numbers, is the model accuracy, The model takes time. is the intermediate objective function value, is the energy value, is the solution consisting of the current pruning ratio.

[0169] The initial temperature is given by , the current number of iterations is t, such as the initial temperature can be set to 1000, the number of iteration terminations can also be set to 1000, the initial temperature and the number of iteration terminations can be set according to actual conditions, and the embodiment of the present invention does not limit this.

[0170] The temperature parameter can be calculated according to the initial temperature value and the current number of iterations by formula :

[0171] ;

[0172] The probability value of receiving a new current pruning ratio can be calculated based on the energy value and temperature parameters by the formula:

[0173] ;

[0174] in, is the old solution consisting of the original current pruning ratios, is a new solution consisting of the new current pruning ratios.

[0175] By determining the final current pruning ratio from the new current pruning ratio and the current pruning ratio before adjustment according to the above probability value, a balance between global search and local search for the pruning ratio is achieved, thereby improving robustness.

[0176] By calculating the above intermediate objective function value, it is possible to balance the accuracy and execution time of the pruned neural network model according to the set search direction.

[0177] S323: Determine the final current pruning ratio from the new current pruning ratio and the current pruning ratio before adjustment according to the probability value, and return to execute step S316.

[0178] After calculating the probability value of receiving the new current pruning ratio, the final current pruning ratio is determined from the new current pruning ratio and the current pruning ratio before adjustment according to the probability value, and it is judged again whether the current number of iterations reaches the preset number of iterations to proceed to the next iteration or determine the target pruning ratio.

[0179] S324: Retrain the pruned neural network model obtained by pruning to obtain a trained neural network model.

[0180] After the to-be-pruned neural network model is pruned according to the target pruning ratio, the pruned neural network model obtained by the pruning is retrained to obtain a trained neural network model.

[0181] S325: Send the trained neural network model to the heterogeneous computing cluster to deploy the trained neural network model using the heterogeneous computing cluster.

[0182] After obtaining the trained neural network model, the trained neural network model is sent to the heterogeneous computing cluster to deploy the trained neural network model using the heterogeneous computing cluster. This allows the model to be pruned according to the pruned model structure that has the highest execution efficiency and accuracy, and then deployed, improving the execution efficiency and accuracy of the compressed model and alleviating the resource pressure on the neural network model during deployment.

[0183] In a specific embodiment of the present invention, after testing the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster, the method may further include the following steps:

[0184] The execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power is recorded to obtain time-consuming record information.

[0185] After testing the execution time of each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster, the execution time of each neural layer in the neural network model to be pruned by each heterogeneous computing power is recorded to obtain the time record information. By recording the execution time corresponding to each neural layer of each heterogeneous computing power, it is convenient to perform heterogeneous computing power analysis based on the detailed execution time records, and improve the accuracy of searching for the target heterogeneous computing power.

[0186] In a specific embodiment of the present invention, adjusting the current pruning ratio of each prunable neural layer may include the following steps:

[0187] Step 1: Find the execution time of each prunable neural layer by the target heterogeneous computing power from the time-consuming record information;

[0188] Step 2: Sort the execution times found to obtain the execution time sequence;

[0189] Step 3: determining the first preset number of execution times at the end of the execution time sequence with the largest execution time as the first execution time;

[0190] Step 4: increasing the current pruning ratio of the prunable neural layer corresponding to each first execution time by a first preset ratio;

[0191] Step 5: Determine the other execution times in the execution time sequence except the first execution times as the second execution times;

[0192] Step 6: lowering the current pruning ratio of the prunable neural layer corresponding to each second execution time by a second preset ratio;

[0193] Among them, the memory size occupied by the model before the pruning ratio is adjusted is the same as the memory size occupied by the model after the pruning ratio is adjusted.

[0194] For the convenience of description, the above six steps can be combined for explanation.

[0195] The execution time of each prunable neural layer by the target heterogeneous computing power is searched from the time record information, and the execution times found are sorted by size to obtain an execution time sequence, and the first preset number of execution times at the end of the execution time sequence with the largest time consumption is determined as the first execution time, and the current pruning ratio of the prunable neural layer corresponding to each first execution time is increased by a first preset ratio, and the other execution times except the first execution time in the execution time sequence are determined as the second execution time, and the current pruning ratio of the prunable neural layer corresponding to each second execution time is decreased by a second preset ratio, and it is ensured that the memory size occupied by the model before the pruning ratio is adjusted is the same as the memory size occupied by the model after the pruning ratio is adjusted. By adjusting the current pruning ratio of each prunable neural layer under the premise of ensuring that the memory size occupied by the model before the pruning ratio is adjusted is the same as the memory size occupied by the model after the pruning ratio is adjusted, the memory occupied by the final pruned neural network model is ensured to remain unchanged, avoiding the impact on subsequent model deployment, and realizing the iterative update of the pruning ratio of each prunable neural layer, thereby improving the execution efficiency and accuracy of the compressed model.

[0196] Search for new solutions to pruning, Middle nerve layer The pruning ratio of the neural layer is increased by X. The pruning ratio decreases by Y. In order to optimize the computational performance of the pruned model, the algorithm will select the neural layer with higher computational time as the , with a higher probability of selecting the neural layer with lower computational time as , and the change of pruning ratio must ensure that the total memory occupied by the model remains unchanged, that is, to ensure ,in Represents all the The total memory occupied by the neural layer before adjustment, Represents all the The total memory occupied by the neural layer before adjustment can be calculated based on The total number of parameters is estimated. The pruning scheme of the new solution is X and Y can be set by yourself, for example, if X is 10%, Y is calculated according to the above formula.

[0197] In a specific embodiment of the present invention, after testing the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster, the method may further include the following steps:

[0198] When it is detected that there is a software update and / or hardware update in the heterogeneous computing cluster, the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster is retested.

[0199] The software and hardware update status can be detected according to a preset time period. After testing the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster, when it is detected that there is only software update, or only hardware update, or both software update and hardware update in the heterogeneous computing cluster, the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster is retested. By retesting the computing power of each heterogeneous example when there is a software and hardware update, the accuracy of the computing power of each heterogeneous computing power obtained by the test is guaranteed.

[0200] See Figure 4, which is an architecture diagram of a model pruning system for a heterogeneous computing cluster in an embodiment of the present invention. It includes 4 modules, and their names and functions are as follows:

[0201] (1) Computing power evaluation module: performs necessary evaluation on the computing power in the heterogeneous computing system and records the computing performance of heterogeneous computing power for different neural layers. It can also be used to collect the real-time memory usage of heterogeneous computing power in the heterogeneous computing system.

[0202] (2) Computing power selection module: Once the artificial intelligence (AI) inference model to be deployed is issued, this module will retrieve the neural layer performance and the real-time memory usage of heterogeneous computing power from the computing power evaluation module, and then select the most suitable heterogeneous computing power for deployment from the heterogeneous computing system, and send the device information to the model pruning search module.

[0203] (3) Model pruning search module: Prune and search the AI ​​inference model to be deployed under multiple constraints to obtain the pruned and compressed AI model.

[0204] (4) Model deployment module: actually deploy the pruned and compressed model.

[0205] The present invention does not limit the application scenarios involved in the pruned neural network model, which can be used to identify data such as images, texts, and audio. For example, when the data to be identified is image data, the current neural network model to be pruned is an image recognition neural network model, which contains multiple neural layers, and each heterogeneous computing power has different execution times for each neural layer in the image recognition neural network model. The image recognition neural network model to be pruned is obtained, and the execution time corresponding to each neural layer in the image recognition neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster is tested. The total time corresponding to each heterogeneous computing power is calculated according to each execution time. The memory size requirement for running the neural network model to be pruned is obtained, and the heterogeneous computing power that meets the memory size requirement and has the smallest total time consumption is determined as the target heterogeneous computing power. The image recognition neural network model to be pruned is pruned using the target heterogeneous computing power according to the preset accuracy requirements and preset time requirements that the pruned image recognition neural network model needs to meet, and then the pruned image recognition neural network model is deployed to the heterogeneous computing cluster, and the deployed image recognition neural network model is used to perform image recognition. By analogy, when the contemporary recognized data is text, audio or other data, it changes accordingly.

[0206] Corresponding to the above method embodiments, the present invention further provides a model pruning device for a heterogeneous computing cluster. The model pruning device for a heterogeneous computing cluster described below and the model pruning method for a heterogeneous computing cluster described above can refer to each other.

[0207] Referring to FIG. 5 , FIG. 5 is a structural block diagram of a model pruning device for a heterogeneous computing cluster according to an embodiment of the present invention. The device may include:

[0208] A to-be-pruned model acquisition module 51 is used to acquire a to-be-pruned neural network model;

[0209] An execution time test module 52 is used to test the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster;

[0210] A total time consumption calculation module 53 is used to calculate the total time consumption corresponding to each heterogeneous computing power according to each execution time consumption;

[0211] A memory requirement acquisition module 54 is used to acquire the memory size requirement for running the neural network model to be pruned;

[0212] A target computing power determination module 55 is used to determine the heterogeneous computing power that meets the memory size requirement and has the minimum total time consumption as the target heterogeneous computing power;

[0213] The model pruning module 56 is used to prune the neural network model to be pruned according to the preset accuracy requirements and preset time requirements that the neural network model after pruning needs to meet by using the target heterogeneous computing power.

[0214] From the above technical solution, it can be seen that a model pruning framework for heterogeneous computing clusters is proposed. By testing the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster, the computing power of each heterogeneous computing power in the heterogeneous computing cluster is evaluated. According to the memory and computing power of each heterogeneous computing power, the appropriate heterogeneous computing power can be selected for the neural network model to be pruned, and the model can be pruned. According to constraints such as model accuracy requirements and execution time requirements, the pruned model structure with the highest execution efficiency and accuracy is searched, which improves the execution efficiency and accuracy of the compressed model and alleviates the resource pressure of the neural network model during deployment.

[0215] In a specific embodiment of the present invention, the device may further include:

[0216] A retraining module is used to prune the neural network model to be pruned by using the target heterogeneous computing power according to the preset accuracy requirements and preset time requirements that the pruned neural network model needs to meet, and then retrain the pruned neural network model obtained by pruning to obtain a trained neural network model;

[0217] The model deployment module is used to send the trained neural network model to a heterogeneous computing cluster so as to deploy the trained neural network model using the heterogeneous computing cluster.

[0218] In a specific implementation of the present invention, the total time consumption calculation module 53 is specifically used to sum up the execution times corresponding to the heterogeneous computing powers to obtain the total times corresponding to the heterogeneous computing powers.

[0219] In a specific embodiment of the present invention, the device may further include:

[0220] The retest module is used to retest the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster after testing the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster, when it is detected that there is a software update and / or hardware update in the heterogeneous computing cluster.

[0221] In a specific embodiment of the present invention, the device may further include:

[0222] The available memory size collection module is used to collect the available memory sizes corresponding to each heterogeneous computing power before determining the heterogeneous computing power that meets the memory size requirements and has the smallest total time consumption as the target heterogeneous computing power;

[0223] The available memory size list generating module is used to generate an available memory size list according to each available memory size.

[0224] In a specific implementation of the present invention, the target computing power determination module 55 includes:

[0225] The total time consumption sequence obtaining submodule is used to sort the total time consumptions and obtain the total time consumption sequence;

[0226] The current minimum total time selection submodule is used to select the current minimum total time from the end of the total time sequence with the smallest total time;

[0227] The current available memory acquisition submodule is used to find the current available memory of the heterogeneous computing power corresponding to the current minimum total time consumption from the available memory size list;

[0228] The second judgment submodule is used to judge whether the current available memory meets the memory size requirement;

[0229] The target heterogeneous computing power determination submodule is used to determine the heterogeneous computing power corresponding to the current minimum total time consumption as the target heterogeneous computing power when it is determined that the current memory meets the memory size requirement;

[0230] The second repeated execution sub-module is used to determine the next total time consumption in the total time consumption sequence that is larger than the current minimum total time consumption as the new current minimum total time consumption when it is determined that the current memory does not meet the memory size requirement, and repeatedly execute the step of searching the current available memory of the heterogeneous computing power corresponding to the current minimum total time consumption from the available memory size list.

[0231] In a specific embodiment of the present invention, the model pruning module 56 includes:

[0232] An initial pruning ratio acquisition submodule is used to obtain a preset number of iterations and a preset initial pruning ratio;

[0233] A current pruning ratio determination submodule, used for determining the initial pruning ratio as the current pruning ratio of the prunable neural layer in each neural layer;

[0234] The first current pruned neural network model obtaining submodule prunes each prunable neural layer according to the current pruning ratio to obtain the current pruned neural network model;

[0235] The first judgment submodule is used to judge whether the current number of iterations reaches the preset number of iterations;

[0236] The model pruning submodule is used to determine the current pruning ratio as the target pruning ratio when it is determined that the current number of iterations reaches the preset number of iterations, and prune the neural network model to be pruned according to the target pruning ratio;

[0237] The occupied memory size acquisition submodule is used to obtain the target memory size occupied by the current pruned neural network model when it is determined that the current number of iterations has not reached the preset number of iterations;

[0238] A pruning ratio adjustment submodule is used to adjust the current pruning ratio of each prunable neural layer to obtain a new current pruning ratio;

[0239] A second current pruned neural network model obtaining submodule is used to prune each prunable neural layer according to a new current pruning ratio to obtain a new current pruned neural network model;

[0240] The model accuracy and time consumption acquisition submodule is used to obtain the model accuracy and model time consumption of the current pruned neural network model;

[0241] A probability value calculation submodule, used to calculate the probability value of receiving a new current pruning ratio when it is determined that the model accuracy meets the preset accuracy requirement and the model time meets the preset time requirement;

[0242] The first repetitive execution submodule is used to determine the final current pruning ratio from the new current pruning ratio and the current pruning ratio before adjustment according to the probability value, and repeatedly execute the step of determining whether the current number of iterations reaches a preset number of iterations.

[0243] In a specific embodiment of the present invention, the probability value calculation submodule includes:

[0244] A current iteration number acquisition unit is used to acquire a preset initial temperature value and a current iteration number;

[0245] A temperature parameter calculation unit, used for calculating the temperature parameter according to the initial temperature value and the current iteration number;

[0246] A weight coefficient acquisition unit, used to acquire a preset model accuracy weight coefficient and a model time consumption weight coefficient;

[0247] An energy value calculation unit, used to calculate the energy value according to the model accuracy weight coefficient, the model time consumption weight coefficient, the model accuracy and the model time consumption;

[0248] The probability value calculation unit is used to calculate the probability value of receiving a new current pruning ratio according to the energy value and the temperature parameter.

[0249] In a specific embodiment of the present invention, the energy value calculation unit is specifically used to calculate the energy value according to the model accuracy weight coefficient, the model time consumption weight coefficient, the model accuracy and the model time consumption through the formula:

[0250] ;

[0251] ;

[0252] in, is the model accuracy weight coefficient, is the model time-consuming weight coefficient, is the model accuracy, The model takes time. is the intermediate objective function value, Is the energy value.

[0253] In a specific embodiment of the present invention, the first current pruned neural network model obtaining submodule includes:

[0254] A norm acquisition unit, used for respectively acquiring a norm of each sub-neural layer in each prunable neural layer;

[0255] A norm sequence obtaining unit is used to sort the norms of each sub-neural layer contained in each prunable neural layer to obtain each norm sequence;

[0256] A norm selection unit, used for selecting each target norm from the end with a smaller norm in each norm sequence according to the current pruning ratio;

[0257] The sub-neural layer pruning unit is used to prune the sub-neural layers corresponding to each target norm.

[0258] In a specific embodiment of the present invention, the device may further include:

[0259] The time consumption recording module is used to record the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the test heterogeneous computing cluster, and obtain the time consumption record information.

[0260] In a specific embodiment of the present invention, the pruning ratio adjustment submodule includes:

[0261] An execution time-consuming search unit is used to search the execution time of each prunable neural layer by the target heterogeneous computing power from the time-consuming record information;

[0262] An execution time sequence obtaining unit is used to sort the execution times found to obtain an execution time sequence;

[0263] A first execution time determination unit, configured to determine a preset number of execution times at one end of the execution time sequence with the largest execution time as a first execution time;

[0264] A pruning ratio increasing unit, used for increasing the current pruning ratio of the prunable neural layer corresponding to each first execution time by a first preset ratio;

[0265] A second execution time determination unit, configured to determine other execution times in the execution time sequence except the first execution times as second execution times;

[0266] A pruning ratio lowering unit, used to lower a current pruning ratio of the prunable neural layer corresponding to each second execution time by a second preset ratio;

[0267] Among them, the memory size occupied by the model before the pruning ratio is adjusted is the same as the memory size occupied by the model after the pruning ratio is adjusted.

[0268] Corresponding to the above method embodiment, referring to FIG6 , FIG6 is a schematic diagram of a model pruning device for a heterogeneous computing cluster provided by the present invention, and the device may include:

[0269] A memory 332, for storing computer programs;

[0270] The processor 322 is used to implement the steps of the model pruning method for a heterogeneous computing cluster of the above method embodiment when executing a computer program.

[0271] Specifically, please refer to FIG. 7, which is a schematic diagram of the specific structure of a model pruning device for a heterogeneous computing cluster provided in this embodiment. The model pruning device for the heterogeneous computing cluster may have relatively large differences due to different configurations or performances, and may include a processor (central processing units, CPU) 322 (for example, one or more processors) and a memory 332, and the memory 332 stores one or more computer programs 342 or data 344. Among them, the memory 332 can be a short-term storage or a persistent storage. The program stored in the memory 332 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the data processing device. Furthermore, the processor 322 may be configured to communicate with the memory 332 to execute a series of instruction operations in the memory 332 on the model pruning device 301 of the heterogeneous computing cluster.

[0272] The model pruning device 301 of the heterogeneous computing cluster may further include one or more power supplies 326 , one or more wired or wireless network interfaces 350 , one or more input and output interfaces 358 , and / or one or more operating systems 341 .

[0273] The steps in the model pruning method for a heterogeneous computing cluster described above can be implemented by the structure of a model pruning device for a heterogeneous computing cluster.

[0274] Corresponding to the above method embodiment, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps can be implemented:

[0275] Obtain the neural network model to be pruned; test the execution time corresponding to each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster; calculate the total time corresponding to each heterogeneous computing power according to each execution time; obtain the memory size requirement for running the neural network model to be pruned; determine the heterogeneous computing power that meets the memory size requirement and has the smallest total time consumption as the target heterogeneous computing power; use the target heterogeneous computing power to prune the neural network model to be pruned according to the preset accuracy requirements and preset time requirements that the neural network model must meet after pruning.

[0276] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0277] For an introduction to the computer-readable storage medium provided by the present invention, please refer to the above method embodiment, and the present invention will not be elaborated here.

[0278] Corresponding to the above method embodiments, the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the model pruning method for a heterogeneous computing cluster as described above.

[0279] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device, computer-readable storage medium, and computer program product disclosed in the embodiment, since they correspond to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0280] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the technical solution and core ideas of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A model pruning method for a heterogeneous computing cluster, characterized in that: include: Obtain the neural network model to be pruned; Test the execution time of each neural layer in the neural network model to be pruned by each heterogeneous computing power in the heterogeneous computing cluster; Calculate the total time consumption corresponding to each heterogeneous computing power based on the execution time consumption; Obtaining the memory size requirement for running the neural network model to be pruned; Determine the heterogeneous computing power that meets the memory size requirement and has the smallest total time consumption as the target heterogeneous computing power; Using the target heterogeneous computing power, the neural network model to be pruned is pruned according to the preset accuracy requirement and the preset time requirement that the neural network model after pruning needs to meet; The target heterogeneous computing power is used to prune the neural network model to be pruned according to the preset accuracy requirement and the preset time requirement that the neural network model after pruning needs to meet, including: Get the preset number of iterations and the preset initial pruning ratio; Determining the initial pruning ratio as the current pruning ratio of the prunable neural layer in each neural layer; Prune each prunable neural layer according to the current pruning ratio to obtain a current pruned neural network model; Determine whether the current number of iterations reaches the preset number of iterations; If yes, the current pruning ratio is determined as the target pruning ratio, and the neural network model to be pruned is pruned according to the target pruning ratio; If not, obtaining the target memory size occupied by the current pruned neural network model; Adjust the current pruning ratio of each prunable neural layer to obtain a new current pruning ratio; Prune each prunable neural layer according to the new current pruning ratio to obtain a new current pruned neural network model; Obtaining the model accuracy and model time consumption of the current pruned neural network model; When it is determined that the model accuracy meets the preset accuracy requirement and the model time meets the preset time requirement, calculating a probability value of receiving the new current pruning ratio; The final current pruning ratio is determined from the new current pruning ratio and the current pruning ratio before adjustment according to the probability value, and the step of determining whether the current number of iterations reaches the preset number of iterations is repeated.

2. The model pruning method for a heterogeneous computing cluster according to claim 1, characterized in that: After pruning the neural network model to be pruned by using the target heterogeneous computing power according to the preset accuracy requirement and the preset time requirement that the neural network model after pruning needs to meet, the method further includes: Retraining the pruned neural network model obtained by pruning to obtain a trained neural network model; The trained neural network model is sent to the heterogeneous computing cluster to deploy the trained neural network model using the heterogeneous computing cluster.

3. The model pruning method for a heterogeneous computing cluster according to claim 1, characterized in that: The total time consumption corresponding to each heterogeneous computing power is calculated based on the execution time consumption of each execution, including: The execution times corresponding to each heterogeneous computing power are summed up to obtain the total execution times corresponding to each heterogeneous computing power.

4. The model pruning method for a heterogeneous computing cluster according to claim 1, characterized in that: After testing the execution time of each neural layer in the to-be-pruned neural network model by each heterogeneous computing power in the heterogeneous computing cluster, the method further includes: When it is detected that there is a software update and / or a hardware update in the heterogeneous computing cluster, the execution time corresponding to each neural layer in the neural network model to be pruned is retested by each heterogeneous computing power in the heterogeneous computing cluster.

5. The model pruning method for a heterogeneous computing cluster according to claim 1, characterized in that: Before determining the heterogeneous computing power that meets the memory size requirement and has the smallest total time consumption as the target heterogeneous computing power, the method further includes: Collect the available memory size corresponding to each heterogeneous computing power; Generates a list of available memory sizes based on the available memory sizes.

6. The model pruning method for a heterogeneous computing cluster according to claim 5, characterized in that: The heterogeneous computing power that meets the memory size requirement and has the smallest total time consumption is determined as the target heterogeneous computing power, including: Sort the total time consumption by size to obtain the total time consumption sequence; Select the current minimum total time consumption from the end with the smaller total time consumption in the total time consumption sequence; Searching the available memory size list for the currently available memory of the heterogeneous computing power corresponding to the currently minimum total time consumption; Determining whether the currently available memory meets the memory size requirement; If yes, the heterogeneous computing power corresponding to the current minimum total time consumption is determined as the target heterogeneous computing power; If not, the next total time consumption in the total time consumption sequence that is greater than the current minimum total time consumption is determined as the new current minimum total time consumption, and the step of searching the current available memory of the heterogeneous computing power corresponding to the current minimum total time consumption from the available memory size list is repeated.

7. The model pruning method for a heterogeneous computing cluster according to claim 1, characterized in that: Calculating the probability value of receiving the new current pruning ratio includes: Get the preset initial temperature value and the current number of iterations; Calculate a temperature parameter according to the initial temperature value and the current number of iterations; Obtain the preset model accuracy weight coefficient and model time consumption weight coefficient; Calculate the energy value according to the model accuracy weight coefficient, the model time consumption weight coefficient, the model accuracy and the model time consumption; A probability value of receiving the new current pruning ratio is calculated according to the energy value and the temperature parameter.

8. The model pruning method for a heterogeneous computing cluster according to claim 7, characterized in that: Calculating the energy value according to the model accuracy weight coefficient, the model time consumption weight coefficient, the model accuracy and the model time consumption includes: The energy value is calculated according to the model accuracy weight coefficient, the model time consumption weight coefficient, the model accuracy and the model time consumption by the formula: ； ； in, is the model accuracy weight coefficient, is the model time-consuming weight coefficient, is the model accuracy, The model takes time. is the intermediate objective function value, Is the energy value.

9. The model pruning method for a heterogeneous computing cluster according to claim 1, characterized in that: Pruning each prunable neural layer according to the current pruning ratio includes: Obtain the norm of each sub-neural layer in each prunable neural layer respectively; Sort the one-norms of each sub-neural layer contained in each prunable neural layer respectively to obtain one-norm sequences; According to the current pruning ratio, each target one-norm is obtained by selecting from the end with the smaller one-norm in each one-norm sequence; Prune the sub-neural layers corresponding to each target norm separately.

10. The model pruning method for a heterogeneous computing cluster according to claim 1, characterized in that: After testing the execution time of each neural layer in the to-be-pruned neural network model by each heterogeneous computing power in the heterogeneous computing cluster, the method further includes: The execution time corresponding to each neural layer in the neural network model to be pruned is recorded for each heterogeneous computing power to obtain time record information.

11. The model pruning method for a heterogeneous computing cluster according to claim 10, characterized in that: Adjust the current pruning ratio of each prunable neural layer, including: Find the execution time of each prunable neural layer by the target heterogeneous computing power from the time-consuming record information; Sort the execution times found by size to get the execution time sequence; Determine the first preset number of execution times of the end with the longest execution time in the execution time sequence as the first execution time; Increase the current pruning ratio of the prunable neural layer corresponding to each first execution time by a first preset ratio; Determine other execution times in the execution time sequence except the first execution times as second execution times; Lowering the current pruning ratio of the prunable neural layer corresponding to each second execution time by a second preset ratio; Among them, the memory size occupied by the model before the pruning ratio is adjusted is the same as the memory size occupied by the model after the pruning ratio is adjusted.

12. A model pruning device for a heterogeneous computing cluster, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the model pruning method for a heterogeneous computing cluster as claimed in any one of claims 1 to 11 when executing the computer program.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the model pruning method for a heterogeneous computing cluster as claimed in any one of claims 1 to 11.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the model pruning method for a heterogeneous computing cluster as described in any one of claims 1 to 11 are implemented.

Images