Image recognition method, device, equipment, medium and pruning method

Abstract

The present invention relates to the field of data processing technology, and discloses an image recognition method, device, equipment, medium and pruning method. According to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations in the training process of the original image recognition model, the parameter average values ​​of different neuron node combinations are determined. The parameter average values ​​of different neuron node combinations are classified and counted to determine the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations. Based on the initial model parameters of the original image recognition model and the parameter average values ​​corresponding to the retained neuron nodes, the model parameters of different pruning models are initialized. Image samples are analyzed and trained using different pruning models to obtain an image recognition model after pruning optimization. The image to be analyzed is input into the image recognition model to determine the target object contained in the image. The calculation and storage costs of the image recognition model are reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an image recognition method, device, equipment, medium and pruning method. Background Art

[0002] As the computational complexity of deep learning models increases, the computational and storage costs of the models gradually increase. How to reduce the computational and storage costs of deep learning models while ensuring their accuracy has become an important goal in current model optimization. Model pruning technology is an effective way to solve this problem. Model pruning optimizes the model structure and reduces parameters to remove unimportant or redundant parameters, thereby reducing the computational and storage costs of the model while ensuring the accuracy of the model.

[0003] However, current model pruning solutions are mostly aimed at optimizing pruning methods and pruning evaluation, but lack optimization of the overall model pruning process. For the trained original model, current solutions mostly directly discard the pruned model parameters and relearn the model parameters during the retraining of the pruned model. This not only ignores the impact of the pruned parameters on the pruned model, but also reduces the efficiency of retraining the pruned model. When using the pruned model for image recognition, the low training efficiency of the entire training process of the image recognition model leads to an increase in the computational and storage costs of the image recognition model.

[0004] It can be seen that how to reduce the computational and storage costs of image recognition models is a problem that technical personnel in this field need to solve. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide an image recognition method, apparatus, device, medium and pruning method, which can solve the problem of increased calculation and storage costs of image recognition models.

[0006] In order to solve the above technical problems, an embodiment of the present invention provides an image recognition model image recognition method, comprising:

[0007] According to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations in the original image recognition model training process, the parameter average values ​​of different neuron node combinations are determined; wherein the different neuron node combinations include the combination of neuron nodes discarded and neuron nodes retained in each training iteration of the model;

[0008] Classify and count the parameter averages of different neuron node combinations to determine the parameter averages corresponding to the neuron nodes retained under different pruning operations;

[0009] Based on the initial model parameters of the original image recognition model and the average values ​​of the parameters corresponding to the neuron nodes retained under different pruning operations, the model parameters of different pruning models are initialized;

[0010] The image samples are analyzed and trained using different pruning models after the model parameters are initialized to obtain the pruned and optimized image recognition model;

[0011] When the image to be analyzed is acquired, the image to be analyzed is input into the image recognition model to determine the target object contained in the image to be analyzed.

[0012] On the one hand, according to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations during the training process of the original image recognition model, the average values ​​of the parameters of different neuron node combinations are determined, including:

[0013] In each training iteration of the original image recognition model, neuron nodes are randomly discarded according to a set ratio to record parameter update values ​​corresponding to different neuron node combinations;

[0014] In each training iteration, when a neuron node combination that is the same as an existing neuron node combination appears, the number of the same neuron node combination is increased by one; wherein the number of different neuron node combinations in the initial state is zero;

[0015] The parameter update value and its number corresponding to each neuron node combination are divided, and the obtained quotient is used as the parameter average value of each neuron node combination.

[0016] On the one hand, randomly discarding neuron nodes according to a set ratio at each training iteration of the original image recognition model to record parameter update values ​​corresponding to different neuron node combinations includes:

[0017] In each training iteration of the original image recognition model, neuron nodes are randomly discarded according to a set ratio, and parameter update values ​​corresponding to the neuron nodes retained in each training iteration are counted;

[0018] The parameter update values ​​corresponding to the neuron nodes retained in each training iteration are accumulated according to the neuron node combination to which they belong, so as to obtain the parameter update value corresponding to each neuron node combination.

[0019] On the one hand, randomly discarding neuron nodes according to a set ratio at each training iteration of the original image recognition model, and counting the parameter update values ​​corresponding to the neuron nodes retained in each training iteration include:

[0020] The parameter update values ​​corresponding to the neuron nodes retained in the previous training iteration and the initial parameter values ​​corresponding to the discarded neuron nodes are used as the model parameters of the current training iteration; wherein the parameter update values ​​corresponding to the neuron nodes retained in the initial state are the initial parameter values ​​of the neuron nodes;

[0021] In the current training iteration, neuron nodes are randomly discarded according to the set ratio, and the image recognition model after randomly discarding neuron nodes is trained using the samples of the current batch, and the parameter update values ​​corresponding to the neuron nodes retained in the current batch are recorded;

[0022] When all batches of samples are trained, the accumulated values ​​of the parameter update values ​​corresponding to the neuron nodes retained in all batches are used as the parameter update values ​​corresponding to the neuron nodes retained in the current training iteration.

[0023] On the one hand, the parameter update values ​​corresponding to the neuron nodes retained in each training iteration are accumulated according to the neuron node combination to which they belong, so as to obtain the parameter update values ​​corresponding to each neuron node combination, including:

[0024] When the neuron node combination in the current training iteration is to discard the i-th neuron node and retain the j-th neuron node, obtain the parameter update value of the j-th neuron node in the current training iteration;

[0025] The neuron node combination under the previous training iteration is to discard the i-th neuron node and retain the parameter update value corresponding to the j-th neuron node, and the parameter update value of the j-th neuron node under the current training iteration is accumulated, and the accumulated value is used as the parameter update value of the neuron node combination under the current training iteration, which is to discard the i-th neuron node and retain the j-th neuron node.

[0026] On the one hand, the average parameter values ​​of different neuron node combinations are classified and counted to determine the average parameter values ​​corresponding to the neuron nodes retained under different pruning operations, including:

[0027] Querying at least one neuron node combination matching the retained neuron node from different neuron node combinations;

[0028] An averaging operation is performed on the parameter average values ​​of at least one matched neuron node combination to obtain the parameter average values ​​corresponding to the retained neuron nodes.

[0029] On the one hand, based on the initial model parameters of the original image recognition model and the average parameter values ​​corresponding to the neuron nodes retained under different pruning operations, the model parameter initialization of different pruning models includes:

[0030] Based on the parameter average values ​​corresponding to the neuron nodes retained under the target pruning operation, the initial model parameters and the set parameter adjustment ratio, the model parameters of the target pruning model after initialization are determined; wherein the target pruning operation is any one of the different pruning operations; the target pruning model is a pruning model obtained by pruning the original image recognition model according to the target pruning operation.

[0031] On the one hand, based on the parameter averages corresponding to the neuron nodes retained under the target pruning operation, the initial model parameters and the set parameter adjustment ratio, the model parameters of the target pruning model after initialization are determined to include:

[0032] Calculate the difference between the average value of the parameters corresponding to the neuron nodes retained under the target pruning operation and the initial model parameters;

[0033] According to the set parameter adjustment ratio, the difference is weightedly added to the initial model parameters to obtain the model parameters after the target pruning model is initialized.

[0034] On the one hand, the image samples are analyzed and trained using different pruning models after the model parameters are initialized to obtain the pruned and optimized image recognition models including:

[0035] The same image sample is input into different pruning models after the model parameters are initialized for analysis to determine the output loss values ​​of different pruning models;

[0036] The pruning model with the smallest output loss value is trained according to the image samples to obtain an image recognition model after pruning optimization.

[0037] On the one hand, after analyzing and training the image samples using different pruning models after the model parameters are initialized to obtain the pruned and optimized image recognition model, it also includes:

[0038] Counting a first time taken by the original image recognition model to analyze the data to be processed and a second time taken by the pruned and optimized image recognition model to analyze the data to be processed;

[0039] When the second time is greater than the first time, prompt information indicating pruning optimization abnormality is output.

[0040] The embodiment of the present invention further provides an image recognition device, comprising a determination unit, a statistics unit, an initialization unit, a training unit and an analysis unit;

[0041] The determination unit is used to determine the parameter average values ​​of different neuron node combinations according to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations in the original image recognition model training process; wherein the different neuron node combinations include the combination of neuron nodes discarded and retained neuron nodes in each training iteration of the model;

[0042] The statistical unit is used to classify and count the parameter average values ​​of different neuron node combinations to determine the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations;

[0043] The initialization unit is used to initialize the model parameters of different pruning models based on the initial model parameters of the original image recognition model and the average values ​​of the parameters corresponding to the neuron nodes retained under different pruning operations;

[0044] The training unit is used to analyze and train image samples using different pruning models after model parameter initialization to obtain an image recognition model after pruning optimization;

[0045] The analysis unit is used to input the image to be analyzed into the image recognition model when the image to be analyzed is acquired, so as to determine the target object contained in the image to be analyzed.

[0046] In one aspect, the determining unit includes a recording unit, an adding unit, and a dividing unit;

[0047] The recording unit is used to randomly discard neuron nodes according to a set ratio during each training iteration of the original image recognition model to record parameter update values ​​corresponding to different neuron node combinations;

[0048] The adding unit is used to increase the number of the same neuron node combination by one when a neuron node combination identical to an existing neuron node combination appears in each training iteration; wherein the numbers of different neuron node combinations in the initial state are all zero;

[0049] The division operation unit is used to perform a division operation on the parameter update value corresponding to each neuron node combination and the number thereof, and use the obtained quotient as the parameter average value of each neuron node combination.

[0050] On the one hand, the recording unit includes a statistical subunit and an accumulation subunit;

[0051] The statistical subunit is used to randomly discard neuron nodes according to a set ratio during each training iteration of the original image recognition model, and to count parameter update values ​​corresponding to the neuron nodes retained in each training iteration;

[0052] The accumulation subunit is used to accumulate the parameter update values ​​corresponding to the neuron nodes retained in each training iteration according to the neuron node combination to which they belong, so as to obtain the parameter update value corresponding to each neuron node combination.

[0053] On the one hand, the statistical subunit is used to use the parameter update values ​​corresponding to the neuron nodes retained in the previous training iteration and the initial parameter values ​​corresponding to the discarded neuron nodes as the model parameters of the current training iteration; wherein the parameter update values ​​corresponding to each neuron node retained in the initial state are the initial parameter values ​​of each neuron node;

[0054] In the current training iteration, neuron nodes are randomly discarded according to the set ratio, and the image recognition model after randomly discarding neuron nodes is trained using the samples of the current batch, and the parameter update values ​​corresponding to the neuron nodes retained in the current batch are recorded;

[0055] When all batches of samples are trained, the accumulated values ​​of the parameter update values ​​corresponding to the neuron nodes retained in all batches are used as the parameter update values ​​corresponding to the neuron nodes retained in the current training iteration.

[0056] On the one hand, the accumulation subunit is used to obtain the parameter update value of the jth neuron node in the current training iteration when the neuron node combination in the current training iteration is to discard the i-th neuron node and retain the j-th neuron node;

[0057] The neuron node combination under the previous training iteration is to discard the i-th neuron node and retain the parameter update value corresponding to the j-th neuron node, and the parameter update value of the j-th neuron node under the current training iteration is accumulated, and the accumulated value is used as the parameter update value of the neuron node combination under the current training iteration, which is to discard the i-th neuron node and retain the j-th neuron node.

[0058] On the one hand, the statistical unit includes a query subunit and an average value calculation subunit;

[0059] The query subunit is used to query at least one neuron node combination matching the retained neuron node from different neuron node combinations;

[0060] The average value operation subunit is used to perform an average value operation on the parameter average values ​​of at least one matched neuron node combination to obtain the parameter average values ​​corresponding to the retained neuron nodes.

[0061] On the one hand, the initialization unit is used to determine the model parameters of the target pruned model after initialization based on the parameter average values ​​corresponding to the neuron nodes retained under the target pruning operation, the initial model parameters and the set parameter adjustment ratio; wherein the target pruning operation is any one of the different pruning operations; the target pruning model is a pruning model obtained by pruning the original image recognition model according to the target pruning operation.

[0062] On the one hand, the initialization unit includes a calculation subunit and a weighted accumulation subunit;

[0063] The calculation subunit is used to calculate the difference between the average value of the parameters corresponding to the neuron nodes retained under the target pruning operation and the initial model parameters;

[0064] The weighted accumulation subunit is used to weightedly accumulate the difference to the initial model parameters according to the set parameter adjustment ratio to obtain the model parameters of the target pruning model after initialization.

[0065] On the one hand, the training unit is used to input the same image sample into different pruning models after the model parameters are initialized for analysis, so as to determine the output loss values ​​of different pruning models;

[0066] The pruning model with the smallest output loss value is trained according to the image samples to obtain an image recognition model after pruning optimization.

[0067] On the one hand, after analyzing and training the image samples using different pruning models after the model parameters are initialized to obtain the pruned and optimized image recognition model, a time statistics unit and an output unit are also included;

[0068] The time statistics unit is used to count the first time spent by the original image recognition model to analyze the data to be processed and the second time spent by the pruned and optimized image recognition model to analyze the data to be processed;

[0069] The output unit is used to output prompt information of pruning optimization abnormality when the second time is greater than the first time.

[0070] The embodiment of the present invention further provides a deep learning model pruning device, including:

[0071] Memory for storing computer programs;

[0072] A processor is used to execute the computer program to implement the steps of the above-mentioned image recognition method.

[0073] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned image recognition method are implemented.

[0074] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned image recognition method when executed by a processor.

[0075] The embodiment of the present invention also provides a deep learning model pruning method, including:

[0076] According to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations during the training process of the original deep learning model, the average parameter values ​​of the different neuron node combinations are determined; wherein the different neuron node combinations include the combination of the neuron nodes discarded and the neuron nodes retained in each training iteration of the model;

[0077] Classify and count the parameter averages of different neuron node combinations to determine the parameter averages corresponding to the neuron nodes retained under different pruning operations;

[0078] Based on the initial model parameters of the original deep learning model and the average parameter values ​​corresponding to the neuron nodes retained under different pruning operations, the model parameters of different pruning models are initialized;

[0079] The sample set is analyzed and trained using different pruning models after model parameter initialization to obtain a deep learning model after pruning optimization.

[0080] On the one hand, after analyzing and training the sample set using different pruning models after model parameter initialization to obtain a deep learning model after pruning optimization, it also includes:

[0081] The data to be processed is analyzed using the deep learning model after pruning optimization to obtain an analysis result; wherein the data to be processed is image data, text data or voice data.

[0082] It can be seen from the above technical scheme that the parameter average values ​​of different neuron node combinations are determined according to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations during the training process of the original image recognition model; wherein, different neuron node combinations include combinations of neuron nodes discarded by each training iteration of the model and neuron nodes retained. The parameter average values ​​of different neuron node combinations are classified and counted to determine the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations. Based on the initial model parameters of the original image recognition model and the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations, the model parameters of different pruning models are initialized. The image samples are analyzed and trained using different pruning models after the model parameters are initialized to obtain the image recognition model after pruning optimization. When the image to be analyzed is obtained, the target object contained in the image to be analyzed can be determined by inputting the image to be analyzed into the image recognition model.

[0083] The beneficial effect of the present invention lies in that by analyzing the impact of the neuron nodes discarded during the training of the original image recognition model on the retained neuron nodes, and based on this, completing the initialization of model parameters of different pruning models, it helps to improve the efficiency of retraining of the pruning model, reduce the demand for computing resources in the entire training process of the model, and effectively reduce the computing and storage costs of the model, so that the image recognition model optimized by pruning can quickly and accurately identify the target objects contained in the image to be analyzed. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0085] Figure 1 A flowchart of an image recognition method provided by an embodiment of the present invention;

[0086] Figure 2 A flow chart of a method for determining parameter average values ​​of different neuron node combinations provided by an embodiment of the present invention;

[0087] Figure 3 A schematic diagram of a forward propagation calculation process before and after randomly discarding neuron nodes in a model training process provided by an embodiment of the present invention;

[0088] Figure 4 A schematic diagram of a back propagation calculation process before and after randomly discarding neuron nodes in a model training process provided by an embodiment of the present invention;

[0089] Figure 5 A schematic diagram of filter pruning provided by an embodiment of the present invention;

[0090] Figure 6 A schematic diagram of the structure of an image recognition device provided by an embodiment of the present invention;

[0091] Figure 7 A flowchart of a deep learning model pruning method provided by an embodiment of the present invention;

[0092] Figure 8 A structural diagram of an image recognition device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0093] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0094] The terms "including" and "having" in the specification of the present invention and the above-mentioned drawings, as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0095] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0096] As deep learning models have shown remarkable results in image recognition, natural language processing, speech recognition and other fields, the computational complexity of the models has also increased exponentially, resulting in an increase in the computational and storage costs of the models. Model pruning technology is an effective way to solve this problem. The model pruning process mainly includes three steps: training the original model, pruning the model, and retraining the pruned model. Among them, retraining is to restore the model accuracy that has been reduced due to model pruning.

[0097] The core of model pruning includes pruning methods, pruning strategies, and pruning evaluation.

[0098] Pruning methods can be divided into soft pruning and hard pruning, structured pruning and parameter pruning. Structured pruning belongs to coarse-grained pruning, such as filter pruning, channel pruning, vector pruning, etc. Parameter pruning belongs to fine-grained pruning, etc.

[0099] Common pruning strategies include pruning strategies based on sparse optimization training and strategies based on random pruning. The former performs sparse training on model parameters and is often used for parameter pruning. It has good performance but higher computational complexity than the latter.

[0100] Pruning evaluation is used to select structural units or parameters to be pruned, such as the importance ranking method in structured pruning and the sparsity method in parameter pruning.

[0101] Current model pruning methods are mostly aimed at optimizing pruning methods and pruning evaluation, but lack optimization of the overall process of the pruning algorithm. For the trained original model, the pruned structural parameters are directly discarded, and the parameters are relearned during the retraining of the pruned model. This not only ignores the impact of the pruned parameters on the pruned model, but also reduces the efficiency of retraining the pruned model, making the model pruning operation require a high amount of computing resources, resulting in an increase in the computing and storage costs of the model.

[0102] Therefore, the embodiments of the present invention provide an image recognition method, device, equipment, medium and deep learning model pruning method, which optimizes the overall process of the pruning algorithm, analyzes the influence of the neuron nodes discarded during the training of the original image recognition model on the retained neuron nodes, and determines the average parameter value corresponding to the retained neuron nodes. Based on the initial model parameters of the original image recognition model and the average parameter values ​​corresponding to the retained neuron nodes under different pruning operations, the model parameters of different pruning models can be initialized quickly, which improves the learning efficiency of the pruning model retraining, thereby effectively reducing the computational and storage costs of the model pruning operation.

[0103] Next, a pruning method for an image recognition model provided by an embodiment of the present invention is introduced in detail. Figure 1 A flowchart of a pruning method for an image recognition model provided by an embodiment of the present invention, the method comprising:

[0104] S101: Determine the average parameter values ​​of different neuron node combinations according to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations during the training of the original image recognition model.

[0105] The original image recognition model is composed of multiple neuron nodes. A neuron node is a neuron node. Each neuron node has its corresponding model parameters. In the embodiment of the present invention, the image recognition model after the pruning operation can be referred to as a pruned model. In order to distinguish the model parameters of the original image recognition model from those of the pruned model, the model parameters of the original image recognition model can be referred to as initial model parameters. For the sake of ease of description, the original image recognition model can be referred to as the original model.

[0106] The training process of the original model will randomly discard a certain proportion of neuron nodes, and the parameter update values ​​of the non-discarded neuron nodes are calculated through forward propagation and back propagation to complete the training of the original model. In order to understand the impact of the discarded neuron nodes on the retained neuron nodes, the neuron nodes discarded in each training iteration of the original model and the retained neuron nodes can be combined as a neuron node. For example, assuming that neuron nodes 1 and 2 are discarded in the current training iteration, and neuron nodes 3, 4, and 5 are retained, then the different neuron node combinations are (1-3), (1-4), (1-5), (2-3), (2-4), and (2-5). Taking (1-3) as an example, it means that neuron node 1 is discarded and neuron node 3 is retained.

[0107] The neuron nodes discarded and retained in each training iteration of the original model may be the same or different. In practical applications, the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations can be accumulated and counted. After the original model training is completed, for each neuron node combination, the parameter update value of the neuron node combination can be divided by the number of the neuron node combination to obtain the parameter average value of the neuron node combination. By analogy, the parameter average values ​​corresponding to all neuron node combinations can be calculated.

[0108] S102: Classify and count the parameter averages of different neuron node combinations to determine the parameter averages corresponding to the neuron nodes retained under different pruning operations.

[0109] In order to select the best pruning model, the model parameters of different pruning models can be initialized first, so as to screen out the best pruning model. In the embodiment of the present invention, the pruning model with the smallest loss value can be used as the best pruning model.

[0110] The training of the pruning model is to update the parameters of the retained neuron nodes. Different neuron node combinations may involve the retention of the same neuron nodes, so at least one neuron node combination matching the retained neuron nodes can be queried from different neuron node combinations; the average value of the parameters of at least one matching neuron node combination is averaged to obtain the average value of the parameters corresponding to the retained neuron nodes.

[0111] Taking the five neuron nodes mentioned above as an example, the neuron node combinations corresponding to the retained neuron node 3 are (1-3) and (2-3), then the parameter average values ​​of the neuron node combination (1-3) and the parameter average values ​​of the neuron node combination (2-3) are averaged, and the average value obtained is the parameter average value corresponding to the retained neuron node 3.

[0112] S103: Initializing model parameters of different pruning models based on the initial model parameters of the original image recognition model and the average values ​​of parameters corresponding to the neuron nodes retained under different pruning operations.

[0113] The initial model parameters of the original image recognition model refer to the model parameters of the original image recognition model after iterative training. In order to distinguish them from the model parameters of the subsequent pruned model, the model parameters of the original image recognition model after iterative training are called initial model parameters.

[0114] Considering that the original model has a large number of training iterations, often hundreds or even thousands of times, the different neuron node combinations generated during the training of the original model can almost cover all pruning situations. Based on the operations of S101 and S102, the average values ​​of the parameters corresponding to the neuron nodes retained under different pruning operations can be determined.

[0115] Different pruning operations can obtain different pruning models. The parameter initialization process of each pruning model is similar. Therefore, any pruning operation among different pruning operations, namely the target pruning operation, is taken as an example for introduction. For the sake of distinction, the pruning model corresponding to the target pruning operation can be called the target pruning model. That is, the target pruning model is the pruning model obtained by pruning the original image recognition model according to the target pruning operation.

[0116] In practical applications, the model parameters of the target pruning model after initialization can be determined based on the parameter averages corresponding to the neuron nodes retained under the target pruning operation, the initial model parameters, and the set parameter adjustment ratio.

[0117] In order to evaluate the impact of the average parameter value corresponding to the neuron nodes retained under the pruning operation on the initial model parameters, the difference between the average parameter value corresponding to the neuron nodes retained under the target pruning operation and the initial model parameters can be calculated; according to the set parameter adjustment ratio, the difference is weightedly added to the initial model parameters to obtain the model parameters after the target pruning model is initialized.

[0118] S104: Analyze and train image samples using different pruning models after model parameter initialization to obtain an image recognition model after pruning optimization.

[0119] In an embodiment of the present invention, the same image sample can be input into different pruning models after the model parameters are initialized for analysis, and the output loss values ​​of different pruning models can be determined through forward reasoning calculation; finally, the model with the smallest output loss value is selected as the optimal pruning model.

[0120] After completing model pruning and model parameter initialization, in order to further improve the accuracy of the pruned model, the pruned model with the smallest output loss value can be trained based on image samples to obtain a pruned and optimized image recognition model.

[0121] In practical applications, image samples can be divided into several batches of samples and input into the pruning model. The output loss value of the model is obtained through forward propagation calculation, and the output loss value is back-propagated. The stochastic gradient descent method is used to optimize the model and update the model parameters, and finally the pruned and optimized image recognition model is obtained.

[0122] S105: When the image to be analyzed is acquired, the image to be analyzed is input into an image recognition model to determine the target object contained in the image to be analyzed.

[0123] In the embodiment of the present invention, the model obtained by performing different pruning operations on the original image recognition model may be referred to as a pruned model, and the model obtained after pruning optimization may be referred to as an image recognition model.

[0124] After obtaining the pruned and optimized image recognition model, if an image to be analyzed is obtained, the image to be analyzed can be directly input into the pruned and optimized image recognition model, and the model will output the analysis result. The analysis result will include the target object contained in the image to be analyzed.

[0125] In the embodiment of the present invention, there is no limitation on the type of the target object. The type of the target object identified by the image recognition model can be set based on the actual recognition requirements. For example, the target object can be a pedestrian, an animal, or a building.

[0126] It should be noted that after obtaining the pruned and optimized image recognition model, the image recognition model can be directly called to analyze the acquired image to be analyzed, without having to perform the above operations S101 to S104 each time.

[0127] It can be seen from the above technical scheme that the parameter average values ​​of different neuron node combinations are determined according to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations during the training process of the original image recognition model; wherein, different neuron node combinations include combinations of neuron nodes discarded by each training iteration of the model and neuron nodes retained. The parameter average values ​​of different neuron node combinations are classified and counted to determine the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations. Based on the initial model parameters of the original image recognition model and the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations, the model parameters of different pruning models are initialized. The image samples are analyzed and trained using different pruning models after the model parameters are initialized to obtain the image recognition model after pruning optimization. When the image to be analyzed is obtained, the target object contained in the image to be analyzed can be determined by inputting the image to be analyzed into the image recognition model. The beneficial effect of the present invention lies in that by analyzing the impact of the neuron nodes discarded during the training of the original image recognition model on the retained neuron nodes, and based on this, completing the initialization of model parameters of different pruning models, it helps to improve the efficiency of retraining of the pruning model, reduce the demand for computing resources in the entire training process of the model, and effectively reduce the computing and storage costs of the model, so that the image recognition model optimized by pruning can quickly and accurately identify the target objects contained in the image to be analyzed.

[0128] The original image recognition model will randomly discard neuron nodes according to the set ratio at each training iteration. By recording the parameter update values ​​corresponding to different neuron node combinations at each training iteration and counting the number of each neuron node combination, the parameter average values ​​of different neuron node combinations can be determined after the original image recognition model completes training.

[0129] Figure 2 A flowchart of a method for determining parameter average values ​​of different neuron node combinations provided by an embodiment of the present invention, the method comprising:

[0130] S201: randomly discarding neuron nodes according to a set ratio at each training iteration of the original image recognition model, and counting parameter update values ​​corresponding to the neuron nodes retained in each training iteration.

[0131] Deep learning networks contain multiple nonlinear hidden layers, which give them powerful expressive power. However, when training data is limited, deep learning networks can easily over-learn and cause overfitting, resulting in poor model generalization ability. Usually, image recognition model training uses the method of randomly discarding neuron nodes (Dropout) to prevent overfitting.

[0132] Dropout refers to the process of randomly selecting a certain proportion of neuron nodes for a certain layer of the neural network in an iteration during the model training process and temporarily discarding them from the network before training the model of the current batch of samples. Model training includes forward propagation and back propagation calculations. In the next iteration, some neuron nodes are randomly hidden until the training is completed. Since it is a temporary discard, each batch is training a different network.

[0133] First, the model structure is modified through Dropout. Some nodes are randomly selected according to the layer, and the forward and backward connections are deleted. Taking the lth hidden layer of the model as an example, some neurons of this layer are randomly deleted according to the drop rate (dropRate), that is, the filter parameters of the connection between this neuron and the previous and next hidden layers are discarded. In actual use, Dropout is usually described as a gating layer placed after the hidden layer of the model, which randomly sets some input units to 0 with the probability of dropRate during training.

[0134] For ease of description, neuron nodes can be referred to as neurons. A neuron is equivalent to a filter, so the model parameters corresponding to the neuron node can be called the filter parameters of the neuron output, referred to as neuron filter parameters.

[0135] Figure 3 A schematic diagram of a forward propagation calculation process before and after randomly discarding neuron nodes during a model training process provided by an embodiment of the present invention. Figure 3 The left side in the middle is a schematic diagram of the forward propagation calculation process before randomly discarding neuron nodes during model training. Figure 3 The middle right side is a schematic diagram of the forward propagation calculation process after randomly discarding neuron nodes during model training. Figure 3 The l-1th hidden layer, lth hidden layer and l+1th hidden layer of the model are taken as examples. Assuming that the lth hidden layer contains 5 neurons, the 0th neuron to the 4th neuron can be used in sequence. to Said that discarded and neurons, then in the current batch sample training, the filter parameter W l [:,1],W l [:,3],W l+1 [1,:],W l +1 [3,:] does not participate in training calculation. W l [:,1] represents the filter parameters of the output of the first neuron in the lth hidden layer; W l [:,3] represents the filter parameters of the output of the third neuron in the lth hidden layer; W l+1[1,:] represents the filter parameters of the first neuron input in the l+1th hidden layer; W l+1 [3,:] represents the filter parameter that discards the input of the third neuron in the l+1th hidden layer. The l+1th hidden layer contains 2 neurons, respectively. and express.

[0136] Then, the Dropout model is trained: a batch of image samples is input into the network model after adding Dropout, and the output loss value J of the model is calculated through forward propagation. The output loss value is the difference between the model output and the manually labeled output, which can be calculated using methods such as cross entropy and mean square error. The output loss value is calculated through back propagation to calculate the gradient of the model parameters W, b, and the stochastic gradient descent optimizer is used to obtain the updated value of W, b.

[0137] Taking the lth hidden layer of the model as an example, the output of the lth layer is y l , the output of the (l+1)th layer is y l+1 , the corresponding filter parameter is W l+1 , b l+1 , σ represents the activation function, and the Dropout gating layer can be regarded as a one-dimensional 0 / 1 sequence vector, denoted by r l+1 , if a neuron node is discarded, the corresponding element is set to 0 to control the output of each neuron. The forward propagation calculation process is:

[0138] .

[0139] In addition, in order to ensure that the output expectation remains unchanged after Dropout, it is usually necessary to scale the output of the neuron with a scaling ratio of 1-dropRate.

[0140] .

[0141] Figure 4 A schematic diagram of a back propagation calculation process before and after randomly discarding neuron nodes in a model training process provided by an embodiment of the present invention. Figure 4 The left side is a schematic diagram of the back propagation calculation process before randomly discarding neuron nodes during model training. Figure 4 The right side of the figure is a schematic diagram of the back propagation calculation process after randomly discarding neuron nodes during model training. Back propagation is to calculate the gradient of the neuron filter parameters of the output loss J on this layer, denoted as . Use stochastic gradient descent to calculate the update parameter W l+1 , λ is the learning rate optimized by stochastic gradient descent, and the back propagation calculation process is as follows:

[0142] ;

[0143] ;

[0144] in, Represents the output y of the l+1th layer l+1 The gradient of It represents the derivation of the activation function.

[0145] It should be noted that the parameter update only updates the corresponding model parameters on the neurons that are not discarded. In the embodiment of the present invention, the neuron combination can be called a discard-retain neuron node pair.

[0146] Then, the updated values ​​of the filter parameters of all discarded-retained neuron node pairs calculated by back propagation after each Dropout are counted. DropW is defined to represent the parameter update value of the discarded-retained neuron node pairs after Dropout, and dropNum is defined to represent the number of discarded-retained neuron node pairs. The parameter update value dropW of other retained neuron nodes corresponding to each neuron node discarded is extracted through the Dropout training information, and then used to generate the initial parameters of the pruning model.

[0147] Assume that after Dropout, the i-th neuron node on a hidden layer is discarded and the j-th neuron node is retained, W[j,:] represents the updated value of the filter parameter of the j-th neuron node after Dropout, and dropW[i][j] is used to represent the updated value of the filter parameter of the j-th neuron node corresponding to the discarded i-th neuron node;

[0148] dropW[i][j]= dropW[i][j]+ W[j,:];

[0149] dropNum[i][j]= dropNum[i][j]+ W[j,:].

[0150] This completes the training of a batch of samples.

[0151] S202: Accumulating the parameter update values ​​corresponding to the neuron nodes retained in each training iteration according to the neuron node combination to which they belong, so as to obtain the parameter update value corresponding to each neuron node combination.

[0152] In an embodiment of the present invention, each time a round of iteration is completed, the parameter update values ​​corresponding to the neuron nodes retained in the previous round of training iteration and the initial parameter values ​​corresponding to the discarded neuron nodes can be used as model parameters of the current training iteration; wherein the parameter update values ​​corresponding to each neuron node retained in the initial state are the initial parameter values ​​of each neuron node.

[0153] In the current training iteration, randomly discard neuron nodes according to the set ratio, use the current batch of samples to train the image recognition model after randomly discarding neuron nodes, and record the parameter update values ​​corresponding to the neuron nodes retained in the current batch. When all batches of samples are trained, the cumulative value of the parameter update values ​​corresponding to the neuron nodes retained in all batches is used as the parameter update value corresponding to the neuron nodes retained in the current training iteration.

[0154] Taking discarding the i-th neuron node and retaining the j-th neuron node as an example, when the neuron node combination in the current training iteration is discarding the i-th neuron node and retaining the j-th neuron node, the parameter update value of the j-th neuron node in the current training iteration can be obtained. The parameter update value corresponding to the neuron node combination in the previous training iteration of discarding the i-th neuron node and retaining the j-th neuron node is accumulated with the parameter update value of the j-th neuron node in the current training iteration, and the accumulated value is used as the parameter update value of the neuron node combination in the current training iteration of discarding the i-th neuron node and retaining the j-th neuron node.

[0155] After completing the training of each batch of samples, the deleted neurons and parameters are restored. At this time, the parameters of the deleted neurons remain the same, while the parameters of the neurons that have not been deleted have been updated. Repeat the previous steps to train the next batch of samples until all batches of samples are trained.

[0156] S203: When a neuron node combination identical to an existing neuron node combination appears in each training iteration, the number of the identical neuron node combination is increased by one.

[0157] In the initial state, the number of different neuron node combinations is zero. Each neuron node combination is processed in a similar manner. Taking any neuron node combination as an example, each time the neuron node combination appears, the number of the neuron node combination is increased by one.

[0158] S204: Performing a division operation on the parameter update value corresponding to each neuron node combination and the number thereof, and using the obtained quotient as the parameter average value of each neuron node combination.

[0159] After training is completed, the parameter average of each drop-retain neuron node pair is calculated according to the formula dropW[i][j] = dropW[i][j] / dropNum[i][j].

[0160] In an embodiment of the present invention, the parameter update values ​​corresponding to the neuron nodes retained in each training iteration are counted, and the parameter update values ​​corresponding to the neuron nodes retained in each training iteration are accumulated according to the neuron node combination to which they belong, so as to obtain the parameter update values ​​corresponding to each neuron node combination. In the case where a neuron node combination identical to an existing neuron node combination appears in each training iteration, the number of the same neuron node combination is increased by one. The parameter update value corresponding to each neuron node combination and its number are divided to obtain the parameter average value of each neuron node combination. The parameter average value of each neuron node combination can effectively reflect the impact of the discarded neuron nodes on the retained neuron nodes. The parameter average value based on the neuron node combination can provide a basis for the subsequent screening model pruning method, thereby facilitating the rapid screening of the optimal pruning model.

[0161] The core of model pruning includes pruning methods, pruning strategies and pruning evaluation. Pruning methods can be divided into structured pruning and parameter pruning, soft pruning and hard pruning by type. Pruning strategies mainly include pruning strategies based on sparse optimization training and strategies based on random pruning. Pruning evaluation is a method to evaluate the importance of pruning structure. There are many types. Commonly used ones include L1 / L2 norm of filter parameters, neuron output activation value, neuron output correlation, information entropy of filter parameters, and sparse training coefficients. Among them, parameter pruning belongs to fine-grained pruning, which prunes some single neuron weight connections, while structured pruning belongs to coarse-grained pruning, including filter pruning, channel pruning, vector pruning, etc. The difference between soft pruning and hard pruning is that the latter directly prunes the convolution kernel, while the former only sets its parameters to 0. Generally, hard pruning and soft pruning are both filter pruning.

[0162] The method of the present invention does not change the Dropout training process of the original model. On this basis, the parameter update values ​​of other retained neuron nodes caused by the Dropout discarded neuron nodes are extracted, and the updated values ​​are used to evaluate the influence of the discarded neuron nodes on the parameters of the retained neuron nodes in the model pruning, so as to adjust the initial parameters of the pruning model. It is mainly aimed at commonly used convolutional neural network models, such as convolutional neural network (Visual Geometry Group, VGG), residual network (ResidualNetwork, Resnet), classic convolutional neural network (InceptionNet), etc. The Dropout layer is added after any hidden layer including convolutional hidden layer and fully connected hidden layer. The sample data set can be a commonly used large data set, such as the visualization database (ImageNet), the standard data set in the field of object detection (VOC), and the image data set (‌ , COCO) etc.

[0163] The dropout-based image recognition model pruning method proposed in the present invention optimizes the overall process of the pruning algorithm, utilizes the influence of the neuron nodes discarded in the dropout on the filter parameters of other non-discarded nodes during the original model training process, and based on this, quickly generates the initial parameters of the neuron filter of the pruned model, and then adaptively adjusts the parameters to adapt the pruning model. Therefore, the method of the present invention is aimed at model pruning whose pruning type is filter pruning. The pruning strategy can select a pruning strategy of random pruning or sparse optimization training. The pruning evaluation method can select any filter importance evaluation method, such as the L1 / L2 norm of the filter parameters, the neuron output activation value, the neuron output correlation, the information entropy of the filter parameters, etc. The method of the present invention realizes the optimization of the overall process of the pruning algorithm, and improves the accuracy of the model by adjusting and adapting the initial parameters of the pruned model, so it can be combined with other filter pruning methods, and a random pruning method can also be used. The former algorithm is relatively complex but has better performance, while the latter is relatively simple but has slightly worse performance.

[0164] In order to describe the technical solution of the present invention more simply and clearly, the present invention adopts a random pruning method to randomly prune neuron nodes from each layer of the model according to the pruning ratio (cuteRate), and then initializes the model parameters and verifies the reasoning performance. The process is repeated for multiple iterations to select the pruned model with the best performance.

[0165] After model pruning, according to the dropW of the discarded-retained neuron node pairs obtained in the previous step, the parameter average cutDropW of the neuron nodes retained after this pruning is calculated. The model pruning randomly prunes the set of neuron nodes in the lth layer as cutSet l+1 , the corresponding set size is cutNum l+1 Taking the retained neuron node i as an example, the filter parameter cutDropW corresponding to the neuron node i is calculated according to the parameters of all discarded neuron nodes l+1 [i,:]:

[0166] .

[0167] Figure 5 A schematic diagram of filter pruning provided by an embodiment of the present invention, after model pruning, the average parameter value of the neuron nodes retained after this pruning is calculated based on the obtained average parameter value of the discarded-retained neuron node pairs. Figure 5 As shown, the model pruning randomly cuts off the 1st and 3rd neuron nodes of the lth layer, and retains the 0th, 2nd, and 4th neuron nodes. Then according to dropW l+1 [1][0],dropW l+1 [3][0] can calculate the average parameter cutDropW of the retained neuron node 0l+1 [0,:]: cutDropW l+1 [0,:]=(dropW l+1 [1][0]+ dropW l+1 [3][0]) / 2.

[0168] According to dropW l+1 [1][2],dropW l+1 [3][2] can calculate the average parameter cutDropW of the retained neuron node 2 l+1 [2,:]:

[0169] cutDropW l+1 [2,:]=(dropW l+1 [1][2]+ dropW l+1 [3][2]) / 2.

[0170] According to dropW l+1 [1][4],dropW l+1 [3][4] The average parameter cutDropW of the retained neuron node 4 can be calculated l+1 [4,:]:

[0171] cutDropW l+1 [4,:]=(dropW l+1 [1][4]+ dropW l+1 [3][4]) / 2.

[0172] Then, the difference between the initial model parameters of the original model (pretW) and the average parameter value after filter pruning (cutDropW) is calculated, the difference is weighted by the step size parameter α and added to the initial model parameters of the original model, which is used as the initial parameters of the pruned model (cutW):

[0173] cutW= pretW+α*(cutDropW- pretW).

[0174] In an embodiment of the present invention, in order to enable the pruned and optimized image recognition model to more accurately identify the category of the target object, after analyzing the data to be processed using the pruned and optimized image recognition model to obtain the analysis results, the model parameters of the pruned and optimized image recognition model can be adjusted based on the target objects included in the analysis results and the target objects actually included in the data to be processed to obtain an adjusted image recognition model.

[0175] By fine-tuning the model parameters of the pruned and optimized image recognition model based on actual results, the accuracy of the image recognition model in identifying target objects can be further improved.

[0176] In order to evaluate whether the computational efficiency of the image recognition model after pruning optimization has been improved, the first time spent by the original image recognition model to analyze the processed data and the second time spent by the image recognition model after pruning optimization to analyze the processed data can be counted. If the second time is greater than the first time, it means that the computational efficiency of the image recognition model after pruning optimization has not been improved, and a prompt message indicating pruning optimization abnormality can be output at this time.

[0177] Figure 6 A schematic diagram of the structure of a pruning device for an image recognition model provided by an embodiment of the present invention, comprising a determination unit 61, a statistics unit 62, an initialization unit 63, a training unit 64 and an analysis unit 65;

[0178] A determination unit 61 is used to determine the average parameter values ​​of different neuron node combinations according to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations during the training process of the original image recognition model; wherein the different neuron node combinations include combinations of neuron nodes discarded and retained in each training iteration of the model;

[0179] A statistical unit 62, used for classifying and statistically analyzing the parameter average values ​​of different neuron node combinations to determine the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations;

[0180] An initialization unit 63, used to initialize model parameters of different pruning models based on initial model parameters of the original image recognition model and average values ​​of parameters corresponding to neuron nodes retained under different pruning operations;

[0181] A training unit 64 is used to analyze and train image samples using different pruning models after model parameter initialization to obtain an image recognition model after pruning optimization;

[0182] The analyzing unit 65 is used to input the image to be analyzed into the image recognition model when the image to be analyzed is acquired, so as to determine the target object contained in the image to be analyzed.

[0183] In some embodiments, the determining unit includes a recording unit, an adding unit, and a dividing unit;

[0184] A recording unit, used to randomly discard neuron nodes according to a set ratio at each training iteration of the original image recognition model to record parameter update values ​​corresponding to different neuron node combinations;

[0185] An adding unit is used to increase the number of the same neuron node combination by one when a neuron node combination identical to an existing neuron node combination appears in each training iteration; wherein the numbers of different neuron node combinations in the initial state are all zero;

[0186] The division operation unit is used to perform a division operation on the parameter update value and the number thereof corresponding to each neuron node combination, and use the obtained quotient as the parameter average value of each neuron node combination.

[0187] In some embodiments, the recording unit includes a statistical subunit and an accumulation subunit;

[0188] A statistical subunit, used to randomly discard neuron nodes according to a set ratio at each training iteration of the original image recognition model, and to count parameter update values ​​corresponding to the neuron nodes retained in each training iteration;

[0189] The accumulation subunit is used to accumulate the parameter update values ​​corresponding to the neuron nodes retained in each training iteration according to the neuron node combination to which they belong, so as to obtain the parameter update value corresponding to each neuron node combination.

[0190] In some embodiments, the statistical subunit is used to use the parameter update values ​​corresponding to the neuron nodes retained in the previous training iteration and the initial parameter values ​​corresponding to the discarded neuron nodes as the model parameters of the current training iteration; wherein the parameter update values ​​corresponding to each neuron node retained in the initial state are the initial parameter values ​​of each neuron node;

[0191] In the current training iteration, neuron nodes are randomly discarded according to the set ratio, and the image recognition model after randomly discarding neuron nodes is trained using the samples of the current batch, and the parameter update values ​​corresponding to the neuron nodes retained in the current batch are recorded;

[0192] When all batches of samples are trained, the accumulated values ​​of the parameter update values ​​corresponding to the neuron nodes retained in all batches are used as the parameter update values ​​corresponding to the neuron nodes retained in the current training iteration.

[0193] In some embodiments, the accumulation subunit is used to obtain the parameter update value of the jth neuron node in the current training iteration when the neuron node combination in the current training iteration is to discard the i-th neuron node and retain the j-th neuron node;

[0194] The neuron node combination under the previous training iteration is to discard the i-th neuron node and retain the parameter update value corresponding to the j-th neuron node, and the parameter update value of the j-th neuron node under the current training iteration is accumulated, and the accumulated value is used as the parameter update value of the neuron node combination under the current training iteration, which is to discard the i-th neuron node and retain the j-th neuron node.

[0195] In some embodiments, the statistics unit includes a query subunit and an average calculation subunit;

[0196] A query subunit, used to query at least one neuron node combination matching the retained neuron node from different neuron node combinations;

[0197] The average value operation subunit is used to perform an average value operation on the parameter average values ​​of at least one matched neuron node combination to obtain the parameter average values ​​corresponding to the retained neuron nodes.

[0198] In some embodiments, the initialization unit is used to determine the model parameters of the target pruned model after initialization based on the parameter average values ​​corresponding to the neuron nodes retained under the target pruning operation, the initial model parameters and the set parameter adjustment ratio; wherein the target pruning operation is any one of the different pruning operations; the target pruning model is a pruning model obtained by performing a pruning operation on the original image recognition model according to the target pruning operation.

[0199] In some embodiments, the initialization unit includes a calculation subunit and a weighted accumulation subunit;

[0200] A calculation subunit, used to calculate the difference between the average value of the parameters corresponding to the neuron nodes retained under the target pruning operation and the initial model parameters;

[0201] The weighted accumulation subunit is used to weightedly accumulate the difference to the initial model parameters according to the set parameter adjustment ratio to obtain the model parameters after the target pruning model is initialized.

[0202] In some embodiments, the training unit is used to input the same image sample into different pruning models after the model parameters are initialized for analysis, so as to determine the output loss values ​​of the different pruning models;

[0203] The pruned model with the smallest output loss value is trained based on the image samples to obtain the pruned and optimized image recognition model.

[0204] In some embodiments, after analyzing and training the image samples using different pruning models after the model parameters are initialized to obtain the pruned and optimized image recognition model, a time statistics unit and an output unit are also included;

[0205] A time statistics unit, used to count the first time spent by the original image recognition model to analyze the data to be processed and the second time spent by the pruned and optimized image recognition model to analyze the data to be processed;

[0206] The output unit is used to output prompt information of pruning optimization abnormality when the second time is greater than the first time.

[0207] Figure 6 The description of the features in the corresponding embodiments can be found in Figure 1 The relevant descriptions of the corresponding embodiments will not be repeated here one by one.

[0208] It can be seen from the above technical scheme that the parameter average values ​​of different neuron node combinations are determined according to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations during the training process of the original image recognition model; wherein, different neuron node combinations include combinations of neuron nodes discarded by each training iteration of the model and neuron nodes retained. The parameter average values ​​of different neuron node combinations are classified and counted to determine the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations. Based on the initial model parameters of the original image recognition model and the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations, the model parameters of different pruning models are initialized. The image samples are analyzed and trained using different pruning models after the model parameters are initialized to obtain the image recognition model after pruning optimization. When the image to be analyzed is obtained, the target object contained in the image to be analyzed can be determined by inputting the image to be analyzed into the image recognition model. The beneficial effect of the present invention lies in that by analyzing the impact of the neuron nodes discarded during the training of the original image recognition model on the retained neuron nodes, and based on this, completing the initialization of model parameters of different pruning models, it helps to improve the efficiency of retraining of the pruning model, reduce the demand for computing resources in the entire training process of the model, and effectively reduce the computing and storage costs of the model, so that the image recognition model optimized by pruning can quickly and accurately identify the target objects contained in the image to be analyzed.

[0209] Figure 7 A flowchart of a deep learning model pruning method provided by an embodiment of the present invention, the method comprising:

[0210] S701: Determine the average parameter values ​​of different neuron node combinations according to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations during the original deep learning model training process.

[0211] Among them, different neuron node combinations include combinations of neuron nodes discarded and retained neuron nodes in each training iteration of the model.

[0212] S702: Classify and count the parameter averages of different neuron node combinations to determine the parameter averages corresponding to the neuron nodes retained under different pruning operations.

[0213] S703: Based on the initial model parameters of the original deep learning model and the average values ​​of the parameters corresponding to the neuron nodes retained under different pruning operations, the model parameters of different pruning models are initialized.

[0214] S704: Analyze and train the sample set using different pruning models after model parameter initialization to obtain a deep learning model after pruning optimization.

[0215] In some embodiments, it also includes:

[0216] The data to be processed is analyzed using the deep learning model after pruning optimization to obtain analysis results; wherein the data to be processed is image data, text data or voice data.

[0217] In the case where the data to be processed is image data, the analysis result is the target object contained in the image data;

[0218] When the data to be processed is text data and the original deep learning model is a text extraction model, the analysis result is the summary information extracted from the text data; when the data to be processed is text data and the original deep learning model is a text translation model, the analysis result is the corresponding translation result in the text data; when the data to be processed is text data and the original deep learning model is a text classification model, the analysis result is the text category to which the text data belongs; when the data to be processed is text data and the original deep learning model is a text question-answering model, the analysis result is the answer information corresponding to the text data;

[0219] When the data to be processed is voice data, the analysis result is a text result obtained by converting the voice data.

[0220] Figure 7 The description of the features in the corresponding embodiments can be found in Figure 1 The relevant descriptions of the corresponding embodiments will not be repeated here one by one.

[0221] It can be seen from the above technical scheme that according to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations during the training process of the original deep learning model, the parameter average values ​​of different neuron node combinations are determined; wherein, different neuron node combinations include combinations of neuron nodes discarded by each training iteration of the model and retained neuron nodes. The parameter average values ​​of different neuron node combinations are classified and counted to determine the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations. Based on the initial model parameters of the original deep learning model and the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations, the model parameters of different pruning models are initialized. The sample set is analyzed and trained using different pruning models after the model parameters are initialized to obtain a deep learning model after pruning optimization. The beneficial effect of the present invention is that by analyzing the influence of the neuron nodes discarded during the training process of the original deep learning model on the retained neuron nodes, and completing the model parameter initialization of different pruning models based on this, it is helpful to improve the efficiency of retraining of the pruning model, reduce the demand for computing resources in the entire training process of the model, and thus effectively reduce the computing and storage costs of the deep learning model.

[0222] Figure 8 A structural diagram of an image recognition device provided by an embodiment of the present invention, such as Figure 8 As shown, the image recognition device includes: a memory 80 for storing a computer program;

[0223] The processor 81 is used to implement the steps of the image recognition method in the above embodiment when executing a computer program.

[0224] The image recognition device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer, or a desktop computer.

[0225] Among them, the processor 81 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 81 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 81 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 81 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 81 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0226] The memory 80 may include one or more computer-readable storage media, which may be non-transitory. The memory 80 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 80 is at least used to store the following computer program 801, wherein the computer program can implement the relevant steps of the image recognition method disclosed in any of the aforementioned embodiments after being loaded and executed by the processor 81. In addition, the resources stored in the memory 80 may also include an operating system 802 and data 803, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 802 may include Windows, Unix, Linux, etc. The data 803 may include but is not limited to the parameter averages of different neuron node combinations, the parameter averages corresponding to the retained neuron nodes, etc.

[0227] In some embodiments, the image recognition device may further include a display screen 82 , an input / output interface 83 , a communication interface 84 , a power supply 85 , and a communication bus 86 .

[0228] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the image recognition device, and may include more or fewer components than those shown in the figure.

[0229] It is understandable that if the image recognition method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or part of the contribution to the current technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk or optical disk and other media that can store program codes.

[0230] Based on this, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned image recognition method are implemented.

[0231] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned image recognition method when executed by a processor.

[0232] The above is a detailed introduction to an image recognition method, device, equipment, computer-readable storage medium, computer program product, and deep learning model pruning method provided by an embodiment of the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referenced to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0233] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0234] The above is a detailed introduction to an image recognition method, device, equipment, computer-readable storage medium, computer program product and deep learning model pruning method provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. An image recognition method, characterized in that: The method is used for the process of retraining the pruning model, including: According to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations in the original image recognition model training process, the parameter average values ​​of different neuron node combinations are determined; wherein the different neuron node combinations include the combination of neuron nodes discarded and neuron nodes retained in each training iteration of the model; Performing classification statistics on the parameter average values ​​of different neuron node combinations to determine the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations; wherein the classification statistics is to query at least one neuron node combination matching the retained neuron nodes from the different neuron node combinations for classification statistics; Initializing the model parameters of different pruning models based on the initial model parameters of the original image recognition model and the average values ​​of the parameters corresponding to the neuron nodes retained under different pruning operations; analyzing and training the image samples using the different pruning models after the model parameters are initialized to obtain the pruned and optimized image recognition model; when the image to be analyzed is obtained, the image to be analyzed is input into the image recognition model to determine the target object contained in the image to be analyzed; Correspondingly, according to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations during the original image recognition model training process, the average values ​​of the parameters of different neuron node combinations are determined to include: In each training iteration of the original image recognition model, neuron nodes are randomly discarded according to a set ratio to record parameter update values ​​corresponding to different neuron node combinations; In each training iteration, when a neuron node combination that is the same as an existing neuron node combination appears, the number of the same neuron node combination is increased by one; wherein the number of different neuron node combinations in the initial state is zero; The parameter update value and its number corresponding to each neuron node combination are divided, and the obtained quotient is used as the parameter average value of each neuron node combination.

2. The image recognition method according to claim 1, characterized in that: In each training iteration of the original image recognition model, neuron nodes are randomly discarded according to a set ratio to record parameter update values ​​corresponding to different neuron node combinations, including: In each training iteration of the original image recognition model, neuron nodes are randomly discarded according to a set ratio, and parameter update values ​​corresponding to the neuron nodes retained in each training iteration are counted; The parameter update values ​​corresponding to the neuron nodes retained in each training iteration are accumulated according to the neuron node combination to which they belong, so as to obtain the parameter update value corresponding to each neuron node combination.

3. The image recognition method according to claim 2, characterized in that: In each training iteration of the original image recognition model, neuron nodes are randomly discarded according to a set ratio, and the parameter update values ​​corresponding to the neuron nodes retained in each training iteration are counted, including: The parameter update values ​​corresponding to the neuron nodes retained in the previous training iteration and the initial parameter values ​​corresponding to the discarded neuron nodes are used as the model parameters of the current training iteration; wherein the parameter update values ​​corresponding to the neuron nodes retained in the initial state are the initial parameter values ​​of the neuron nodes; In the current training iteration, neuron nodes are randomly discarded according to the set ratio, and the image recognition model after randomly discarding neuron nodes is trained using the samples of the current batch, and the parameter update values ​​corresponding to the neuron nodes retained in the current batch are recorded; When all batches of samples are trained, the accumulated values ​​of the parameter update values ​​corresponding to the neuron nodes retained in all batches are used as the parameter update values ​​corresponding to the neuron nodes retained in the current training iteration.

4. The image recognition method according to claim 2, characterized in that: The parameter update values ​​corresponding to the neuron nodes retained in each training iteration are accumulated according to the neuron node combination to which they belong, so as to obtain the parameter update values ​​corresponding to each neuron node combination, including: When the neuron node combination in the current training iteration is to discard the i-th neuron node and retain the j-th neuron node, obtain the parameter update value of the j-th neuron node in the current training iteration; The neuron node combination under the previous training iteration is to discard the i-th neuron node and retain the parameter update value corresponding to the j-th neuron node, and the parameter update value of the j-th neuron node under the current training iteration is accumulated, and the accumulated value is used as the parameter update value of the neuron node combination under the current training iteration, which is to discard the i-th neuron node and retain the j-th neuron node.

5. The image recognition method according to claim 1, characterized in that: Determining the average parameter values ​​corresponding to the neuron nodes retained under different pruning operations includes: An averaging operation is performed on the parameter average values ​​of at least one matched neuron node combination to obtain the parameter average values ​​corresponding to the retained neuron nodes.

6. The image recognition method according to claim 1, characterized in that: Based on the initial model parameters of the original image recognition model and the average values ​​of the parameters corresponding to the neuron nodes retained under different pruning operations, the model parameter initialization of different pruning models includes: Based on the parameter average values ​​corresponding to the neuron nodes retained under the target pruning operation, the initial model parameters and the set parameter adjustment ratio, the model parameters of the target pruning model after initialization are determined; wherein the target pruning operation is any one of the different pruning operations; the target pruning model is a pruning model obtained by pruning the original image recognition model according to the target pruning operation.

7. The image recognition method according to claim 6, characterized in that: Based on the average parameter values ​​corresponding to the neuron nodes retained under the target pruning operation, the initial model parameters and the set parameter adjustment ratio, the model parameters of the target pruning model after initialization are determined, including: Calculate the difference between the average value of the parameters corresponding to the neuron nodes retained under the target pruning operation and the initial model parameters; According to the set parameter adjustment ratio, the difference is weightedly added to the initial model parameters to obtain the model parameters after the target pruning model is initialized.

8. The image recognition method according to claim 1, characterized in that: The image samples are analyzed and trained using different pruning models after the model parameters are initialized to obtain the pruned and optimized image recognition models including: The same image sample is input into different pruning models after the model parameters are initialized for analysis to determine the output loss values ​​of different pruning models; The pruning model with the smallest output loss value is trained according to the image samples to obtain an image recognition model after pruning optimization.

9. The image recognition method according to claim 1, characterized in that: After analyzing and training the image samples using different pruning models after the model parameters are initialized to obtain the pruned and optimized image recognition model, the method further includes: Counting a first time taken by the original image recognition model to analyze the data to be processed and a second time taken by the pruned and optimized image recognition model to analyze the data to be processed; When the second time is greater than the first time, prompt information indicating pruning optimization abnormality is output.

10. An image recognition device, characterized in that: It includes a determination unit, a statistics unit, an initialization unit, a training unit and an analysis unit; the device is used for the retraining process of the pruning model; The determination unit is used to determine the parameter average values ​​of different neuron node combinations according to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations in the original image recognition model training process; wherein the different neuron node combinations include the combination of neuron nodes discarded and retained neuron nodes in each training iteration of the model; The statistical unit is used to perform classification statistics on the parameter average values ​​of different neuron node combinations to determine the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations; wherein the classification statistics is to query at least one neuron node combination matching the retained neuron node from the different neuron node combinations for classification statistics; The initialization unit is used to initialize the model parameters of different pruning models based on the initial model parameters of the original image recognition model and the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations; the training unit is used to analyze and train the image samples using the different pruning models after the model parameters are initialized to obtain the image recognition model after pruning optimization; The analysis unit is used to input the image to be analyzed into the image recognition model when the image to be analyzed is acquired, so as to determine the target object contained in the image to be analyzed; Correspondingly, according to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations during the original image recognition model training process, the average values ​​of the parameters of different neuron node combinations are determined to include: In each training iteration of the original image recognition model, neuron nodes are randomly discarded according to a set ratio to record parameter update values ​​corresponding to different neuron node combinations; In each training iteration, when a neuron node combination that is the same as an existing neuron node combination appears, the number of the same neuron node combination is increased by one; wherein the number of different neuron node combinations in the initial state is zero; The parameter update value and its number corresponding to each neuron node combination are divided, and the obtained quotient is used as the parameter average value of each neuron node combination.

11. An image recognition device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the image recognition method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image recognition method according to any one of claims 1 to 9 are implemented.

13. A pruning method for a deep learning model, characterized in that: The method is used for the retraining process of the pruning model, including: According to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations during the training process of the original deep learning model, the average parameter values ​​of the different neuron node combinations are determined; wherein the different neuron node combinations include the combination of the neuron nodes discarded and the neuron nodes retained in each training iteration of the model; Performing classification statistics on the parameter average values ​​of different neuron node combinations to determine the parameter average values ​​corresponding to the neuron nodes retained under different pruning operations; wherein the classification statistics is to query at least one neuron node combination matching the retained neuron nodes from the different neuron node combinations for classification statistics; Initializing the model parameters of different pruning models based on the initial model parameters of the original deep learning model and the average values ​​of the parameters corresponding to the neuron nodes retained under different pruning operations; wherein the pruning operation adopts a random pruning operation; The sample set is analyzed and trained using different pruning models after model parameter initialization to obtain a deep learning model after pruning optimization. Analyze the data to be processed by using the deep learning model after pruning optimization to obtain an analysis result; wherein the data to be processed is text data or voice data; Correspondingly, according to the parameter update values ​​of different neuron node combinations and the number of different neuron node combinations during the original deep learning recognition model training process, the average parameter values ​​of different neuron node combinations are determined to include: In each training iteration of the original deep learning recognition model, neuron nodes are randomly discarded according to a set ratio to record parameter update values ​​corresponding to different neuron node combinations; In each training iteration, when a neuron node combination that is the same as an existing neuron node combination appears, the number of the same neuron node combination is increased by one; wherein the number of different neuron node combinations in the initial state is zero; The parameter update value and its number corresponding to each neuron node combination are divided, and the obtained quotient is used as the parameter average value of each neuron node combination.

Images