Training method, device, equipment and storage medium for neural network model

Abstract

The embodiment of the present application provides a training method, device, equipment and storage medium for a neural network model, the method comprising: in the current iteration, processing a sample image through a neural network model to obtain a network loss; determining a back propagation stop layer for the current iteration; updating parameters of multiple network layers in the neural network model based on the network loss, wherein the parameters of at least one network layer in the neural network model whose depth is less than the back propagation stop layer remain unchanged. The embodiment of the present application can effectively reduce the training time of the neural network and save resources.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method, apparatus, device and storage medium for a neural network model. Background Art

[0002] A general neural network model is composed of dozens to hundreds of layers, and its training is achieved through multiple rounds of iterations until the neural network model converges and the training is completed. In each round of iteration, forward propagation, back propagation and parameter update are included, among which forward propagation is the process of image processing, back propagation is the process of calculating gradients based on network loss, and parameter update is the process of updating parameters based on the gradients calculated in the back propagation process. In actual training, the multiple rounds of iterative training process of the neural network model requires a lot of time and resources. Summary of the invention

[0003] The embodiments of the present application disclose a technical solution for a neural network model.

[0004] In a first aspect, an embodiment of the present application provides a method for training a neural network model, comprising, in a current iteration, processing a sample image through the neural network model to obtain a network loss; determining a back propagation stop layer for the current iteration; and updating parameters of multiple network layers in the neural network model based on the network loss, wherein parameters of at least one network layer in the neural network model whose depth is less than the back propagation stop layer remain unchanged.

[0005] It can be seen that in the training process of the neural network model of the present application, in the current round of iteration, the sample image is first processed by forward propagation to obtain the network loss (i.e., the loss function), and then the back propagation stop layer of the current round is determined. Finally, the parameters of multiple network layers of the neural network model are updated based on the network loss, and the parameters of the network layers with a depth less than the back propagation stop layer remain unchanged. Through the embodiments of the present application, the back propagation stops when it reaches the back propagation stop layer, and there is no need to continue to perform gradient propagation to the network layer with a depth less than the back propagation stop layer. There is no need to update the parameters of the network layer with a depth less than the back propagation stop layer. Therefore, without affecting the performance of the trained neural network model, the present application reduces the number of back propagation layers, saves computing resources, reduces the training time of the neural network model, and improves training efficiency.

[0006] Based on the first aspect, in a possible implementation manner, determining the back propagation stop layer of the current iteration includes: determining the back propagation stop layer according to the round number of the current iteration.

[0007] It can be understood that the neural network model needs to go through many rounds of iterative training before it finally converges. In each round of iterative training, the back propagation stop layer can be determined according to the number of the current iteration.

[0008] Based on the first aspect, in a possible implementation, determining the back propagation stop layer according to the number of rounds of the current iteration includes: performing a modulo operation on the total number of layers of the neural network model by the number of rounds of the current iteration to obtain the layer number of the back propagation stop layer.

[0009] It can be seen that in a certain round of iteration, the total number of layers of the neural network model can be modulo calculated according to the number of rounds of the current iteration to determine the layer number of the back propagation stop layer. For example, if the current iteration round is the 69th round and the total number of layers of the neural network model is 80 layers, the current iteration round is modulo calculated on the total number of layers of the neural network model, and the result is 69, then the layer number of the back propagation stop layer is 69, that is, back propagation to the 69th layer, and the network layers smaller than the 69th layer are not back propagated; if the current iteration round is the 85th round and the total number of layers of the neural network model is 80 layers, the current iteration round is modulo calculated on the total number of layers of the neural network model, and the result is 5, then the layer number of the back propagation stop layer is 5, that is, back propagation to the 5th layer, and the network layers smaller than the 5th layer are not back propagated. Therefore, the implementation of the embodiment of the present application reduces the number of layers of back propagation, reduces the amount of calculation and the time of back propagation, thereby saving computing resources and reducing the training time of the neural network model.

[0010] Based on the first aspect, in a possible implementation, determining the back propagation stop layer of the current iteration includes: determining the back propagation stop layer of the current iteration based on the gradient and weight, and / or clock number of at least one network layer of the neural network model in the current iteration.

[0011] It can be seen that the neural network model includes dozens or even hundreds of layers. The back propagation process of the neural network model is essentially the process of calculating the gradient of each network layer, and the parameter update process is the process of updating the weight of each network layer according to the gradient of each network layer. Therefore, in the training iteration process of the neural network model, the back propagation stop layer of the current iteration can be determined based on the gradient and weight of at least one network layer of the neural network model in the current iteration, and / or the number of clocks. Among them, the number of clocks in each network layer of the neural network model is pre-set.

[0012] Based on the first aspect, in a possible implementation, determining the back propagation stop layer of the current iteration according to the gradient and weight, and / or clock number of at least one network layer of the neural network model in the current iteration includes: determining the target network layer of the current iteration according to the clock number of at least one network layer of the neural network model; determining the back propagation stop layer of the current iteration based on the gradient and weight of at least one network layer having a depth less than the target network layer in the current iteration.

[0013] It can be seen that when each network layer of the neural network model is set with a clock number, the back propagation stop layer of the current iteration is determined. First, the target network layer of the current iteration is determined according to the clock number of each network layer of the neural network model; for the network layer with a depth greater than the target network layer, back propagation is directly performed according to the traditional neural network model training method; for the network layer with a depth less than the target network layer, the back propagation stop layer of the current iteration is determined according to its gradient, weight and the number of rounds of the current iteration.

[0014] Based on the first aspect, in a possible implementation, determining the target network layer of the current iteration according to the clock number of at least one network layer of the neural network model includes: determining the target network layer of the current iteration based on the number of rounds of the current iteration and the clock number of each network layer in the neural network model.

[0015] It can be understood that the target network layer of the current iteration can be determined based on the number of rounds of the current iteration and the preset number of clocks of each layer. For example, in one embodiment, the number of clocks of each network layer can be modulo-operated by the number of rounds of the current iteration, and the network layer with a modulo operation result of 0 is used as the preset target network layer, wherein the preset target network layer may include one network layer or multiple network layers, and then the smallest network layer among the preset target network layers is used as the target network layer. For example, the number of clocks of each network layer is modulo-operated by the number of rounds of the current iteration, if the network layers with a modulo operation result of 0 include the 4th layer, the 8th layer, and the 12th layer, then the preset target network layers include the 4th layer, the 8th layer, and the 12th layer, and then the smallest layer among the preset target network layers is taken as the target network layer, that is, the 4th layer is the target network layer.

[0016] Based on the first aspect, in a possible implementation, the method further includes: if the target network layer does not exist in the current iteration, determining the back propagation stop layer of the current iteration according to the gradient and weight of at least one network layer in the neural network model in the current iteration.

[0017] It can be understood that in the above embodiment, if the current iteration round number performs a modulo operation on the clock number of each network layer, if there is no network layer whose modulo operation result is 0, that is, the modulo operation results are not 0, then there is no target network layer. In the case where there is no target network layer in the current iteration, in the current iteration, the back propagation stop layer of the current iteration is determined directly according to the gradients and weights of each network layer in the neural network model.

[0018] Based on the first aspect, in a possible implementation, determining the back propagation stop layer of the current iteration according to the gradient and weight, and / or clock number of at least one network layer of the neural network model in the current iteration includes: determining the back propagation stop layer of the current iteration according to the ratio of the gradient and weight of at least one network layer of the neural network model in the current iteration and a preset threshold.

[0019] It can be understood that when the clock number is not set for each network layer of the neural network model, the back propagation stop layer of the current iteration can be determined directly based on the ratio of the gradient to the weight of each network layer in the neural network model and the preset threshold.

[0020] Based on the first aspect, in a possible implementation, the back propagation stop layer of the current iteration is determined according to the ratio of the gradient to the weight of at least one network layer of the neural network model in the current iteration and a preset threshold, including: if in the current network layer of the neural network model, the ratio of the normalized gradient value to the normalized weight value is less than or equal to the preset threshold, determining that the current network layer is the back propagation stop layer.

[0021] It can be understood that in one embodiment, when backpropagating to a certain network layer, if the ratio of the normalized gradient value to the normalized weight value in the current network layer of the neural network model is less than or equal to the preset threshold, the current network layer is determined to be the backpropagation stop layer; if the ratio of the normalized gradient value to the normalized weight value is greater than the preset threshold, the backpropagation continues to propagate to the next layer. Among them, the normalized gradient value is the average of the absolute values ​​of multiple gradients of the current layer, and the normalized weight value is the average of the absolute values ​​of multiple weights of the current layer.

[0022] Based on the first aspect, in a possible implementation, the number of clocks of at least one network layer in the neural network model is set based on at least one of a first strategy and a second strategy; wherein, the first strategy includes: the larger the number of layers of the neural network model, the larger the set number of clocks; and the second strategy includes: for at least one network layer in the neural network model that is more important for training, a smaller number of clocks is set.

[0023] It can be understood that in a possible implementation method, the smaller the clock number, the higher the frequency of the result of 0 when the number of iterations is modulo the clock number. The clock number of a shallower network layer is set to be smaller, so that it is easy to reach the network layer during back propagation, and the deeper layer larger than the network layer will also perform back propagation and parameter update. Therefore, the clock number of each network layer can be set according to the first strategy; there may be one or more important layers in the neural network model. For such a network layer, a smaller clock number will be set, so that the number of iterations modulo the smaller clock number will result in a larger number of times being 0, which is relatively easy to achieve, so the number of iterations will be larger, thereby increasing the accuracy of training and making the neural network model stabilize earlier.

[0024] Based on the first aspect, in a possible implementation, the updating of parameters of multiple network layers in the neural network model based on the network loss, wherein the parameters of at least one network layer whose depth is less than the back propagation stop layer remain unchanged, comprises: performing back propagation based on the network loss, calculating the gradients of multiple network layers in the neural network model until the back propagation stop layer is reached; and updating the parameters of the multiple network layers based on the gradients of the multiple network layers, wherein the parameters of at least one network layer whose depth is less than the back propagation stop layer remain unchanged.

[0025] It can be understood that after determining the back propagation stop layer, back propagation can be performed. Back propagation is performed based on the network loss obtained by forward propagation, and the back propagation stops at the back propagation stop layer. The back propagation process is actually a process of calculating the gradients of multiple network layers; based on the calculated gradients of multiple network layers, the parameters of the multiple network layers are updated, among which, only the parameters of the network layers with a depth greater than the back propagation stop layer are updated, and the parameters of the network layers with a depth less than the back propagation stop layer remain unchanged.

[0026] Based on the first aspect, in a possible implementation, before performing the current iteration, it also includes: performing at least one round of iterative training on the neural network model, wherein during the at least one round of iterative training, parameters of all network layers of the neural network model are updated.

[0027] It can be understood that in the training process of the neural network, incomplete back propagation (incomplete back propagation, that is, in each round of iteration, only some network layers are back propagated and gradients are calculated) and parameter updates are not performed from the first round of iteration. Instead, at least one round of complete back propagation (complete back propagation, that is, in each round of iteration, all network layers are back propagated and gradients are calculated) and parameter updates are performed in the initial stage of training to ensure that all network layers of the neural network model can be trained. In actual training, for different neural network models, the number of complete back propagation rounds can be set according to actual conditions.

[0028] In a second aspect, an embodiment of the present application provides an image processing method, comprising: acquiring an image; inputting the image into a trained neural network model for processing to obtain a processing result; wherein the neural network model is trained by the method described in the first aspect or any embodiment of the first aspect.

[0029] It can be seen that, firstly, an image is acquired, and then the acquired image is input into a trained neural network model for processing to obtain a processing result. The neural network model is trained by the method described in the first aspect or any one of the embodiments of the first aspect. The implementation of the embodiment of the present application shortens the training time of the neural network model, and the acquired image is directly input into the trained neural network model to obtain the desired processing result, thereby shortening the entire image processing time.

[0030] In a third aspect, the present application also provides a training device for a neural network model, including:

[0031] A processing unit, used for processing the sample image by the neural network model in a current iteration to obtain a network loss;

[0032] A determination unit, used to determine the back propagation stop layer of the current iteration;

[0033] An updating unit is used to update the parameters of multiple network layers in the neural network model based on the network loss, wherein the parameters of at least one network layer in the neural network model whose depth is less than the back propagation stop layer remain unchanged.

[0034] Based on the third aspect, in a possible implementation manner, the determining unit is specifically configured to: determine the back propagation stop layer according to the number of rounds of the current iteration.

[0035] Based on the third aspect, in a possible implementation, the determination unit is specifically used to: perform a modulo operation on the total number of layers of the neural network model according to the number of rounds of the current iteration to obtain the layer number of the back propagation stop layer.

[0036] Based on the third aspect, in a possible implementation, the determination unit is specifically used to determine the back propagation stop layer of the current iteration based on the gradient and weight, and / or clock number of at least one network layer of the neural network model in the current iteration.

[0037] Based on the third aspect, in a possible implementation, the determination unit is specifically used to: determine the target network layer of the current iteration based on the clock number of at least one network layer of the neural network model; determine the back propagation stop layer of the current iteration based on the gradient and weight of at least one network layer whose depth is less than the target network layer in the current iteration.

[0038] Based on the third aspect, in a possible implementation, the determination unit is specifically used to: determine the target network layer of the current iteration based on the number of rounds of the current iteration and the number of clocks of each network layer in the neural network model.

[0039] Based on the third aspect, in a possible implementation, the determination unit is specifically used to: if the target network layer does not exist in the current iteration, determine the back propagation stop layer of the current iteration according to the gradient and weight of at least one network layer in the neural network model in the current iteration.

[0040] Based on the third aspect, in a possible implementation, the determination unit is specifically used to determine the back propagation stop layer of the current iteration based on the ratio of the gradient to the weight of at least one network layer of the neural network model in the current iteration and a preset threshold.

[0041] Based on the third aspect, in a possible implementation, the determination unit is specifically used to: if in the current network layer of the neural network model, the ratio of the gradient normalization value to the weight normalization value is less than or equal to a preset threshold, determine that the current network layer is the back propagation stop layer, wherein the gradient normalization value is the average of the absolute values ​​of multiple gradients of the current network layer, and the weight normalization value is the average of the absolute values ​​of multiple weights of the current network layer.

[0042] Based on the third aspect, in a possible implementation, the number of clocks of at least one network layer in the neural network model is set based on at least one of a first strategy and a second strategy; wherein, the first strategy includes: the larger the number of network layers, the larger the set number of clocks, and the second strategy includes: for at least one network layer in the neural network model that is more important for training, a smaller number of clocks is set.

[0043] Based on the third aspect, in a possible implementation, the update unit is specifically used to: perform back propagation based on the network loss, calculate the gradients of multiple network layers in the neural network model until the back propagation stop layer is reached; based on the gradients of the multiple network layers, update the parameters of the multiple network layers, wherein the parameters of at least one network layer whose depth is less than the back propagation stop layer remain unchanged.

[0044] Based on the third aspect, in a possible implementation, the device also includes a preprocessing unit, which is used to: perform at least one round of iterative training on the neural network model, wherein during the at least one round of iterative training, parameters of all network layers of the neural network model are updated.

[0045] Each functional unit in the apparatus of the third aspect is used to implement the method described in the first aspect and any specific implementation manner of the first aspect.

[0046] In a fourth aspect, an embodiment of the present application provides an image processing device, comprising: an acquisition unit, for acquiring an image; an image processing unit, for inputting the image into a trained neural network model for processing and obtaining a processing result; the neural network model is trained by the method described in the first aspect or any embodiment of the first aspect.

[0047] Each functional unit in the apparatus of the fourth aspect is used to execute the method described in the second aspect or any possible implementation manner of the second aspect.

[0048] In a fifth aspect, an embodiment of the present application provides a training device for a neural network model, comprising a memory and a processor, wherein the memory is used to store instructions, and the processor is used to call the instructions stored in the memory to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0049] In a sixth aspect, an embodiment of the present application provides an image processing device, comprising a memory and a processor, wherein the memory is used to store instructions, and the processor is used to call the instructions stored in the memory to execute the method described in the second aspect or any possible implementation manner of the second aspect.

[0050] In the seventh aspect, an embodiment of the present application provides a non-volatile storage medium for storing program instructions. When the program instructions are applied to a training device for a neural network model, they can be used to implement the method described in the first aspect or any possible implementation manner of the first aspect.

[0051] In an eighth aspect, an embodiment of the present application provides a computer program product, the computer program product including program instructions, when the computer program product is executed by a training device of a neural network model, the training device of the neural network model executes the method described in the first aspect. The computer program product may be a software installation package, and when it is necessary to use the method provided by any possible design of the first aspect, the computer program product may be downloaded and executed on the training device of the neural network model to implement the method described in the first aspect or any possible implementation of the first aspect.

[0052] In a ninth aspect, an embodiment of the present application provides a non-volatile storage medium for storing program instructions. When the program instructions are applied to an image processing device, they can be used to implement the method described in the second aspect or any possible implementation of the second aspect.

[0053] In a tenth aspect, an embodiment of the present application provides a computer program product, the computer program product including program instructions, when the computer program product is executed by an image processing device, the image processing device executes the method described in the second aspect. The computer program product may be a software installation package, and when it is necessary to use the method provided by any possible design of the second aspect, the computer program product may be downloaded and executed on the image processing device to implement the method described in the second aspect or any possible implementation of the second aspect.

[0054] It can be seen that the embodiments of the present application provide a training method, apparatus, device and storage medium for a neural network model, wherein the training method for the neural network model includes: first performing at least one round of iterative training on the neural network model, and in at least one round of iterative training, backpropagating and updating parameters of all network layers of the neural network model; then, in each subsequent round of iterative training, first processing the neural network model through forward propagation to obtain network loss, and then determining the backpropagation stop layer of the current iteration, and based on the network loss, backpropagating multiple network layers in the neural network model until the backpropagation stop layer, and then updating the parameters of multiple network layers, wherein the parameters of the network layer with a depth less than the backpropagation stop layer are not kept unchanged, and after multiple rounds of iterative training, the neural network model is finally converged, so that without affecting the performance of the trained neural network model, by backpropagating and updating parameters of some network layers of the neural network model, the amount of calculation is reduced, computing resources are saved, thereby shortening the training time of the neural network model and improving the training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 A schematic diagram showing changes in the normalized gradient values ​​and normalized weight values ​​of each layer during the training process of the neural network model provided in this application;

[0057] Figure 2 A schematic diagram of a training method for a neural network model provided in an embodiment of the present application;

[0058] Figure 3 A schematic diagram of another training method for a neural network model provided in an embodiment of the present application;

[0059] Figure 4 A schematic diagram of a method for determining a back propagation stop layer for current iterative training provided in an embodiment of the present application;

[0060] Figure 5 A schematic diagram of an example flow chart of a training process of a neural network model provided in an embodiment of the present application;

[0061] Figure 6 A schematic diagram of another method for determining the back propagation stop layer of the current iterative training provided in an embodiment of the present application;

[0062] Figure 7 A schematic diagram of an image processing method provided in an embodiment of the present application;

[0063] Figure 8 A schematic diagram of a training device for a neural network model provided in an embodiment of the present application;

[0064] Fig. 9 A schematic diagram of a training device for a neural network model provided in an embodiment of the present application;

[0065] Fig.10 A schematic diagram of an image processing device provided in an embodiment of the present application;

[0066] Fig.11 A schematic diagram of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0068] It should be noted that the terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0069] It should be noted that when used in this specification and the appended claims, the terms "comprises" and any of their variations are intended to cover non-exclusive inclusions. For example, a system, product or device including a series of units / devices is not limited to the listed units / devices, but may optionally include units / devices not listed, or may also optionally include other units / devices inherent to these products or devices.

[0070] It will also be understood that the term “if” may be interpreted as “when” or “upon” or “in response to determining” or “in response to detecting” or “in the event of…” depending on the context.

[0071] The neural network training process needs to go through the process of forward propagation, back propagation and parameter update and iteration until convergence (the convergence can be judged by whether the value of the loss function no longer decreases or tends to be stable), so as to obtain a trained neural network. Forward propagation is the process of propagating from the first layer to the last layer of the neural network layer, which is the image processing process. Back propagation is the process of propagating from the last layer to the first layer of the neural network layer. In the process of back propagation, the gradient values ​​of each layer of the neural network are mainly calculated, and the parameters (such as weights, biases, etc.) are updated according to the gradient values ​​in each layer. The parameters in the neural network are updated through back propagation, so that the output value of the neural network is constantly close to the target value. In this way, after multiple iterative training, the neural network model finally converges. However, due to the large number of parameters in the neural network, the training process for general neural networks takes a long time, especially in large-scale training, which consumes huge time and resources.

[0072] Analyze the training process of the classic neural network model deep residual network ResNet50, refer to Figure 1 , Figure 1 (1) in the figure is a schematic diagram showing how the normalized gradient values ​​of each layer change with the number of iterations. Figure 1 (2) is a schematic diagram of the change of the normalized weight value of each layer with the number of iterations. The change of the normalized gradient value and the normalized weight value of each layer can reflect the change trend of the gradient and weight during the training process. Analysis shows that: during the training process, as the number of iterations increases, the gradient and weight gradually decrease and finally tend to be stable, and the weight of the shallow layer tends to be stable earlier than the weight of the deep layer.

[0073] Based on the above findings, the embodiments of the present application propose not to perform complete back propagation, but to stop the back propagation at a certain layer in the neural network in a round of iteration during the training process. By reducing the number of propagation layers in the back propagation without affecting the performance of the trained neural network model, the amount of computation in the neural network training process is reduced, thereby reducing the overall training time and improving the network training efficiency.

[0074] The embodiment of the present application provides a training method for a neural network model, wherein incomplete back propagation is performed in a certain iteration, that is, the back propagation stops at a certain middle layer of the neural network model. The present application refers to the layer where the back propagation stops in the neural network model as the back propagation stop layer. Figure 2 The method includes but is not limited to the following steps:

[0075] S201. In the current iteration, the sample image is processed by the neural network model to obtain the network loss.

[0076] The current iteration may be the first iteration, an intermediate iteration, or the last iteration in a multi-round iterative training process of the neural network model.

[0077] In one possible implementation, in at least one initial round of iterative training, all network layers of the neural network model are back-propagated and parameter updated to ensure that all network layers can be trained. In subsequent training, in each round of iteration, the sample image is first processed by the neural network model to obtain the network loss, that is, forward propagation is performed from the first layer to the last layer of the neural network model; then, the back-propagation is stopped at a certain layer.

[0078] In another possible implementation, each round of iterative training starting from the first round performs back propagation and parameter update of some network layers, that is, in each round of iterative training, the sample image is first processed by the neural network model to obtain the network loss, that is, forward propagation is performed from the first layer to the last layer of the neural network model; then, the back propagation is stopped at a certain layer.

[0079] S202: Determine the back propagation stop layer of the current iteration.

[0080] The back propagation stop layer of the current iteration can be determined, wherein the back propagation stop layer determines the cutoff condition for back propagation. That is, the incomplete back propagation is propagated from the last layer of the neural network to the stop layer of the neural network, wherein the stop layer can be any layer of the neural network.

[0081] Specifically, the back propagation stop layer of the current iteration may be determined before the entire back propagation of the current iteration is performed. Alternatively, the judgment may be performed layer by layer, that is, when back propagating to a certain network layer, it is determined whether the current network layer is the back propagation stop layer of the current iteration, etc. The embodiment of the present disclosure does not limit the timing of determining the back propagation stop layer.

[0082] S203, updating parameters of multiple network layers in the neural network model based on the network loss, wherein parameters of at least one network layer in the neural network model whose depth is less than the back propagation stop layer remain unchanged.

[0083] After determining the back propagation stop layer of the current iteration, the gradient of the neural network model can be back propagated based on the network loss, starting from the last layer of the neural network model and propagating layer by layer until the back propagation stop layer is reached. After that, the gradient is no longer propagated to the network layer with a smaller depth. In this way, the gradient values ​​of multiple network layers including the back propagation stop layer can be obtained. For example, the multiple network layers include the back propagation stop layer and at least one network layer behind it (i.e., with a larger depth); then, the parameters of the multiple network layers can be updated according to the gradient values ​​of the multiple network layers obtained by back propagation. The network layers with a depth less than the back propagation stop layer are not back propagated, and their parameters remain unchanged accordingly.

[0084] It can be seen that the training method of the neural network model provided by the embodiment of the present disclosure, in the current iterative training of the neural network model, first, the sample image is processed by the neural network model to obtain the network loss, and then the back propagation stop layer of the current iteration is determined, and finally, the multiple network layers of the neural network model are back propagated based on the network loss to the back propagation stop layer, and then the parameters of these multiple network layers are updated. Therefore, by implementing the embodiment of the present application, the amount of calculation is reduced by reducing the number of back propagation layers, thereby saving computing resources, greatly reducing the training time of the neural network model, and improving the training efficiency.

[0085] refer to Figure 3 , Figure 3 A training method for a neural network model provided in an embodiment of the present application includes but is not limited to the following steps:

[0086] S301. In the current iteration, the sample image is processed by the neural network model to obtain the network loss.

[0087] The current iteration may be the first iteration, an intermediate iteration, or the last iteration in a multi-round iterative training process of the neural network model.

[0088] In one possible implementation, incomplete back propagation is performed from the first round of iterative training. In another possible implementation, in the training process of the neural network model, when the iterative training starts, the gradient changes are relatively large, and the gradient information of each layer is relatively important. Therefore, performing complete back propagation at the beginning of training is conducive to maintaining the performance of the trained neural network model. For example, m rounds of iterative training are performed on the neural network model from the first round. During the iterative training process, each round performs complete forward propagation and complete back propagation, and m is an integer greater than 1; from the m+i (i is a positive integer) round to the subsequent iterative training, each round performs incomplete back propagation and parameter update. In each subsequent round of iterative training, back propagation stops at a certain middle layer. It should be noted that in the above possible implementation, the number of rounds m of complete back propagation can be set according to actual conditions. For neural network models of different applications, the number of layers is different, and the amount of data processed is different, so the number of rounds m set can be different and can be set according to specific conditions.

[0089] S302: Determine the back propagation stop layer based on the result obtained by taking the modulus of the total number of layers of the neural network model based on the current number of iterations.

[0090] For example, the modulo the total number of layers of the neural network model with the current iteration number, the modulo result is the layer number of the back propagation stop layer of this round. For example, the total number of layers of the neural network model is 80, and the value of m is 40. When the number of iteration rounds is greater than 40, if the current iteration round is the 60th round, the back propagation of this iteration stops at the 60th layer, and if the current iteration round is the 86th round, the back propagation of this iteration stops at the 6th layer. For another example, the back propagation stop layer is determined by further processing the modulo result, or based on the modulo result and other factors.

[0091] S303, updating parameters of multiple network layers in the neural network model based on the network loss, wherein the parameters of at least one network layer in the neural network model whose depth is less than the back propagation stop layer remain unchanged.

[0092] After determining the back propagation stop layer of the current iteration, back propagation can be performed on multiple network layers of the neural network model based on the network loss, starting from the last layer of the neural network model to the back propagation stop layer, and then the parameters of these multiple network layers are updated. For network layers with a depth less than the back propagation stop layer, back propagation and parameter update are not performed.

[0093] In some embodiments, multiple rounds of iterative training are required to converge the neural network model. If the current iteration is the first round or an intermediate round of the entire iterative training process, each subsequent round of iteration will require forward propagation, partial back propagation and partial parameter update.

[0094] In order to more clearly understand the solution of the embodiment of the present application, the solution of the embodiment of the present application is described below by taking examples.

[0095] For example, assuming that the neural network model includes 66 network layers, and complete back propagation is performed from the first round to the mth round, and m is set to 80, then during the training process of the neural network model, from the 1st round to the 80th round, the neural network model undergoes complete forward propagation and complete back propagation; starting from the 81st round, the neural network model undergoes complete forward propagation and incomplete back propagation until the neural network model converges; if the current iteration round is the 88th round, the back propagation of this iteration stops at the 22nd layer, and if the current iteration round is the 120th round, the back propagation of this iteration stops at the 54th layer.

[0096] For another example, suppose the neural network model includes 66 network layers, and incomplete back propagation is performed from the first round. If the current iteration round is the 5th round, the back propagation of this iteration stops at the 5th layer. If the current iteration round is the 65th round, the back propagation of this iteration stops at the 65th layer. If the current iteration round is the 120th round, the back propagation of this iteration stops at the 54th layer.

[0097] It should be noted that, in an embodiment of the present application, the number of layers of the neural network model can be set starting from serial number 0, that is, the 66-layer neural network model can be numbered from 0 to 65. If the current iteration round number is the 66th round, the back propagation of this round of iteration stops at the 0th layer.

[0098] It can be seen that during the training process of the neural network model, the number of layers for back propagation and parameter updating is reduced. For those layers that do not perform back propagation, there is no need to calculate the gradient of the parameters, which means that the amount of calculation and calculation time in the training process are reduced, shortening the training time while also saving computing resources. In large-scale training, the communication time of gradients on different graphics cards GPUs is reduced, thereby increasing the training rate. In addition, in this embodiment, the back propagation stop layer is determined according to the number of rounds of the current iteration. The method is simple and easy to calculate, and is suitable for occasions where the accuracy of the neural network model is not required to be high.

[0099] refer to Figure 4 , Figure 4 A method for determining the back propagation stop layer of the current iterative training provided in an embodiment of the present application, wherein the back propagation cutoff is judged layer by layer, and the method includes but is not limited to:

[0100] S401, calculating the gradient of the current network layer.

[0101] Generally speaking, back propagation is carried out from the last layer to the first layer of the neural network model. The back propagation process is the process of calculating the gradients of each network layer of the neural network model. When back propagation reaches the current network layer, the gradient of the current network layer is calculated.

[0102] It should be noted that in the embodiment of the present disclosure, determining the back propagation stop layer occurs during the back propagation process, that is, determining the back propagation stop layer and back propagation are not two independent processes, but the back propagation stop layer is determined while the back propagation is being performed.

[0103] It should be noted that in the embodiment of the present application, the number of layers of the neural network model can be set to start from serial number 0, and the back propagation is carried out from the last layer of the neural network model to the 0th layer. The current network layer here can be the last layer of the neural network model, or a middle layer.

[0104] S402: Determine whether the current network layer is a back propagation stop layer of the current iteration according to the gradient and weight of the current network layer.

[0105] After calculating the gradient of the current network layer, determine whether the current network layer is the back propagation stop layer based on the gradient and weight of the current network layer.

[0106] In a possible implementation, whether the current network layer is a back-propagation stop layer can be determined by judging whether the parameters of the current network layer (such as gradients and weights, or other types of parameters) meet set conditions.

[0107] In an optional example, whether the current network layer is a back propagation stop layer is determined based on the ratio of the gradient to the weight in the current network layer and a preset threshold. For example, if the ratio of the gradient to the weight in the current network layer is less than or equal to the preset threshold, the current network layer is determined to be the back propagation stop layer of the current iteration, and if the ratio is greater than the preset threshold, the back propagation continues to propagate to the next layer.

[0108] If there are multiple gradients or weights in the current network layer, the ratios of the multiple gradients or weights can be calculated respectively, and the multiple ratios obtained can be averaged, maximized, or processed in other ways to obtain the final ratio. Alternatively, a target gradient or target weight can be selected from multiple gradients or multiple weights, and a ratio can be obtained based on the selected target gradient or target weight, which is not limited in the embodiments of the present disclosure.

[0109] In an optional example, the network layer of the neural network model includes multiple weights, and multiple gradients are calculated. According to the relationship between the ratio of the normalized value of the gradient to the normalized value of the weight in the current network layer and the preset threshold, it is determined whether the current network layer is the back propagation stop layer, wherein the normalized value of the gradient is the average processing result of the absolute values ​​of the multiple gradients of the current network layer, and the normalized value of the weight is the average processing result of the absolute values ​​of the multiple weights of the current network layer. The average processing here can be arithmetic average, weighted average, or geometric average, etc. For example, based on the multiple gradients and multiple weights of the current network layer, the normalized value of the gradient and the normalized value of the weight in the current network layer are calculated; the ratio of the normalized value of the gradient to the normalized value of the weight in the current network layer is calculated; the ratio is compared with the preset threshold, if the ratio is less than or equal to the preset threshold, the current network layer is determined to be the back propagation stop layer of the current iteration, if the ratio is greater than the preset threshold, the back propagation continues to propagate to the next layer.

[0110] In an optional example, it is also possible to determine whether the current network layer is a back propagation stop layer based on the relationship between the ratio of the gradient variance value to the weight variance value in the current network layer and a preset threshold. For example, based on multiple gradients and multiple weights of the current network layer, the gradient variance value and the weight variance value in the current network layer are calculated; the ratio of the gradient variance value to the weight variance value in the current network layer is calculated; the ratio is compared with the preset threshold, and if the ratio is less than or equal to the preset threshold, the current network layer is determined to be the back propagation stop layer of the current iteration, and if the ratio is greater than the preset threshold, the back propagation continues to propagate to the next layer.

[0111] In the disclosed embodiment, the operations of S401 and S402 may be performed for each network layer starting from the last layer of the neural network model until the back propagation stop layer of the current iteration is found.

[0112] It should be noted that the preset threshold is pre-set, and each round of iterative training corresponds to a preset threshold, wherein the preset thresholds of different rounds may be the same, or the preset thresholds of different rounds may be different. In some optional embodiments, the preset threshold is adjusted according to a certain strategy. For example, the preset threshold is set so that the smaller the number of iterative rounds, the smaller the layer number of the back propagation stop layer, so that the performance of the trained neural network model can be improved.

[0113] In a possible implementation, the frequency of adjusting the preset threshold may be set, and the frequency of adjusting the preset threshold refers to how often the threshold is adjusted or how often it is adjusted or in what manner. For example, the preset threshold may be adjusted once per round.

[0114] In a possible implementation, an adjustment function of the preset threshold value may be set, and different adjustment functions may result in different ways of changing the preset threshold value. In one example, the preset threshold value increases as the number of iterations increases. For example, the preset threshold value may change in a linear growth manner, an exponential growth manner, or other ways. The preset threshold value may be set by a linear function, an exponential function, or other functions. For example, the preset threshold value is set to the function value of a certain function, the number of iterations is used as an independent variable, and the preset threshold value changes as the number of iterations changes.

[0115] It can be seen that in each round of iterative training, the back propagation stop layer of the current iteration can be determined according to the gradient and weight of each network layer. Specifically, it can be: when back propagating to the current network layer, the gradient of the current network layer is calculated; according to the relationship between the ratio of the gradient to the weight of the current network layer and the preset threshold, the back propagation stop layer of the current iteration is determined: if the ratio of the gradient to the weight of the current network layer is less than or equal to the preset threshold, the current network layer is determined to be the back propagation stop layer; if the ratio of the gradient to the weight of the current network layer is greater than the preset threshold, the back propagation continues to propagate to the next layer; the above steps are repeated until the back propagation stop layer of the current iteration is determined. In the process of back propagation, the ratio of gradient to weight is constantly changing. According to the calculated ratio of the gradient to weight of each network layer, the back propagation stop layer is determined, which reduces the overall training time of the neural network model, improves the training efficiency, and can obtain a more accurate neural network model.

[0116] based on Figure 4 The method for determining the back propagation stop layer of the current iteration, the embodiment of the present application provides a flowchart of an example of the neural network model training process, reference Figure 5 .

[0117] The neural network model is trained after multiple rounds of iterations and finally reaches convergence. In the first round of incomplete back propagation iteration, the initial threshold is set in advance. First, a complete forward propagation is performed to obtain the network loss; then back propagation is performed. In the back propagation, the gradient norm value of each layer is calculated, and the back propagation stop layer is determined according to the relationship between the ratio of the gradient norm value of each layer to the weight norm value and the initial threshold. Specifically, starting from the last layer, the ratio of the gradient norm value of the last layer to the weight norm value is compared with the initial threshold. If it is greater than the initial threshold, it continues to propagate to the next layer until the ratio of the gradient norm value to the weight norm value of the layer is less than or equal to the initial threshold, then the back propagation is terminated at this layer; finally, the parameters are updated according to the gradient value calculated during the back propagation process to complete the first round of iterations; in the second round of iterative training, the initial threshold is adjusted, and the complete forward propagation and incomplete back propagation are performed in the same way as above, and the parameters are updated again to complete the second round of iterations; and iterative training is performed in this way until the neural network model converges.

[0118] In some cases, the back propagation stop layer is determined only by using the relationship between the ratio of the gradient normalized value to the weight normalized value of each layer and the threshold. In the back propagation process, the ratio of the gradient norm value to the weight norm value of a certain layer is always less than the threshold, resulting in the inability to update the layers before the layer. In this case, the convergence of the neural network may be affected. Based on this, the embodiment of the present application also provides a method for determining the back propagation stop layer of the current iterative training, referring to Figure 6 As shown, the method includes but is not limited to:

[0119] S501. Determine the target network layer of the current iteration according to the clock number of each network layer of the neural network model.

[0120] One or more layers of the neural network model may be set with corresponding clock numbers, and the target network layer of the current iteration is determined according to the clock numbers of each network layer of the neural network model.

[0121] In one possible implementation, the modulo operation is performed on the clock number of each network layer through the current iteration round number, and the network layer with a modulo result of 0 (the modulo result can also be set to other values) can be set as a candidate target layer, and then the network layer with the smallest layer number is determined as the target network layer from at least one candidate target layer. For example, in the current iteration process, the layers with a modulo operation of 0 for the clock number of the current iteration round number are the 8th layer, the 10th layer, and the 11th layer, that is, the candidate target layers are the 8th layer, the 10th layer, and the 11th layer, then the network layer with the smallest layer number, the 8th layer, is determined as the target layer. It should be noted that the target network layer can also be determined by other means, and this application does not impose specific restrictions on this.

[0122] The number of clocks of the network layer can be set based on various strategies. In some embodiments, the number of clocks in the neural network model is set based on at least one of the first strategy and the second strategy, wherein the first strategy includes: the larger the number of network layers, the larger the number of clocks set. The smaller the number of clocks, the higher the frequency of the result of the modulo operation of the number of iterative rounds on the number of clocks is 0. The number of clocks of the shallower network layer is set to be smaller, so that the frequency of reaching the network layer during back propagation is high, and the deeper layer greater than the network layer will also perform back propagation and parameter update. The second strategy includes: for at least one network layer with higher training importance in the neural network model, a smaller number of clocks is set. There may be one or some more important layers in the neural network model. For such a network layer, a smaller number of clocks will be set, so that the number of iterative rounds modulo operation of the smaller number of clocks will result in a higher number of times of 0, so that at least one network layer with higher training importance performs back propagation more frequently, increasing the accuracy of training and making the neural network model tend to be stable earlier.

[0123] In a possible implementation, the number of clocks can be set by a function. For example, the number of clocks is set by a linear function or a proportional function, with the layer number of the network layer as the independent variable of the linear function or the proportional function, and the number of clocks as the dependent variable (function value) of the linear function or the proportional function, to set the number of clocks for multiple network layers of the neural network.

[0124] In a possible implementation, the clock number may be randomly generated by a random function. For example, the random function is set to generate a random number according to at least one of the first strategy or the second strategy.

[0125] It should be noted that the number of clocks of one or more network layers may also be set in other ways, for example, manually. The present application does not specifically limit the way of setting the number of clocks.

[0126] S502: Determine a back propagation stop layer for the current iteration based on the gradient and weight of at least one network layer having a depth less than the target network layer in the current iteration.

[0127] If there is a target network layer in the current iteration, then back propagation is directly performed for at least one network layer whose depth is greater than the target network layer. In the back propagation process, it is only necessary to calculate multiple gradient values ​​in at least one network layer, without calculating the gradient norm value and the ratio of the gradient norm value to the weight norm value. For network layers whose depth is less than the target network layer, the back propagation stop layer of the current iteration is determined based on the gradients and weights of each layer. For example, if the target network layer is determined to be the 8th layer, back propagation is performed for all layers after the 8th layer (from the 8th layer to the last layer). In fact, it is propagated from the last layer to the 8th layer. These network layers only need to calculate the gradient values ​​of each layer. For all layers before the 8th layer, the back propagation stop layer of the current iteration is determined based on the ratio of the gradient norm value of each layer to the weight norm value.

[0128] In a possible implementation, for a network layer whose depth is less than the target network layer, the back propagation stop layer of the current iteration can be determined based on the ratio of the gradient to the weight in the current network layer and a preset threshold. First, the gradient of the current network layer is calculated, and then the ratio of the gradient to the weight in the current network layer is calculated, and then the ratio is compared with the preset threshold. If the ratio is less than or equal to the preset threshold, the current network layer is determined to be the back propagation stop layer of the current iteration. If the ratio is greater than the preset threshold, the back propagation continues to propagate to the next layer. Then the ratio of the gradient to the weight of the next layer is compared with the preset threshold, and whether it is the back propagation stop layer is determined based on the comparison result, until the back propagation stop layer is determined.

[0129] If there are multiple gradients or weights in the current network layer, the ratios of the multiple gradients or weights can be calculated respectively, and the multiple ratios obtained can be averaged, maximized, or processed in other ways to obtain the final ratio. Alternatively, a target gradient or target weight can be selected from multiple gradients or multiple weights, and a ratio can be obtained based on the selected target gradient or target weight, which is not limited in the embodiments of the present disclosure.

[0130] In a more specific possible implementation, for a network layer whose depth is less than the target network layer, the back propagation stop layer of the current iteration can be determined based on the ratio of the gradient normalization value to the weight normalization value in the current network layer and a preset threshold. The gradient normalization value is the average value of the absolute values ​​of multiple gradients of the current network layer, and the weight normalization value is the average processing result of the absolute values ​​of multiple weights of the current network layer. The average processing here can be arithmetic average, weighted average, or geometric average, etc. For example, based on multiple gradients and multiple weights of the current network layer, the gradient normalization value and the weight normalization value in the current network layer are calculated; the ratio of the gradient normalization value to the weight normalization value in the current network layer is calculated; the ratio is compared with the preset threshold. If the ratio is greater than the preset threshold, the back propagation continues to propagate to the next layer until the ratio of the network layer is less than or equal to the preset threshold, and the network layer is determined to be the back propagation stop layer of the current iteration.

[0131] It should be noted that, for a network layer whose depth is less than the target network layer, the back propagation stop layer of the current iteration can also be determined based on the ratio of the gradient variance value to the weight variance value in the current network layer and a preset threshold. The method is described in the above implementation method and will not be repeated here for the sake of brevity.

[0132] It should also be noted that the preset threshold in the above implementation is variable. For each round of iteration, the preset threshold may be different. For the setting of the threshold, refer to the description of the preset threshold in S402 above. For the sake of brevity of the specification, it will not be repeated here.

[0133] S503: If the target network layer does not exist in the current iteration, determine the back propagation stop layer of the current iteration according to the gradient and weight of at least one network layer in the neural network model in the current iteration.

[0134] If the target network layer does not exist in the current iteration, that is, if there is no layer whose result modulo the clock number of the current iteration round is 0, that is, the modulo the clock number of each network layer of the iteration round number is not 0, then the back propagation stop layer of the current iteration is determined directly according to the gradient and weight of each network layer.

[0135] If there are multiple gradients or weights in the current network layer, the ratios of the multiple gradients or weights can be calculated respectively, and the multiple ratios obtained can be averaged, maximized, or processed in other ways to obtain the final ratio. Alternatively, a target gradient or target weight can be selected from multiple gradients or multiple weights, and a ratio can be obtained based on the selected target gradient or target weight, which is not limited in the embodiments of the present disclosure.

[0136] In a possible implementation, the back propagation stop layer of the current iteration can be determined based on the relationship between the ratio of the gradient norm value and the weight norm value of each network layer and a preset threshold. For the specific method, refer to the description in S402, which will not be repeated here for the sake of brevity.

[0137] In another possible implementation, the back propagation stop layer of the current iteration can be determined based on the relationship between the ratio of the gradient variance value and the weight variance value of each network layer and a preset threshold. The specific method is described in S402, and will not be repeated here for the sake of brevity.

[0138] It should also be noted that the embodiments of the present application also provide an algorithm optimization and correction method during the training process. 1) In traditional neural network training, during back propagation, in addition to the gradient affecting the change of weights, in the training of some applications, there are other factors that affect the change of weights, such as the parameters weightdecay and momentum that affect the convergence speed. In the present application, due to the incomplete back propagation form adopted, some layers in each round may not be back propagated and no gradients are generated, so the present application removes the weight changes caused by weightdecay and momentum. 2) In incomplete back propagation, compared with normal training, the number of updates of parameters of certain layers is reduced, which will cause information loss to a certain extent, and thus may affect the accuracy. Therefore, we can compensate for the impact on accuracy by increasing the number of training rounds. By appropriately increasing the number of training rounds, we can increase the training of the model, which helps to compensate for the impact of incomplete back propagation on accuracy.

[0139] In large-scale training, multiple graphics card processors (GPUs) are generally used simultaneously to process data. However, during the back-propagation process, all gradients generated on different GPUs need to be communicated, which still consumes a lot of communication time. Therefore, reducing the number of gradients that need to be communicated will significantly improve the communication time of large-scale training. This embodiment is orthogonal to other communication methods (does not affect each other). By using incomplete back-propagation, the number of gradients that need to be updated can be reduced. Combined with other communication schemes, it can effectively reduce the communication time of large-scale training and improve the acceleration ratio of large-scale training.

[0140] It can be seen that in the embodiment of the present application, the number of clocks is set in each layer of the neural network model. First, the target network layer of the current iteration is determined according to the number of clocks of each network layer of the neural network model; then, for the network layer whose depth is greater than the target network layer, back propagation is directly performed to calculate the gradient. For the network layer whose depth is less than the target network layer, the back propagation stop layer of the current iteration is determined according to the gradient and weight; if there is no target network layer, the back propagation stop layer of the current iteration is directly determined according to the gradient and weight. The target network layer is first determined by the number of clocks, and then the back propagation stop layer is determined by the preset threshold, which avoids the defects in the above scheme, reduces the training time of the neural network model, saves computing resources, and can obtain an accurate neural network model. The embodiment of the present application is suitable for the training of any neural network model.

[0141] refer to Figure 7 , Figure 7 An image processing method provided in an embodiment of the present application includes but is not limited to:

[0142] S701, acquiring an image.

[0143] Acquire an image. This application does not limit the image acquisition method and image type. For example, it can be an image captured by a camera, or an image copied by a mobile device, or an image acquired by other means; it can be a face image, an object image, a plant image, or other images, etc.

[0144] S702: Input the image into the trained neural network model for processing to obtain a processing result.

[0145] The image is input into the neural network model for processing to obtain a processing result, wherein the neural network model is pre-trained.

[0146] In a possible implementation, the neural network model training process may be: starting from the first round of iterative training, the neural network model is subjected to incomplete back propagation until the neural network model converges. In each round of iterative training, the sample image is first processed by the neural network model to obtain the network loss; then the back propagation stop layer of the current iteration is determined; finally, the back propagation and parameter update of multiple network layers in the neural network model are performed based on the network loss.

[0147] In a possible implementation, the neural network model training process may be: first, at least one round of iterative training is performed, wherein, in at least one round of iterative training, all network layers of the neural network model are back-propagated and parameter updates are performed. In each subsequent round of iterative training, the sample image is processed by forward propagation to obtain the network loss, and then the back-propagation stop layer of the current iteration is determined, and the multiple network layers in the neural network model are back-propagated and parameter updates are performed based on the network loss.

[0148] Among them, the method for determining the back propagation stop layer of the current iteration can be, for example, to determine the back propagation stop layer according to the number of rounds of the current iteration; or, to determine the back propagation stop layer of the current iteration according to the ratio of the gradient to the weight of at least one network layer of the neural network model in the current iteration and a preset threshold; or, to first determine the target network layer of the current iteration according to the number of clocks of at least one network layer of the neural network model, and then determine the back propagation stop layer of the current iteration based on the gradient and weight of at least one network layer having a depth less than the target network layer in the current iteration.

[0149] It can be seen that the image is first acquired, and then the image is processed using the trained neural network model to obtain the processing result. Among them, in the initial iterative training of the neural network model, each round is to back propagate and update the parameters of all network layers to ensure that all network layers are trained; in the subsequent multiple rounds of iterative training, in each round of iteration, the sample image is first processed by forward propagation to obtain the network loss, and then the back propagation stop layer of the current iteration is determined, and then the back propagation is performed to the stop layer based on the network loss. Finally, according to the gradient calculated in the back propagation, the parameters of multiple network layers in the neural network model are updated to the stop layer, and after multiple rounds of such training, the neural network converges. Therefore, to implement the embodiment of the present application, it is necessary to pre-train the neural network model, use the trained neural network model to process the image, obtain the processing result, and shorten the image processing time by reducing the training time of the neural network model, thereby improving the image processing efficiency while ensuring the image processing performance.

[0150] refer to Figure 8 , Figure 8 A schematic diagram of a training device 60 for a neural network model provided in an embodiment of the present application, the device 60 includes:

[0151] The processing unit 601 is used to process the sample image through the neural network model in the current iteration to obtain the network loss;

[0152] A determination unit 602, used to determine a back propagation stop layer of a current iteration;

[0153] The updating unit 603 is used to update the parameters of multiple network layers in the neural network model based on the network loss, wherein the parameters of at least one network layer in the neural network model whose depth is less than the back propagation stop layer remain unchanged.

[0154] In a specific embodiment, the determination unit 602 is configured to determine the stop layer according to the number of rounds of the current iteration.

[0155] In a specific embodiment, the determination unit 602 is specifically configured to perform a modulo operation on the total number of layers of the neural network model according to the number of rounds of the current iteration to obtain the layer number of the back propagation stop layer.

[0156] In a specific embodiment, the determination unit 602 is specifically used to determine the back propagation stop layer of the current iteration based on the gradient and weight, and / or clock number of at least one network layer of the neural network model in the current iteration.

[0157] In a specific embodiment, the determination unit 602 is specifically used to determine the target network layer of the current iteration based on the clock number of at least one network layer of the neural network model; and determine the back propagation stop layer of the current iteration based on the gradient and weight of at least one network layer whose depth is less than the target network layer in the current iteration.

[0158] In a specific embodiment, the determination unit 602 is specifically used to determine the target network layer of the current iteration based on the number of rounds of the current iteration and the number of clocks of each network layer in the neural network model.

[0159] In a specific embodiment, the determination unit 602 is specifically used to determine the back propagation stop layer of the current iteration according to the gradient and weight of at least one network layer in the neural network model in the current iteration if the target network layer does not exist in the current iteration.

[0160] In a specific embodiment, the determination unit 602 is specifically used to determine the back propagation stop layer of the current iteration based on the ratio of the gradient to the weight of at least one network layer of the neural network model in the current iteration and a preset threshold.

[0161] In a specific embodiment, the determination unit 602 is specifically used to determine that the current network layer is a backpropagation stop layer if the ratio of the gradient normalization value to the weight normalization value in the current network layer of the neural network model is less than or equal to a preset threshold, wherein the gradient normalization value is the average of the absolute values ​​of multiple gradients of the current network layer, and the weight normalization value is the average of the absolute values ​​of multiple weights of the current network layer.

[0162] In a specific embodiment, the number of clocks of at least one network layer in a neural network model is set based on at least one of a first strategy and a second strategy; wherein the first strategy includes: the larger the number of network layers, the larger the set number of clocks; and the second strategy includes: for at least one network layer in the neural network model that is more important for training, a smaller number of clocks is set.

[0163] In a specific embodiment, the determination unit 602 is specifically configured to determine whether the layer is a stop layer according to multiple gradients and multiple weights of the layer if there is no target layer.

[0164] In a specific embodiment, the determination unit 602 is specifically used to determine whether the layer is a stop layer based on the gradient normalization value and weight normalization value of the layer; wherein the gradient normalization value is the average value of the absolute values ​​of multiple gradients of the layer, and the weight normalization value is the average value of the absolute values ​​of multiple weights of the layer.

[0165] In a specific embodiment, the preset threshold is adjusted according to a first preset strategy, and the first preset strategy includes an adjustment frequency of the preset threshold and / or an adjustment function of the preset threshold.

[0166] In a specific embodiment, the adjustment frequency is once per iteration round.

[0167] In a specific embodiment, the adjustment function is an increasing function, and the preset threshold is a function value of the increasing function; the preset threshold is adjusted according to a first preset strategy, including: the preset threshold gradually increases with the increase in the number of iterations.

[0168] In a specific embodiment, the updating unit 603 is specifically used to perform back propagation based on the network loss, calculate the gradients of multiple network layers in the neural network model until the back propagation stop layer is reached; based on the gradients of the multiple network layers, update the parameters of the multiple network layers, wherein the parameters of at least one network layer whose depth is less than the back propagation stop layer remain unchanged.

[0169] In a specific embodiment, the device also includes a preprocessing unit 604, which is used to perform at least one round of iterative training on the neural network model before performing a current iteration, wherein during at least one round of iterative training, parameters of all network layers of the neural network model are updated.

[0170] Each functional module of the above device 60 is used to implement Figure 2 or Figure 3 or Figure 4 or Figure 6 The method described in the embodiment, for specific content, please refer to Figure 2 or Figure 3 or Figure 4 or Figure 6 For the sake of brevity, the descriptions in the related contents of the embodiments are not repeated here.

[0171] refer to Fig. 9 , Fig. 9 A schematic diagram of a neural network model training device 700 provided in an embodiment of the present application includes at least: a processor 710, a communication interface 720 and a memory 730, wherein the processor 710, the communication interface 720 and the memory 730 are coupled via a bus 740.

[0172] The processor 710 calls the program code in the memory 730 to run Figure 8 The processing unit 601, the determination unit 602, the update unit 603, and the preprocessing unit 604 in the embodiment. In practical applications, the processor 710 may include one or more general-purpose processors, wherein the general-purpose processor may be any type of device capable of processing electronic instructions, including a central processing unit (CPU), a microprocessor, a microcontroller, a main processor, a controller, and an ASIC (Application Specific Integrated Circuit), etc. The processor 710 reads the program code stored in the memory 730, and cooperates with the communication interface 720 to execute part or all of the steps of the training method of the quantized neural network model in the above-mentioned embodiment of the present application.

[0173] The communication interface 720 may be a wired interface (e.g., an Ethernet interface) for communicating with other computing nodes or devices. When the communication interface 720 is a wired interface, the communication interface 720 may use a protocol family above TCP / IP, such as RAAS protocol, Remote Function Call (RFC) protocol, Simple Object Access Protocol (SOAP) protocol, Simple Network Management Protocol (SNMP) protocol, Common Object Request Broker Architecture (CORBA) protocol, and distributed protocols, etc.

[0174] The memory 730 may store program codes and program data. The program codes include the code of the processing unit 601, the code of the determination unit 602, the code of the update unit 603, and the code of the preprocessing unit 604. The program data includes: the number of iterations, the gradient value, the gradient normalization value, the weight normalization value, the threshold value, the number of clocks, etc. In practical applications, the memory 730 may include a volatile memory (Volatile Memory), such as a random access memory (Random Access Memory, RAM); the memory may also include a non-volatile memory (Non-Volatile Memory), such as a read-only memory (Read-Only Memory, ROM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, HDD) or a solid-state drive (Solid-State Drive, SSD). The memory may also include a combination of the above-mentioned types of memory.

[0175] refer to Fig.10 , Fig.10 A schematic diagram of an image processing device 80 provided in an embodiment of the present application, the device 80 includes:

[0176] The acquisition unit 801 is used to acquire images. The present application does not limit the image acquisition method and image type. It can be an image captured by a camera, an image copied by a mobile device, or an image acquired by other means; it can be a face image, an object image, a plant image, or other images, etc.

[0177] The image processing unit 802 is used to input the image into the trained neural network model for processing to obtain a processing result.

[0178] In a possible implementation, the neural network model training process may be: starting from the first round of iterative training, the neural network model is subjected to incomplete back propagation until the neural network model converges. In each round of iterative training, the sample image is first processed by the neural network model to obtain the network loss; then the back propagation stop layer of the current iteration is determined; finally, the back propagation and parameter update of multiple network layers in the neural network model are performed based on the network loss.

[0179] In a possible implementation, the neural network model training process may be: first, at least one round of iterative training is performed, wherein, in at least one round of iterative training, all network layers of the neural network model are back-propagated and parameter updates are performed. In each subsequent round of iterative training, the sample image is processed by forward propagation to obtain the network loss, and then the back-propagation stop layer of the current iteration is determined, and the multiple network layers in the neural network model are back-propagated and parameter updates are performed based on the network loss.

[0180] Among them, the method for determining the back propagation stop layer of the current iteration can be, for example, to determine the back propagation stop layer according to the number of rounds of the current iteration; or, to determine the back propagation stop layer of the current iteration according to the ratio of the gradient to the weight of at least one network layer of the neural network model in the current iteration and a preset threshold; or, to first determine the target network layer of the current iteration according to the number of clocks of at least one network layer of the neural network model, and then determine the back propagation stop layer of the current iteration based on the gradient and weight of at least one network layer having a depth less than the target network layer in the current iteration.

[0181] Each functional module of the above device 80 is used to implement Figure 7 The method described in the embodiment, for specific content, please refer to Figure 7 For the sake of brevity, the descriptions in the related contents of the embodiments are not repeated here.

[0182] refer to Fig.11 , Fig.11 A schematic diagram of an image processing device 900 provided in an embodiment of the present application at least includes: a processor 910, a communication interface 920 and a memory 930, wherein the processor 910, the communication interface 920 and the memory 930 are coupled via a bus 940.

[0183] The processor 910 calls the program code in the memory 930 to run Fig.10 The acquisition unit 801 and the image processing unit 802 in the image processing unit 801 are shown in FIG. 8 . In practical applications, the processor 910 may include one or more general-purpose processors, wherein the general-purpose processor may be any type of device capable of processing electronic instructions, including a central processing unit (CPU), a microprocessor, a microcontroller, a main processor, a controller, and an ASIC (Application Specific Integrated Circuit). The processor 910 reads the program code stored in the memory 930, and cooperates with the communication interface 920 to execute some or all of the steps of the image processing method in the above-mentioned embodiment of the present application.

[0184] The communication interface 920 may be a wired interface (e.g., an Ethernet interface) for communicating with other computing nodes or devices. When the communication interface 920 is a wired interface, the communication interface 920 may use a protocol family above TCP / IP, such as RAAS protocol, Remote Function Call (RFC) protocol, Simple Object Access Protocol (SOAP) protocol, Simple Network Management Protocol (SNMP) protocol, Common Object Request Broker Architecture (CORBA) protocol, and distributed protocols, etc.

[0185] The memory 930 may store program codes and program data. The program code package includes the code of the acquisition unit 801 and the code of the image processing unit 802. The program data includes: images, processing results, etc. In practical applications, the memory 930 may include a volatile memory (Volatile Memory), such as a random access memory (Random Access Memory, RAM); the memory may also include a non-volatile memory (Non-Volatile Memory), such as a read-only memory (Read-Only Memory, ROM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, HDD) or a solid-state drive (Solid-State Drive, SSD). The memory may also include a combination of the above-mentioned types of memory.

[0186] The embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by hardware (such as a processor, etc.) to implement part or all of the steps of any method executed by the neural network training device in the embodiment of the present application.

[0187] The embodiment of the present application also provides a computer program product. When the computer program product is read and executed by a computer, the training device of a neural network executes part or all of the steps of the training method of the neural network in the embodiment of the present application.

[0188] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. The computer program is executed by hardware (such as a processor, etc.) to implement part or all of the steps of any method executed by the image processing device in the embodiment of the present application.

[0189] The embodiment of the present application also provides a computer program product. When the computer program product is read and executed by a computer, an image processing device executes part or all of the steps of the image processing method in the embodiment of the present application.

[0190] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server, or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a storage disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state storage disk, SolidState Disk, SSD), etc. In the embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in one embodiment, reference may be made to the relevant description of other embodiments.

[0191] In the several embodiments provided in the present application, it should be understood that the disclosed device can also be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0192] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0193] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0194] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0195] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A training method for a neural network model, characterized in that: include: In the current iteration, the sample image is processed by the neural network model to obtain a network loss; Determining the target network layer of the current iteration according to the clock number of at least one network layer of the neural network model; The number of clocks of at least one network layer in the neural network model is set based on at least one of a first strategy and a second strategy; wherein the first strategy includes: the larger the number of network layers, the larger the number of clocks set; and the second strategy includes: for at least one network layer with higher training importance in the neural network model, a smaller number of clocks is set; Determine a back propagation stop layer of the current iteration based on the gradient and weight of at least one network layer having a depth less than the target network layer in the current iteration; The parameters of multiple network layers in the neural network model are updated based on the network loss, wherein the parameters of at least one network layer in the neural network model whose depth is less than the back-propagation stop layer remain unchanged.

2. The method according to claim 1, characterized in that The step of determining the target network layer of the current iteration according to the clock number of at least one network layer of the neural network model comprises: Based on the number of rounds of the current iteration and the number of clocks of each network layer in the neural network model, a target network layer of the current iteration is determined.

3. The method according to claim 1 or 2, characterized in that: The determining the back propagation stop layer of the current iteration based on the gradient and weight of at least one network layer with a depth smaller than the target network layer in the current iteration comprises: The back propagation stop layer of the current iteration is determined according to a ratio of a gradient to a weight in the current iteration of at least one network layer whose depth is less than that of the target network layer and a preset threshold.

4. The method according to claim 1 or 2, characterized in that: The updating of parameters of multiple network layers in the neural network model based on the network loss, wherein the parameters of at least one network layer in the neural network model having a depth less than the back propagation stop layer remain unchanged, comprises: Performing back propagation based on the network loss to obtain the gradients of multiple network layers in the neural network model until reaching the back propagation stop layer; Based on the gradients of the multiple network layers, parameters of the multiple network layers are updated, wherein parameters of at least one network layer in the neural network model whose depth is less than the back propagation stop layer remain unchanged.

5. The method according to claim 1 or 2, characterized in that: Before performing the current iteration, the process further includes: The neural network model is subjected to at least one round of iterative training, wherein during the at least one round of iterative training, parameters of all network layers of the neural network model are updated.

6. An image processing method, characterized in that: include: Get the image; The image is input into a trained neural network model for processing to obtain a processing result; wherein the neural network model is trained by the method described in any one of claims 1-5.

7. A training device for a neural network model, characterized in that: include: A processing unit, used for processing the sample image by the neural network model in a current iteration to obtain a network loss; A determination unit, configured to determine a target network layer of the current iteration according to a clock number of at least one network layer of the neural network model; The number of clocks of at least one network layer in the neural network model is set based on at least one of a first strategy and a second strategy; wherein the first strategy includes: the larger the number of network layers, the larger the number of clocks set; and the second strategy includes: for at least one network layer with higher training importance in the neural network model, a smaller number of clocks is set; Determine a back propagation stop layer of the current iteration based on the gradient and weight of at least one network layer having a depth less than the target network layer in the current iteration; An updating unit is used to update the parameters of multiple network layers in the neural network model based on the network loss, wherein the parameters of at least one network layer in the neural network model whose depth is less than the back propagation stop layer remain unchanged.

8. An image processing device, characterized in that: include: An acquisition unit, used for acquiring an image; An image processing unit, used for inputting the image into a trained neural network model for processing and obtaining a processing result; The neural network model is obtained by training through the method described in any one of claims 1-5.

9. A training device for a neural network model, characterized in that: The device includes a memory and a processor, the memory is used to store instructions, and the processor is used to call the instructions stored in the memory to execute the method according to any one of claims 1-5.

10. An image processing device, characterized in that: The device includes a memory and a processor, the memory is used to store instructions, and the processor is used to call the instructions stored in the memory to execute the method according to claim 6.

11. A computer-readable storage medium, characterized in that: The method comprises program instructions, and when the program instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

Images