Image processing model training method, device, apparatus, medium and program product

Abstract

The present disclosure relates to a method, device, equipment, medium and program product for training an image processing model. The image processing model comprises at least two network components. The method comprises: obtaining, by the image processing model, at least two pieces of prediction results corresponding to at least two training images in a training image group one by one; determining, according to the at least two pieces of prediction results and at least two pieces of label information corresponding to the at least two training images one by one, at least two pieces of gradient stability information corresponding to the at least two network components, wherein the gradient stability information corresponding to any network component of the at least two network components is used to represent the stability of the gradient of the network component; and training the image processing model according to the at least two pieces of gradient stability information.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision technology, and in particular to a training method, apparatus, electronic device, storage medium, and program product for an image processing model. Background Technology

[0002] Training image processing models for dense prediction relies heavily on large-scale datasets. However, training image processing models with large datasets often requires significant time. Since batch training linearly reduces the number of training iterations and thus significantly reduces training time, researchers have focused on batch training in recent years. For example, the LARS and LAMB algorithms based on batch training allow for training ResNet50 classification models on ImageNet in minutes. However, the LARS and LAMB algorithms have not achieved satisfactory results for image processing models used for dense prediction, thus limiting researchers' use of large-scale datasets. Summary of the Invention

[0003] This disclosure provides a training technique for an image processing model.

[0004] According to one aspect of this disclosure, a method for training an image processing model is provided, the image processing model comprising at least two network components, the method comprising:

[0005] The image processing model is used to obtain at least two prediction results that correspond one-to-one with at least two training images in the training image group.

[0006] Based on the at least two prediction results and at least two label information corresponding to the at least two training images, at least two gradient stability information corresponding to the at least two network components are determined, wherein the gradient stability information corresponding to any one of the at least two network components is used to represent the stability of the gradient of the network component.

[0007] The image processing model is trained based on the at least two gradient stability information items.

[0008] In this embodiment of the disclosure, the image processing model includes at least two network components. The image processing model obtains at least two prediction results corresponding one-to-one with at least two training images in the training image group. Based on the at least two prediction results and at least two label information corresponding one-to-one with the at least two training images, at least two gradient stability information corresponding one-to-one with the at least two network components is determined. The gradient stability information corresponding to any one of the at least two network components is used to represent the stability of the gradient of the network component. The image processing model is trained based on the at least two gradient stability information, thereby improving the stability and generalization ability of the image processing model.

[0009] In one possible implementation, the gradient stability information includes the variance of the gradient;

[0010] The step of determining at least two gradient stability information corresponding to at least two network components based on the at least two prediction results and at least two label information corresponding to at least two training images includes: determining at least two first gradients corresponding to at least two training images based on the at least two prediction results and at least two label information corresponding to at least two training images; and for any one of the at least two network components, determining the variance of the gradient corresponding to the network component based on the at least two first gradients.

[0011] The step of training the image processing model based on the at least two gradient stability information includes: training the image processing model based on the variance of at least two gradients that correspond one-to-one with the at least two network components.

[0012] In this implementation, at least two first gradients corresponding to the at least two training images are determined based on the at least two prediction results and at least two label information corresponding to the at least two training images. For any of the at least two network components, the variance of the gradient corresponding to the network component is determined based on the at least two first gradients. The image processing model is then trained based on the variance of the at least two gradients corresponding to the at least two network components. This allows for an accurate measurement of the stability of the gradients of the network components through the variance of the gradients corresponding to the network components, thereby improving the training effect of the image processing model.

[0013] In one possible implementation, determining the variance of the gradient corresponding to the network component based on the at least two first gradients includes:

[0014] The training image group is divided into a first training image subgroup and a second training image subgroup, wherein the first training image subgroup and the second training image subgroup each include at least one training image;

[0015] Based on the first gradient corresponding to the training images in the first training image subgroup, determine the second gradient corresponding to the first training image subgroup;

[0016] Based on the first gradient corresponding to the training images in the second training image subgroup, determine the third gradient corresponding to the first training image subgroup;

[0017] Backpropagation is performed based on the second gradient to determine the fourth gradient corresponding to the network component;

[0018] Backpropagation is performed based on the third gradient to determine the fifth gradient corresponding to the network component.

[0019] Based on the fourth and fifth gradients, the variance of the gradient corresponding to the network component is determined.

[0020] In this implementation, the training image group is divided into a first training image subgroup and a second training image subgroup. A second gradient corresponding to the first training image subgroup is determined based on the first gradient corresponding to the training images in the first training image subgroup. A third gradient corresponding to the first training image subgroup is determined based on the first gradient corresponding to the training images in the second training image subgroup. Backpropagation is performed based on the second gradient to determine the fourth gradient corresponding to the network component. Backpropagation is performed based on the third gradient to determine the fifth gradient corresponding to the network component. The variance of the gradients corresponding to the network component is determined based on the fourth and fifth gradients. This allows for faster calculation of the variance of the gradients corresponding to the network components in the image processing model, thereby further improving the training speed of the image processing model and saving computational overhead.

[0021] In one possible implementation, training the image processing model based on the variance of at least two gradients corresponding one-to-one with the at least two network components includes:

[0022] Based on the variance of at least two gradients corresponding to at least two network components, at least two first target learning rates corresponding to at least two network components are determined; wherein, the first target learning rate corresponding to any one of the at least two network components is negatively correlated with the variance of the gradient corresponding to the network component.

[0023] The image processing model is trained according to the at least two first objective learning rates.

[0024] In this implementation, at least two first target learning rates corresponding to the at least two network components are determined based on the variance of at least two gradients corresponding to each of the at least two network components. The first target learning rate corresponding to any one of the at least two network components is negatively correlated with the variance of the gradient corresponding to that network component. The image processing model is trained based on these at least two first target learning rates. Thus, during the training process of the image processing model, the learning rate of each network component is dynamically updated based on the variance of the gradients corresponding to each network component in the image processing model. This allows the variance of the gradients corresponding to each network component to tend towards consistency through parameter updates.

[0025] In one possible implementation, the at least two network components include a target network component and at least one non-target network component;

[0026] The step of determining at least two first target learning rates corresponding to at least two network components based on the variance of at least two gradients corresponding to each of the at least two network components includes:

[0027] For any non-target network component, a first target learning rate is determined based on the variance of the gradient corresponding to the target network component, the variance of the gradient corresponding to the non-target network component, and a preset initial learning rate.

[0028] For the target network component, a first target learning rate is determined based on the preset initial learning rate.

[0029] In this implementation, for any non-target network component, a first target learning rate is determined based on the variance of the gradient corresponding to the target network component, the variance of the gradient corresponding to the non-target network component, and a preset initial learning rate. For the target network component, a first target learning rate is determined based on the preset initial learning rate. This allows for dynamic adjustment of the learning rate for non-target network components, enabling parameter updates to make the variance of the gradient corresponding to the non-target network component more consistent with the variance of the gradient corresponding to the target network component.

[0030] In one possible implementation, determining the first target learning rate corresponding to the non-target network component based on the variance of the gradient corresponding to the target network component, the variance of the gradient corresponding to the non-target network component, and a preset initial learning rate includes:

[0031] Determine a first ratio between the variance of the gradient corresponding to the target network component and the variance of the gradient corresponding to the non-target network component;

[0032] Based on the first ratio and the preset initial learning rate, the first target learning rate corresponding to the non-target network component is determined.

[0033] In this implementation, a first ratio is determined between the variance of the gradient corresponding to the target network component and the variance of the gradient corresponding to the non-target network component. Based on the first ratio and a preset initial learning rate, a first target learning rate is determined for the non-target network component. This allows for the determination of a more suitable learning rate for each network component in the image processing model, thereby improving the convergence speed and generalization ability of the image processing model.

[0034] In one possible implementation, determining the first target learning rate corresponding to the non-target network component based on the first ratio and a preset initial learning rate includes:

[0035] Obtain the second ratio of the second target learning rate corresponding to the non-target network component to the preset initial learning rate in the previous training round;

[0036] Based on the first ratio, the second ratio, and the preset initial learning rate, a first target learning rate is determined for the non-target network component.

[0037] In this implementation, by obtaining the second ratio of the second target learning rate corresponding to the non-target network component to the preset initial learning rate in the previous training round, and determining the first target learning rate corresponding to the non-target network component based on the first ratio, the second ratio and the preset initial learning rate, the problem of unstable variance of the gradient corresponding to the network component can be addressed, thereby improving the training effect of the image processing model.

[0038] In one possible implementation, before obtaining at least two prediction results corresponding one-to-one with at least two training images in the training image set through the image processing model, the method further includes:

[0039] From the training image set, a training image group is randomly determined, wherein the number of training images in the training image set is greater than the number of training images in the training image group.

[0040] In this implementation, by randomly selecting a small batch of training images from the training image set for the t-th iteration of the image processing model, the training efficiency of the image processing model can be improved.

[0041] According to one aspect of this disclosure, a training apparatus for an image processing model is provided, the image processing model comprising at least two network components, the apparatus comprising:

[0042] The prediction module is used to obtain at least two prediction results that correspond one-to-one with at least two training images in the training image group through the image processing model;

[0043] The first determining module is used to determine at least two gradient stability information corresponding to at least two network components based on the at least two prediction results and at least two label information corresponding to the at least two training images, wherein the gradient stability information corresponding to any one of the at least two network components is used to represent the stability of the gradient of the network component.

[0044] The training module is used to train the image processing model based on the at least two gradient stability information items.

[0045] In one possible implementation, the gradient stability information includes the variance of the gradient;

[0046] The first determining module is configured to: determine at least two first gradients corresponding to the at least two training images based on the at least two prediction results and at least two label information corresponding to the at least two training images; and for any one of the at least two network components, determine the variance of the gradient corresponding to the network component based on the at least two first gradients.

[0047] The training module is used to train the image processing model based on the variance of at least two gradients that correspond one-to-one with the at least two network components.

[0048] In one possible implementation, the first determining module is used to:

[0049] The training image group is divided into a first training image subgroup and a second training image subgroup, wherein the first training image subgroup and the second training image subgroup each include at least one training image;

[0050] Based on the first gradient corresponding to the training images in the first training image subgroup, determine the second gradient corresponding to the first training image subgroup;

[0051] Based on the first gradient corresponding to the training images in the second training image subgroup, determine the third gradient corresponding to the first training image subgroup;

[0052] Backpropagation is performed based on the second gradient to determine the fourth gradient corresponding to the network component;

[0053] Backpropagation is performed based on the third gradient to determine the fifth gradient corresponding to the network component.

[0054] Based on the fourth and fifth gradients, the variance of the gradient corresponding to the network component is determined.

[0055] In one possible implementation, the training module is used for:

[0056] Based on the variance of at least two gradients corresponding to at least two network components, at least two first target learning rates corresponding to at least two network components are determined; wherein, the first target learning rate corresponding to any one of the at least two network components is negatively correlated with the variance of the gradient corresponding to the network component.

[0057] The image processing model is trained according to the at least two first objective learning rates.

[0058] In one possible implementation, the at least two network components include a target network component and at least one non-target network component;

[0059] The training module is used for:

[0060] For any non-target network component, a first target learning rate is determined based on the variance of the gradient corresponding to the target network component, the variance of the gradient corresponding to the non-target network component, and a preset initial learning rate.

[0061] For the target network component, a first target learning rate is determined based on the preset initial learning rate.

[0062] In one possible implementation, the training module is used for:

[0063] Determine a first ratio between the variance of the gradient corresponding to the target network component and the variance of the gradient corresponding to the non-target network component;

[0064] Based on the first ratio and the preset initial learning rate, the first target learning rate corresponding to the non-target network component is determined.

[0065] In one possible implementation, the training module is used for:

[0066] Obtain the second ratio of the second target learning rate corresponding to the non-target network component to the preset initial learning rate in the previous training round;

[0067] Based on the first ratio, the second ratio, and the preset initial learning rate, a first target learning rate is determined for the non-target network component.

[0068] In one possible implementation, the device further includes:

[0069] The second determining module is used to randomly determine a training image group from the training image set, wherein the number of training images in the training image set is greater than the number of training images in the training image group.

[0070] According to one aspect of this disclosure, an electronic device is provided, comprising: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the executable instructions stored in the memory to perform the method described above.

[0071] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the above-described method.

[0072] According to one aspect of this disclosure, a computer program product is provided, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in an electronic device, a processor in the electronic device performs the above-described method.

[0073] In this embodiment of the disclosure, the image processing model includes at least two network components. The image processing model obtains at least two prediction results corresponding one-to-one with at least two training images in the training image group. Based on the at least two prediction results and at least two label information corresponding one-to-one with the at least two training images, at least two gradient stability information corresponding one-to-one with the at least two network components is determined. The gradient stability information corresponding to any one of the at least two network components is used to represent the stability of the gradient of the network component. The image processing model is trained based on the at least two gradient stability information, thereby improving the stability and generalization ability of the image processing model.

[0074] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0075] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0076] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0077] Figure 1 A flowchart illustrating the training method of the image processing model provided in the embodiments of this disclosure is shown.

[0078] Figure 2A block diagram of a training apparatus for an image processing model provided in an embodiment of this disclosure is shown.

[0079] Figure 3 A block diagram of an electronic device 1900 provided in an embodiment of this disclosure is shown. Detailed Implementation

[0080] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0081] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0082] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0083] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0084] This disclosure provides a training method, apparatus, electronic device, storage medium, and program product for an image processing model. The image processing model includes at least two network components. The model obtains at least two prediction results corresponding to at least two training images in a training image set. Based on the at least two prediction results and at least two label information corresponding to the at least two training images, at least two gradient stability information items corresponding to the at least two network components are determined. The gradient stability information corresponding to any one of the at least two network components is used to represent the stability of the gradient of that network component. The image processing model is trained based on the at least two gradient stability information items, thereby improving the stability and generalization ability of the image processing model.

[0085] Related technologies struggle to achieve satisfactory training results when training image processing models using large-scale training image sets. However, by employing the image processing model training method provided in this disclosure, even in application scenarios requiring large-scale training image sets, it is possible to train image processing models with high stability and generalization ability.

[0086] The training method of the image processing model provided in the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0087] Figure 1 A flowchart illustrating a training method for an image processing model provided in an embodiment of this disclosure is shown. In one possible implementation, the entity executing the image processing model training method may be an image processing model training device. For example, the image processing model training method may be executed by a terminal device, a server, or other electronic equipment. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, or wearable device, etc. In some possible implementations, the image processing model training method may be implemented by a processor calling computer-readable instructions stored in memory. Figure 1 As shown, the training method of the image processing model includes steps S11 to S13.

[0088] In step S11, at least two prediction results are obtained through the image processing model, which correspond one-to-one with at least two training images in the training image group.

[0089] In step S12, based on the at least two prediction results and at least two label information corresponding to the at least two training images, at least two gradient stability information corresponding to the at least two network components are determined, wherein the gradient stability information corresponding to any one of the at least two network components is used to represent the stability of the gradient of the network component.

[0090] In step S13, the image processing model is trained based on the at least two gradient stability information items.

[0091] The image processing model in this embodiment can be any model used for image processing. In one possible implementation, the image processing model can be used for dense prediction. For example, the image processing model can be used for image processing tasks such as object detection, semantic segmentation, instance segmentation, and panoptic segmentation.

[0092] In one possible implementation, the image processing model can be a deep learning-based image processing model. The training method for the image processing model provided in this disclosure can be applied to image processing models with various network structures. For example, the image processing model can employ various CNN (Convolutional Neural Network) network structures, various Transformer network structures, etc. The training method for the image processing model provided in this disclosure is insensitive to hyperparameters and can achieve good results without extensive parameter tuning.

[0093] In this embodiment of the disclosure, the network components of the image processing model can represent the components constituting the image processing model. Network components can also be called modules, network modules, components, or groups, etc., and are not limited thereto. The image processing model includes at least two network components. The at least two network components can include network components of different types or network components of the same type, and are not limited thereto. For example, the at least two network components can include at least two of the following: a backbone network, a feature pyramid network, a region candidate network, a detector, and a segmenter. In one example, the parameters of the image processing model can be represented by w, where w = {w (i) |i∈[1,h]}, where h represents the number of network components in the image processing model, h is an integer greater than or equal to 2, w (i) These can represent the learnable parameters of the i-th network component in the image processing model.

[0094] In this embodiment of the disclosure, the training image group can be a small batch, and the training image group includes at least two training images. In one example, the training image group can adopt S... t It indicates that, among them, S t The training image includes b images, where b is an integer greater than or equal to 2. In this embodiment of the disclosure, different sets of training images can be used for training in different iterations during the training process of the image processing model.

[0095] In one possible implementation, before obtaining at least two prediction results corresponding one-to-one with at least two training images in the training image set through the image processing model, the method further includes: randomly determining a training image set from the training image set, wherein the number of training images in the training image set is greater than the number of training images in the training image set.

[0096] In this implementation, the training image set can be adopted. Let x represent the number of training images in the training image set S, where n is an integer greater than or equal to 3.k Let y represent the k-th training image in the training image set S. k x represents k The label information. In this implementation, during the t-th iteration of the image processing model, a small batch, i.e., training image group S, can be randomly sampled from the training image set S. t That is, in the t-th iteration of the image processing model, the training image set S can be used. t The image processing model is trained.

[0097] In this implementation, by randomly selecting a small batch of training images from the training image set for the t-th iteration of the image processing model, the training efficiency of the image processing model can be improved.

[0098] In another possible implementation, during the t-th iteration of the image processing model, a training image group can be determined from the training image set according to a preset rule for selecting a mini-batch. That is, in this implementation, the rule for selecting the training image group does not have to be random. For example, the training image set can be divided into 10 training image groups on average. In the first iteration, the first training image group is used to train the image processing model; in the second iteration, the second training image group is used; and so on, in the eleventh iteration, the first training image group is used to train the image processing model, and so on.

[0099] In another possible implementation, the training image set can be used as a training image group. That is, in this implementation, the entire training image set can be used for training in each iteration of the image processing model.

[0100] In this embodiment of the disclosure, for any training image in the training image group, inputting the training image into the image processing model allows the image processing model to obtain a prediction result corresponding to the training image. Inputting each training image in the training image group into the image processing model respectively yields a prediction result corresponding one-to-one with each training image in the training image group.

[0101] In this embodiment of the disclosure, for any one of the at least two network components, gradient stability information corresponding to the network component can be determined based on the at least two prediction results and at least two label information corresponding one-to-one with the at least two training images. Thus, the gradient stability information corresponding to each of the at least two network components can be determined. The gradient stability information corresponding to any network component can be any information capable of representing the stability of the gradient of the network component. For example, the gradient stability information corresponding to the network component can be represented by indicators that can measure the degree of dispersion, such as variance, standard deviation, mean square error, and root mean square error, and is not limited here.

[0102] In one possible implementation, the gradient stability information includes the variance of the gradient; determining at least two gradient stability information corresponding to at least two network components based on the at least two prediction results and at least two label information corresponding to the at least two training images includes: determining at least two first gradients corresponding to the at least two training images based on the at least two prediction results and at least two label information corresponding to the at least two training images; for any of the at least two network components, determining the variance of the gradient corresponding to the network component based on the at least two first gradients; training the image processing model based on the at least two gradient stability information includes: training the image processing model based on the variance of the at least two gradients corresponding to the at least two network components.

[0103] In this implementation, for any training image in the training image group, a first gradient corresponding to the training image can be obtained based on the prediction result corresponding to the training image and the label information corresponding to the training image. Here, the first gradient can represent the gradient corresponding to the training image.

[0104] In one example, in the t-th iteration of the image processing model, the training image set S can be determined using Equation 1. t The j-th training image x j The corresponding loss function value:

[0105]

[0106] Among them, w t Let L represent the parameters of the image processing model, L() represent the loss function corresponding to the image processing model, and y j Represents the training image x j The corresponding tag information.

[0107] calculate For wt The gradient of the training image x can be obtained. j The corresponding first gradient r j,t Training image group S t The first gradients corresponding to the b training images can be denoted as r. 1,t ,...,r j,t ,...,r b,t .

[0108] In this implementation, for any one of the at least two network components, the variance of the gradient corresponding to the network component can be determined based on the first gradient corresponding to each training image in the training image group. In this implementation, the gradient stability information corresponding to the network component is represented by the variance of the gradient corresponding to the network component. The smaller the variance of the gradient corresponding to the network component, the higher the stability of the gradient of the network component; the larger the variance of the gradient corresponding to the network component, the lower the stability of the gradient of the network component.

[0109] In this implementation, at least two first gradients corresponding to the at least two training images are determined based on the at least two prediction results and at least two label information corresponding to the at least two training images. For any of the at least two network components, the variance of the gradient corresponding to the network component is determined based on the at least two first gradients. The image processing model is then trained based on the variance of the at least two gradients corresponding to the at least two network components. This allows for an accurate measurement of the stability of the gradients of the network components through the variance of the gradients corresponding to the network components, thereby improving the training effect of the image processing model.

[0110] As an example of this implementation, the first gradient corresponding to the training image in the training image group can be determined by at least two nodes of the distributed training system, thereby further improving the training speed of the image processing model.

[0111] As an example of this implementation, determining the variance of the gradient corresponding to the network component based on the at least two first gradients includes: dividing the training image group into a first training image subgroup and a second training image subgroup, wherein the first training image subgroup and the second training image subgroup each include at least one training image; determining a second gradient corresponding to the first training image subgroup based on the first gradient corresponding to the training image in the first training image subgroup; determining a third gradient corresponding to the first training image subgroup based on the first gradient corresponding to the training image in the second training image subgroup; performing backpropagation based on the second gradient to determine a fourth gradient corresponding to the network component; performing backpropagation based on the third gradient to determine a fifth gradient corresponding to the network component; and determining the variance of the gradient corresponding to the network component based on the fourth gradient and the fifth gradient.

[0112] In this example, the first training image subgroup and the second training image subgroup are each subsets of the training image group, and the union of the first and second training image subgroups constitutes the training image group. The first and second training image subgroups have no overlap. The number of training images in the first and second training image subgroups can be the same or different. In one example, the training image group can be divided equally into the first and second training image subgroups.

[0113] In one example, the training image group S t The corresponding gradient g t Equation 2 can be used to determine:

[0114]

[0115] Where, r j,t S represents the training image group t The training image x j The corresponding first gradient, b represents the training image group S t The number of training images.

[0116] In this example, the second gradient can represent the gradient corresponding to the first training image subgroup. The second gradient corresponding to the first training image subgroup can be determined based on the average, median, and sum of the first gradients corresponding to each training image in the first training image subgroup. In one example, Equation 3 can be used to determine the second gradient G corresponding to the first training image subgroup. t,1 :

[0117]

[0118] In this example, the third gradient can represent the gradient corresponding to the second training image subgroup. The third gradient corresponding to the second training image subgroup can be determined based on the average, median, and sum of the first gradients corresponding to each training image in the second training image subgroup. In one example, Equation 4 can be used to determine the third gradient G corresponding to the second training image subgroup. t,2 :

[0119]

[0120] Based on the second gradient G corresponding to the first training image subgroup t,1 By performing backpropagation, the fourth gradient corresponding to the i-th network component in the image processing model can be determined. The fourth gradient can represent the gradient corresponding to the network component determined based on the first training image subgroup; that is, the fourth gradient can represent the gradient of the network component corresponding to the first training image subgroup. The third gradient G is based on the second training image subgroup. t,2 By performing backpropagation, the fifth gradient corresponding to the i-th network component in the image processing model can be determined. Here, the fifth gradient represents the gradient corresponding to the network component determined based on the second training image subgroup; that is, the fifth gradient can represent the gradient of the network component corresponding to the second training image subgroup. For the i-th network component in the image processing model, based on the fourth gradient corresponding to the network component... The fifth gradient corresponding to the network component The variance of the gradient corresponding to the network component can be determined.

[0121] In this example, by dividing the training image group into a first training image subgroup and a second training image subgroup, a second gradient corresponding to the first training image subgroup is determined based on the first gradient corresponding to the training images in the first training image subgroup, and a third gradient corresponding to the first training image subgroup is determined based on the first gradient corresponding to the training images in the second training image subgroup. Backpropagation is performed based on the second gradient to determine the fourth gradient corresponding to the network component, and backpropagation is performed based on the third gradient to determine the fifth gradient corresponding to the network component. The variance of the gradients corresponding to the network component is determined based on the fourth gradient and the fifth gradient. This allows for faster calculation of the variance of the gradients corresponding to the network components in the image processing model, thereby further improving the training speed of the image processing model and saving computational overhead.

[0122] In one example, determining the variance of the gradient corresponding to the network component based on the fourth gradient and the fifth gradient corresponding to the network component includes: determining the variance of the gradient corresponding to the network component based on the cosine similarity between the fourth gradient and the fifth gradient corresponding to the network component.

[0123] In one example, the fourth gradient corresponding to the i-th network component in the image processing model With the fifth gradient The cosine similarity between them can be denoted as: In this example, the variance of the gradient corresponding to the network component is negatively correlated with the cosine similarity.

[0124] In one example, Equation 5 can be used to determine the variance of the gradient corresponding to the i-th network component in the image processing model.

[0125]

[0126] Where, η t This indicates the preset initial learning rate.

[0127] In another example, Equation 6 can be used to determine the variance of the gradient corresponding to the i-th network component in the image processing model.

[0128]

[0129] As another example of this implementation, when dividing the training image group into subgroups, it can be divided into three or more training image subgroups. The variance of at least two gradients corresponding to at least two network components can be determined based on the gradients corresponding to at least three training image subgroups. For example, the training image group can be divided into K training image subgroups, where K is an integer greater than or equal to 3. A method similar to that described above for determining the gradients corresponding to each training image subgroup can be used to determine the gradient corresponding to each training image subgroup. Backpropagation can be performed based on the gradients corresponding to each training image subgroup to obtain K gradients corresponding to each network component. For any network component, the average value of the K gradients corresponding to that network component can be determined, and the variance of the gradient corresponding to that network component can be determined based on the difference between the K gradients and the average value.

[0130] As an example of this implementation, training the image processing model based on the variance of at least two gradients corresponding to the at least two network components includes: determining at least two first target learning rates corresponding to the at least two network components based on the variance of the at least two gradients corresponding to the at least two network components; wherein the first target learning rate corresponding to any one of the at least two network components is negatively correlated with the variance of the gradient corresponding to the network component; and training the image processing model based on the at least two first target learning rates.

[0131] In this example, the first target learning rate corresponding to any one of the at least two network components can represent the target learning rate corresponding to the network component in this iteration.

[0132] In one example, the first target learning rate for each network component in the image processing model can be determined independently. In this example, the first target learning rates for network components of the same type in the image processing model can be different or the same.

[0133] In another example, the first target learning rate for network components of the same type in the image processing model can be determined simultaneously. In this example, the first target learning rate for network components of the same type in the image processing model can be the same.

[0134] In this example, the first target learning rate corresponding to any one of the at least two network components is negatively correlated with the variance of the gradient corresponding to that network component. That is, the larger the variance of the gradient corresponding to the network component, the smaller the first target learning rate corresponding to that network component; the smaller the variance of the gradient corresponding to the network component, the larger the first target learning rate corresponding to that network component.

[0135] In this example, for any network component in the image processing model, the parameters of the network component are updated according to the first target learning rate corresponding to that network component. In different iterations, the target learning rate for the same network component may be different or the same.

[0136] In this example, at least two first target learning rates corresponding to the at least two network components are determined based on the variance of at least two gradients corresponding to each of the at least two network components. The first target learning rate corresponding to any one of the at least two network components is negatively correlated with the variance of the gradient corresponding to that network component. The image processing model is trained based on these at least two first target learning rates. Thus, during the training process of the image processing model, the learning rate of each network component is dynamically updated based on the variance of the gradients corresponding to each network component in the image processing model. This allows the variance of the gradients corresponding to each network component to tend towards consistency through parameter updates.

[0137] In one example, the at least two network components include a target network component and at least one non-target network component; determining at least two first target learning rates corresponding to the at least two network components based on the variance of at least two gradients corresponding to the at least two network components includes: for any non-target network component, determining a first target learning rate corresponding to the non-target network component based on the variance of the gradient corresponding to the target network component, the variance of the gradient corresponding to the non-target network component, and a preset initial learning rate; for the target network component, determining a first target learning rate corresponding to the target network component based on the preset initial learning rate.

[0138] In this example, one network component in the image processing model can be designated as the target network component. Furthermore, all network components in the image processing model other than the target network component can be designated as non-target network components.

[0139] In this example, a preset initial learning rate can be used as the first target learning rate for the target network component. For any non-target network component, if the variance of the gradient corresponding to the non-target network component is greater than the variance of the gradient corresponding to the target network component, the first target learning rate for the non-target network component can be set to be less than the preset initial learning rate; if the variance of the gradient corresponding to the non-target network component is equal to the variance of the gradient corresponding to the target network component, the first target learning rate for the non-target network component can be set to be equal to the preset initial learning rate; if the variance of the gradient corresponding to the non-target network component is less than the variance of the gradient corresponding to the target network component, the first target learning rate for the non-target network component can be set to be greater than the preset initial learning rate.

[0140] In this example, for any non-target network component, a first target learning rate is determined based on the variance of the gradient corresponding to the target network component, the variance of the gradient corresponding to the non-target network component, and a preset initial learning rate. For the target network component, a first target learning rate is determined based on the preset initial learning rate. This allows for dynamic adjustment of the learning rate for the non-target network component, enabling parameter updates to make the variance of the gradient corresponding to the non-target network component more consistent with the variance of the gradient corresponding to the target network component.

[0141] In one example, the backbone network of the image processing model can be identified as the target network component, and the other network components in the image processing model besides the backbone network can be identified as non-target network components. Based on this example, by adjusting the learning rates of the other network components, the variance of the gradients of the other network components can be made to converge with the variance of the gradients of the backbone network.

[0142] In another example, determining at least two first target learning rates corresponding to at least two network components based on the variances of at least two gradients corresponding to each of the at least two network components includes: determining at least two first target learning rates corresponding to each of the at least two network components based on a preset correspondence between the gradient variance and the first target learning rate, and the variances of the at least two gradients. In this example, the correspondence between the gradient variance and the first target learning rate can be preset. For any of the at least two network components, the first target learning rate corresponding to the network component can be determined based on the variance of the gradient corresponding to the network component and the correspondence. In this example, when determining the first target learning rate corresponding to the network components in the image processing model, it is not necessary to divide the network components of the image processing model into target network components and non-target network components.

[0143] In one example, determining the first target learning rate corresponding to the non-target network component based on the variance of the gradient corresponding to the target network component, the variance of the gradient corresponding to the non-target network component, and a preset initial learning rate includes: determining a first ratio of the variance of the gradient corresponding to the target network component to the variance of the gradient corresponding to the non-target network component; and determining the first target learning rate corresponding to the non-target network component based on the first ratio and the preset initial learning rate.

[0144] In this example, for any non-target network component, the first ratio can represent the ratio between the variance of the gradient corresponding to the target network component and the variance of the gradient corresponding to the non-target network component during this training round. In this example, the first target learning rate corresponding to the non-target network component is positively correlated with the first ratio. The larger the first ratio, the larger the first target learning rate corresponding to the non-target network component; the smaller the first ratio, the smaller the first target learning rate corresponding to the non-target network component.

[0145] In this example, by determining a first ratio of the variance of the gradient corresponding to the target network component to the variance of the gradient corresponding to the non-target network component, and by determining a first target learning rate corresponding to the non-target network component based on the first ratio and a preset initial learning rate, a more suitable learning rate can be determined for each network component in the image processing model, thereby improving the convergence speed and generalization ability of the image processing model.

[0146] In one example, Equations 7 and 8 can be used to determine the first target learning rate corresponding to the i-th network component in the image processing model.

[0147]

[0148]

[0149] Where, η t This represents the preset initial learning rate. This represents the variance of the gradient corresponding to the target network component. This represents the variance of the gradient corresponding to the i-th network component.

[0150] In one example, determining the first target learning rate corresponding to the non-target network component based on the first ratio and a preset initial learning rate includes: obtaining a second ratio between the second target learning rate corresponding to the non-target network component and the preset initial learning rate in the previous training round; and determining the first target learning rate corresponding to the non-target network component based on the first ratio, the second ratio, and the preset initial learning rate. The second target learning rate corresponding to the non-target network component can represent the target learning rate corresponding to the non-target network component in the previous training round. The second ratio can represent the ratio between the second target learning rate corresponding to the non-target network component and the preset initial learning rate.

[0151] In one example, Equations 9 and 10 can be used to determine the first target learning rate corresponding to the i-th network component in the image processing model.

[0152]

[0153]

[0154] Where, η t This indicates the preset initial learning rate; This represents the variance of the gradient corresponding to the target network component. Let represent the variance of the gradient corresponding to the i-th network component; α represents the momentum factor, α∈[0,1); This represents the ratio of the target learning rate to the preset initial learning rate for the i-th network component in the previous training round.

[0155] In this example, by obtaining the second ratio of the second target learning rate corresponding to the non-target network component to the preset initial learning rate in the previous training round, and determining the first target learning rate corresponding to the non-target network component based on the first ratio, the second ratio and the preset initial learning rate, the problem of unstable variance of the gradient corresponding to the network component can be addressed, thereby improving the training effect of the image processing model.

[0156] In one example, the target learning rate for each network component in the image processing model can be updated every τ iterations, where τ can be an integer greater than or equal to 1.

[0157] In another example, determining the first target learning rate for the non-target network component based on the variance of the gradient corresponding to the target network component, the variance of the gradient corresponding to the non-target network component, and a preset initial learning rate includes: determining a third ratio between the variance of the gradient corresponding to the non-target network component and the variance of the gradient corresponding to the target network component; and determining the first target learning rate for the non-target network component based on the third ratio and the preset initial learning rate. In this example, the first target learning rate for the non-target network component is negatively correlated with the third ratio. The larger the third ratio, the smaller the first target learning rate for the non-target network component; the smaller the third ratio, the larger the first target learning rate for the non-target network component.

[0158] As another example of this implementation, training the image processing model based on the variance of the at least two gradients includes: determining, from the at least two network components, the network components that will have their parameters updated in the current iteration and the network components that will not have their parameters updated in the current iteration, based on the variance of the at least two gradients that correspond one-to-one with the at least two network components, and updating the parameters of the network components that will have their parameters updated in the current iteration.

[0159] In another possible implementation, determining at least two gradient stability information corresponding to at least two network components based on the at least two prediction results and at least two label information corresponding to the at least two training images includes: determining at least two first gradients corresponding to the at least two training images based on the at least two prediction results and at least two label information corresponding to the at least two training images; determining the standard deviation of at least two gradients corresponding to the at least two network components based on the at least two first gradients; and training the image processing model based on the at least two gradient stability information includes: training the image processing model based on the standard deviation of the at least two gradients.

[0160] The training method for the image processing model provided in this disclosure can be adapted to different optimizers, such as SGD (Stochastic Gradient Descent), AdamW, etc. In one possible implementation, Stochastic Gradient Descent (SGD) can be used to update the parameters of the i-th network component in the image processing model. in, This represents the gradient corresponding to the i-th network component.

[0161] The image processing model training method provided in this disclosure can be applied to computer vision, detection and segmentation (e.g., object detection, medical image segmentation), batch optimization, dense prediction and other application scenarios, and is not limited thereto.

[0162] The training method of the image processing model provided in this disclosure embodiment is illustrated below through a specific application scenario. In this application scenario, during the t-th iteration of the image processing model, a small batch, i.e., the training image group S, can be randomly sampled from the training image set S. t The training image group S t Each training image in the dataset is input into the image processing model to obtain a result corresponding to the training image group S. t The prediction results correspond one-to-one with each training image in the dataset. For the training image group S... t The training image x j Based on the training image x j The corresponding prediction results, and the training image x j The corresponding label information can be used to obtain the training image x. j The corresponding first gradient r j,t The training image group S can be... tThe training images are divided into a first training image subgroup and a second training image subgroup. Based on the first gradient corresponding to each training image in the first training image subgroup, the second gradient G corresponding to the first training image subgroup can be determined. t,1 Based on the first gradient corresponding to each training image in the second training image subgroup, the third gradient G corresponding to the second training image subgroup can be determined. t,2 Based on the second gradient G corresponding to the first training image subgroup t,1 By performing backpropagation, the fourth gradient corresponding to the i-th network component in the image processing model can be determined. Based on the third gradient G corresponding to the second training image subgroup t,2 By performing backpropagation, the fifth gradient corresponding to the i-th network component in the image processing model can be determined. Furthermore, the variance of the gradient corresponding to the i-th network component in the image processing model can be determined using Equation 5 above. The first target learning rate corresponding to the i-th network component in the image processing model can be determined using Equations 9 and 10 above. After determining the first target learning rate for each network component in the image processing model, the parameters of each network component can be updated according to the first target learning rate.

[0163] The image processing model training method provided in this application scenario can be quickly adapted to the business needs of different scenarios, which can accelerate the production efficiency of image processing models, speed up the launch of downstream businesses, and reduce production time costs.

[0164] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0165] In addition, this disclosure also provides a training device for an image processing model, an electronic device, a computer-readable storage medium, and a computer program product. All of the above can be used to implement any of the image processing model training methods provided in this disclosure. The corresponding technical solutions and effects can be found in the relevant descriptions in the method section, and will not be repeated here.

[0166] Figure 2 A block diagram of a training apparatus for an image processing model provided in an embodiment of this disclosure is shown. The image processing model includes at least two network components. For example... Figure 2 As shown, the training device for the image processing model includes:

[0167] Prediction module 21 is used to obtain at least two prediction results that correspond one-to-one with at least two training images in the training image group through an image processing model;

[0168] The first determining module 22 is used to determine at least two gradient stability information corresponding to at least two network components based on the at least two prediction results and at least two label information corresponding to the at least two training images, wherein the gradient stability information corresponding to any one of the at least two network components is used to represent the stability of the gradient of the network component.

[0169] Training module 23 is used to train the image processing model based on the at least two gradient stability information items.

[0170] In one possible implementation, the gradient stability information includes the variance of the gradient;

[0171] The first determining module 22 is configured to: determine at least two first gradients corresponding to the at least two training images based on the at least two prediction results and at least two label information corresponding to the at least two training images; and for any one of the at least two network components, determine the variance of the gradient corresponding to the network component based on the at least two first gradients.

[0172] The training module 23 is used to train the image processing model based on the variance of at least two gradients that correspond one-to-one with the at least two network components.

[0173] In one possible implementation, the first determining module 22 is used to:

[0174] The training image group is divided into a first training image subgroup and a second training image subgroup, wherein the first training image subgroup and the second training image subgroup each include at least one training image;

[0175] Based on the first gradient corresponding to the training images in the first training image subgroup, determine the second gradient corresponding to the first training image subgroup;

[0176] Based on the first gradient corresponding to the training images in the second training image subgroup, determine the third gradient corresponding to the first training image subgroup;

[0177] Backpropagation is performed based on the second gradient to determine the fourth gradient corresponding to the network component;

[0178] Backpropagation is performed based on the third gradient to determine the fifth gradient corresponding to the network component.

[0179] Based on the fourth and fifth gradients, the variance of the gradient corresponding to the network component is determined.

[0180] In one possible implementation, the training module 23 is used for:

[0181] Based on the variance of at least two gradients corresponding to at least two network components, at least two first target learning rates corresponding to at least two network components are determined; wherein, the first target learning rate corresponding to any one of the at least two network components is negatively correlated with the variance of the gradient corresponding to the network component.

[0182] The image processing model is trained according to the at least two first objective learning rates.

[0183] In one possible implementation, the at least two network components include a target network component and at least one non-target network component;

[0184] The training module 23 is used for:

[0185] For any non-target network component, a first target learning rate is determined based on the variance of the gradient corresponding to the target network component, the variance of the gradient corresponding to the non-target network component, and a preset initial learning rate.

[0186] For the target network component, a first target learning rate is determined based on the preset initial learning rate.

[0187] In one possible implementation, the training module 23 is used for:

[0188] Determine a first ratio between the variance of the gradient corresponding to the target network component and the variance of the gradient corresponding to the non-target network component;

[0189] Based on the first ratio and the preset initial learning rate, the first target learning rate corresponding to the non-target network component is determined.

[0190] In one possible implementation, the training module 23 is used for:

[0191] Obtain the second ratio of the second target learning rate corresponding to the non-target network component to the preset initial learning rate in the previous training round;

[0192] Based on the first ratio, the second ratio, and the preset initial learning rate, a first target learning rate is determined for the non-target network component.

[0193] In one possible implementation, the device further includes:

[0194] The second determining module is used to randomly determine a training image group from the training image set, wherein the number of training images in the training image set is greater than the number of training images in the training image group.

[0195] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation and technical effects can be referred to the description of the above method embodiments. For the sake of brevity, they will not be repeated here.

[0196] This disclosure also provides a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the above-described method. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium.

[0197] This disclosure also proposes a computer program including computer-readable code, wherein when the computer-readable code is run in an electronic device, a processor in the electronic device executes the above-described method.

[0198] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in an electronic device, the processor in the electronic device executes the above-described method.

[0199] This disclosure also provides an electronic device, including: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the executable instructions stored in the memory to perform the above-described method.

[0200] Electronic devices can be provided as terminals, servers, or other forms of devices.

[0201] Figure 3 A block diagram of an electronic device 1900 provided in an embodiment of this disclosure is shown. For example, the electronic device 1900 may be provided as a server. (Refer to...) Figure 3 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0202] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output (I / O) interface 1958. Electronic device 1900 can operate on an operating system stored in memory 1932, such as Microsoft Server operating system (Windows Server). TM Apple's graphical user interface-based operating system (Mac OSX) TM ), a multi-user, multi-process computer operating system (Unix) TM Linux is a free and open-source Unix-like operating system. TM ), the open-source Unix-like operating system (FreeBSD) TM (or similar.)

[0203] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0204] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0205] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0206] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0207] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0208] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0209] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0210] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0211] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0212] The computer program product can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0213] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0214] If the technical solution of this disclosure involves personal information, the product applying the technical solution of this disclosure has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this disclosure involves sensitive personal information, the product applying the technical solution of this disclosure has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to indicate that the user has entered the scope of personal information collection and that personal information will be collected. If the user voluntarily enters the collection scope, it is deemed to have consented to the collection of their personal information; or on the personal information processing device, with clear signs / information informing the user of the personal information processing rules, authorization is obtained from the user through pop-up information or by asking the user to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0215] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for training an image processing model, characterized in that, The image processing model comprises at least two network components, and the method comprises: obtaining, by the image processing model, at least two pieces of prediction results corresponding to at least two training images in a training image group one by one; determining, according to the at least two pieces of prediction results and at least two pieces of label information corresponding to the at least two training images one by one, at least two pieces of gradient stability information corresponding to the at least two network components, wherein the gradient stability information corresponding to any network component of the at least two network components is used to represent the stability of the gradient of the network component; training the image processing model according to the at least two pieces of gradient stability information; wherein the gradient stability information comprises the variance of the gradient; The determining, according to the at least two pieces of prediction results and at least two pieces of label information corresponding to the at least two training images one by one, at least two pieces of gradient stability information corresponding to the at least two network components comprises: determining, according to the at least two pieces of prediction results and at least two pieces of label information corresponding to the at least two training images one by one, at least two pieces of first gradients corresponding to the at least two training images; dividing the training image group into a first training image subgroup and a second training image subgroup, wherein the first training image subgroup and the second training image subgroup each comprise at least one training image; determining a second gradient corresponding to the first training image subgroup according to the first gradient corresponding to the training image in the first training image subgroup; determining a third gradient corresponding to the first training image subgroup according to the first gradient corresponding to the training image in the second training image subgroup; for any network component of the at least two network components, determining a fourth gradient corresponding to the network component based on the second gradient through back propagation; determining a fifth gradient corresponding to the network component based on the third gradient through back propagation; determining the variance of the gradient corresponding to the network component according to the fourth gradient and the fifth gradient.

2. The method of claim 1, wherein, The training the image processing model according to the at least two pieces of gradient stability information comprises training the image processing model according to the variance of the at least two gradients corresponding to the at least two network components one by one.

3. The method according to claim 1 or 2, characterized in that, The training the image processing model according to the variance of the at least two gradients corresponding to the at least two network components one by one comprises: determining, according to the variance of the at least two gradients corresponding to the at least two network components one by one, at least two pieces of first target learning rates corresponding to the at least two network components one by one; wherein the first target learning rate corresponding to any network component of the at least two network components is negatively correlated with the variance of the gradient corresponding to the network component; training the image processing model according to the at least two pieces of first target learning rates.

4. The method of claim 3, wherein, The at least two network components comprise a target network component and at least one non-target network component; The determining, according to the variance of the at least two gradients corresponding to the at least two network components one by one, at least two pieces of first target learning rates corresponding to the at least two network components one by one comprises: For any non-target network component, a first target learning rate corresponding to the non-target network component is determined according to a variance of a gradient corresponding to the target network component, a variance of a gradient corresponding to the non-target network component, and a preset initial learning rate; For the target network component, a first target learning rate corresponding to the target network component is determined according to the preset initial learning rate.

5. The method of claim 4, wherein, The first target learning rate corresponding to the non-target network component is determined according to the variance of the gradient corresponding to the target network component, the variance of the gradient corresponding to the non-target network component, and the preset initial learning rate, including: A first ratio of the variance of the gradient corresponding to the target network component to the variance of the gradient corresponding to the non-target network component is determined; The first target learning rate corresponding to the non-target network component is determined according to the first ratio and the preset initial learning rate.

6. The method of claim 5, wherein, The first target learning rate corresponding to the non-target network component is determined according to the first ratio and the preset initial learning rate, including: A second ratio of a second target learning rate corresponding to the non-target network component to the preset initial learning rate in a previous round of training is obtained; The first target learning rate corresponding to the non-target network component is determined according to the first ratio, the second ratio, and the preset initial learning rate.

7. The method according to claim 1 or 2, characterized in that, Before the at least two prediction results corresponding to the at least two training images in the training image group are obtained by the image processing model, the method further includes: A training image group is randomly determined from a training image set, where a number of training images in the training image set is greater than a number of training images in the training image group.

8. A training device for an image processing model, characterized in that, The image processing model includes at least two network components, and the device includes: A prediction module is configured to obtain, by an image processing model, at least two prediction results corresponding to at least two training images in a training image group; A first determination module is configured to determine, according to the at least two prediction results and at least two pieces of label information corresponding to the at least two training images, at least two pieces of gradient stability information corresponding to the at least two network components, where gradient stability information corresponding to any network component in the at least two network components is used to represent stability of a gradient of the network component; A training module is configured to train the image processing model according to the at least two pieces of gradient stability information. The gradient stability information includes a variance of a gradient. The first determination module is configured to: Determine, according to the at least two prediction results and the at least two pieces of label information corresponding to the at least two training images, at least two first gradients corresponding to the at least two training images; Divide the training image group into a first training image subgroup and a second training image subgroup, where the first training image subgroup and the second training image subgroup each include at least one training image; Determine a second gradient corresponding to the first training image subgroup according to a first gradient corresponding to a training image in the first training image subgroup; and Determine a second gradient corresponding to the second training image subgroup according to a first gradient corresponding to a training image in the second training image subgroup. determine a third gradient corresponding to the first training image sub-group according to the first gradient corresponding to the training image in the second training image sub-group; for any network component in the at least two network components, determine a fourth gradient corresponding to the network component based on the second gradient by back propagation; determine a fifth gradient corresponding to the network component based on the third gradient by back propagation; determine a variance of the gradient corresponding to the network component according to the fourth gradient and the fifth gradient.

9. An electronic device, comprising: comprise: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the executable instructions stored in the memory to execute the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by the processor, implement the method of any one of claims 1 to 7.

11. A computer program product, characterised in that, comprise computer readable code, or a non-volatile computer readable storage medium carrying computer readable code, when the computer readable code is run in an electronic device, a processor in the electronic device executes the method of any one of claims 1 to 7.

Images