Computing method, apparatus, computer device, and storage medium

Abstract

The present disclosure relates to a computing method, device, computer equipment and storage medium. The method comprises: performing recovery processing on a sample image based on a generation network model to generate a first output image; calculating a first gradient according to the first output image and a first loss function; calculating a second gradient according to a first discrimination result of the first output image by a discrimination network model; calculating a second gradient bias according to a second discrimination result of a label image corresponding to the sample image by the discrimination network model and a second loss function; taking a difference between the second gradient and the second gradient bias as a third gradient; calculating a target gradient according to the first gradient and the third gradient; and updating parameters of the generation network model according to the target gradient to complete model training of the generation network model. The model training speed can be improved, the conflict between the optimization directions of the two loss functions in the training process can be balanced, and the precision and accuracy of the model obtained by training can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and in particular, to a computing method and device, computer equipment and storage medium. BACKGROUND

[0002] With the development of neural network technology, neural network technology has also been widely developed and applied in image reconstruction and recovery. In order to obtain a better generator that can recover the input image and output a generated image closer to the real image, the generator is obtained by synchronously training the generator and the discriminator. In the related art, the generator is obtained by using a reconstruction loss function and an adversarial loss function to train the model. However, the output image of the trained generator has problems such as edge deformation, false texture, color deviation, etc. SUMMARY

[0003] Therefore, it is necessary to provide a computing method, device, computer equipment and storage medium to solve the above technical problems.

[0004] According to an aspect of the present disclosure, a computing method is provided, which includes:

[0005] recovering the sample image based on a generation network model to generate a first output image;

[0006] calculating a first gradient according to the first output image and a first loss function;

[0007] calculating a second gradient according to a first discrimination result of the first output image by a discrimination network model;

[0008] calculating a second gradient bias according to a second discrimination result of a label image corresponding to the sample image by the discrimination network model and a second loss function;

[0009] taking a difference between the second gradient and the second gradient bias as a third gradient;

[0010] calculating a target gradient according to the first gradient and the third gradient;

[0011] updating parameters of the generation network model according to the target gradient to complete model training of the generation network model.

[0012] In a possible implementation, calculating a target gradient according to the first gradient and the third gradient includes:

[0013] adjust the first gradient and / or the third gradient according to a first p-norm of the first gradient and a third p-norm of the third gradient, so that the p-norms of the adjusted first gradient and the third gradient are consistent;

[0014] a sum of the first gradient and the third gradient with consistent p-norms is taken as the target gradient.

[0015] In a possible implementation, the adjusting of the first gradient and / or the third gradient according to the first p-norm of the first gradient and the third p-norm of the third gradient, so that the p-norms of the adjusted first gradient and the third gradient are consistent, includes:

[0016] a target p-norm is determined according to the first p-norm and the third p-norm, the target p-norm being greater than or equal to a minimum value of the first p-norm and the third p-norm and less than or equal to a maximum value of the first p-norm and the third p-norm;

[0017] the third gradient and the first gradient that do not have consistent p-norms with the target p-norm are adjusted, so that the p-norms of the adjusted first gradient and the third gradient are consistent.

[0018] In a possible implementation, the target gradient is calculated according to the first gradient and the third gradient, including:

[0019] Before the target gradient is calculated according to the first gradient and the third gradient, the first gradient and the third gradient are adjusted according to a gradient adjustment model.

[0020] In a possible implementation, the method includes:

[0021] After the parameters of the generation network model are updated according to the target gradient, a second output image is generated by performing recovery processing on a sample image based on the updated generation network model;

[0022] a first loss value is calculated according to the second output image and a first loss function;

[0023] an updated second loss value is calculated according to a discrimination result of the second output image by a discrimination network model and a second loss function;

[0024] a third loss value of the gradient adjustment model is calculated according to the updated first loss value and the updated second loss value;

[0025] the gradient adjustment model is updated according to the third loss value, and the model training of the loss value adjustment model is completed.

[0026] In a possible implementation, the first loss function comprises a reconstruction loss function, and the second loss function comprises an adversarial loss function.

[0027] According to another aspect of the present disclosure, a computing device is provided, and the device comprises:

[0028] a first image acquisition module configured to perform recovery processing on a sample image based on a generation network model to generate a first output image;

[0029] a first gradient acquisition module configured to calculate a first gradient according to the first output image and a first loss function;

[0030] a second gradient acquisition module configured to calculate a second gradient according to a first discrimination result of the first output image by a discrimination network model;

[0031] a bias acquisition module configured to calculate a second gradient bias according to a second discrimination result of a label image corresponding to the sample image by the discrimination network model and a second loss function;

[0032] a third gradient acquisition module configured to take a difference between the second gradient and the second gradient bias as a third gradient;

[0033] a target gradient acquisition module configured to calculate a target gradient according to the first gradient and the third gradient;

[0034] a first update module configured to update parameters of the generation network model according to the target gradient to complete model training of the generation network model.

[0035] In a possible implementation, the target gradient acquisition module comprises:

[0036] a first gradient adjustment submodule configured to adjust the first gradient and / or the third gradient according to a first p-norm of the first gradient and a third p-norm of the third gradient, so that the p-norms of the adjusted first gradient and the third gradient are consistent;

[0037] a gradient calculation submodule configured to take a sum of the first gradient and the third gradient with consistent p-norms as the target gradient.

[0038] In a possible implementation, the first gradient adjustment submodule comprises:

[0039] a target norm determination sub-module configured to determine a target p-norm according to the first p-norm and the third p-norm, the target p-norm being greater than or equal to the minimum of the first p-norm and the third p-norm and being less than or equal to the maximum of the first p-norm and the third p-norm;

[0040] an adjustment sub-module configured to adjust the p-norm of the third gradient and the gradient whose p-norm is inconsistent with the target p-norm in the first gradient, so that the p-norm of the adjusted first gradient and the third gradient is consistent.

[0041] In a possible implementation, the target gradient obtaining module comprises:

[0042] a second gradient adjustment sub-module configured to adjust the first gradient and the third gradient according to a gradient adjustment model before the target gradient is calculated according to the first gradient and the third gradient.

[0043] In a possible implementation, the apparatus further comprises:

[0044] a second image obtaining module configured to, after the parameters of the generative network model are updated according to the target gradient, perform recovery processing on a sample image based on the updated generative network model to generate a second output image;

[0045] a first calculation module configured to calculate a first loss value according to the second output image and a first loss function;

[0046] a second calculation module configured to calculate an updated second loss value according to a discrimination result of the second output image by a discrimination network model and a second loss function;

[0047] a third calculation module configured to calculate a third loss value of the gradient adjustment model according to the updated first loss value and the updated second loss value;

[0048] a second updating module configured to update the gradient adjustment model according to the third loss value, so as to complete model training of the loss value adjustment model.

[0049] In a possible implementation, the first loss function comprises a reconstruction loss function, and the second loss function comprises an adversarial loss function.

[0050] According to another aspect of the present disclosure, there is provided an artificial intelligence chip, which comprises the above computing apparatus.

[0051] According to an aspect of the present disclosure, there is provided another electronic device, which comprises the above artificial intelligence chip.

[0052] According to another aspect of the present disclosure, a board card is provided, comprising a memory device, an interface device and a control device, and the above-mentioned artificial intelligence chip;

[0053] The artificial intelligence chip is connected with the memory device, the control device and the interface device respectively.

[0054] The memory device is configured to store data.

[0055] The interface device is configured to realize data transmission between the artificial intelligence chip and an external device.

[0056] The control device is configured to monitor the state of the artificial intelligence chip.

[0057] The memory device comprises a plurality of groups of storage units, each group of storage units is connected with the artificial intelligence chip through a bus, and the storage unit is a DDR SDRAM.

[0058] The chip comprises a DDR controller configured to control data transmission and data storage of each storage unit.

[0059] The interface device is a standard PCIE interface.

[0060] According to an aspect of the present disclosure, a computer readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the above-mentioned method.

[0061] The computing method and device provided by the embodiments of the present disclosure can improve the model training speed, balance the conflict of the optimization direction of the two loss functions in the training process, and improve the precision and accuracy of the trained model, so that the output image can be edge deformation-free, error-free texture-free, and color-bias-free, closer to the real image.

[0062] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0063] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present disclosure and serve to explain the principles of the present disclosure.

[0064] Figure 1 A flowchart of a computing method according to an embodiment of the present disclosure is shown.

[0065] Figure 2 A schematic diagram of gradient adjustment by implementation mode one according to an embodiment of the present disclosure is shown.

[0066] Figure 3 FIG. 1 is a schematic diagram illustrating a process of adjusting a gradient and updating a gradient adjustment model according to an embodiment of the present disclosure.

[0067] Figure 4 FIG. 1 is a schematic diagram illustrating a process of adjusting a gradient and updating a gradient adjustment model according to an embodiment of the present disclosure.

[0068] Figure 5 FIG. 1 is a schematic diagram illustrating a process of adjusting a gradient and updating a gradient adjustment model according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some, but not all, of the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present disclosure.

[0070] It should be understood that the terms "first", "second", "third", and "fourth" in the claims, specification, and drawings of the present disclosure are used to distinguish different objects, and are not used to describe a particular order. The terms "include" and "contain" used in the specification and claims of the present disclosure indicate the presence of described features, whole, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, whole, steps, operations, elements, components, and / or sets thereof.

[0071] It should also be understood that the terms used in the present disclosure specification are only for the purpose of describing specific embodiments, and are not intended to limit the present disclosure. As used in the specification and claims of the present disclosure, the singular forms "a", "an", and "the" are intended to include plural forms, unless the context clearly indicates otherwise. It should be further understood that the term "and / or" used in the specification and claims of the present disclosure refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0072] As used in the specification and claims, the term "if" can be interpreted as "when" or "upon" or "in response to a determination" or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if detected [the described condition or event]" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the described condition or event]" or "in response to detecting [the described condition or event]" depending on the context.

[0073] In the related art, in the synchronous optimization training process of the generator and the discriminator, the reconstruction loss function and the adversarial loss function are used for update optimization. However, due to the fact that the optimization directions of the two loss functions are not completely consistent, and even there may be a large conflict, in order to solve the influence caused by this conflict, in the related art, the proportion coefficients of the two loss functions are generally set according to experience. When the proportion coefficient of the reconstruction loss function is increased, the output image of the generator contains more blur and less error; when the proportion coefficient of the adversarial loss function is increased, the output image of the generator contains less blur and more error. The proportion of each loss function selected according to experience cannot be well balanced. At the same time, the adversarial loss function comes from the constantly optimized discriminator, and the numerical range itself may change. The fixed proportion coefficient may not effectively balance the proportion of the two loss functions with dynamically changing numerical ranges in determining the optimization direction.

[0074] To solve the above technical problems, the present disclosure provides a computing method and device, which can improve the model training speed, balance the conflict of the optimization directions of the two loss functions in the training process, and improve the precision and accuracy of the trained model, so that the output image is closer to the real image without edge deformation, error texture, and color deviation. Figure 1 A flowchart of a computing method according to an embodiment of the present disclosure is shown. As shown in Figure 1 The method can be applied to a processor, and the method includes steps S11-S17.

[0075] In step S11, a sample image is recovered based on a generation network model to generate a first output image.

[0076] The generation network model can be a model created according to an image reconstruction task, which can be a generator. After the image is input into the generation network model, the generation network model can output a network output image (such as the first output image, the second output image, the output image, etc. described herein). In order to train a generation network model that meets the needs, a discriminant network model needs to be used to complete the training in the training process. The discriminant network model can be a discriminator, which is used to make a class label probability judgment on the image output by the generation network model, and the output discriminant result includes the probability of the input image belonging to different class labels (also referred to as class probability herein). The class labels include: true (True), which can be represented by 1, representing that the image input into the discriminant network is a real image taken; false (False), which can be represented by 0, representing that the image input into the discriminant network is not a real image taken. Before the model training, a plurality of sample images and a label image (ground truth image) corresponding to each sample image can be obtained. The class label of the label image is "true". The class label of the first output image should be "false".

[0077] In step S12, a first gradient is calculated according to the first output image and a first loss function.

[0078] In this embodiment, the first loss function can be a reconstruction loss function, which can include a mean square error loss function (MSELoss), an L1 loss function (L1Loss), etc. After detecting the first output image output by the generation network model, a first loss value corresponding to the first output image can be calculated according to the first loss function, and then a first gradient is calculated according to the first loss value.

[0079] In step S13, a second gradient is calculated according to a first discrimination result of the first output image by the discrimination network model and a second loss function.

[0080] In this embodiment, the second loss function can include an adversarial loss, which is also denoted as GAN loss. The adversarial loss can include a binary cross entropy loss (BCELoss), etc. The first output image can be input to the discrimination network model to obtain a first discrimination result of the first output image by the discrimination network model, and then a second loss value corresponding to the first output image is calculated according to the first discrimination result, the second loss function and positive and negative class labels (i.e. the class labels “true” and “false” described above). Further, a second gradient is calculated according to the second loss value.

[0081] In step S14, a second gradient bias is calculated according to a second discrimination result of a label image corresponding to the sample image by the discrimination network model and the second loss function.

[0082] In this embodiment, the label image corresponding to the first output image can be input to the discrimination network model to obtain a second discrimination result of the label image by the discrimination network model, and then a third loss value corresponding to the label image is calculated according to the second discrimination result, the second loss function and positive and negative class labels (i.e. the class labels “true” and “false” described above). Further, the second gradient bias is calculated according to the third loss value.

[0083] In step S15, a third gradient is calculated as a difference between the second gradient and the second gradient bias.

[0084] In step S16, a target gradient is calculated according to the first gradient and the third gradient.

[0085] In the embodiment, in the case that the target gradient is zero, it is determined that the first output image generated by the generation network model and the corresponding label image have reached consistency (such as the similarity reaching a specified degree, being completely identical, etc.), and the update optimization of the generation network model and the discrimination network model can be stopped, and the model training is ended. The third gradient is taken as the basis for calculating the target gradient in the subsequent step, which can ensure that the target gradient of zero is given when the first output image generated by the generation network model and the corresponding label image are consistent, so that the model training can be smoothly stopped.

[0086] In the related art, the update optimization of the generation network model and the discrimination network model is also performed by using the adversarial loss function and the reconstruction loss function, but a non-zero gradient is generated regardless of whether the first output image generated by the generation network model and the corresponding label image are consistent, and it is not determined when the model training can be stopped. The following takes the adversarial loss in the related art as an example to illustrate the problem in the related art. The adversarial loss is a binary cross entropy loss (BCELoss). The specific problems are as follows:

[0087] The BCELoss is as follows: wherein y is a discrimination label (which is True or False), is a category probability output by the category network model. In the actual training process it is almost impossible to reach 0 or 1, and it is a floating point number in the range of (0, 1). At this time, it is easy to find that the gradient of the BCELoss to is always 0. And because is obtained by passing the first output image out through the discrimination network model, it is easy to find that the second gradient of the first output image out is not 0, so is not 0. This means that even if the first output image generated by the generation network model is completely the same as the label image gt, the second gradient calculated based on the adversarial loss function will also be a non-zero gradient, which causes the generation network model to be unable to stay in the ideal state position where the first output image out is consistent with the label image gt.

[0088] The reconstruction loss function is realized by calculating the norm after the difference, which can ensure that the first gradient of the generation network model is 0 when the first output image out is consistent with the corresponding label image gt, so that the model training can be stopped when the generation network model is in the ideal state.

[0089] In the related art, the generation network model is determined based on the first gradient and the second gradient to determine the total gradient, and then the generation network model is updated and optimized based on the total gradient. However, since the total gradient in the related art is determined by weighted summation of the first gradient and the second gradient, and the second gradient is always greater than zero, the total gradient is always greater than zero, and the model training cannot be stopped when the generation network model is in an ideal state.

[0090] In the embodiments of the present disclosure, in order to make the adversarial loss function l rec and the reconstruction loss function l adv can stay at the position of the ideal state of the generation network model, that is, the parameter combination of the generation network model that makes the first output image out consistent with the corresponding label image gt, the second gradient bias is introduced to force the generation network model to not receive any non-zero gradient when the first output image out is consistent with the corresponding label image gt.

[0091] Then, when l rec and l adv are simply combined in the form of addition to calculate the target gradient (that is, the sum of the first gradient and the third gradient is directly added as the target gradient), l rec is used as the gradient of the label image gt, and the second gradient bias The target gradient obtained by the parameters w of the generation network model may be:

[0092] wherein, l rec is the reconstruction loss function, is the gradient of the first output image out with respect to the parameter w, is the first gradient. is the third gradient, is a gradient calculation function for calculating the third gradient according to the second gradient and the second gradient bias.

[0093] In one possible implementation, when the gradient calculation function is , when the first output image out is consistent with the label image gt, , then

[0094] In one possible implementation, the way of calculating the target gradient according to the first gradient and the third gradient can include way one and way two:

[0095] Way one, as above, the target gradient is the first gradient, is a second gradient.

[0096] Mode two:

[0097] The first gradient and / or the third gradient can be adjusted according to the first p-norm of the first gradient and the third p-norm of the third gradient, so that the p-norms of the adjusted first gradient and the third gradient are consistent. The sum of the first gradient and the third gradient with consistent p-norms is taken as the target gradient.

[0098] The adjustment of the first gradient and / or the third gradient according to the first p-norm of the first gradient and the third p-norm of the third gradient can include any one of the following implementation modes one, two and three:

[0099] Implementation mode one: the smaller one of the first p-norm and the third p-norm is determined as a target p-norm, and then one of the first gradient and the third gradient with a p-norm that is not the target p-norm is adjusted, so that the p-norms of the first gradient and the third gradient are both the target p-norm after adjustment.

[0100] Implementation mode two: the larger one of the first p-norm and the third p-norm is determined as a target p-norm, and then one of the first gradient and the third gradient with a p-norm that is not the target p-norm is adjusted, so that the p-norms of the first gradient and the third gradient are both the target p-norm after adjustment.

[0101] Implementation mode three: the target p-norm is determined according to the first p-norm and the third p-norm through a preset mode, and then the first gradient and the third gradient are adjusted, so that the p-norms of the first gradient and the third gradient are both the target p-norm after adjustment. The preset mode can include: determining the average of the first p-norm and the third p-norm as the target p-norm, weighting and summing the first p-norm and the third p-norm to obtain the target p-norm, weighting and summing the first p-norm and the third p-norm to determine a value in a value range interval composed of the first p-norm and the third p-norm, that is, the target p-norm is less than or equal to the larger one of the first p-norm and the third p-norm and greater than or equal to the smaller one of the first p-norm and the third p-norm.

[0102] The adjustment of the first gradient and / or the third gradient refers to multiplying the gradient by a scalar coefficient, and the p-norms of the two gradients after multiplication by the coefficient are consistent, both being the target p-norm.

[0103] In this way, the p-norm of the first gradient and the third gradient can be adjusted to the target p-norm through the above-mentioned implementation modes one, two and three, so that the model training process can be stably performed, and the occurrence of model training collapse can be avoided. For example, Figure 2 A schematic diagram of gradient adjustment by the embodiment of the present disclosure through implementation mode one is shown. As shown in Figure 2 Before adjustment, when there is a large included angle between the directions of the first gradient and the third gradient, the large gradient G2 (the larger one of the first gradient and the third gradient) with a larger value and a larger p-norm is projected to the small gradient G1 (the smaller one of the first gradient and the third gradient) with a smaller value and a smaller p-norm. The length of the projection g2 of the large gradient G2 may exceed the length g1 of the small gradient G1 itself, which makes the direction of the target gradient after addition contain the opposite direction of the small gradient G1 and the perpendicular direction of the small gradient G1.

[0104] In the gradient descent method, the gradient direction is in the direction in which the corresponding loss function value changes fastest, and the normal direction changes slowly. When the "before adjustment" case shown in Figure 2 The component in the opposite direction of the small gradient G1 will cause the loss function corresponding to the small gradient G1 to increase after being applied to update the network parameters, and the loss function is not sensitive enough in the perpendicular direction to offset the influence of the reverse component. That is, the effect of the loss function corresponding to the small gradient G1 is covered by the loss function corresponding to the large gradient G2. The increase of the loss function represents that more errors occur under the corresponding evaluation standard. The steepest gradient descent direction refers to the parameter adjustment direction in which the loss function value decreases fastest. This direction generally refers to the reverse direction of the derivative of the loss function with respect to the parameters of the network model in network optimization. The derivative of the loss function with respect to the parameters is generally denoted as the gradient direction. This direction itself is the steepest ascent direction of the loss function, and in the process of optimizing the parameters, the original parameters are subtracted by a step size limited component in this direction, so as to realize gradient descent. In Figure 2 In the example of

[0105] The large gradient G2 can be adjusted as in the above-mentioned implementation mode two, and the adjusted large gradient G2 is as shown in Figure 2The p-norm of the small gradient G1 and the p-norm of the adjusted large gradient G2' are consistent. Since the p-norm of the small gradient G1 and the p-norm of the adjusted large gradient G2' are consistent, even if the angle between the two is large, the length of the projection of one gradient (G1 or G2') to the other gradient will not exceed the length of the gradient being projected. At this time, the target gradient calculated based on G1 and G2', and the parameter update amount of the generation network model calculated according to the target gradient, will not contain the steepest ascent direction of any one of the loss functions (i.e. the first loss function and the second loss function), which alleviates the imbalance between the two loss functions.

[0106] The first gradient and the third gradient both contain ① the partial derivative of the corresponding loss function with respect to the first output image out and ② the partial derivative of the first output image out with respect to the network parameter w. In the above-mentioned one illustrative example of the second implementation manner, the partial derivative of the loss function with respect to w can be modified. Taking the p-norm as the 1-norm as an example, a simple implementation manner can be: wherein, represents the 1-norm of the first loss function represents the 1-norm of the second loss function the ratio between them. is the first gradient, is the third gradient.

[0107] Manner three:

[0108] The first gradient and / or the second gradient can be adjusted by the gradient adjustment model (coordinator) before the target gradient is calculated by the above-mentioned manner two. Then, the second adjustment of the p-norm consistency of the above-mentioned manner two is performed, and finally the target gradient is calculated according to the first gradient and the third gradient obtained after the two adjustments. After the update optimization of the generation network model is completed according to the target gradient, the gradient adjustment model is updated.

[0109] Figure 3 The process of adjusting the gradient by the gradient adjustment model and updating the gradient adjustment model in the embodiment of the present disclosure is shown in the following schematic diagram: Figure 3As shown, the first adjustment of the first gradient and / or the second gradient may include: after executing steps S11-S15 above, inputting the first and second gradients into the gradient adjustment model to obtain the adjusted first gradient and adjusted third gradient output by the model. Then, the steps of method two above are continued to calculate the target gradient. The gradient adjustment model can adjust the first and third gradients, and it can be optimized synchronously with the generative network model and the class network model throughout the entire model training process, ensuring the accuracy of the adjustment of the first and third gradients. The adjustment of the first and / or third gradients by the gradient adjustment model can balance the influence of the two gradients on parameter updates, allowing the generative network model to be gradually updated and optimized.

[0110] In step S17, the parameters of the generative network model are updated according to the target gradient to complete the training of the generative network model.

[0111] After calculating the target gradient, a check can be performed to determine if the target gradient is zero. If the target gradient is zero, it can be determined that the generative network model has been able to output an image consistent with the corresponding label image, and the model can terminate. If the target gradient is not zero, it can be determined that the training of the generative network model needs to continue, and the parameters of the generative network model should be updated based on the target gradient.

[0112] After updating the generative network model, the first output image and the corresponding label image are input into the discriminative network model, and the parameters of the discriminative network model are updated based on the output results.

[0113] In one possible implementation, after updating the parameters of the generative network model in step S17, a new generative network model is obtained. Then, as follows... Figure 3 As shown, updating the gradient adjustment model includes: generating the sample images used by the network model in this update (i.e., ... Figure 3 The first row of input is the sample image of the generative network model and Figure 3 The second row of input samples for the generative network model is the same sample image. This is then input into a new generative network model to obtain the second output image. Next, the first and second loss values ​​corresponding to the second output image are calculated (the calculation process is similar to that in steps S12 and S13 above, and will not be repeated here). Based on the first and second loss values, the third loss value of the gradient adjustment model is calculated, and then... coor The gradient of the gradient adjustment model is calculated, and the parameters of the gradient adjustment model are updated and optimized based on the gradient of the gradient adjustment model.

[0114] In a possible implementation, the third loss value is calculated according to the first loss value and the second loss value of the second output image, and the third loss value is calculated by the following formula:

[0115] l coor = ReLU(l rec,new -l rec,old ) + ReLU(l adv,new -l adv,old )

[0116] wherein, l rec,old represents the first loss value of the first output image corresponding to the same sample image as the second output image, that is, the first loss value of the first row in the formula (1). Figure 3 l adv,old represents the second loss value of the first output image corresponding to the same sample image as the second output image, that is, the second loss value of the first row in the formula (2). Figure 3 l rec,new represents the first loss value of the second output image, that is, the first loss value of the second row in the formula (1). Figure 3 l adv,new represents the second loss value of the second output image, that is, the second loss value of the second row in the formula (2). Figure 3

[0117] ReLU(l rec,new -l rec,old ) is greater than zero, which proves that the loss value of the first loss function corresponding to the same sample image after the generation network model is updated is larger, and the optimization gradient adjustment model needs to be updated. Similarly, the value of ReLU(l adv,new -l adv,old ) is greater than zero, which proves that the loss value of the second loss function corresponding to the same sample image after the generation network model is updated is larger, and the optimization gradient adjustment model needs to be updated. The value of l coor can represent: if the new loss function is smaller than the old loss function, relu l coor is 0, which means that both loss functions can be smaller in value after one update, which means that both are optimized, and this time the loss function can not be considered, and the gradient adjustment model can not be updated; if the new loss function is larger than the old loss function, relu l coor is not 0, and the optimization gradient adjustment model needs to be updated this time.

[0118] The present disclosure also provides a computing device, which is applied to a processor and includes:

[0119] The first image acquisition module is configured to perform recovery processing on the sample image based on the generation network model to generate a first output image. ​

[0120] a first gradient obtaining module configured to calculate a first gradient according to the first output image and a first loss function;

[0121] a second gradient obtaining module configured to calculate a second gradient according to a first discrimination result of the first output image by the discrimination network model;

[0122] a bias obtaining module configured to calculate a second gradient bias according to a second discrimination result of a label image corresponding to the sample image by the discrimination network model and a second loss function;

[0123] a third gradient obtaining module configured to take a difference between the second gradient and the second gradient bias as a third gradient;

[0124] a target gradient obtaining module configured to calculate a target gradient according to the first gradient and the third gradient;

[0125] a first updating module configured to update parameters of the generation network model according to the target gradient, so as to complete model training of the generation network model.

[0126] In a possible implementation, the target gradient obtaining module comprises:

[0127] a first gradient adjusting submodule configured to adjust the first gradient and / or the third gradient according to a first p-norm of the first gradient and a third p-norm of the third gradient, so that the p-norms of the adjusted first gradient and the third gradient are consistent;

[0128] a gradient calculating submodule configured to take a sum of the first gradient and the third gradient with consistent p-norms as the target gradient.

[0129] In a possible implementation, the first gradient adjusting submodule comprises:

[0130] a target norm determining submodule configured to determine a target p-norm according to the first p-norm and the third p-norm, the target p-norm being greater than or equal to a minimum value of the first p-norm and the third p-norm and less than or equal to a maximum value of the first p-norm and the third p-norm;

[0131] an adjusting submodule configured to adjust the gradient with inconsistent p-norm between the third gradient and the first gradient and the target p-norm, so that the p-norms of the adjusted first gradient and the third gradient are consistent.

[0132] In a possible implementation, the target gradient obtaining module comprises:

[0133] The second gradient adjustment submodule is configured to adjust the first gradient and the third gradient according to a gradient adjustment model before calculating a target gradient according to the first gradient and the third gradient.

[0134] In a possible implementation, the apparatus further includes:

[0135] The second image acquisition module is configured to perform recovery processing on a sample image based on the updated generative network model to generate a second output image after updating the parameters of the generative network model according to the target gradient.

[0136] The first calculation module is configured to calculate a first loss value according to the second output image and a first loss function.

[0137] The second calculation module is configured to calculate an updated second loss value according to a discrimination result of the second output image by the discrimination network model and a second loss function.

[0138] The third calculation module is configured to calculate a third loss value of the gradient value adjustment model according to the updated first loss value and the updated second loss value.

[0139] The second update module is configured to update the gradient adjustment model according to the third loss value, so as to complete model training of the loss value adjustment model.

[0140] In a possible implementation, the first loss function includes a reconstruction loss function, and the second loss function includes an adversarial loss function.

[0141] It should be noted that, for each method embodiment described above, in order to simply describe, each method embodiment is described as a series of action combinations, but those skilled in the art should know that the present disclosure is not limited by the order of actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0142] It should be further noted that although Figure 1 the steps in the flowchart are displayed in sequence according to the arrows, these steps are not necessarily executed in sequence according to the arrows. Unless otherwise specified in this document, the execution of these steps has no strict order limitation, and these steps can be executed in other orders. Moreover, Figure 1At least one of the steps in the above method can include a plurality of sub-steps or a plurality of stages, which are not necessarily performed at the same time, but can be performed at different times, and the order of the execution of the sub-steps or stages is not necessarily sequential, but can be performed alternately or alternately with at least one of the other steps or the sub-steps or stages of the other steps.

[0143] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present disclosure can also be implemented in other ways. For example, the division of units / modules described in the above embodiments is only a logical functional division, and another division mode can be used in actual implementation. For example, a plurality of units, modules or components can be combined, or can be integrated into another system, or some features can be omitted or not executed.

[0144] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present disclosure can be integrated into one unit / module, or each unit / module can exist physically, or two or more units / modules can be integrated together. The integrated unit / module can be realized in the form of hardware or in the form of a software program module.

[0145] Figure 4 is a structural diagram of a combination processing device 1200 according to an embodiment of the present disclosure. As shown in Figure 4 The combination processing device 1200 includes a computing processing device 1202, an interface device 1204, other processing devices 1206 and a storage device 1208, as shown in the above method. Figure 1 The computing processing device can include one or more computing devices 1210, which can be configured to perform the operations of each step in the computing method described herein in conjunction with the above

[0146] In different embodiments, the computing processing device of the present disclosure can be configured to perform user-specified operations. In an exemplary application, the computing processing device can be implemented as a single-core artificial intelligence processor or a multi-core artificial intelligence processor. Similarly, one or more computing devices included in the computing processing device can be implemented as an artificial intelligence processor core or part of the hardware structure of an artificial intelligence processor core. When a plurality of computing devices are implemented as an artificial intelligence processor core or part of the hardware structure of an artificial intelligence processor core, the computing processing device of the present disclosure can be considered to have a single-core structure or a homogeneous multi-core structure.

[0147] In an exemplary operation, the computing processing device of the present disclosure can interact with other processing devices through an interface device to collectively accomplish a user-specified operation. Depending on the implementation, the other processing devices of the present disclosure can include one or more types of processors including general and / or special purpose processors such as a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an artificial intelligence processor, etc. These processors can include, but are not limited to, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned before, only in terms of the computing processing device of the present disclosure, it can be considered as having a single-core structure or a homogeneous multi-core structure. However, when considering the computing processing device and the other processing devices collectively, both of them can be considered as forming a heterogeneous multi-core structure.

[0148] In one or more embodiments, the other processing devices can serve as an interface of external data and control for the computing processing device of the present disclosure (which can be embodied as a related operation device for artificial intelligence such as neural network operation), and perform basic controls including but not limited to data carrying, starting and / or stopping of the computing device, etc. In additional embodiments, the other processing devices can also cooperate with the computing processing device to collectively accomplish an operation task.

[0149] In one or more embodiments, the interface device can be used to transmit data and control instructions between the computing processing device and the other processing devices. For example, the computing processing device can obtain input data from the other processing devices via the interface device and write into the storage device (or memory) on the computing processing device chip. Further, the computing processing device can obtain control instructions from the other processing devices via the interface device and write into the control cache on the computing processing device chip. Alternatively or optionally, the interface device can also read data in the storage device of the computing processing device and transmit to the other processing devices.

[0150] Additionally or alternatively, the combination processing apparatus of the present disclosure can further comprise a storage device. As shown in the figures, the storage device is connected with the computing processing apparatus and the other processing apparatus respectively. In one or more embodiments, the storage device can be used to save data of the computing processing apparatus and / or the other processing apparatus. For example, the data can be data that cannot be saved completely in the internal or on-chip storage device of the computing processing apparatus or the other processing apparatus.

[0151] In some embodiments, the present disclosure further discloses a chip (e.g. Figure 5 shown in FIG. 13B). In one implementation, the chip is a System on Chip (SoC), and integrates one or more combination processing apparatuses as shown in Figure 4 FIG. 13A. The chip can be connected with other related components through an external interface device (e.g. Figure 5 shown in FIG. 13B). The related components can be, for example, a camera, a display, a mouse, a keyboard, a network card or a wifi interface. In some application scenarios, other processing units (e.g. video codec) and / or interface modules (e.g. DRAM interface) can be integrated on the chip. In some embodiments, the present disclosure further discloses a chip package structure comprising the above chip. In some embodiments, the present disclosure further discloses a board card comprising the above chip package structure. The board card will be described in detail below. Figure 5

[0152] Figure 5 is a structural schematic diagram of a board card 1300 according to an embodiment of the present disclosure. As shown in Figure 5 FIG. 13C, the board card comprises a storage device 1304 for storing data, which comprises one or more storage units 1310. The storage device can be connected and perform data transmission with the controller device 1308 and the chip 1302 described above through, for example, a bus. Further, the board card further comprises an external interface device 1306 configured for data relay or switching function between the chip (or the chip in the chip package structure) and an external device 1312 (e.g. a server or a computer, etc.). For example, data to be processed can be transmitted from the external device to the chip through the external interface device. For another example, the computing result of the chip can be transmitted back to the external device via the external interface device. According to different application scenarios, the external interface device can have different interface forms, for example, it can adopt a standard PCIE interface, etc.

[0153] ​In one or more embodiments, the control device in the board card of the present disclosure can be configured to regulate the state of the chip. To this end, in one application scenario, the control device can include a micro controller unit (MCU) for regulating the working state of the chip.

[0154] According to the above description Figure 4 and Figure 5 According to the above description, those skilled in the art can understand that the present disclosure also discloses an electronic device or apparatus, which can include one or more of the above-mentioned board cards, one or more of the above-mentioned chips, and / or one or more of the above-mentioned combined processing apparatuses.

[0155] According to different application scenarios, the electronic device or apparatus of the present disclosure can include a server, a cloud server, a server cluster, a data processing apparatus, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a PC device, an Internet of Things terminal, a mobile terminal, a mobile phone, a vehicle recording device, a navigation device, a sensor, a camera, a camera, a video camera, a projector, a watch, a headset, a mobile storage, a wearable device, a visual terminal, an autonomous driving terminal, a vehicle, a household appliance, and / or a medical device. The vehicle includes an airplane, a ship and / or a vehicle; the household appliance includes a television, an air conditioner, a microwave oven, a refrigerator, an electric rice cooker, a humidifier, a washing machine, an electric lamp, a gas stove, an exhaust hood; the medical device includes a nuclear magnetic resonance instrument, a B-ultrasound instrument and / or an electrocardiograph. The electronic device or apparatus of the present disclosure can also be applied to the fields of Internet, Internet of Things, data center, energy, transportation, public management, manufacturing, education, power grid, telecommunications, finance, retail, construction site, medical treatment, etc. Further, the electronic device or apparatus of the present disclosure can also be used in cloud, edge, terminal and other application scenarios related to artificial intelligence, big data and / or cloud computing. In one or more embodiments, the electronic device or apparatus with high computing power according to the present disclosure scheme can be applied to a cloud device (such as a cloud server), while the electronic device or apparatus with low power consumption can be applied to a terminal device and / or an edge device (such as a smart phone or a camera). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or the edge device are compatible with each other, so that according to the hardware information of the terminal device and / or the edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or the edge device, so as to complete the unified management, scheduling and cooperative work of end-cloud integration or cloud-edge integration.

[0156] It should be noted that, for the purpose of simplicity, the present disclosure expresses some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art can understand that the solutions of the present disclosure are not limited to the order of the actions described. Therefore, those skilled in the art can understand that some of the steps can be performed in other orders or simultaneously according to the disclosure or teaching of the present disclosure. Further, those skilled in the art can understand that the embodiments described in the present disclosure can be regarded as optional embodiments, i.e., the actions or modules involved are not necessarily essential for the implementation of one or more solutions of the present disclosure. In addition, the description of some embodiments in the present disclosure also focuses on different solutions. Therefore, those skilled in the art can understand that the parts not described in detail in one embodiment of the present disclosure can also refer to the relevant description of other embodiments.

[0157] In specific implementation aspects, those skilled in the art can understand that the several embodiments disclosed in the present disclosure can also be implemented in other manners not disclosed herein based on the disclosure and teaching of the present disclosure. For example, as to each unit in the foregoing electronic device or apparatus embodiments, the division thereof is made on the basis of logical functions, and there can be another division manner in actual implementation. For another example, a plurality of units or components can be combined or integrated into another system, or some features or functions of the units or components can be selectively disabled. As to the connection relationship between different units or components, the connection discussed in the foregoing can be direct or indirect coupling between units or components. In some scenarios, the foregoing direct or indirect coupling involves communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0158] In the present disclosure, the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units. The foregoing components or units can be located in the same place or distributed on a plurality of network units. In addition, according to actual needs, some or all of the units can be selected to achieve the purpose of the solutions described in the embodiments of the present disclosure. In addition, in some scenarios, a plurality of units in the embodiments of the present disclosure can be integrated into one unit or physically exist separately.

[0159] In some implementation scenarios, the above-mentioned integrated units can be implemented in the form of software program modules. If implemented in the form of software program modules and sold or used as independent products, the integrated units can be stored in a computer readable memory. Based on this, when the scheme of the present disclosure is embodied in the form of a software product (for example, a computer readable storage medium), the software product can be stored in a memory, which can include a number of instructions to make a computer device (for example, a personal computer, a server or a network device, etc.) execute part or all of the steps of the method described in the embodiments of the present disclosure. The aforementioned memory can include but is not limited to a U disk, a flash disk, a read-only memory (Read Only Memory, ROM), a random access memory (Random Access Memory, RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0160] In some other implementation scenarios, the above-mentioned integrated units can also be implemented in the form of hardware, that is, specific hardware circuits, which can include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit can include but is not limited to physical devices, and the physical devices can include but are not limited to transistors or memristors, etc. In view of this, various devices described herein (for example, computing devices or other processing devices) can be implemented by appropriate hardware processors, such as CPUs, GPUs, FPGAs, DSPs and ASICs, etc. Further, the aforementioned storage units or storage devices can be any appropriate storage medium (including magnetic storage medium or magneto-optical storage medium, etc.), which can be, for example, a variable resistance memory (Resistive Random Access Memory, RRAM), a dynamic random access memory (Dynamic Random Access Memory, DRAM), a static random access memory (Static Random Access Memory, SRAM), an enhanced dynamic random access memory (Enhanced Dynamic Random Access Memory, EDRAM), a high bandwidth memory (High Bandwidth Memory, HBM), a hybrid memory cube (Hybrid Memory Cube, HMC), a ROM and a RAM, etc.

[0161] While several embodiments of the present disclosure have been shown and described herein, it is to be understood that the embodiments are merely exemplary. Numerous changes, substitutions, and alterations can be made to the embodiments without departing from the spirit and scope of the disclosure. It is understood that various alternatives to the embodiments of the disclosure described herein can be employed in practicing the present disclosure. The claims are intended to cover all such alternatives and equivalents. It is understood that every embodiment that is described applies to each and every combination of features described above and below.

[0162] The above detailed description of the embodiments of the present disclosure has been presented for the purposes of illustration and description. It is understood that the description is not intended to limit the embodiments of the disclosure to the forms disclosed herein. Consequently, many modifications and variations will be apparent to practitioners skilled in the art. It is therefore understood that the scope of the disclosure should be limited only by the claims and their equivalents.

Claims

1. A calculation method, characterized in that, The method includes: The sample image is restored based on the generative network model to generate the first output image; The first gradient is calculated based on the first output image and the first loss function; The second gradient is calculated based on the first discrimination result and the second loss function of the discriminant network model for the first output image; The second gradient bias is calculated based on the second discrimination result of the discriminant network model on the label image corresponding to the sample image and the second loss function; The difference between the second gradient and the bias of the second gradient is taken as the third gradient; The target gradient is calculated based on the first gradient and the third gradient. The parameters of the generative network model are updated according to the target gradient to complete the training of the generative network model.

2. The method according to claim 1, characterized in that, The target gradient is calculated based on the first gradient and the third gradient, including: Based on the first p-norm of the first gradient and the third p-norm of the third gradient, the first gradient and / or the third gradient are adjusted so that the p-norms of the adjusted first gradient and the third gradient are consistent. The sum of the first and third gradients with consistent p-norm is taken as the target gradient.

3. The method according to claim 2, characterized in that, Adjusting the first gradient and / or the third gradient based on the first p-norm of the first gradient and the third p-norm of the third gradient, so that the p-norms of the adjusted first gradient and the third gradient are consistent, includes: Based on the first p-norm and the third p-norm, a target p-norm is determined, wherein the target p-norm is greater than or equal to the minimum value of the first p-norm and the third p-norm, and less than or equal to the maximum value of the first p-norm and the third p-norm; The third gradient and the gradients in the first gradient whose p-norms are inconsistent with the target p-norm are adjusted so that the p-norms of the adjusted first gradient are consistent with those of the third gradient.

4. The method according to any one of claims 1-3, characterized in that, The target gradient is calculated based on the first gradient and the third gradient, including: Before calculating the target gradient based on the first gradient and the third gradient, the first gradient and the third gradient are adjusted according to the gradient adjustment model.

5. The method according to claim 4, characterized in that, The method includes: After updating the parameters of the generative network model according to the target gradient, the sample image is restored based on the updated generative network model to generate a second output image. The first loss value is calculated based on the second output image and the first loss function; Based on the discrimination result of the discriminant network model on the second output image and the second loss function, the second loss value for updating is calculated; The third loss value of the gradient value adjustment model is calculated based on the first loss value and the second loss value used for updating; The gradient adjustment model is updated based on the third loss value, and the model training of the loss value adjustment model is now complete.

6. The method according to claim 4, characterized in that, The first loss function includes the reconstruction loss function, and the second loss function includes the adversarial loss function.

7. A computing device, characterized in that, The device includes: The first image acquisition module is used to restore the sample image based on the generative network model and generate the first output image; The first gradient acquisition module is used to calculate the first gradient based on the first output image and the first loss function; The second gradient acquisition module is used to calculate the second gradient based on the first discrimination result and the second loss function of the discrimination network model on the first output image; The bias acquisition module is used to calculate the second gradient bias based on the second discrimination result of the discrimination network model on the label image corresponding to the sample image and the second loss function; The third gradient acquisition module is used to take the difference between the second gradient and the second gradient bias as the third gradient; The target gradient acquisition module is used to calculate the target gradient based on the first gradient and the third gradient; The first update module is used to update the parameters of the generative network model according to the target gradient, so as to complete the model training of the generative network model.

8. The apparatus according to claim 7, characterized in that, The target gradient acquisition module includes: The first gradient adjustment submodule is used to adjust the first gradient and / or the third gradient according to the first p-norm of the first gradient and the third p-norm of the third gradient, so that the p-norms of the first gradient and the third gradient are consistent after adjustment. The gradient calculation submodule is used to take the sum of the first and third gradients with consistent p-norm as the target gradient.

9. The apparatus according to claim 8, characterized in that, The first gradient adjustment submodule includes: The target p-norm determination submodule is used to determine a target p-norm based on the first p-norm and the third p-norm, wherein the target p-norm is greater than or equal to the minimum value of the first p-norm and the third p-norm, and less than or equal to the maximum value of the first p-norm and the third p-norm; The adjustment submodule is used to adjust the gradients in the third gradient and the first gradient whose p-norms are inconsistent with the target p-norm, so that the p-norms of the adjusted first gradient are consistent with those of the third gradient.

10. The apparatus according to any one of claims 7-9, characterized in that, The target gradient acquisition module includes: The second gradient adjustment submodule is used to adjust the first gradient and the third gradient according to the gradient adjustment model before calculating the target gradient based on the first gradient and the third gradient.

11. The apparatus according to claim 10, characterized in that, The device further includes: The second image acquisition module is used to update the parameters of the generator network model according to the target gradient, and then perform recovery processing on the sample image based on the updated generator network model to generate a second output image. The first calculation module is used to calculate the first loss value based on the second output image and the first loss function; The second calculation module is used to calculate the second loss value for updating based on the discrimination result of the discrimination network model on the second output image and the second loss function. The third calculation module is used to calculate the third loss value of the model based on the first loss value for updating and the second loss value for updating; The second update module is used to update the gradient adjustment model according to the third loss value, thus completing the model training of the loss value adjustment model.

12. The apparatus according to claim 11, characterized in that, The first loss function includes the reconstruction loss function, and the second loss function includes the adversarial loss function.

13. An artificial intelligence chip, characterized in that, The chip includes a computing device as described in any one of claims 7-12.

14. An electronic device, characterized in that, The electronic device includes the artificial intelligence chip as described in claim 13.

15. A circuit board, characterized in that, The board includes: storage devices, interface devices, and control devices, as well as the artificial intelligence chip as described in claim 13; The artificial intelligence chip is connected to the storage device, the control device, and the interface device, respectively. The storage device is used to store data; The interface device is used to realize data transmission between the artificial intelligence chip and external devices; The control device is used to monitor the state of the artificial intelligence chip; The storage device includes: multiple sets of storage units, each set of storage units being connected to the artificial intelligence chip via a bus, and the storage unit being: DDR SDRAM; The chip includes: a DDR controller for controlling data transmission and data storage of each memory cell; The interface device is a standard PCIe interface.

16. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images