IMAGE PROCESSING METHOD, DEVICE, ELECTRONIC DEVICE, AND STORAGE MEDIUM

Abstract

The embodiments of the present disclosure disclose an image processing method, an apparatus, an electronic device, and a storage medium, which include the steps of obtaining an original image to be processed, inputting the original image into a first image processing model, processing the original image with the first image processing model to generate a target image, where the first image processing model and the second image processing model are generated by alternately training online, the monitoring information in the training process of the first image processing model includes at least a part of the image generated by the second image processing model during the training process, and the model scale of the first image processing model is smaller than the model scale of the second image processing model, and outputting the target image.

Description

[Technical field]

[0001] TECHNICAL FIELD The present disclosure relates to an image processing technology field, for example, an image processing method, device, electronic device, and storage medium.

[0002] This application claims priority to a Chinese patent application filed with the State Intellectual Property Office of China on July 22, 2022, bearing application number 202210873377.2, the entire contents of which are incorporated herein by reference. [Background technology]

[0003] Some image generators can generate images in one image domain based on images in another image domain, for example, generate high-resolution images based on low-resolution images, etc. Their unique image generation capabilities allow them to meet a wide range of application scenarios. Summary of the Invention [Problem to be solved by the invention]

[0004] Embodiments of the present disclosure provide an image processing method, apparatus, electronic device, and storage medium. [Means for solving the problem]

[0005] In a first aspect, an embodiment of the present disclosure comprises: obtaining an original image to be processed; inputting the original image into a first image processing model; A step of processing the original image with the first image processing model to generate a target image, the first image processing model and the second image processing model are generated by alternately training online, the monitoring information in the training process of the first image processing model includes at least a part of the image generated by the second image processing model during the training process, and the model scale of the first image processing model is smaller than the model scale of the second image processing model; and outputting the target image.

[0006] In a second aspect, an embodiment of the present disclosure comprises: an image acquisition module configured to acquire an original image to be processed; an input module configured to input the original image into a first image processing model; A generation module configured to process the original image with the first image processing model to generate a target image, the first image processing model and the second image processing model are generated by alternately training online, the monitoring information in the training process of the first image processing model includes at least a part of the image generated by the second image processing model during the training process, and the model scale of the first image processing model is smaller than the model scale of the second image processing model; and an output module configured to output the target image.

[0007] In a third aspect, embodiments of the present disclosure include one or more processors; a storage device configured to store one or more programs; When the one or more programs are executed by one or more processors, they cause the one or more processors to execute the image processing method described in any one of the embodiments of the present disclosure.

[0008] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium including computer-executable instructions, which, when executed by a computer processor, are used to perform the image processing method according to any one of the embodiments of the present disclosure. [Brief description of the drawings]

[0009] Throughout the drawings, the same or similar reference numbers refer to the same or similar elements, It is understood that the drawings are schematic and that elements and components are not necessarily drawn to scale. [Figure 1] FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic flowchart illustrating the generation of a pseudo-labeled image in an image processing method provided by an embodiment of the present disclosure. [Diagram 3] FIG. 3 is a schematic diagram of a training framework for a second image processing model in an image processing method provided by an embodiment of the present disclosure. [Figure 4] FIG. 4 is a schematic diagram of a training framework in an image processing method provided by an embodiment of the present disclosure, where a first image processing model uses pseudo-labeled images as supervised information. [Diagram 5] FIG. 5 is a schematic diagram of a training framework in an image processing method provided by an embodiment of the present disclosure, where a first image processing model uses a first image as surveillance information. [Figure 6] FIG. 6 is a diagram illustrating an overall training framework of a first image processing model in an image processing method provided by an embodiment of the present disclosure. [Figure 7] FIG. 7 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure. [Figure 8] FIG. 8 is a schematic structural diagram of an electronic device provided according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Training an image generator requires a large amount of high-quality paired data so that the network can learn the mapping relationship between different image domains. However, the cost of creating paired images is very high, and they need to be retouched one by one according to the retouching instructions, which increases the cost of creating training data.

[0011] To address the above situation, embodiments of the present disclosure provide an image processing method, apparatus, electronic device, and storage medium.

[0012] Hereinafter, the embodiments of the present disclosure will be described with reference to the drawings. Although specific embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be realized in various forms and should not be construed as being limited to the embodiments described herein, but rather are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and do not limit the scope of protection of the present disclosure.

[0013] It should be understood that the steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. Also, method embodiments may include additional steps and / or omit the performance of illustrated steps. The scope of the present disclosure is not limited in this respect.

[0014] As used herein, the term "including" and variations thereof are open-ended, meaning "including, but not limited to." The term "based on" means "based at least in part on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one other embodiment," and the term "some embodiments" means "at least some embodiments." Definitions of other terms are provided below.

[0015] It should be noted that concepts such as "first" and "second" referred to in this disclosure are used only to distinguish different devices, modules, or units, and are not used to limit the order or interdependence of functions performed by these devices, modules, or units.

[0016] It should be noted that the modifications "one" and "multiple" referred to in this disclosure are exemplary and not limiting, and should be understood by one of ordinary skill in the art as "one or more," unless the context clearly indicates otherwise.

[0017] It is to be understood that the data of the embodiments of the present disclosure (including, but not limited to, the data itself, acquisition or use of the data) is subject to the requirements of each respective law, regulation and applicable regulation.

[0018] 1 is a schematic flow chart of an image processing method provided by an embodiment of the present disclosure. The embodiment of the present disclosure can perform image processing based on a small-scale model after distilling the small-scale model with a large-scale model. The method can be performed by an image processing device, which can be realized in the form of software and / or hardware and can be disposed in an electronic device such as a computer device.

[0019] As shown in FIG. 1, the image processing method provided by this embodiment includes: An original image to be processed is obtained, step S110; An original image is input to a first image processing model (step S120); Step S130: processing the original image with a first image processing model to generate a target image, the first image processing model and the second image processing model are generated by alternately training online, the monitoring information during the training process of the first image processing model includes at least a part of the image generated by the second image processing model during the training process, and the model scale of the first image processing model is smaller than the model scale of the second image processing model; and outputting the target image. S140.

[0020] In an embodiment of the present disclosure, the image processing method can generate a target image in one image domain based on an original image in another image domain. The process of generating the target image based on the original image may be performed by a first image processing model.

[0021] Here, the first image processing model and the second image processing model may be generators of Generative Adversarial Networks (GAN) or other generators capable of generating an image in one image domain based on an image in another image domain. The model scale of the first image processing model being smaller than the model scale of the second image processing model may mean that the model width (also referred to as the number of channels of the model) of the first image processing model is smaller than the model width of the second image processing model and / or the model depth (also referred to as the number of network layers of the model) of the first image processing model is smaller than the model depth of the second image processing model. The first image processing model is considered to be a simple model with a small scale, and the second image processing model is considered to be a complex model with a large scale.

[0022] Generally, the training effect of a large model is better than that of a small model when trained based on the same true labeled data pair. That is, the training effect of the second image processing model is better than that of the first image processing model when trained based on the same true labeled data pair. By training the first image processing model using at least a part of the image generated by the second image processing model as monitoring information (i.e., model distilling the first image processing model with the second image processing model), the performance of the first image processing model can be made closer to that of the second image processing model.

[0023] Here, the first image processing model may be optimized only by the second image processing model, that is, the monitoring information of the first image processing model may all come from the second image processing model. For example, the first image processing model may be optimized using images generated by the second image processing model based on labeled samples and at least a portion of pseudo-labeled images generated based on unlabeled samples as monitoring information. In such a training method, the second image processing model may be called a teacher generator, and the first image processing model may be called a student generator. Model distillation can achieve model size compression, which is advantageous for deploying a small-scale, high-performance first image processing model on a resource-limited device.

[0024] If, after the training of the second image processing model is completed, the images generated by the second image processing model are used as monitoring information to train the first image processing model, it will require a long time and a large amount of calculations to train the first image processing model from scratch to approach the trained second image processing model and to train the first image processing model to have performance equivalent to that of the second image processing model.

[0025] In view of this, in an embodiment of the present disclosure, the first image processing model and the second image processing model can be generated by online alternating training. The online alternating training process may, for example, include a step in which the second image processing model is trained based on a true labeled data pair obtained in a current round, and a step in which the first image processing model is trained by imitating the training process of the current round of the second image processing model with an image generated by the second image processing model after the training of the current round as monitoring information. Here, the true labeled data pair is a data pair composed of a labeled sample and a true labeled image, the image domain of the labeled sample is the same as the image domain of the original image, and the image domain of the true labeled image is the same as the image domain of the target image.

[0026] In the training process of the second image processing model, it is considered that the first image processing model undergoes one distillation every time the training is repeated, so that the first image processing model is trained to gradually follow the training of the second image processing model. By training in a gradual alternating manner in this way, the distillation of the model from the second image processing model to the first image processing model can be completed with a small amount of calculation. In the embodiment of the present disclosure, such a method of training in a gradual alternating manner can be called online distillation. In practical applications, compared with the case where the first image processing model is distilled after the training of the second image processing model is completed, the amount of model calculation can be reduced by 30% by training the first image processing model by online distillation.

[0027] In the embodiment of the present disclosure, the training process of the second image processing model can refer to the entire process from the start of training of the second image processing model to the completion of training. In the training process of the second image processing model, in addition to training based on true labeled data pairs, pseudo-labeled images can also be generated based on unlabeled samples. Here, the image domain of the unlabeled samples is the same as the image domain of the original samples, and the image domain of the pseudo-labeled images is the same as the image domain of the target image. That is, in addition to the iterative training of each round following the second image processing model, the first image processing model may perform its own training based on at least a part of the pseudo-labeled images. Therefore, the amount of training data of the first image processing model can be increased and the generalization of the first image processing model can be improved without adding the cost of creating additional pair data.

[0028] The pseudo-labeled image can provide monitoring information to the first image processing model, but cannot provide monitoring information to the second image processing model. That is, the first image processing model can be trained based on the pseudo-labeled image, but the second image processing model cannot be trained based on the pseudo-labeled image. The first image processing model may have performance variations after independent training based on the pseudo-labeled image. Therefore, in order to reduce such performance variations, independent training can be performed based on a part of high-quality pseudo-labeled images. However, when training is performed based on all pseudo-labeled images, the pseudo-labeled images in the embodiment of the present disclosure are generated in the training process of the second image processing model, so that when the second image processing model trains again based on the true labeled data pair after generating the pseudo-labeled image, the first image processing model is trained following the second image processing model. Therefore, in order to obtain the effects of cost reduction and efficiency improvement, the variation introduced into the first image processing model by the pseudo label may be corrected in a timely manner, and the image generation quality of the first image processing model may be ensured. The first image processing model obtained in this manner can ensure excellent image processing performance while achieving weight reduction.

[0029] In the embodiment of the present disclosure, the first image processing model can be applied to a lightweight terminal device. The lightweight terminal device can be considered to be a terminal device with limited resources, such as a terminal device such as a mobile phone. The first image processing model after completion of training has a small model scale, good model generalization and generated image quality, which effectively reduces the difficulty of model deployment in a mobile terminal device with limited resources and other lightweight Internet of Things devices. The first image processing model after completion of training can generate a high-quality target image based on an original image.

[0030] An embodiment of the present disclosure realizes collaborative compression of model dimension and training data dimension in the GAN training process as follows.

[0031] By alternately training the second image processing model and the first image processing model through online distillation, the first image processing model is gradually induced to learn the optimization process of the second image processing model step by step, so that the first image processing model can output images with similar image quality to the second image processing model with less computational effort, and the compression of model dimension is completed.

[0032] The second image processing model introduces a large amount of unlabeled samples during the training process to generate pseudo-labeled images, and trains the first image processing model based on at least a part of the pseudo-labeled images, thereby transforming the traditional training method of training based on true labeled data pairs into a method of training based on true labeled data pairs together with pseudo-labeled images, thereby compressing the demand for true labeled data pairs, i.e., realizing compression of the training data dimension. The pseudo-labeled images can provide additional supervision information for the training of the first image processing model, which increases the amount of training data for the first image processing model without adding the cost of creating additional pair data, improves the generalization of the first image processing model, and is more advantageous in that the model learns the structural features of the image domain of the generated image.

[0033] In the embodiment of the present disclosure, the first image processing model is a small-scale model, and the second image processing model is a large-scale model. When trained based on the same true labeled data pair, the large-scale model generally has a better training effect than the small-scale model. The first image processing model and the second image processing model are trained alternately online, and the first image processing model is trained using images generated during the training process of the second image processing model as monitoring information, and the first image processing model imitates each round of the iterative training process of the second image processing model, and the training proceeds step by step. By proceeding with the training step by step, the distillation of the model from the second image processing model to the first image processing model can be completed with a small amount of calculation, and the performance of the first image processing model is closer to that of the second image processing model.

[0034] In the training process of the second image processing model, in addition to training based on true labeled data pairs, pseudo-labeled images can also be generated based on unlabeled samples. That is, in addition to following each round of the iterative training process of the second image processing model, the first image processing model can also perform its own training based on at least a part of the pseudo-labeled images, so as to increase the amount of training data of the first image processing model and improve the generalization of the first image processing model without adding the cost of creating additional pair data.

[0035] The pseudo-labeled image can provide the first image processing model with monitoring information for training, but cannot provide the second image processing model with monitoring information for training, so that the performance of the first image processing model may vary after being trained based on the pseudo-labeled image. However, since the pseudo-labeled image is generated in the training process of the second image processing model, when the second image processing model is trained again based on the true labeled data pair after generating the pseudo-labeled image, the first image processing model continues to train according to the second image processing model. Therefore, the variation introduced into the first image processing model by the pseudo label can be corrected in a timely manner, the image generation quality of the first image processing model can be ensured, and the effects of cost reduction and efficiency improvement can be obtained.

[0036] The embodiments of the present disclosure can be combined with each of the image processing methods provided in the above embodiments. The second image processing model can be trained based on true labeled data pairs in the training process, and generate pseudo-labeled images based on unlabeled samples. In the image processing method provided by this embodiment, the generation process of pseudo-labeled images is described. The second image processing model trained based on the current true labeled data can be used to generate pseudo-labeled data based on unlabeled data, thereby providing additional supervision information for the training of the first image processing model.

[0037] 2 is a schematic flow chart showing the generation of a pseudo-labeled image in the image processing method provided by the embodiment of the present disclosure. As shown in FIG. 2, in the image processing method provided by the embodiment, the pseudo-labeled image can be generated according to the following steps.

[0038] S210, the second image processing model and the classifier obtain unlabeled samples as inputs of the second image processing model in the process of performing adversarial training based on true labeled data pairs.

[0039] In an embodiment of the present disclosure, the second image processing model may be a generator in a GAN, and may perform adversarial training together with a classifier in the GAN. Illustratively, FIG. 3 is a schematic diagram of a training framework for a second image processing model in an image processing method provided by an embodiment of the present disclosure. Referring to FIG. 3, in some embodiments, the step of the second image processing model and the classifier performing adversarial training based on a true labeled data pair includes: True labeled data pairs

number

number

[0040] Here, the true labeled data pair

number

number

number

[0041] where x is each labeled sample, y is each true labeled image, and G T (x) is each first image p generated by the second image processing model based on each labeled sample. t l and

number

[0042] Referring again to FIG. 3, the second image processing model G T and the discriminator D is the true labeled data pair

number

number

[0043] Here, the meaning of the same symbols can be referred to above. By training the second image processing model based on the reconstruction loss, the output of the second image processing model can be made closer to the true labeled image.

[0044] In this case, the second image processing model G T The complete optimization loss function for can be expressed as

number

[0045] In the process of performing adversarial training based on the true labeled data pair, the second image processing model and the classifier may acquire an unlabeled sample as an input of the second image processing model, for example, in an interval for performing adversarial training based on the true labeled data pair. Here, acquiring an unlabeled sample may be, for example, randomly extracting an unlabeled sample from an unlabeled sample set. Here, the number of unlabeled samples acquired each time may be at least one. Since the acquisition of the unlabeled sample is performed in the interval of the adversarial training, that is, the second image processing model can continue to perform adversarial training even after generating the candidate pseudo-label. That is, after the first image processing model is trained based on the pseudo-labeled image, the optimization process of the second image processing model can continue to be imitated, and the generalization of the first image processing model can be improved, as well as the variation that may be introduced by the pseudo-labeled image can be compensated for, and the performance of the first image processing model can be ensured.

[0046] In some embodiments, after performing successive adversarial training based on a predetermined percentage (e.g., one-third, one-half, etc.) of true labeled data pairs, unlabeled samples can be obtained as input to the second image processing model during an interval period before performing adversarial training based on the remaining percentage of true labeled data pairs. By performing pre-training based on a predetermined percentage of true labeled data in advance, the pseudo-labeled images generated by the second image processing model can be more aligned with the structural features of the image domain of the generated image, and to some extent provide better supervision information to the first image processing model.

[0047] In some embodiments, in the process of performing adversarial training on the second image processing model and the classifier based on true labeled data pairs, the step of obtaining unlabeled samples as input to the second image processing model may include a step of obtaining a predetermined number of true labeled data pairs each time and obtaining unlabeled samples as input to the second image processing model after performing adversarial training on the second image processing model and the classifier.

[0048] In these embodiments, obtaining unlabeled samples at intervals for performing adversarial training based on true labeled data pairs may be obtaining unlabeled samples after performing adversarial training every time a preset number (e.g., one, two, etc.) of true labeled data pairs are obtained, where the preset number is to some extent inversely proportional to the degree of compression of the training data of the first image processing model. For example, performing adversarial training every time one true labeled data pair is obtained and then obtaining one unlabeled sample to generate candidate pseudo-labeled images can compress 50% of the first image processing model's demand for true labeled data pairs when all the candidate pseudo-labeled images are used to train the first image processing model, and performing adversarial training every time two true labeled data pairs are obtained and then obtaining one unlabeled sample to generate candidate pseudo-labeled images can compress 33% of the first image processing model's demand for true labeled data pairs when all the candidate pseudo-labeled images are used to train the first image processing model. On the other hand, if the preset number is reduced, the training process of the first image processing model simulating the second image processing model may require a large amount of calculation. Therefore, the size of the preset number can be set according to the actual situation, taking into consideration the balance between the compression amount of the training data of the first image processing model and the training calculation amount.

[0049] S220, a second image processing model generates candidate pseudo-labeled images based on the unlabeled samples.

[0050] In this embodiment, the second image processing model can generate candidate pseudo-labels based on unlabeled samples based on each current model parameter during training. The second image processing model can also continue to perform adversarial training based on true labeled data pairs after generating pseudo-labeled images based on the acquired unlabeled samples. By continuing to optimize the second image processing model based on the supervision information provided by the true labeled data pairs, the first image processing model continues to mimic the optimization process of the second image processing model.

[0051] S230, the candidate pseudo-labeled images are selected by a classifier to obtain a final pseudo-labeled image.

[0052] Unlabeled sample set

number

number

[0053] 2nd image processing model G T In the training process of the second image processing model G, there exists a classifier D that is in an adversarial relationship with the first image processing model G. T The quality of the generated image can be properly judged by the pseudo-labeled image p t u If the classifier D judges that the image is close to the actual image, then p t u The quality of the classifier D is high, and t u If it can be determined that p is not a real image, t u is of poor quality.

[0054] In this embodiment, a high-quality pseudo-labeled image can be selected from a large number of candidate pseudo-labels by a discriminator and transmitted to a first image processing model for training. Here, the selected high-quality pseudo-labeled image can be inputted into the first image processing model for training immediately or non-immediately. For example, after accumulating a certain number of pseudo-labeled images, they can be inputted into the first image processing model at once to perform training, etc. Here, the input timing of the pseudo-labeled image is not strictly limited, and other methods of inputting the pseudo-labeled image into the first image processing model can also be applied here, and this is not exhaustive.

[0055] In some embodiments, the step of selecting the candidate pseudo-labeled images by the classifier may include the steps of evaluating the authenticity of the candidate pseudo-labeled images by the classifier to obtain an evaluation result, and selecting the candidate pseudo-labeled images based on predetermined evaluation criteria and the evaluation result.

[0056] Here, the candidate pseudo-labeled image p t u Rate the authenticity of the image and give it a score S t uObtain and

number

[0057] In these embodiments, in order to ensure the quality of the pseudo-labeled images, a classifier can be used to select candidate pseudo-labeled images, and the selected high-quality pseudo-labeled images can improve the generalization of the student generator. The selection method contributes to discovering the structural features of unlabeled samples of the same style, which can be complemented with the true labeled data pair to train the first image processing model, thus alleviating the expensive and time-consuming process of generating and selecting training data, reducing costs and improving efficiency.

[0058] The embodiments of the present disclosure have described a process for generating pseudo-labeled images. A second image processing model trained on the current true labeled data can be used to generate pseudo-labeled data based on unlabeled data, thereby providing additional supervision information for training the first image processing model. The image processing method provided by the embodiments of the present disclosure and the image processing method provided by the above embodiments belong to the same disclosure idea, and the above embodiments can be referred to for technical details not described in detail in this embodiment. In this embodiment and the above embodiments, the same technical features have the same beneficial effects.

[0059] The embodiments of the present disclosure can be combined with each of the image processing methods provided in the above embodiments. In the image processing method provided by this embodiment, a training step of a first image processing model is described. During the training process of the first image processing model, not only can the first image generated by the second image processing model based on the labeled sample be used as supervisory information, but also at least a part of the pseudo-labeled image generated by the second image processing model based on the unlabeled sample can be trained as supervisory information. Through the labeled distillation loss of the first image and the unlabeled distillation loss of the pseudo-labeled image between the first image processing model and the second image processing model, the first image processing model can be trained to improve the generalization of the one image processing model and the quality of the generated image.

[0060] Illustratively, Fig. 4 is a schematic diagram of a training framework when the first image processing model uses pseudo-labeled images as supervision information in the image processing method provided by the embodiment of the present disclosure. Referring to Fig. 4, in some embodiments, when the first image processing model uses pseudo-labeled images as supervision information, the first image processing model is trained according to the following steps:

[0061] First image processing model G s The input of the pseudo-labeled image pt u The unlabeled sample x corresponding to j u and the first image processing model G s Therefore, the unlabeled sample x j u Based on the second image p s u Generate a pseudo-labeled image p t u and the second image p s u Based on the distillation loss (unlabeled distillation loss L kd u (which can be called the distillation loss L kd u A first image processing model is trained based on the

[0062] Here, the pseudo-labeled image p t u and the second image p s u The reconstitution loss between the distillation loss L kd u and / or, in some embodiments, the pseudo-labeled image p t u and the second image p s u The step of determining the distillation loss based on the second image processing model G T Let p be a pseudo-labeled image. t u The feature image φ in the process of generating j (p t u ), and the first image processing model G S is the second image p s u The feature image φ in the process of generating j (p t u ) based on the perceptual loss L preu and determining the perceptual loss L preu The distillation loss L kd u and using the same as above.

[0063] Here, the perceptual loss L preu Using the pseudo-labeled image p t u and the second image p s u When measuring the difference between preu is the feature reconstruction loss L fea and the style reconstruction loss L style may include at least one of the following:

[0064] Feature reconstruction loss L fea By p t u andp s u We can encourage the feature representations L to have similar feature representations, which can be obtained from the φ-measurements of pre-trained networks, e.g., φ-measurements of a super-resolution test sequence network (Visual Geometry Group, VGG). Here, the feature reconstruction loss L fea can be defined as follows:

number

number

[0065] Style reconstruction loss L style The color, texture, general pattern, etc. t u andp s uis introduced to penalize the difference in style features of the two, where the style reconstruction loss L style can be defined as follows:

number

[0066] It is contemplated that the unlabeled distillation loss may include a reconstruction loss and / or a perceptual loss between the pseudo-labeled image and the second image. When the unlabeled distillation loss includes a reconstruction loss and a perceptual loss, the unlabeled distillation loss may be a sum or weighted sum of the two, etc.

[0067] In some embodiments, the first image processing model G S The model is trained according to the following steps: s u and, accordingly, determining the distillation loss L kd u Based on the first image processing model G S and further comprising the step of training the distillation loss L kd u and total variation loss L tv The method may further include training a first image processing model based on

[0068] Here, the total variable loss L tv By introducing the first image processing model G S We can improve the spatial smoothness of the output image of the vector. To balance the above losses, we use three hyperparameters, λ fea , λ style , λ tv In this case, the total unlabeled distillation loss L kd u (p t u ,ps u ) can be defined as follows:

number

[0069] In some embodiments, during the training process, the second image processing model G T The images generated by

number

[0070] First image processing model G S The first image p t l The labeled sample x corresponding to i and the first image processing model G S Labeled sample x by i Based on the third image p s lThen, the first image p t l and the third image p s l Based on the distillation loss (labeled distillation loss L kd l (which can be called the distillation loss L kd l Based on the first image processing model G S Train.

[0071] Here, the labeled distillation loss L kd l The calculation process of the unlabeled distillation loss L kd u The calculation process of can be seen here. Here, the labeled distillation loss L kd l In the first image p t l and the third image p s l The perceptual loss may include the reconstruction loss and / or perceptual loss between the feature reconstruction loss L fea and the style reconstruction loss L style At least one of the following may be included: kd l and the third image p s l Total variable loss L tv Based on the first image processing model G S You can also train.

[0072] First image processing model G S In the training process, the labeled distillation loss L kd l and unlabeled distillation losses L kd u When both are included, the distillation loss L of the entire first image processing model kd can be defined as follows:

number

[0073] Illustratively, Fig. 6 is an overall training framework diagram of the first image processing model in the image processing method provided by the embodiment of the present disclosure. Referring to Fig. 6, in the training process of the first image processing model, the collaborative compression of model dimension and training data dimension is realized.

[0074] Referring to FIG. 6, the second image processing model and the classifier can perform adversarial training based on the true labeled data pair, and each time the second image processing model is iteratively optimized, the first image processing model can be induced to imitate the optimization process of the second image processing model through the labeled distillation loss, thereby realizing online distillation of the first image processing model. Here, the first image processing model can be called a student generator, and the second image processing model can be called a teacher generator. Through online distillation, the model dimension can be compressed, which is advantageous for deploying a small-scale, high-performance first image processing model on a resource-limited device.

[0075] In the adversarial training process, the second image processing model can obtain unlabeled samples to generate candidate pseudo-labeled images. Then, the candidate pseudo-labeled images can be screened by a discriminator to obtain high-quality pseudo-labeled images. The screened pseudo-labeled images can introduce an unlabeled distillation loss into the first image processing model to train the first image processing model. By generating pseudo-labeled images, the amount of training data for the first image processing model can be increased without adding the cost of creating additional pair data, thereby realizing compression of the dimension of the training data and improving the generalization of the first image processing model.

[0076] The embodiment of the present disclosure has described the training step of the first image processing model. In the training process of the first image processing model, not only the first image generated by the second image processing model based on the labeled sample can be used as monitoring information, but also the pseudo-labeled image generated by the second image processing model based on the unlabeled sample can be trained as monitoring information. The first image processing model can be trained by the labeled distillation loss for the first image and the unlabeled distillation loss for the pseudo-labeled image between the first image processing model and the second image processing model to improve the generalization of the first image processing model and the quality of the generated image. The image processing method provided by the embodiment of the present disclosure and the image processing method provided by the above embodiment belong to the same disclosure idea, and the above embodiment can be referred to for technical details not described in detail in this embodiment. In this embodiment and the above embodiment, the same technical features have the same beneficial effects.

[0077] 7 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure. The image processing device provided by this embodiment can perform image processing based on the small-scale model after distilling the small-scale model through the large-scale model.

[0078] As shown in FIG. 7, the image processing device provided by the embodiment of the present disclosure includes: an image acquisition module 710 configured to acquire an original image to be processed; an input module 720 configured to input an original image to the first image processing model; A generation module 730 configured to process an original image with a first image processing model to generate a target image, the first image processing model and the second image processing model are generated by alternately training online, the monitoring information in the training process of the first image processing model includes at least a part of the image generated by the second image processing model in the training process, and the model scale of the first image processing model is smaller than the model scale of the second image processing model; and an output module 740 configured to output the target image.

[0079] In some embodiments, the second image processing model is trained based on true labeled data pairs in a training process and generates pseudo-labeled images based on unlabeled samples, and accordingly, the image processing device may include a model training module, and the model training module may include a pseudo-label generation unit. The pseudo label generation unit may be configured to generate the pseudo labeled image based on the following steps: The second image processing model and the classifier obtain unlabeled samples as inputs of the second image processing model in a process of performing adversarial training based on the true labeled data pairs; A second image processing model generates candidate pseudo-labeled images based on the unlabeled samples; A classifier selects the candidate pseudo-labeled images to obtain the final pseudo-labeled image.

[0080] In some embodiments, the pseudo label generation unit comprises: Each time a preset number of true labeled data pairs are obtained, the second image processing model and the classifier can be configured to perform adversarial training and then obtain unlabeled samples as input to the second image processing model.

[0081] In some embodiments, the pseudo label generation unit comprises: Evaluate the authenticity of the candidate pseudo-labeled images using a classifier to obtain an evaluation result; The system can be configured to select candidate pseudo-labeled images based on preset evaluation criteria and evaluation results.

[0082] In some embodiments, the model training module may include a second image processing model training unit; The second image processing model training unit may be configured to perform adversarial training on the second image processing model and the classifier based on the true labeled data pair; The second image processing model training unit is Take the labeled samples in the true-labeled data pairs as input for the second image processing model, A second image processing model generates a first image based on the labeled examples; Among the true labeled data pairs, obtain the true labeled images corresponding to the labeled samples, A classifier determines whether the first image and the true labeled image are of the same type; A second image processing model is trained based on the classification results. The classifier can be configured to train the classifier based on the determination that the objects are not of the same type.

[0083] In some embodiments, the second image processing model training unit: Based on the first image and the true labeled image, we determine the reconstruction loss. The method may be further configured to train a second image processing model based on the reconstruction loss.

[0084] In some embodiments, the model training module may include a first image processing model training unit; When the first image processing model uses the pseudo-labeled image as supervision information, the first image processing model training unit can be configured to train the first image processing model according to the following steps:

[0085] Obtain unlabeled samples corresponding to the pseudo-labeled images as inputs of a first image processing model; generating a second image based on the unlabeled samples using a first image processing model; Determine the distillation loss based on the pseudo-labeled image and the second image, A first image processing model is trained based on the distillation loss.

[0086] In some embodiments, the first image processing model training unit comprises: Determine a perceptual loss based on a feature image in the process in which the second image processing model generates a pseudo-labeled image and a feature image in the process in which the first image processing model generates a second image; It can be configured to use the perceptual loss as the distillation loss.

[0087] In some embodiments, the perceptual loss includes at least one of a feature reconstruction loss and a style reconstruction loss.

[0088] In some embodiments, the first image processing model training unit may be further configured as follows. determining a total variation loss based on the second image; Accordingly, training the first image processing model based on the distillation loss includes training the first image processing model based on the distillation loss and the total variation loss.

[0089] In some embodiments, the images generated by the second image processing model during the training process include: The second image processing model further includes a first image generated based on the labeled samples in the true-labeled data pairs; When the first image processing model uses the first image as the monitoring information, the first image processing model training unit can be configured to train the first image processing model according to the following steps:

[0090] Obtaining labeled samples corresponding to a first image as an input of a first image processing model; generating a third image based on the labeled samples using the first image processing model; determining a distillation loss based on the first image and the third image; A first image processing model is trained based on the distillation loss.

[0091] The image processing device provided by the embodiments of the present disclosure can execute the image processing method provided by any of the embodiments of the present disclosure, and has functional modules and beneficial effects corresponding to the executing method.

[0092] It should be noted that each unit and module included in the above-mentioned device is simply partitioned according to functional logic, but is not limited to the above partitions as long as the corresponding functions can be realized. Furthermore, the specific names of each functional unit are intended to make them easier to distinguish from one another, and are not used to limit the scope of protection of the embodiments of the present disclosure.

[0093] Referring now to FIG. 8, a schematic structural diagram of an electronic device (e.g., a terminal device or server in FIG. 8) 800 suitable for implementing the embodiments of the present disclosure is shown. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDA), tablet computers (PAD), portable media players (PMP), and in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital televisions (TV) and desktop computers. The electronic device shown in FIG. 8 is merely an example and does not limit the functions and scope of use of the embodiments of the present disclosure.

[0094] As shown in Fig. 8, the electronic device 800 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 801 capable of performing various appropriate operations and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. The RAM 803 further stores various programs and data necessary for the operation of the electronic device 800. The processing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0095] Typically, the following devices can be connected to the I / O interface 805: input devices 806, including, for example, a touch panel, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 807, including, for example, a Liquid Crystal Display (LCD), speaker, vibrator, etc.; storage devices 808, including, for example, a magnetic tape, hard disk, etc.; and communication devices 809. The communication devices 809 can enable the electronic device 800 to communicate and exchange data with other devices, wirelessly or via wires. Although FIG. 8 illustrates the electronic device 800 with various devices, it should be understood that it is not necessary to implement or include all of the devices shown. Instead, more or fewer devices can be implemented or included.

[0096] In particular, according to an embodiment of the present disclosure, the process described with reference to the flowcharts may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product including a computer program embodied in a non-transitory computer-readable medium, the computer program including program code for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication device 809, or may be installed from the storage device 808 or the ROM 802. When the computer program is executed by the processing device 801, it performs the above functions defined in the image processing method of the embodiment of the present disclosure.

[0097] The electronic device provided by the embodiment of the present disclosure and the image processing method provided by the above embodiment belong to the same disclosed concept, and the technical details not described in detail in this embodiment can be referred to the above embodiment. This embodiment has the same beneficial effects as the above embodiment.

[0098] An embodiment of the present disclosure provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method provided by the above embodiment.

[0099] It should be noted that the above-mentioned computer readable medium of the present disclosure may be a computer readable signal medium, a computer readable storage medium, or any combination of the above two. The computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor apparatus or device, or any combination thereof. Examples of computer readable storage media include, but are not limited to, an electrical connection with one or more wires, a portable computer magnetic disk, a hard drive, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory (FLASH), an optical fiber, a portable compact magnetic disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable medium may be any tangible medium that contains or stores a program, which can be used by or in combination with an apparatus or device for executing instructions. In this disclosure, a computer-readable signal medium can take many forms, including, but not limited to, a data signal in baseband or propagated as part of a carrier in which computer-readable program code is embodied. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transmit a program for use by or in conjunction with an apparatus or device for executing instructions. The program code embodied on a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wire, optical cable, radio frequency (RF), etc., or any suitable combination thereof.

[0100] In some embodiments, clients and servers may communicate using any now known or later developed network protocol, such as HyperText Transfer Protocol (HTTP), and may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include local area networks (LANs), wide area networks (WANs), networks connected between wide area networks (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any now known or later developed networks.

[0101] The computer-readable medium may be included in the electronic device, or may exist independently of the electronic device.

[0102] The computer-readable medium includes one or more programs embedded therein, and when the one or more programs are executed by an electronic device, the electronic device: The method comprises obtaining an original image to be processed, inputting the original image into a first image processing model, processing the original image with the first image processing model to generate a target image, in which the first image processing model and the second image processing model are generated by alternately training online, the monitoring information in the training process of the first image processing model includes at least a part of the image generated by the second image processing model in the training process, the model scale of the first image processing model is smaller than the model scale of the second image processing model, and outputting the target image.

[0103] The storage medium may be a non-transitory storage medium.

[0104] Computer program code for carrying out the operations of the present disclosure can be written in one or more programming languages, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can run completely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer or to an external computer (e.g., an Internet connection via an Internet Service Provider) via any kind of network, such as a Local Area Network (LAN) or Wide Area Network (WAN).

[0105] The flowcharts and block diagrams in the drawings illustrate possible architectures, functions, and operations of methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, a program segment, or a portion of code, including one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions shown in the blocks may be executed in a different order than that shown in the figures. For example, two blocks shown in succession may in fact be executed substantially in parallel or may be executed in the reverse order depending on the functionality involved. It should be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented using a combination of specialized hardware and computer instructions.

[0106] The units according to the embodiments of the present disclosure may be realized by software or hardware. The names of the units and modules may not limit the units and modules themselves.

[0107] The functions described herein may be performed, at least in part, by one or more hardware logic components. For example, but not limited to, typical types of hardware logic components that may be used include Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Parts (ASSPs), System on Chip (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0108] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program used by or in conjunction with an instruction execution device or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device or device, or any suitable combination thereof. Examples of machine-readable storage media include an electrical connection by one or more wires, a portable computer disk, a hard drive, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0109] According to one or more embodiments of the present disclosure, obtaining an original image to be processed; inputting the original image into a first image processing model; processing the original image with the first image processing model to generate a target image; The first image processing model and the second image processing model are generated by alternately training online, and the monitoring information in the training process of the first image processing model includes at least a part of the image generated by the second image processing model in the training process, and the model scale of the first image processing model is smaller than the model scale of the second image processing model; and outputting the target image.

[0110] According to one or more embodiments of the present disclosure, In some embodiments, the second image processing model is trained based on true labeled data pairs in a training process, and the image processing method further includes generating pseudo-labeled images based on unlabeled samples, the pseudo-labeled images being: The second image processing model and the classifier perform adversarial training based on the true labeled data pairs, and in the process, obtain unlabeled samples as inputs of the second image processing model; generating candidate pseudo-labeled images based on the unlabeled samples using the second image processing model; and selecting the candidate pseudo-labeled images using the classifier to obtain a final pseudo-labeled image.

[0111] According to one or more embodiments of the present disclosure, there is provided an image processing method further comprising the steps of:

[0112] In some embodiments, in the process of performing adversarial training on the second image processing model and the classifier based on the true labeled data pairs, the step of obtaining unlabeled samples as inputs to the second image processing model includes: The method includes a step of acquiring unlabeled samples as input to the second image processing model after performing adversarial training on the second image processing model and a classifier each time a predetermined number of true labeled data pairs are acquired.

[0113] According to one or more embodiments of the present disclosure, there is provided an image processing method further comprising the steps of:

[0114] In some embodiments, the step of selecting the candidate pseudo-labeled images using a classifier comprises: evaluating the authenticity of the candidate pseudo-labeled image by the classifier to obtain an evaluation result; and selecting the candidate pseudo-labeled images based on a preset evaluation criterion and the evaluation result.

[0115] According to one or more embodiments of the present disclosure, there is provided an image processing method, further comprising the steps of:

[0116] In some embodiments, the step of performing adversarial training of the second image processing model and the classifier based on the true labeled data pairs includes: obtaining labeled samples in the true labeled data pairs as inputs to the second image processing model; generating a first image based on the labeled samples using the second image processing model; obtaining a true labeled image corresponding to the labeled sample from the true labeled data pair; determining, by the classifier, whether the first image and the true labeled image are of the same type; training the second image processing model based on the images determined to be the same type by the classifier; and training the classifier with the goal of determining that the objects are not of the same type by the classifier.

[0117] According to one or more embodiments of the present disclosure, there is provided an image processing method, further comprising the steps of:

[0118] In some embodiments, the second image processing model and the classifier are subjected to adversarial training based on the true labeled data pairs, determining a reconstruction loss based on the first image and the true labeled image; and training the second image processing model based on the reconstruction loss.

[0119] According to one or more embodiments of the present disclosure, there is provided an image processing method, further comprising the steps of:

[0120] In some embodiments, when the first image processing model uses the pseudo-labeled images as surveillance information, the first image processing model: obtaining unlabeled samples corresponding to the pseudo-labeled images as inputs of the first image processing model; generating a second image based on the unlabeled samples using the first image processing model; determining a distillation loss based on the pseudo-labeled image and the second image; training the first image processing model based on the distillation loss.

[0121] According to one or more embodiments of the present disclosure, there is provided an image processing method, further comprising the steps of:

[0122] In some embodiments, determining a distillation loss based on the pseudo-labeled image and the second image comprises: determining a perceptual loss based on a feature image in the process in which the second image processing model generates a pseudo-labeled image and a feature image in the process in which the first image processing model generates a second image; and using the perceptual loss as a distillation loss.

[0123] According to one or more embodiments of the present disclosure, there is provided an image processing method, further comprising the steps of:

[0124] In some embodiments, the perceptual loss includes at least one of a feature reconstruction loss and a style reconstruction loss.

[0125] According to one or more embodiments of the present disclosure, there is provided an image processing method further comprising the steps of:

[0126] In some embodiments, the first image processing model comprises: determining a total variation loss based on the second image; Accordingly, training the first image processing model based on the distillation loss includes training a first image processing model based on the distillation loss and the total variation loss.

[0127] According to one or more embodiments of the present disclosure, there is provided an image processing method, further comprising the steps of:

[0128] In some embodiments, the images generated by the second image processing model in a training process include: the second image processing model further comprises a first image generated based on the labeled samples in the true labeled data pairs; Accordingly, when the first image processing model uses the first image as surveillance information, the first image processing model is trained based on the following steps. obtaining, as an input of the first image processing model, labeled samples corresponding to the first image; generating a third image based on the labeled samples using the first image processing model; determining a distillation loss based on the first image and the third image; The first image processing model is trained based on the distillation loss.

[0129] According to one or more embodiments of the present disclosure, the first image processing model is applied to a lightweight terminal device.

[0130] According to one or more embodiments of the present disclosure, an image acquisition module configured to acquire an original image to be processed; an input module configured to input the original image into a first image processing model; A generation module configured to process the original image with the first image processing model to generate a target image, the first image processing model and the second image processing model are generated by alternately training online, the monitoring information in the training process of the first image processing model includes at least a part of the image generated by the second image processing model in the training process, and the model scale of the first image processing model is smaller than the model scale of the second image processing model; an output module configured to output the target image.

[0131] Those skilled in the art should understand that the disclosure scope of the present disclosure is not limited to the embodiments consisting of the specific combinations of the above technical features, but should also cover other embodiments formed by any combination of the above technical features or their equivalent features without departing from the concept disclosed above. For example, embodiments formed by replacing the above features with technical features having similar functions (but not limited to these) disclosed in the present disclosure are also covered.

[0132] Also, although operations are shown in a particular order, this should not be understood as requiring that these operations be performed in the particular order shown, or in sequential order. In certain situations, multitasking and parallel processing may be advantageous. Similarly, although certain implementation details are included in the above description, these should not be construed as limiting the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

[0133] Although the subject matter has been described in language specific to structural features and / or logical operations of methods, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or operations described above, but rather, the specific features and operations described above are merely example forms of implementing the claims.

Claims

1. 1. An image processing method, comprising: obtaining an original image to be processed; inputting the original image into a first image processing model; processing the original image with the first image processing model to generate a target image; the first image processing model and the second image processing model are generated by alternately training online; The monitoring information during the training process of the first image processing model includes at least a portion of the images generated by the second image processing model during the training process; a model scale of the first image processing model is smaller than a model scale of the second image processing model; Steps and and outputting the target image. In a training process, the second image processing model is trained based on true labeled data pairs and generates pseudo-labeled images based on unlabeled samples; The pseudo-labeled image is The second image processing model and the classifier are a process of performing adversarial training based on the true labeled data pairs, the step of acquiring unlabeled samples as inputs of the second image processing model, the unlabeled samples being acquired at intervals of performing adversarial training based on the true labeled data pairs; generating candidate pseudo-labeled images based on the unlabeled samples using the second image processing model; selecting the candidate pseudo-labeled images using the classifier to obtain a final pseudo-labeled image; The method is generated based on

2. In the process of performing adversarial training on the second image processing model and the classifier based on the true labeled data pair, the step of obtaining an unlabeled sample as an input of the second image processing model includes: performing adversarial training on the second image processing model and the classifier each time a predetermined number of true labeled data pairs are obtained, and then obtaining unlabeled samples as inputs to the second image processing model; The method of claim 1 , comprising:

3. The step of selecting the candidate pseudo-labeled images by the classifier includes: evaluating the authenticity of the candidate pseudo-labeled image by the classifier to obtain an evaluation result; selecting the candidate pseudo-labeled images based on a preset evaluation criterion and the evaluation results; The method of claim 1 , comprising:

4. The step of performing adversarial training of the second image processing model and the classifier based on the true labeled data pairs includes: obtaining labeled samples in the true labeled data pairs as inputs to the second image processing model; generating a first image based on the labeled samples using the second image processing model; obtaining a true labeled image corresponding to the labeled sample from the true labeled data pair; determining, by the classifier, whether the first image and the true labeled image are of the same type; training the second image processing model based on the determination by the classifier that the images are of the same type; training the classifier with the goal of determining that the objects are not of the same type by the classifier; The method of claim 1 , comprising:

5. In the process of performing adversarial training on the second image processing model and the classifier based on the true labeled data pair, determining a reconstruction loss based on the first image and the true labeled image; training the second image processing model based on the reconstruction loss; The method of claim 4 further comprising:

6. When the first image processing model uses the pseudo-labeled image as surveillance information, the first image processing model: obtaining unlabeled samples corresponding to the pseudo-labeled images as inputs of the first image processing model; generating a second image based on the unlabeled samples using the first image processing model; determining a distillation loss based on the pseudo-labeled image and the second image; training the first image processing model based on the distillation loss; The method of claim 1 , wherein the training is based on

7. Determining a distillation loss based on the pseudo-labeled image and the second image includes: determining a perceptual loss based on a feature image in the process in which the second image processing model generates a pseudo-labeled image and a feature image in the process in which the first image processing model generates the second image; using the perceptual loss as a distillation loss; The method of claim 6, comprising:

8. The perceptual loss includes at least one of a feature reconstruction loss and a style reconstruction loss. The method of claim 7.

9. The first image processing model is determining a total variation loss based on the second image; Accordingly, the step of training the first image processing model based on the distillation loss includes training the first image processing model based on the distillation loss and the total variation loss. The method according to claim 6.

10. The images generated by the second image processing model during the training process are the second image processing model further comprises a first image generated based on the labeled samples in the true labeled data pairs; Accordingly, when the first image processing model uses a first image as monitoring information, the first image processing model: obtaining labeled samples corresponding to the first image as an input of the first image processing model; generating a third image based on the labeled samples using the first image processing model; determining a distillation loss based on the first image and the third image; training the first image processing model based on the distillation loss; The method of claim 1 , wherein the training is based on

11. The first image processing model is applied to a lightweight terminal device. The method of claim 1.

12. An image processing device, an image acquisition module configured to acquire an original image to be processed; an input module configured to input the original image into a first image processing model; a generation module configured to process the original image with the first image processing model to generate a target image, the first image processing model and the second image processing model are generated by alternately training online; The monitoring information during the training process of the first image processing model includes at least a portion of the images generated by the second image processing model during the training process; a model scale of the first image processing model is smaller than a model scale of the second image processing model; A generation module; an output module configured to output the target image; In a training process, the second image processing model is trained based on true labeled data pairs and generates pseudo-labeled images based on unlabeled samples; The image processing device includes: The second image processing model and the classifier further include a pseudo-label generation unit, which is used to obtain unlabeled samples as inputs of the second image processing model in a process of performing adversarial training based on the true labeled data pairs; The unlabeled samples are obtained at intervals for performing adversarial training based on the true labeled data pairs; generating candidate pseudo-labeled images based on the unlabeled samples using the second image processing model; selecting the candidate pseudo-labeled images using the classifier to obtain a final pseudo-labeled image; Image processing device.

13. An electronic device, one or more processors; a storage device configured to store one or more programs; The one or more programs, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 11. electronic equipment.

14. A storage medium containing computer-executable instructions, comprising: The computer-executable instructions, when executed by a computer processor, perform the method according to any one of claims 1 to 11. storage medium.

Images