Image processing model training method, image processing method and related device

Abstract

The present application discloses a training method for an image processing model, an image processing method, and related devices. The image processing model includes a generative adversarial module, and the method includes: obtaining a source domain image and a target domain image; wherein the source domain image is composed of virtual data, and the target domain image is composed of real data; the generative adversarial module includes a first generator corresponding to the source domain image and a second generator corresponding to the target domain image; inputting the source domain image into the first generator to obtain a first intermediate image corresponding to the source domain image; inputting the first intermediate image into the second generator to obtain a reconstructed source domain image corresponding to the source domain image, and determining a first reconstruction loss of the reconstructed source domain image relative to the source domain image; adjusting the parameters in the first generator and the second generator based on the first reconstruction loss to obtain a trained generative adversarial module. The above scheme can improve the accuracy of the generative adversarial module in converting virtual data into data close to real data.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing model training method, an image processing method and related devices. Background Art

[0002] With the continuous development of artificial intelligence, image processing models have been further applied in image processing. Image processing models can perform semantic segmentation on images, thereby associating each pixel in the image with a category label, allowing for rapid identification of the category of the object in the image. However, in some areas where obtaining real data is difficult, there is a lack of data support for training image processing models. While obtaining virtual data is relatively easy and can provide sufficient data, the conversion of virtual data to real data is often inaccurate, which in turn affects the training effect of image processing models. In view of this, how to improve the accuracy of generative adversarial modules in converting virtual data to real data has become an urgent problem to be solved. Summary of the Invention

[0003] The main technical problem solved by this application is to provide a training method for an image processing model, an image processing method and related devices, which can improve the accuracy of the generative adversarial module in converting virtual data into data close to real data.

[0004] To solve the above technical problems, the first aspect of the present application provides a training method for an image processing model, wherein the image processing model includes a generative adversarial module, and the method includes: obtaining a source domain image and a target domain image; wherein the source domain image is composed of virtual data, and the target domain image is composed of real data; the generative adversarial module includes a first generator corresponding to the source domain image and a second generator corresponding to the target domain image; inputting the source domain image into the first generator to obtain a first intermediate image corresponding to the source domain image; inputting the first intermediate image into the second generator to obtain a reconstructed source domain image corresponding to the source domain image, and determining a first reconstruction loss of the reconstructed source domain image relative to the source domain image; adjusting parameters in the first generator and the second generator based on the first reconstruction loss to obtain the trained generative adversarial module.

[0005] To solve the above technical problems, the second aspect of the present application provides a training method for an image processing model, wherein the image processing model includes a semantic segmentation module, and the method includes: obtaining a source domain image and a target domain image; wherein the source domain image is composed of virtual data, and the target domain image is composed of real data; converting the source domain image into a first image, and converting the target domain image into a second image; wherein the texture of the first image is consistent with that of the target domain image and the features of the first image are consistent with those of the source domain image, and the texture of the second image is consistent with that of the source domain image and the features of the second image are consistent with those of the target domain image; and using the first image and the second image to train the semantic segmentation module to obtain the trained semantic segmentation module.

[0006] In order to solve the above technical problems, the third aspect of this application provides an image processing method, including: obtaining image data to be processed; inputting the image data to be processed into an image processing model to obtain a semantic segmentation result corresponding to the image to be processed; wherein, the image processing model is obtained based on the method described in the second aspect above.

[0007] To solve the above technical problems, the fourth aspect of the present application provides an electronic device, which includes: a memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method described in the first aspect, the second aspect, or the third aspect.

[0008] In order to solve the above technical problems, the fifth aspect of the present application provides a computer-readable storage medium on which program data is stored. When the program data is executed by a processor, the method of the first aspect, the second aspect, or the third aspect is implemented.

[0009] The above scheme obtains a source domain image composed of virtual data and a target domain image composed of real data. The generative adversarial module in the image processing model includes a first generator corresponding to the source domain image and a second generator corresponding to the target domain image. The first generator is used to convert the source domain image into a first intermediate image so that the first intermediate image is converted to be close to the target domain image. The first intermediate image is input into the second generator so that the first intermediate image is converted to be close to the source domain image, thereby generating a reconstructed source domain image, determining a first reconstruction loss of the reconstructed source domain image relative to the source domain image, and adjusting the parameters in the first generator and the second generator based on the first reconstruction loss, wherein the first generator and the second generator have completely opposite conversion directions, and the two interact and adjust with each other, thereby improving the accuracy of the generative adversarial module in converting virtual data into data close to real data. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. Among them:

[0011] Figure 1 This is a flowchart of an embodiment of a training method for an image processing model of the present application;

[0012] Figure 2 It is a topological diagram of an embodiment of the training method of the image processing model of the present application;

[0013] Figure 3 This is a flowchart of another embodiment of the training method of the image processing model of the present application;

[0014] Figure 4 This is a flowchart of another embodiment of the training method of the image processing model of the present application;

[0015] Figure 5 This is a topological diagram of an implementation method corresponding to the training of the generative adversarial module of this application;

[0016] Figure 6 This is a topological diagram of another embodiment corresponding to the training of the generative adversarial module of the present application;

[0017] Figure 7 This is a flowchart of another embodiment of the training method of the image processing model of the present application;

[0018] Figure 8 This is a flowchart of another embodiment of the training method of the image processing model of the present application;

[0019] Figure 9 This is a flowchart of an embodiment of the image processing method of the present application;

[0020] Figure 10 This is a schematic structural diagram of an embodiment of the electronic device of the present application;

[0021] Figure 11 It is a structural diagram of an embodiment of the computer storage medium of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] The terms "system" and "network" are often used interchangeably in this document. The term "and / or" is simply a description of an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " generally indicates that the related objects are in an "or" relationship. Furthermore, "multiple" in this document means two or more than two.

[0024] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of a training method for an image processing model of the present application. The image processing model includes a generative adversarial module. The method includes:

[0025] S101: Obtain a source domain image and a target domain image, wherein the source domain image is composed of virtual data and the target domain image is composed of real data, and the generative adversarial module includes a first generator corresponding to the source domain image and a second generator corresponding to the target domain image.

[0026] Specifically, when training image processing models, there are some areas where it is difficult to obtain real data. For example, in remote sensing, autonomous driving, medical diagnosis and other fields, remote sensing images usually have low resolution and poor training effect when used for image processing model training. Due to the immaturity of autonomous driving technology, images can only be collected by humans instead of autonomous driving vehicles, resulting in low image collection efficiency and difficulty in fully covering different road conditions. In the field of medical diagnosis, it is also difficult to obtain relatively high-quality real data due to privacy and randomness of collection. Due to various objective factors, it is difficult to obtain training sets for training image processing models in many fields.

[0027] Furthermore, a dataset composed of virtual data is used as the source domain image, which is used as the main data source for training the image processing model, and a dataset composed of real data is used as the target domain image, which is used to provide a basis for conversion of the source domain image. Since the number of source domain images far exceeds that of target domain images and there are visual differences between the source domain images and the target domain images, the data distribution between the two domain images does not match and there is domain offset in the initial stage.

[0028] In one application, a small amount of real-world data is collected to form a target domain image, while a large amount of virtual data is used to form a source domain image using a synthetic dataset. The target domain image and the source domain image belong to the same field, including but not limited to remote sensing, autonomous driving, and medical diagnosis.

[0029] S102: Input the source domain image into the first generator to obtain a first intermediate image corresponding to the source domain image.

[0030] Specifically, the source domain image is input into the first generator to obtain a first intermediate image, wherein the first generator converts the source domain image into a target domain image. Therefore, it is expected that the first intermediate image obtained has a texture close to that of the target domain image and the features still retain the features of the source domain image, so that the image processed by the first generator can be close to the target domain image in style, thereby obtaining a large amount of data that can be used for in-depth training of the image processing model, thereby improving the training effect of the model.

[0031] S103: Input the first intermediate image into the second generator to obtain a reconstructed source domain image corresponding to the source domain image, and determine a first reconstruction loss of the reconstructed source domain image relative to the source domain image.

[0032] Specifically, the first intermediate image is input into the second generator, wherein the second generator converts the target domain image into the source domain image, so it is expected that the texture of the obtained second intermediate image is close to that of the source domain image.

[0033] Furthermore, the second generator converts the target domain image into the source domain image. Therefore, when the first intermediate image is input into the second generator, the first intermediate image is inverted to obtain the reconstructed source domain image. The source domain image and the reconstructed source domain image are compared to obtain the first reconstruction loss.

[0034] In one application scenario, see Figure 2 , Figure 2This is a topological diagram of an embodiment of the training method of the image processing model of the present application. A source domain image generated by virtual data is input into the first generator so that the first generator converts the virtual data into a first intermediate image so that the texture of the first intermediate image approaches the target domain image, that is, the texture of the virtual data is converted into a texture close to the real data. To test the conversion effect of the first generator, the first intermediate image is input into the second generator so that the second generator converts the first intermediate image as data similar to the target domain image, thereby generating a reconstructed source domain image. The similarity in texture and features between the source domain image and the reconstructed source domain image is compared to determine the loss of the reconstructed source domain image relative to the source domain image. Among them, the first generator and the second generator have completely opposite conversion directions, and the two interact and adjust with each other, thereby improving the accuracy of the generative adversarial module in converting virtual data into data close to real data.

[0035] S104: Adjust the parameters in the first generator and the second generator based on the first reconstruction loss to obtain a trained generative adversarial module.

[0036] Specifically, the parameters in the first generator and the second generator are adjusted based on the first reconstruction loss. When the convergence condition corresponding to the first reconstruction loss is met, the parameters in the first generator and the second generator are fixed to obtain the trained generative adversarial module.

[0037] In one application, the first reconstruction loss corresponds to a reconstruction loss threshold. When the number of adjustments to the first generator and the second generator meets the number threshold and the first reconstruction loss is less than the reconstruction loss threshold, the parameters in the first generator and the second generator are determined to obtain a trained generative adversarial module so that after the source domain image passes through the generative adversarial module, the source domain image is converted into image data that is close to the target domain image in texture and still retains the characteristics of the source domain image, thereby obtaining a large amount of image data that can be used for training.

[0038] The above scheme obtains a source domain image composed of virtual data and a target domain image composed of real data. The generative adversarial module in the image processing model includes a first generator corresponding to the source domain image and a second generator corresponding to the target domain image. The first generator is used to convert the source domain image into a first intermediate image so that the first intermediate image is converted to be close to the target domain image. The first intermediate image is input into the second generator so that the first intermediate image is converted to be close to the source domain image, thereby generating a reconstructed source domain image, determining a first reconstruction loss of the reconstructed source domain image relative to the source domain image, and adjusting the parameters in the first generator and the second generator based on the first reconstruction loss, wherein the first generator and the second generator have completely opposite conversion directions, and the two interact and adjust with each other, thereby improving the accuracy of the generative adversarial module in converting virtual data into data close to real data.

[0039] See also Figure 3 , Figure 3 This is a flow chart of another embodiment of the training method of the image processing model of the present application. The image processing model includes a generative adversarial module and a semantic segmentation module. The generative adversarial module includes a first generator corresponding to the source domain image and a second generator corresponding to the target domain image, as well as a first discriminator corresponding to the source domain image. The method includes:

[0040] S301: Obtain a source domain image and a target domain image, wherein the source domain image is composed of virtual data and the target domain image is composed of real data.

[0041] Specifically, source domain images composed of virtual data and target domain images composed of real data are obtained, wherein the source domain images constitute a source domain image set, and the target domain images constitute a target domain image set. The number of images in the source domain image set far exceeds the number of images in the target domain image set. The source domain images can be obtained from virtual or synthetic datasets.

[0042] Furthermore, the source domain image and the target domain image are preprocessed, and the preprocessing includes but is not limited to rotating, shearing and scaling the images to obtain updated source domain images and target domain images, thereby unifying the sizes of the source domain images and the target domain images, and facilitating the management and comparison of image data.

[0043] In one application, source domain images and target domain images are preprocessed based on server performance and network input requirements. When the server performance is lower than a threshold, the source domain images and target domain images are scaled to unify the sizes of the source domain images and the target domain images.

[0044] In one application scenario, when the image processing model is applied to the field of autonomous driving, the source domain image is obtained from GTA5, and the target domain image is acquired after being captured by a camera device. The source domain image and the target domain image are scaled to unify the sizes of the source domain image and the target domain image.

[0045] S302: Input the source domain image into the first generator to obtain a first intermediate image corresponding to the source domain image.

[0046] Specifically, the source domain image is input into the first generator to obtain a first intermediate image, wherein the first generator converts the source domain image into a target domain image. Therefore, it is expected that the first intermediate image obtained still has the features of the source domain image and the texture is close to the target domain image, that is, the picture is still the picture of the source domain image and the image style is close to the target domain image, thereby realizing the style migration from virtual data to real data, so that the image processed by the first generator can be close to the target domain image in style.

[0047] S303: Input the first intermediate image into the first discriminator to obtain a first generative adversarial loss of the first intermediate image relative to the target domain image.

[0048] Specifically, it is expected that the texture of the first intermediate image is close to that of the target domain image, so that the first discriminator cannot determine that the first intermediate image is virtual data, and at the same time it is expected that the features of the first intermediate image remain consistent with the source domain image. Therefore, the first intermediate image is input into the first discriminator, so that the first discriminator compares the textures of the first intermediate image and the target domain image to obtain a first generative adversarial loss, and the first generator and the first discriminator can be adjusted based on the first generative adversarial loss.

[0049] S304: Input the first intermediate image into the second generator to obtain a reconstructed source domain image corresponding to the source domain image, and determine a first reconstruction loss of the reconstructed source domain image relative to the source domain image.

[0050] Specifically, the first intermediate image is input into the second generator to obtain a reconstructed source domain image, wherein the second generator converts the target domain image into the source domain image. Therefore, when the first intermediate image is input into the second generator, the first intermediate image is inverted to obtain the reconstructed source domain image. The similarity between the source domain image and the reconstructed source domain image is compared to obtain the first reconstruction loss, and the first generator and the first generator can be adjusted based on the first reconstruction adversarial loss.

[0051] S305: Adjust the parameters of the first generator and the first discriminator based on the first generative adversarial loss, and adjust the parameters of the first generator and the second generator based on the first reconstruction loss to obtain a trained generative adversarial module.

[0052] Specifically, a first generative adversarial loss is generated between the first generator and the first discriminator, and the parameters in the first generator and the first discriminator are adjusted based on the first generative adversarial loss. A first reconstruction loss is generated between the first generator and the second generator, and the parameters in the first generator and the second generator are adjusted based on the first reconstruction loss.

[0053] Furthermore, when the first generative adversarial loss and the first reconstruction loss meet the convergence condition, a trained generative adversarial module is obtained, so that the trained generative adversarial module converts the source domain image into image data that is stylistically consistent with the target domain image through the first generator, and it is difficult for the first discriminator to distinguish between image data derived from virtual data and image data derived from real data, thereby enriching the data source for training.

[0054] It should be noted that after the second generator is adjusted and optimized, the target domain image is input into the trained generative adversarial module. The second generator converts the target domain image into the source domain image. Therefore, when the target domain image is input into the trained generative adversarial module, image data that is converted from the target domain image and is consistent in style with the source domain image can be obtained.

[0055] S306: Input the source domain image into the trained generative adversarial module to obtain a first image, and input the target domain image into the trained generative adversarial module to obtain a second image.

[0056] Specifically, the texture of the first image is consistent with the target domain image, and the features of the first image are consistent with the source domain image. The texture of the second image is consistent with the source domain image, and the features of the second image are consistent with the target domain image. Texture reflects the style of an image, and features reflect the content of an image. That is, the source domain image is fed into the trained generative adversarial module to obtain a first image that is consistent in style and content with the target domain image. The target domain image is fed into the trained generative adversarial module to obtain a second image that is consistent in style and content with the source domain image.

[0057] Furthermore, the essential content of the first image is still the source domain image but the style is similar to the target domain image, which is a source domain image of the target domain type. The essential content of the second image is still the target domain image but the style is similar to the source domain image, which is a target domain image of the source domain type. Therefore, the data distribution between the two domain images can be more uniform and the domain offset is reduced from the data level.

[0058] S307: Train the semantic segmentation module using the first image and the second image to obtain a trained semantic segmentation module, and determine the trained image processing model.

[0059] Specifically, the first image is obtained based on the source domain image. Since the number of source domain images is large, sufficient training data can be obtained. The semantic segmentation module is trained using the first image to obtain a semantic segmentation module after preliminary training. The second image is used to fine-tune the semantic segmentation module after preliminary training to further optimize the semantic segmentation module to obtain a trained semantic segmentation module.

[0060] In this embodiment, a source domain image composed of virtual data and a target domain image composed of real data are obtained, and a generative adversarial module is trained using the source domain image and the target domain image, so that the trained generative adversarial module can convert the source domain image into a first image whose texture and features are consistent with the target domain image, thereby converting the source domain image into the first image, expanding the data whose texture is close to the target domain image, compensating for the problem of insufficient target domain image, reducing domain offset from the data level, and converting the target domain image into a second image whose texture and features are consistent with the source domain image, and using the first image and the second image as a mixed training set to train the semantic segmentation module, thereby improving the semantic segmentation accuracy of the semantic segmentation module when performing semantic segmentation on the image.

[0061] See also Figure 4 , Figure 4 This is a flow chart of another embodiment of the training method of the image processing model of the present application. The image processing model includes a generative adversarial module and a semantic segmentation module. The generative adversarial module includes a first generator corresponding to the source domain image and a second generator corresponding to the target domain image. The generative adversarial module also includes a first discriminator corresponding to the source domain image and a second discriminator corresponding to the target domain image. The method includes:

[0062] S401: Obtain a source domain image and a target domain image, wherein the source domain image is composed of virtual data and the target domain image is composed of real data.

[0063] Specifically, source domain images composed of virtual data and target domain images composed of real data are obtained, wherein the source domain images constitute a source domain image set, and the target domain images constitute a target domain image set. The number of images in the source domain image set far exceeds the number of images in the target domain image set. The source domain images can be obtained from virtual or synthetic datasets.

[0064] S402: Input the source domain image into the first generator to obtain a first intermediate image corresponding to the source domain image.

[0065] Specifically, the source domain image is input into the first generator to obtain a first intermediate image, wherein the first generator converts the source domain image into a target domain image. Therefore, it is expected that the first intermediate image still has the features of the source domain image while the texture is close to the target domain image.

[0066] S403: Input the first intermediate image into the second generator to obtain a reconstructed source domain image corresponding to the source domain image, and determine a first reconstruction loss of the reconstructed source domain image relative to the source domain image.

[0067] Specifically, the first intermediate image is input into the second generator to obtain a reconstructed source domain image, wherein the second generator converts the target domain image into the source domain image. Therefore, when the first intermediate image is input into the second generator, the first intermediate image is inverted to obtain the reconstructed source domain image, and the similarity between the source domain image and the reconstructed source domain image is compared to obtain the first reconstruction loss.

[0068] In one application scenario, see Figure 5 , Figure 5 This is a topological diagram of an implementation method corresponding to the training of the generative adversarial module of the present application. The first intermediate image is input into the second generator to obtain a reconstructed source domain image, and the first reconstruction loss is determined based on the similarity between the source domain image and the reconstructed source domain image.

[0069] S404: Input the target domain image into the second generator to obtain a second intermediate image corresponding to the target domain image.

[0070] Specifically, the target domain image is input into the second generator to obtain a second intermediate image, wherein the second generator converts the target domain image into the source domain image. Therefore, it is expected that the second intermediate image still has the features of the target domain image while the texture is close to the source domain image.

[0071] S405: Input the second intermediate image into the first generator to obtain a reconstructed target domain image corresponding to the target domain image, and determine a second reconstruction loss of the reconstructed target domain image relative to the target domain image.

[0072] Specifically, the second intermediate image is input into the first generator to obtain a reconstructed target domain image, wherein the first generator converts the source domain image into the target domain image. Therefore, when the second intermediate image is input into the first generator, the second intermediate image is inverted to obtain the reconstructed target domain image, and the similarity between the target domain image and the reconstructed target domain image is compared to obtain the second reconstruction loss.

[0073] In one application scenario, see Figure 6 , Figure 6 This is a topological diagram of another implementation corresponding to the training of the generative adversarial module of the present application. The second intermediate image is input into the first generator to obtain a reconstructed target domain image, and the second reconstruction loss is determined based on the similarity between the target domain image and the reconstructed target domain image.

[0074] S406: Input the first intermediate image into the first discriminator to obtain a first generative adversarial loss of the first intermediate image relative to the target domain image, and input the second intermediate image into the second discriminator to obtain a second generative adversarial loss of the second intermediate image relative to the source domain image.

[0075] Specifically, the source domain image is input into the first generator to obtain a first intermediate image, wherein the first generator converts the source domain image into the target domain image. Therefore, it is expected that the texture of the first intermediate image is close to that of the target domain image, so that the first discriminator cannot determine that the first intermediate image is virtual data, and at the same time, it is expected that the features of the first intermediate image remain consistent with the source domain image. Therefore, the first intermediate image is input into the first discriminator, so that the first discriminator compares the textures of the first intermediate image and the target domain image to obtain a first generative adversarial loss.

[0076] Furthermore, the target domain image is input into the second generator to obtain a second intermediate image, wherein the second generator converts the target domain image into the source domain image. Therefore, it is expected that the texture of the second intermediate image is close to that of the source domain image, so that the second discriminator cannot determine that the second intermediate image is real data, and at the same time, it is expected that the features of the second intermediate image remain consistent with the target domain image. Therefore, the second intermediate image is input into the second discriminator so that the second discriminator compares the texture of the second intermediate image and the source domain image to obtain a second generative adversarial loss.

[0077] S407: Based on the first generative adversarial loss, the first reconstruction loss, the second generative adversarial loss, and the second reconstruction loss, adjust the parameters of the first generator, the second generator, the first discriminator, and the second discriminator to obtain a trained generative adversarial module.

[0078] Specifically, the first reconstruction loss and the second reconstruction loss reflect the completeness and accuracy of feature preservation when performing image conversion between the two domains of the first generator and the second generator. The first generative adversarial loss reflects the accuracy of the first generator when converting the texture of the source domain image to the texture of the target domain image. The second generative adversarial loss reflects the accuracy of the second generator when converting the texture of the target domain image to the texture of the source domain image. Through the first generative adversarial loss, the first reconstruction loss, the second generative adversarial loss and the second reconstruction loss, a more accurate total loss corresponding to the generative adversarial module is determined, and the parameters of the generative adversarial module are adjusted according to the total loss. After meeting the first convergence condition, the trained generative adversarial module is obtained.

[0079] In one application method, the steps of adjusting the parameters in the first generator, the second generator, the first discriminator and the second discriminator based on the first generative adversarial loss, the first reconstruction loss, the second generative adversarial loss and the second reconstruction loss to obtain the trained generative adversarial module include: determining the total loss corresponding to the generative adversarial module based on the first generative adversarial loss, the first reconstruction loss, the second generative adversarial loss and the second reconstruction loss; adjusting the parameters in the first generator, the second generator, the first discriminator and the second discriminator based on the total loss; and obtaining the trained generative adversarial module in response to satisfying the first convergence condition.

[0080] Specifically, the source domain image is input into the generative adversarial module and the target domain image is input into the generative adversarial module, and the first generative adversarial loss and the first reconstruction loss corresponding to the source domain image, as well as the second generative adversarial loss and the second reconstruction loss corresponding to the target domain image are respectively obtained. Then, the total loss of the generative adversarial model is determined based on the losses determined respectively by the two data sources to improve the accuracy of the total loss. Based on the total loss, the parameters in the first generator, the second generator, the first discriminator and the second discriminator are adjusted until the first convergence condition corresponding to the generative adversarial module is met, thereby fixing the parameters in the first generator, the second generator, the first discriminator and the second discriminator to obtain the trained generative adversarial module.

[0081] In a specific application scenario, the step of determining the total loss corresponding to the generative adversarial module based on the first generative adversarial loss, the first reconstruction loss, the second generative adversarial loss, and the second reconstruction loss includes: performing a weighted sum of the first generative adversarial loss and the second generative adversarial loss, and adding the sum to the first reconstruction loss and the second reconstruction loss, thereby determining the total loss corresponding to the generative adversarial module. The above process is expressed as follows:

[0082]

[0083]

[0084] Among them, L (G,D,R) Denotes the total loss corresponding to the generated adversarial module, L (G,D) represents the total generative adversarial loss, represents the first generation adversarial loss, represents the second generation adversarial loss, L R represents the total reconstruction loss, λ and μ are the source domain image generation adversarial loss coefficient and the target domain image generation adversarial coefficient, respectively, and λ and μ add up to 1.

[0085] Furthermore, the first generative adversarial loss and the second generative adversarial loss are obtained by maximizing the discriminator loss and minimizing the generator loss, so that the first intermediate image generated by the optimized first generator can make it difficult for the first discriminator to distinguish it from the target domain image through texture, and the second intermediate image generated by the optimized second generator can make it difficult for the second discriminator to distinguish it from the source domain image through texture, thereby reducing the domain shift between the two domain datasets and making the data distribution between the two domain datasets more balanced.

[0086] S408: Inputting the source domain image into the trained generative adversarial module to obtain a first image, and inputting the target domain image into the trained generative adversarial module to obtain a second image.

[0087] Specifically, the texture of the first image is consistent with the target domain image, and the features of the first image are consistent with the source domain image. The texture of the second image is consistent with the source domain image, and the features of the second image are consistent with the target domain image. Texture reflects the style of an image, and features reflect the content of an image. That is, the source domain image is fed into the trained generative adversarial module to obtain a first image that is consistent in style and content with the target domain image. The target domain image is fed into the trained generative adversarial module to obtain a second image that is consistent in style and content with the source domain image.

[0088] Furthermore, the essential content of the first image is still the source domain image but the style is similar to the target domain image, which is a source domain image of the target domain type. The essential content of the second image is still the target domain image but the style is similar to the source domain image, which is a target domain image of the source domain type. Therefore, the data distribution between the two domain images can be more uniform and the domain offset is reduced from the data level.

[0089] S409: Train the semantic segmentation module using the first image and the second image to obtain a trained semantic segmentation module, and determine the trained image processing model.

[0090] In one application, true labels corresponding to targets of different categories are set in a first image, wherein the true labels corresponding to the first image match the true labels in the source domain image. True labels corresponding to targets of different categories are set in a second image, wherein the true labels corresponding to the second image match the true labels in the target domain image. A training set consisting of the first image group is input into a semantic segmentation module so that the semantic segmentation module recognizes targets in the input image and sets pixels corresponding to targets of the same category to the same color. The difference between the image output by the semantic segmentation module and the input first image is compared, and the parameters of the semantic segmentation module are adjusted based on the difference. When the convergence condition is met, a preliminarily trained semantic segmentation module is obtained. Furthermore, the second image is input into the preliminarily trained semantic segmentation module to fine-tune the preliminarily trained semantic segmentation module using a small amount of the second image, thereby training the semantic segmentation module based on two types of training data and improving the semantic segmentation accuracy of the trained semantic segmentation module.

[0091] In this embodiment, a source domain image composed of virtual data and a target domain image composed of real data are obtained, the source domain image is input into a generative adversarial module, and the target domain image is input into a generative adversarial module, and a first generative adversarial loss and a first reconstruction loss corresponding to the source domain image, as well as a second generative adversarial loss and a second reconstruction loss corresponding to the target domain image are respectively obtained, and then the total loss of the generative adversarial model is determined based on the losses determined respectively by the two data sources, so as to improve the accuracy of the total loss, and the parameters in the generative adversarial module are adjusted based on the total loss so that the trained generative adversarial module can convert the source domain image into a first image whose texture is consistent with the target domain image and whose features are consistent with the source domain image, thereby expanding the data with texture close to the target domain image by converting the source domain image into the first image, compensating for the problem of insufficient target domain image, reducing domain offset from the data level, and converting the target domain image into a second image with texture consistent with the source domain image and features consistent with the target domain image, and using the first image and the second image as a mixed training set to train the semantic segmentation module, thereby improving the semantic segmentation accuracy when the semantic segmentation module performs semantic segmentation on the image.

[0092] See also Figure 7 , Figure 7 This is a flow chart of another embodiment of the training method of the image processing model of the present application. The image processing model includes a semantic segmentation module. The method includes:

[0093] S701: Obtain a source domain image and a target domain image, wherein the source domain image is composed of virtual data and the target domain image is composed of real data.

[0094] Specifically, when training image processing models, there are some areas where it is difficult to obtain real data. For example, in remote sensing, autonomous driving, medical diagnosis and other fields, remote sensing images usually have low resolution and poor training effect when used for image processing model training. Due to the immaturity of autonomous driving technology, images can only be collected by humans instead of autonomous driving vehicles, resulting in low image collection efficiency and difficulty in fully covering different road conditions. In the field of medical diagnosis, it is also difficult to obtain relatively high-quality real data due to privacy and randomness of collection. Due to various objective factors, it is difficult to obtain training sets for training image processing models in many fields.

[0095] Furthermore, a dataset consisting of virtual data is used as the source domain image, which is used as the main data source for training image processing models. However, since the number of source domain images far exceeds that of target domain images and there are visual differences between source and target domain images, there is a data distribution mismatch between the two domain images and domain shift in the initial stage.

[0096] S702: Convert the source domain image into a first image, and convert the target domain image into a second image, wherein the texture of the first image is consistent with that of the target domain image and the features of the first image are consistent with those of the source domain image, and the texture of the second image is consistent with that of the source domain image and the features of the second image are consistent with those of the target domain image.

[0097] Specifically, the source domain image is converted into a first image whose texture and features are consistent with the target domain image, and the target domain image is converted into a second image whose texture and features are consistent with the source domain image.

[0098] Furthermore, the essential content of the first image is still the source domain image but the style is similar to the target domain image, which is a source domain image of the target domain type. The essential content of the second image is still the target domain image but the style is similar to the source domain image, which is a target domain image of the source domain type. Therefore, the data distribution between the two domain images can be more uniform and the domain offset is reduced from the data level. For fields where sufficient real data cannot be obtained, the first image can be used to train the semantic segmentation model.

[0099] S703: Train a semantic segmentation module using the first image and the second image to obtain a trained semantic segmentation module.

[0100] Specifically, the first image is obtained based on the source domain image. Since there are a large number of source domain images, sufficient training data can be obtained. The semantic segmentation module is trained using the first image to obtain a preliminarily trained semantic segmentation module. The preliminarily trained semantic segmentation module is then fine-tuned using the second image to further optimize the semantic segmentation module to obtain a trained semantic segmentation module. The first image and the second image are annotated with true labels.

[0101] In one application method, a large number of first images are first used to train the semantic segmentation module, the first image is input into the semantic segmentation module to obtain a first label prediction image, the cross entropy loss is used as the loss of the predicted label on the first label prediction image relative to the true label of the first image, the semantic segmentation module is adjusted based on the loss, and then a small number of second models are used to optimize the adjusted semantic segmentation module, the second image is input into the adjusted semantic segmentation module to obtain a second label prediction image, the cross entropy loss is used as the loss of the predicted label on the second label prediction image relative to the true label of the second image, the semantic segmentation module is adjusted based on the loss, thereby obtaining a semantic segmentation module that is more accurate in semantic segmentation of images in both domains.

[0102] In one application scenario, a large convolution kernel is used to replace the existing small convolution kernel in the semantic segmentation module, thereby expanding the segmentation range of the semantic segmentation module and improving the segmentation accuracy.

[0103] The above scheme obtains a source domain image composed of virtual data and a target domain image composed of real data, converts the source domain image into a first image and converts the target domain image into a second image, wherein the texture of the first image is consistent with the target domain image and the features of the first image are consistent with the source domain image, and the texture of the second image is consistent with the source domain image and the features of the second image are consistent with the target domain image, so that the virtual data and the real data can be converted to each other in style, reducing domain offset, using the first image to train the semantic segmentation module to obtain a semantic segmentation module after preliminary training, and using the second image to fine-tune the semantic segmentation module after preliminary training to obtain a semantic segmentation module that is more accurate in semantic segmentation of both domain images.

[0104] See also Figure 8 , Figure 8 This is a flow chart of another embodiment of the training method of the image processing model of the present application. The image processing model includes a generative adversarial module and a semantic segmentation module. The generative adversarial module includes a first generator and a first discriminator corresponding to the source domain image, and a second generator and a second discriminator corresponding to the target domain image. The method includes:

[0105] S801: Obtain a source domain image and a target domain image, wherein the source domain image is composed of virtual data and the target domain image is composed of real data.

[0106] Specifically, source domain images composed of virtual data and target domain images composed of real data are obtained, wherein the source domain images constitute a source domain image set, and the target domain images constitute a target domain image set. The number of images in the source domain image set far exceeds the number of images in the target domain image set. The source domain images can be obtained from virtual or synthetic datasets.

[0107] Furthermore, the source domain image and the target domain image are preprocessed, and the preprocessing includes but is not limited to rotating, shearing and scaling the images to obtain updated source domain images and target domain images, thereby unifying the sizes of the source domain images and the target domain images, and facilitating the management and comparison of image data.

[0108] In one application, source domain images and target domain images are preprocessed based on server performance and network input requirements. When the server performance is lower than a threshold, the source domain images and target domain images are scaled to unify the sizes of the source domain images and the target domain images.

[0109] In one application scenario, when the image processing model is applied to the field of autonomous driving, the source domain image is obtained from GTA5, and the target domain image is acquired after being captured by a camera device. The source domain image and the target domain image are scaled to unify the sizes of the source domain image and the target domain image.

[0110] S802: extracting common categories of the source domain image and the target domain image, and unifying labels corresponding to the common categories in the source domain image and the target domain image, wherein the common categories are types of objects commonly contained in the source domain image and the target domain image.

[0111] Specifically, common categories in the source domain image and the target domain image are obtained, and the same labels are set for the common types in the source domain image and the target domain image, so that the labels of the source domain image and the target domain image are unified, so that when the semantic segmentation module is subsequently trained, the unified labels can be used as a reference to improve the accuracy of the semantic segmentation module training. When applied to the field of autonomous driving, the types of targets include but are not limited to pedestrians, vehicles, roads, trees, buildings and the sky. The types of targets can also be customized in other fields.

[0112] S803: Train the generative adversarial module using the source domain image and the target domain image to obtain a trained generative adversarial module.

[0113] Specifically, the source domain image corresponds to a first generator and a first discriminator, and the target domain image corresponds to a second generator and a second discriminator. The source domain image is input into the first generator and passed through the first discriminator to obtain a first generative adversarial loss. The target domain image is input into the second generator and passed through the second discriminator to obtain a second generative adversarial loss. The parameters of the first generator, first discriminator, second generator and second discriminator are adjusted according to the first generative adversarial loss and the second generative adversarial loss. After repeated training until the convergence condition is met, the trained generative adversarial module is obtained.

[0114] In one application method, the generative adversarial module includes a first generator and a first discriminator corresponding to the source domain image, and a second generator and a second discriminator corresponding to the target domain image; the step of training the generative adversarial module using the source domain image and the target domain image to obtain the trained generative adversarial module includes: training the first generator, the first discriminator, the second generator and the second discriminator using the source domain image and the target domain image to obtain the trained generative adversarial module.

[0115] Specifically, the generative adversarial module consists of two groups of generators and discriminators, where the source domain image corresponds to the first generator and the first discriminator, and the target domain image corresponds to the second generator and the second discriminator. The first generator and the first discriminator are trained using the source domain image, and the second generator and the second discriminator are trained using the target domain image, thereby improving the image processing and identification capabilities of each group of generators and discriminators, thereby determining the parameters of the generative adversarial module.

[0116] In an application scenario, a source domain image is input into a first generator to obtain a first intermediate image corresponding to the source domain image; and a target domain image is input into a second generator to obtain a second intermediate image corresponding to the target domain image; the first intermediate image is input into a first discriminator to obtain a first generative adversarial loss of the first intermediate image relative to the target domain image, the first intermediate image is input into a second generator to obtain a reconstructed source domain image, and the first reconstruction loss of the reconstructed source domain image relative to the source domain image is determined; and the second intermediate image is input into a second discriminator to obtain a second generative adversarial loss of the second intermediate image relative to the source domain image, the second intermediate image is input into the first generator to obtain a reconstructed target domain image, and the second reconstruction loss of the reconstructed target domain image relative to the target domain image is determined; based on the first generative adversarial loss, the first reconstruction loss, the second generative adversarial loss and the second reconstruction loss, the parameters in the first generator, the second generator, the first discriminator and the second discriminator are adjusted to obtain a trained generative adversarial module.

[0117] Specifically, the source domain image is input into the first generator to obtain a first intermediate image, wherein the first generator converts the source domain image into the target domain image, and therefore the first intermediate image is expected to have a texture close to that of the target domain image, so that the first discriminator cannot determine that the first intermediate image is virtual data, and at the same time it is expected that the features of the first intermediate image remain consistent with the source domain image, so the first intermediate image is input into the first discriminator so that the first discriminator compares the textures of the first intermediate image and the target domain image to obtain a first generative adversarial loss, and the first intermediate image is input into the second generator to obtain a reconstructed source domain image, wherein the second generator converts the target domain image into the source domain image, therefore, when the first intermediate image is input into the second generator, the first intermediate image is inverted to obtain a reconstructed source domain image, and the similarity between the source domain image and the reconstructed source domain image is compared to obtain a first reconstruction loss.

[0118] Furthermore, the target domain image is input into the second generator to obtain a second intermediate image, wherein the second generator converts the target domain image into the source domain image, and therefore the second intermediate image is expected to have a texture close to that of the source domain image, so that the second discriminator cannot determine that the second intermediate image is real data, and at the same time it is expected that the features of the second intermediate image remain consistent with the target domain image, and therefore the second intermediate image is input into the second discriminator so that the second discriminator compares the texture of the second intermediate image and the source domain image to obtain a second generative adversarial loss, and the second intermediate image is input into the first generator to obtain a reconstructed target domain image, wherein the first generator converts the source domain image into the target domain image, and therefore, when the second intermediate image is input into the first generator, the second intermediate image is inverted to obtain a reconstructed target domain image, and the similarity between the target domain image and the reconstructed target domain image is compared to obtain a second reconstruction loss.

[0119] Among them, the first reconstruction loss and the second reconstruction loss reflect the completeness and accuracy of feature retention when the first generator and the second generator perform image conversion between the two domains. The first generative adversarial loss reflects the accuracy of the first generator when converting the texture of the source domain image to the texture of the target domain image. The second generative adversarial loss reflects the accuracy of the second generator when converting the texture of the target domain image to the texture of the source domain image. Through the first generative adversarial loss, the first reconstruction loss, the second generative adversarial loss and the second reconstruction loss, a more accurate total loss corresponding to the generative adversarial module is determined, and the parameters of the generative adversarial module are adjusted according to the total loss. After meeting the first convergence condition, the trained generative adversarial module is obtained.

[0120] In a specific application scenario, the step of determining the total loss corresponding to the generative adversarial module based on the first generative adversarial loss, the first reconstruction loss, the second generative adversarial loss, and the second reconstruction loss includes: performing a weighted summation of the first generative adversarial loss and the second generative adversarial loss, and adding the sum to the first reconstruction loss and the second reconstruction loss, thereby determining the total loss corresponding to the generative adversarial module. The above process can be referred to Formula (1) and Formula (2) in the above embodiment, and will not be repeated here.

[0121] Furthermore, the first generative adversarial loss and the second generative adversarial loss are obtained by maximizing the discriminator loss and minimizing the generator loss, so that the first intermediate image generated by the optimized first generator can make it difficult for the first discriminator to distinguish it from the target domain image through texture, and the second intermediate image generated by the optimized second generator can make it difficult for the second discriminator to distinguish it from the source domain image through texture, thereby reducing the domain shift between the two domain datasets and making the data distribution between the two domain datasets more balanced.

[0122] S804: Input the source domain image into the trained generative adversarial module to obtain a first image, and input the target domain image into the trained generative adversarial module to obtain a second image.

[0123] Specifically, the texture of the first image is consistent with the target domain image, and the features of the first image are consistent with the source domain image. The texture of the second image is consistent with the source domain image, and the features of the second image are consistent with the target domain image. Texture reflects the style of an image, and features reflect the content of an image. That is, the source domain image is fed into the trained generative adversarial module to obtain a first image that is consistent in style and content with the target domain image. The target domain image is fed into the trained generative adversarial module to obtain a second image that is consistent in style and content with the source domain image.

[0124] Furthermore, the essential content of the first image is still the source domain image but the style is similar to the target domain image, which is a source domain image of the target domain type. The essential content of the second image is still the target domain image but the style is similar to the source domain image, which is a target domain image of the source domain type. Therefore, the data distribution between the two domain images can be more uniform and the domain offset is reduced from the data level.

[0125] S805: Determine the true label of the first image based on the label corresponding to the common category in the source domain image, and determine the true label of the second image based on the label corresponding to the common category in the target domain image.

[0126] Specifically, in the first image, the true label of the first image is determined based on the label corresponding to the common category in the source domain image, and in the second image, the true label of the second image is determined based on the label corresponding to the common category in the target domain image, that is, the label corresponding to the category of the target is set for each category of the target on the first image and the second image respectively.

[0127] S806: Input the first image and the second image into the semantic segmentation module respectively to obtain an output image, and adjust the parameters in the semantic segmentation module based on the loss between the predicted label on the output image and the corresponding true label.

[0128] Specifically, the first image and the second image are respectively input into the semantic segmentation module to obtain an output image, and the loss between the predicted label on the output image and the corresponding true label on the input first image or the second image is obtained, so as to adjust the parameters in the semantic segmentation module based on the loss.

[0129] Furthermore, by training the semantic segmentation module and the generative adversarial module separately, after completing the training of the generative adversarial module, the semantic segmentation module is trained using the first image and the second image generated by the generative adversarial module as a mixed training set, thereby improving the accuracy of the semantic segmentation module in classifying pixels on the image, enabling the semantic segmentation module to determine the category of the target and give the accurate label corresponding to the target, and setting the pixels corresponding to targets of the same category to the same color.

[0130] In one application, a first image is input into a semantic segmentation module to obtain a first label prediction image; based on the loss of the predicted label on the first label prediction image relative to the true label of the first image, the parameters in the semantic segmentation module are adjusted; a second image is input into the adjusted semantic segmentation module to obtain a second label prediction image; based on the loss of the predicted label on the second label prediction image relative to the true label of the second image, the parameters in the semantic segmentation module are adjusted.

[0131] Specifically, a large number of first images are first used to train the semantic segmentation module, the first image is input into the semantic segmentation module to obtain a first label prediction image, the cross entropy loss is used as the loss of the predicted label on the first label prediction image relative to the true label of the first image, the semantic segmentation module is adjusted based on the loss, and then a small number of second models are used to optimize the adjusted semantic segmentation module, the second image is input into the adjusted semantic segmentation module to obtain a second label prediction image, the cross entropy loss is used as the loss of the predicted label on the second label prediction image relative to the true label of the second image, the semantic segmentation module is adjusted based on the loss, so as to obtain a semantic segmentation module that is more accurate in semantic segmentation of images in both domains.

[0132] In one application scenario, a large convolution kernel is used to replace the existing small convolution kernel in the semantic segmentation module, thereby expanding the segmentation range of the semantic segmentation module and improving the segmentation accuracy.

[0133] S807: In response to satisfying the second convergence condition, obtaining a trained semantic segmentation module.

[0134] Specifically, when the preset second convergence condition is met, a trained semantic segmentation module is obtained.

[0135] In one application method, the first image and the second image are respectively input into the semantic segmentation module to train the semantic segmentation module. When the first image and the second image are sequentially input into the semantic segmentation module as one cycle, the semantic segmentation module is iteratively updated in multiple cycles until the semantic segmentation accuracy of the semantic segmentation module meets the convergence condition to obtain the trained semantic segmentation module.

[0136] In this embodiment, the source domain image is input into the first generator to obtain the first generative adversarial loss through the first discriminator, and the target domain image is input into the second generator to obtain the second generative adversarial loss through the second discriminator. At the same time, the first reconstruction loss and the second reconstruction loss after the source domain image and the target domain image are reconstructed are obtained, thereby obtaining the total loss corresponding to the generative adversarial module. After completing the training of the generative adversarial module, the source domain image and the target domain image are respectively input into the trained generative adversarial module to obtain the first image and the second image. The first image and the second image are then used to train the semantic segmentation module respectively to improve the semantic segmentation accuracy of the semantic segmentation module.

[0137] See also Figure 9 , Figure 9 : is a flow chart of an embodiment of an image processing method of the present application, the method comprising:

[0138] S901: Obtain image data to be processed.

[0139] Specifically, image data to be processed including an object to be processed is acquired, wherein the category of the object to be processed is included in the common category of the source domain image and the target domain image in the above embodiment.

[0140] S902: Input the image data to be processed into a semantic segmentation module to obtain a semantic segmentation result corresponding to the image to be processed.

[0141] Specifically, the semantic segmentation module is obtained based on the method described in any of the above embodiments. The trained semantic segmentation module can perform semantic segmentation on image data composed of virtual data or image data composed of real data, and obtain relatively accurate semantic segmentation results.

[0142] See also Figure 10 , Figure 10 This is a structural diagram of an embodiment of an electronic device of the present application. The electronic device 100 includes a memory 1001 and a processor 1002 coupled to each other, wherein the memory 1001 stores program data (not shown), and the processor 1002 calls the program data to implement the training method or image processing method of the image processing model in any of the above embodiments. For an explanation of the relevant content, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0143] See also Figure 11 , Figure 11 This is a structural diagram of an embodiment of the computer storage medium of the present application. The computer storage medium 110 stores program data 1100. When the program data 1100 is executed by the processor, the training method of the image processing model or the image processing method in any of the above embodiments is implemented. For the description of the relevant content, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0144] It should be noted that the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0145] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0146] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0147] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A training method for an image processing model, characterized in that: The image processing model includes a generative adversarial module, and the method includes: Obtain a source domain image and a target domain image; wherein the source domain image is composed of virtual data, and the target domain image is composed of real data; the generative adversarial module includes a first generator corresponding to the source domain image and a second generator corresponding to the target domain image, as well as a first discriminator corresponding to the source domain image and a second discriminator corresponding to the target domain image; Inputting the source domain image into the first generator to obtain a first intermediate image corresponding to the source domain image; inputting the first intermediate image into the second generator to obtain a reconstructed source domain image corresponding to the source domain image, and determining a first reconstruction loss of the reconstructed source domain image relative to the source domain image; Inputting the target domain image into the second generator to obtain a second intermediate image corresponding to the target domain image; inputting the second intermediate image into the first generator to obtain a reconstructed target domain image corresponding to the target domain image, and determining a second reconstruction loss of the reconstructed target domain image relative to the target domain image; Inputting the first intermediate image into the first discriminator to obtain a first generative adversarial loss of the first intermediate image relative to the target domain image, and inputting the second intermediate image into the second discriminator to obtain a second generative adversarial loss of the second intermediate image relative to the source domain image; Based on the first generative adversarial loss, the first reconstruction loss, the second generative adversarial loss, and the second reconstruction loss, the parameters in the first generator, the second generator, the first discriminator, and the second discriminator are adjusted to obtain the trained generative adversarial module.

2. The image processing model training method according to claim 1, characterized in that: The step of adjusting parameters in the first generator, the second generator, the first discriminator, and the second discriminator based on the first generative adversarial loss, the first reconstruction loss, the second generative adversarial loss, and the second reconstruction loss to obtain the trained generative adversarial module includes: Determining a total loss corresponding to the generative adversarial module based on the first generative adversarial loss, the first reconstruction loss, the second generative adversarial loss, and the second reconstruction loss; adjusting parameters in the first generator, the second generator, the first discriminator, and the second discriminator based on the total loss; In response to satisfying the first convergence condition, the trained generative adversarial module is obtained.

3. The image processing model training method according to claim 2, characterized in that: The step of determining a total loss corresponding to the generative adversarial module based on the first generative adversarial loss, the first reconstruction loss, the second generative adversarial loss, and the second reconstruction loss includes: A weighted sum is performed on the first generative adversarial loss and the second generative adversarial loss, and the sum is added to the first reconstruction loss and the second reconstruction loss to determine the total loss corresponding to the generative adversarial module.

4. The method for training an image processing model according to any one of claims 1 to 3, characterized in that: The image processing model also includes a semantic segmentation module; After the step of obtaining the trained generative adversarial module, the method further includes: Inputting the source domain image into the trained generative adversarial module to obtain a first image, and inputting the target domain image into the trained generative adversarial module to obtain a second image; wherein the texture of the first image is consistent with that of the target domain image and the features of the first image are consistent with those of the source domain image, and the texture of the second image is consistent with that of the source domain image and the features of the second image are consistent with those of the target domain image; The semantic segmentation module is trained using the first image and the second image to obtain the trained semantic segmentation module, and the trained image processing model is determined.

5. A training method for an image processing model, characterized in that: The image processing model includes a semantic segmentation module, and the method includes: Obtaining a source domain image and a target domain image; wherein the source domain image is composed of virtual data, and the target domain image is composed of real data; Converting the source domain image into a first image and converting the target domain image into a second image; wherein the texture of the first image is consistent with that of the target domain image and the features of the first image are consistent with those of the source domain image, and the texture of the second image is consistent with that of the source domain image and the features of the second image are consistent with those of the target domain image; Training the semantic segmentation module using the first image and the second image to obtain the trained semantic segmentation module; Wherein, the image processing model further includes a generative adversarial module; Before the step of training the semantic segmentation module using the first image and the second image to obtain the trained semantic segmentation module, the method further includes: Training the generative adversarial module using the source domain image and the target domain image to obtain the trained generative adversarial module; The step of converting the source domain image into a first image and converting the target domain image into a second image comprises: Inputting the source domain image into the trained generative adversarial module to obtain the first image, and inputting the target domain image into the trained generative adversarial module to obtain the second image; The generative adversarial module includes a first generator and a first discriminator corresponding to the source domain image, and a second generator and a second discriminator corresponding to the target domain image; The step of training the generative adversarial module using the source domain image and the target domain image to obtain the trained generative adversarial module includes: Using the source domain image and the target domain image to train the first generator, the first discriminator, the second generator, and the second discriminator to obtain the trained generative adversarial module; The step of training the first generator, the first discriminator, the second generator, and the second discriminator using the source domain image and the target domain image to obtain the trained generative adversarial module includes: Inputting the source domain image into the first generator to obtain a first intermediate image corresponding to the source domain image; and inputting the target domain image into the second generator to obtain a second intermediate image corresponding to the target domain image; Inputting the first intermediate image into the first discriminator to obtain a first generative adversarial loss of the first intermediate image relative to the target domain image, inputting the first intermediate image into the second generator to obtain a reconstructed source domain image, and determining a first reconstruction loss of the reconstructed source domain image relative to the source domain image; and Inputting the second intermediate image into the second discriminator to obtain a second generative adversarial loss of the second intermediate image relative to the source domain image, inputting the second intermediate image into the first generator to obtain a reconstructed target domain image, and determining a second reconstruction loss of the reconstructed target domain image relative to the target domain image; Based on the first generative adversarial loss, the first reconstruction loss, the second generative adversarial loss, and the second reconstruction loss, the parameters in the first generator, the second generator, the first discriminator, and the second discriminator are adjusted to obtain the trained generative adversarial module.

6. The image processing model training method according to claim 5, characterized in that: After the step of obtaining the source domain image and the target domain image, the method further includes: Extracting common categories between the source domain image and the target domain image, and unifying labels corresponding to the common categories in the source domain image and the target domain image; wherein the common categories are types of objects commonly contained in the source domain image and the target domain image.

7. The image processing model training method according to claim 6, characterized in that: The step of training the semantic segmentation module using the first image and the second image to obtain the trained semantic segmentation module includes: Determining the true label of the first image based on the label corresponding to the common category in the source domain image, and determining the true label of the second image based on the label corresponding to the common category in the target domain image; Inputting the first image and the second image into the semantic segmentation module respectively to obtain output images, and adjusting parameters in the semantic segmentation module based on the loss between the predicted labels on the output images and the corresponding true labels; In response to satisfying the second convergence condition, the trained semantic segmentation module is obtained.

8. The image processing model training method according to claim 7, characterized in that: The step of inputting the first image and the second image into the semantic segmentation module respectively to obtain an output image, and adjusting parameters in the semantic segmentation module based on the loss between the predicted label on the output image and the corresponding true label includes: Inputting the first image into the semantic segmentation module to obtain a first label prediction image; adjusting parameters in the semantic segmentation module based on a loss of a predicted label on the first label prediction image relative to a true label of the first image; Inputting the second image into the adjusted semantic segmentation module to obtain a second label prediction image; Parameters in the semantic segmentation module are adjusted based on the loss of the predicted label on the second label prediction image relative to the true label of the second image.

9. An image processing method, characterized in that: The method comprises: Obtaining image data to be processed; The image data to be processed is input into a semantic segmentation module to obtain a semantic segmentation result corresponding to the image to be processed; wherein the semantic segmentation module is obtained based on the method according to any one of claims 5 to 8.

10. An electronic device, characterized in that: include: A memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method according to any one of claims 1-4, 5-8 or 9.

11. A computer-readable storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the method according to any one of claims 1 to 4 or 5 to 8 or 9 is implemented.

Images