Image processing model training method and device, image processing method and device, electronic equipment, medium and product

Abstract

The invention provides an image processing model training method and device, an image processing method and device, electronic equipment, a medium and a product, and the training method comprises the steps: carrying out the feature extraction and down-sampling of a first image, and obtaining the hidden coding feature of the first image; performing up-sampling on the hidden coding features to obtain a first mask graph of the first image; performing image generation processing on the hidden coding features to obtain a second image; and training the image processing model according to the first image, the label of the first image, the first mask pattern and the second image. According to the embodiment of the invention, the network training effect can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a training method and device for an image processing model, an image processing method and device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Image segmentation is the process of dividing an image into several specific regions with unique properties and extracting the target of interest. Human segmentation is an application of image segmentation, which distinguishes people from backgrounds at the pixel level. Human segmentation includes portrait segmentation and general portrait segmentation. Portrait segmentation is the segmentation of half-length portraits, which is mainly suitable for video calls and conference scenarios. Since portrait segmentation is widely deployed on web pages, mobile phones, and edge devices, portrait segmentation needs to have extremely fast inference speed while taking into account segmentation accuracy. General portrait segmentation is the segmentation of full-length and half-length portraits. With the help of deep learning technology, the effect and efficiency of general portrait segmentation have made rapid breakthroughs, but the segmentation accuracy of portrait edges is still not ideal. Summary of the invention

[0003] The present disclosure provides a training method and device for an image processing model, an image processing method and device, an electronic device, a computer-readable storage medium, and a computer program product.

[0004] In a first aspect, the present disclosure provides a method for training an image processing model, the method comprising:

[0005] Perform feature extraction and down-sampling on the first image to obtain latent coding features of the first image; perform up-sampling on the latent coding features to obtain a first mask map of the first image; perform image generation processing on the latent coding features to obtain a second image; and train the image processing model based on the first image and its label, the first mask map and the second image.

[0006] In a second aspect, the present disclosure provides an image processing method, the method comprising:

[0007] The fifth image is processed by an image processing model to obtain a mask map of the fifth image; the fifth image is segmented according to the mask map of the fifth image to obtain a sixth image of the target object in the fifth image; wherein the image processing model is trained according to the above-mentioned training method.

[0008] In a third aspect, the present disclosure provides a training device for an image processing model, the device comprising:

[0009] A downsampling module, used for performing feature extraction and downsampling on the first image to obtain latent coding features of the first image;

[0010] an upsampling module, configured to upsample the latent coding features to obtain a first mask image of the first image;

[0011] An image generation module, used for performing image generation processing on the latent coding feature to obtain a second image;

[0012] A training module is used to train the image processing model according to the first image and the label of the first image, the first mask image and the second image.

[0013] In a fourth aspect, the present disclosure provides an image processing device, the device comprising:

[0014] A processing module, used for processing the fifth image through an image processing model to obtain a mask image of the fifth image;

[0015] A segmentation module is used to perform image segmentation on the fifth image according to the mask image of the fifth image to obtain a sixth image of the target object in the fifth image; wherein the image processing model is trained according to the above-mentioned training method.

[0016] In a fifth aspect, the present disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and one or more of the computer programs are executed by the at least one processor so that the at least one processor can execute the above-mentioned image processing model training method or image processing method.

[0017] In a sixth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-mentioned image processing model training method or image processing method.

[0018] In the seventh aspect, the present disclosure provides a computer program product, which includes a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned image processing model training method or image processing method.

[0019] The training method of the image processing model provided by the embodiment of the present disclosure can extract and downsample the first image features to obtain latent coding features; upsample the latent coding features to obtain a first mask map; and generate a second image based on the latent coding features, thereby assisting the supervised training of the image processing model; and train the image processing model by determining the network loss value based on the first image and label, the first mask map and the second image, thereby improving the training effect and improving the segmentation accuracy of the image processing model.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:

[0022] Figure 1 An application scenario diagram of the training method and device for the image processing model provided in the embodiments of the present disclosure;

[0023] Figure 2 A flowchart of a method for training an image processing model provided in an embodiment of the present disclosure;

[0024] Figure 3 A schematic diagram of a label of a training method for an image processing model provided in an embodiment of the present disclosure;

[0025] Figure 4 A network diagram of a training method for an image processing model provided in an embodiment of the present disclosure;

[0026] Figure 5 A schematic diagram of region expansion in a training method for an image processing model provided in an embodiment of the present disclosure;

[0027] Figure 6 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0028] Figure 7 A block diagram of a training device for an image processing model provided in an embodiment of the present disclosure;

[0029] Figure 8 A block diagram of an image processing device provided by an embodiment of the present disclosure;

[0030] Fig. 9A block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0032] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0033] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0034] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0035] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0036] In the technical solution disclosed in this disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution complies with relevant national laws and regulations (for example, the "Information Security Technology Personal Information Security Specification", etc.). For example: corresponding prescribed measures are taken for personal information access control; prescribed restrictions are imposed on the display of personal information; the purpose of using personal information does not exceed the scope of direct or reasonable association; when using personal information, clear identity orientation is eliminated to avoid precise positioning of specific individuals.

[0037] The background mating algorithm in the related art is a portrait segmentation method in natural scenes. It needs to use the background image as a reference. Therefore, when shooting the image, it is necessary to keep the background picture as still as possible, that is, try not to shake it, which is inconvenient for practical application. To this end, the relevant technical personnel proposed to use the context switching (CS) module to select useful information, and then obtain the segmented image through the decoder. However, this segmentation will be mixed with the background pixel information, resulting in defects in the edge area of ​​the image and poor segmentation accuracy.

[0038] The disclosed embodiments provide a method for training an image processing model, which can generate a second image with the same object as the first image during network training, and assist in the training of the image processing model to improve the effect of network training, thereby improving the accuracy of image segmentation by the trained image processing model, and making the segmented image more accurate and natural.

[0039] Figure 1 A diagram of an application scenario of the method and apparatus for training an image processing model provided in an embodiment of the present disclosure.

[0040] like Figure 1 As shown, the application scenario of the embodiment of the present disclosure may include a terminal device 101, a network 103 and a server 102. The network 103 is used to provide a medium for a communication link between the terminal device 101 and the server 102. The network 103 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0041] The user can use the terminal device 101 to interact with the server 102 through the network 103 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0042] The terminal device 101 may be any electronic device having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0043] The server 102 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal device 101. The background management server may analyze and process received data such as user requests, and feed back processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0044] It should be noted that the training method and apparatus of the image processing model provided in the embodiments of the present disclosure can be executed by the server 102. Accordingly, the training method and apparatus of the image processing model provided in the embodiments of the present disclosure can be set in the server 102. The training method and apparatus of the image processing model provided in the embodiments of the present disclosure can also be executed by a server or a server cluster that is different from the server 102 and can communicate with the terminal device 101 and / or the server 102. Accordingly, the training method and apparatus of the image processing model provided in the embodiments of the present disclosure can also be set in a server or a server cluster that is different from the server 102 and can communicate with the terminal device 101 and / or the server 102.

[0045] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0046] Figure 2 A flowchart of a method for training an image processing model provided in an embodiment of the present disclosure. Figure 2 , the method comprising:

[0047] Step S21, performing feature extraction and downsampling on the first image to obtain latent coding features of the first image;

[0048] Step S22, upsampling the latent coding features to obtain a first mask image of the first image;

[0049] Step S23, performing image generation processing on the latent coding feature to obtain a second image;

[0050] Step S24: training the image processing model according to the first image and the label of the first image, the first mask image and the second image.

[0051] For example, the image processing model is used to segment the image to obtain a mask map of the portrait in the image. The image processing model may be a convolutional neural network, such as a U-type network (Unet). In the case of a U-type network, the image processing model may include an encoding network for feature extraction and downsampling and a decoding network for upsampling, and the encoding network and the decoding network each include multiple network layers. The present disclosure does not limit the specific network structure of the image processing model.

[0052] In some possible implementations, the first image may be a sample in a training set, the first image includes a portrait area and a background area of ​​a target object, the target object may be a person or other types of targets, and the portrait area may include areas corresponding to a face, hair, and a portion of a human body. The present disclosure does not limit the specific type of the target object.

[0053] In some possible implementations, a label may be generated for the first image in the training set, and the label of each first image includes a second mask map and a third mask map. Among them, the first image may be preliminarily segmented by a segmentation algorithm in the relevant technology (e.g., HFGI (High-Fidelity GAN Inversion)) or other segmentation methods to obtain the second mask map. Due to the segmentation accuracy limitation of the preliminary segmentation, the second mask map is expressed as a mask map including the face, hair, and hair edges (e.g., hair strands) of the target object. Furthermore, the hair edge area in the second mask map may be removed by a preset removal method (e.g., manual removal) to obtain a third mask map, which is a mask map including the face and hair (e.g., hair trunk) of the target object. The present disclosure does not limit the specific method for obtaining the second mask map and the third mask map.

[0054] In some possible implementations, after obtaining the second mask map and the third mask map, the image processing model can be pre-trained with the first image and its third mask map, so that the image processing model has preliminary image segmentation capabilities; the image processing model is then pre-trained with the first image and its second mask map to improve the segmentation accuracy of the image processing model, so as to improve the training effect of subsequent training and increase the convergence speed of the network model. The present disclosure does not limit the specific training method of pre-training.

[0055] In the formal training stage, in step S21, feature extraction and downsampling can be performed on the first image to obtain latent code features of the first image. The first image can be input into the coding network of the image processing model, and feature extraction is performed through the feature extraction layer of the coding network to obtain image features; the image features are sequentially downsampled through multiple downsampling layers of the coding network, and each downsampling layer will obtain downsampled portrait features and background features, and finally output the latent code features (latent code) of the first image.

[0056] In the example, the feature extraction layer can be implemented using the mobilenetv3 network (a lightweight convolutional neural network model); each downsampling layer may include a 3*3 convolution layer (with a step size of 2 and a convolution kernel size of 64) and an activation layer (such as a Prelu layer). For example, when the downsampling layer is 4 layers and the feature extraction layer outputs 512*512 image features, the 4 downsampling layers output 256*256, 128*128, 64*64, and 32*32 features in sequence, and the 32*32 features output by the last downsampling layer are the hidden coding features. It should be understood that those skilled in the art can set the network type of the feature extraction layer, the specific type and number of downsampling layers according to actual conditions, and the present disclosure does not limit this.

[0057] In some possible implementations, after obtaining the latent coding features of the first image, the latent coding features may be upsampled in step S22 to obtain a first mask image of the first image. The latent coding features may be input into a decoding network of the image processing model, and sequentially upsampled through multiple upsampling layers of the decoding network, and each layer may obtain a more refined portrait feature with an upsampling multiple (e.g., 2 times), and the output result of the last upsampling layer is processed by a segmentation convolution layer of the decoding network to output a first mask image.

[0058] In the example, each upsampling layer may include a 3*3 convolution layer (with a step size of 1 / 2 and a convolution kernel size of 64) and an activation layer (such as a Prelu layer); the final segmentation convolution layer may be a 1*1*3 convolution layer. For example, when there are 4 upsampling layers and the hidden coding features are 32*32 features, the 4 upsampling layers output 64*64, 128*128, 256*256, and 512*512 features in sequence; and then processed by the segmentation convolution layer, a 512*512 first mask image is output.

[0059] In some possible implementations, in step S23, the latent coding features may be processed for image generation to obtain a second image. The latent coding features may be input into a preset image generation model for processing to generate a second image that retains the face identification (ID), that is, the second image also includes the target object. The image generation model may use StyleGAN (a type of adversarial generative network, GAN), and by setting the noise to 0, a more beautiful and natural face image may be generated. The present disclosure does not limit the specific network type of the image generation model.

[0060] By introducing an image generation model to generate a second image of the same object to assist the supervised training of the image processing model, the segmentation results of the image processing model can be made more beautiful and natural, thereby improving the segmentation effect.

[0061] In some possible implementations, in step S24, the image processing model is trained according to the first image and the label of the first image, the first mask image, and the second image. According to the difference between the first mask image and the label of the first image, the difference between the first image and the second image, the difference between the first mask image and the second image, etc., multiple network loss values ​​of the image processing model can be determined respectively, and the network parameters of the image processing model can be reversely adjusted according to the network loss values; after multiple rounds of iterations, when the training conditions (such as network convergence) are met, the trained image processing model can be obtained.

[0062] According to the training method of the image processing model of the embodiment of the present disclosure, it is possible to extract and downsample the first image features to obtain latent coding features; upsample the latent coding features to obtain a first mask map; and generate a second image identical to the object in the first image based on the latent coding features, thereby assisting the supervised training of the image processing model; and train the image processing model by determining the network loss value based on the first image and label, the first mask map, and the second image, thereby improving the training effect and improving the segmentation accuracy of the image processing model.

[0063] The following is a detailed description of the training method of the image processing model according to the embodiment of the present disclosure.

[0064] As mentioned above, a training set of the image processing model can be obtained first, the training set includes multiple first images, each first image includes a portrait area and a background area of ​​the target object; and a label of the first image is generated, the label of each first image includes a second mask image and a third mask image, the second mask image is a mask image including the face, hair and hair edges (such as hair strands) of the target object, and the third mask image is a mask image including the face and hair (such as hair trunk) of the target object.

[0065] Figure 3 A schematic diagram of labels for a training method for an image processing model provided in an embodiment of the present disclosure. Figure 3 : is an illustration of the third mask image, which is a binary mask image. The white area (1) is the portrait area, including the face, hair and part of the body; the black area (0) is the background area.

[0066] In some possible implementations, an image processing model may be constructed, for example, using a U-shaped network, including an encoding network part for feature extraction and downsampling and a decoding network part for upsampling. During training, an attention layer may also be added to the image processing model to enhance attention and improve network training effects.

[0067] Figure 4 A network diagram of a training method for an image processing model provided in an embodiment of the present disclosure. Figure 4 As shown, the image processing model includes an encoding network 41 and a decoding network 42; an attention layer 43 is included during training, and an image generation model 44 is set to generate a second image.

[0068] In the example, the encoding network 41 includes a feature extraction layer (for example, using a mobilenetv3 network) and a multi-layer downsampling layer ( Figure 4 In the example, 4 layers are taken, namely, downsampling layers conv1, conv2, conv3, conv4); the decoding network 42 includes multiple upsampling layers ( Figure 4In the figure, 4 layers are taken as an example, namely the upsampling layer conv5, conv6, conv7, conv8) and the segmentation convolution layer (for example, a 1*1*3 convolution layer).

[0069] In the example, the attention layer 43 includes multiple layers of 1*1 convolutional layers, global pooling layers and activation layers (such as sigmoid layers) in sequence. The present disclosure does not limit the specific network structure of the attention layer 43.

[0070] In some possible implementations, in the formal training phase, in step S21, the first image can be feature extracted and downsampled by the image processing model to obtain latent code features of the first image. The first image can be input into the encoding network of the image processing model, and feature extracted by the feature extraction layer of the encoding network to obtain image features; the image features are sequentially downsampled through multiple downsampling layers of the encoding network, and each downsampling layer will obtain the downsampled portrait features and background features, and finally output the latent code features of the first image.

[0071] In some possible implementations, in step S22, the latent coding features of the first image are upsampled by the image processing model to obtain a first mask map of the first image. The latent coding features can be input into the decoding network of the image processing model, and sequentially upsampled through multiple upsampling layers of the decoding network, and each layer will obtain a more refined portrait feature that is upsampled twice, and the output result of the last upsampling layer is processed by the segmentation convolution layer of the decoding network to output the first mask map.

[0072] In the example, each upsampling layer also inputs the shared features of the downsampling layer with the corresponding feature size, for example Figure 4 The shared features output by the downsampling layer conv4 are input to the upsampling layer conv5; the shared features output by the downsampling layer conv3 are input to the upsampling layer conv6; the shared features output by the downsampling layer conv2 are input to the upsampling layer conv7; the shared features output by the downsampling layer conv1 are input to the upsampling layer conv8. The present disclosure does not limit the specific processing method.

[0073] In some possible implementations, during training, the image processing model further includes an attention layer to enhance attention and improve network training effects. According to the training method of the image processing model of the embodiment of the present disclosure, the method further includes: enhancing attention to the label of the first image through the attention layer to obtain attention weight information.

[0074] In some possible implementations, the attention enhancement process includes:

[0075] determining a second difference image between the second mask image and the first mask image;

[0076] normalizing the second mask image according to the second difference image to obtain a fourth mask image;

[0077] The fourth mask image is input into the attention layer for processing to obtain the attention weight information.

[0078] For example, during the network training process, the attention mechanism can be used to strengthen attention. For any round of iteration, the second mask map (initial label) of the first image can be used to subtract the first mask map output by the image processing model in the previous training state to obtain a difference image between the second mask map and the first mask map (referred to as the second difference image), and the second difference image is a binary mask image including the hair edge area (such as hair strands and edges).

[0079] In some possible implementations, the mean of the second difference image can be calculated (removing 255 and 0), and the pixel value of the pixel point in the second mask image with a pixel value of 255 is set to the mean, thereby standardizing the second mask image and obtaining a fourth mask image; the fourth mask image is then input into the attention layer and processed sequentially through multiple network layers of the attention layer.

[0080] In the example, a 1*1*64 convolution layer is used to extract mask features from the fourth mask image, and then a 1*1*256 convolution layer is used to increase the channel dimension, and then a 1*1*64 convolution layer is used to reduce the dimension. This expansion and contraction can increase the nonlinearity of the feature, which can also be understood as extracting more possible mask feature information; then a global pooling layer is used to perform spatial compression to obtain 1*1*64 features, and an activation layer (such as a sigmoid layer) is used to obtain 64-dimensional weight information, that is, the attention weight information output by the attention layer; and then the attention weight information can be input into an upsampling layer of the decoding network for processing. The present disclosure does not limit the specific network structure of the attention layer and the specific upsampling layer for inputting the attention weight information.

[0081] In this way, the image processing model can pay attention to the information of the object's hair and hair edge area, making the image processing model sensitive to the data distribution of unclear hair and edge segmentation, thereby improving the network's segmentation accuracy for hair edges.

[0082] In some possible implementations, the step of upsampling the latent coding features in step S22 to obtain a first mask image of the first image may include:

[0083] Upsampling the latent coding features of the first image to obtain first sampling features;

[0084] Upsampling the first sampling feature based on the attention weight information to obtain a second sampling feature;

[0085] The second sampled features are upsampled to obtain the first mask image.

[0086] For example, the latent coding features can be input into the decoding network of the image processing model, and upsampled by the upsampling layer before adding the attention weight information to obtain the first sampling feature; the first sampling feature is upsampled by the upsampling layer after adding the attention weight information to obtain the second sampling feature; and then upsampled by the upsampling layer after adding the attention weight information and the final segmentation convolution layer processing to output the first mask map.

[0087] In the example, Figure 4 As shown in the figure, the upsampling layer that adds the attention weight information is conv6, the latent coding feature is input into the upsampling layer conv5 for processing, and the first sampling feature is output; the first sampling feature is input into the upsampling layer conv6 for processing, and the second sampling feature is output; the second sampling feature is processed by the upsampling layer conv7 and the upsampling layer conv8 and the segmentation convolution layer in turn, and the first mask map is output. In this way, the image processing model can improve its ability to extract information about the edge area of ​​the hair and improve the accuracy of the segmentation of the edge area.

[0088] In some possible implementations, in step S23, the latent coding features may be processed for image generation to obtain a second image. The latent coding features may be input into a preset image generation model for processing to generate a second image that retains the face identification (ID), that is, the second image also includes the target object. The image generation model may use StyleGAN (a type of adversarial generative network GAN), and by setting the noise to 0, a more beautiful and natural face image may be generated. The present disclosure does not limit the specific network type of the image generation model.

[0089] By introducing an image generation model to generate a second image of the same object to assist the supervised training of the image processing model, the attention of the image processing model can be properly distributed on the face part, preventing the generation of holes, making the segmentation result tend to be more beautiful and natural, and improving the segmentation effect.

[0090] In some possible implementations, in step S24, the image processing model may be trained according to the first image and the label of the first image, the first mask image, and the second image. According to the difference between the first mask image and the label of the first image, the difference between the first image and the second image, the difference between the first mask image and the second image, etc., multiple network loss values ​​of the image processing model may be determined respectively, and the network parameters of the image processing model may be reversely adjusted according to the network loss values ​​to complete an iterative training; after multiple rounds of iterative training, when the training conditions (such as network convergence) are met, a trained image processing model may be obtained.

[0091] In some possible implementations, step S24 may include:

[0092] Determine a first loss value and a fourth loss value according to the first mask image and the label of the first image;

[0093] determining a second loss value according to the first image and the second image;

[0094] Determining a third loss value according to the first mask image and the second image;

[0095] The image processing model is trained according to the first loss value, the second loss value, the third loss value and the fourth loss value.

[0096] For example, the network loss value of the image processing model may include a first loss value, a second loss value, a third loss value, and a fourth loss value. The first loss value is used to characterize the difference between the first mask image output by the network and the labeled second mask image; the second loss value is used to characterize the difference in object identity between the first image and the generated second image; the third loss value is used to characterize the difference between the first mask image output by the network and the generated second image; and the fourth loss value is used to characterize the difference in edge information between the first mask image output by the network and the label.

[0097] In some possible implementations, the step of determining the first loss value and the fourth loss value according to the first mask map and the label of the first image may include: determining the first loss value according to the difference between the first mask map and the second mask map.

[0098] In the example, the first loss value L1 can be calculated by the mean square error loss function mse, which is expressed as:

[0099] L1 = mse (net(x), label1) (1)

[0100] In formula (1), x represents the first image, net(x) represents the first mask image output by the image processing model, and label1 represents the second mask image.

[0101] In some possible implementations, the step of determining the second loss value according to the first image and the second image may include:

[0102] Performing face recognition on the first image to obtain a third image;

[0103] The second loss value is determined according to a distance between the third image and the second image.

[0104] For example, a trained face recognition network may be used to perform face recognition on the first image, and the recognized third image may be output. The face recognition network may be, for example, arcface (also known as Insight Face, an open source face recognition network model), and the present disclosure does not limit the specific network type of the face recognition network.

[0105] In some possible implementations, the second loss value may be determined based on the distance between the third image and the second image generated by the image generation model, where the distance is, for example, a cosine (cosine) distance. The second loss value L2 is expressed as:

[0106] L2 = 1- cos (arcface (x), face) (2)

[0107] In formula (2), x represents the first image, arcface(x) represents the third image, and face represents the second image.

[0108] By introducing the second loss value, the network can reduce the difference in object identity between the first image and the generated second image during training, making the face of the generated image closer to the face of the original image and reducing unnatural parts in the generated image (such as only focusing on the edge part while the face may have holes), thereby improving the effect of network training.

[0109] In some possible implementations, the step of determining a third loss value according to the first mask image and the second image includes:

[0110] Determining a fourth image of the target object according to the first mask image and the first image;

[0111] Performing face detection on the fourth image and the second image respectively to determine a first face region in the fourth image and a second face region in the second image;

[0112] Expanding the first face region in the fourth image to obtain a first expanded region; and expanding the second face region in the second image to obtain a second expanded region;

[0113] The third loss value is determined according to a difference between the first extension area and the second extension area.

[0114] For example, with Figure 3 The label shown is similar, and the image processing model outputs a first mask map that is a binary mask map, where the white area (1) is the portrait area and the black area (0) is the background area. The first mask map can be multiplied with the first image to remove the background area in the first image, retain the portrait area in the first image, and segment the fourth image of the target object from the first image.

[0115] In some possible implementations, a trained face detection network may be used to perform face detection on the fourth image and the second image, respectively, to determine a first face region in the fourth image and a second face region in the second image. The face detection network may be, for example, a yolo5face face detector, and the present disclosure does not limit the specific network type of the face detection network.

[0116] Figure 5 A schematic diagram of region expansion in a training method of an image processing model provided in an embodiment of the present disclosure. Figure 5 As shown, the face area may correspond to the image area defined by the black face frame, and may be represented by the coordinates of two diagonal vertices of the face frame.

[0117] In some possible implementations, an expansion ratio may be preset, such as 1.5, 2, 3, etc., indicating that the face area is expanded outward by 1.5 times, 2 times, or 3 times, so as to expand the image area to the hair and part of the human body area. The present disclosure does not limit the specific value of the expansion ratio.

[0118] In some possible implementations, according to the expansion ratio, the first face region can be expanded in the fourth image to obtain a first expanded region; and the second face region can be expanded in the second image to obtain a second expanded region. According to the coordinates of two diagonal vertices of the face region, such as the upper left and lower right vertex coordinates, the width-to-length ratio of the face region can be maintained, and the face region can be expanded in the corresponding direction (such as the upper left and lower right) according to the expansion ratio, and expanded to the following: Figure 5 The upper left and lower right white vertices shown obtain the expanded area.

[0119] In some possible implementations, the third loss value L3 may be determined according to the difference between the first extension area and the second extension area, and is expressed as:

[0120] L3 = || yolo(net(x)*x), yolo(face)|| (3)

[0121] In formula (3), net(x)*x represents the fourth image; face represents the second image; yolo(net(x)*x) represents the first extended region in the fourth image; yolo(face) represents the second extended region in the second image; ||…|| represents L1 loss, that is, mean absolute error loss.

[0122] By expanding the face area outward, the area involved in the comparison can be expanded to the edge pixels of the portrait, thereby improving the network's segmentation accuracy for edge pixels during training; and L1 comparison can better output natural and beautiful faces and improve the network training effect.

[0123] In some possible implementations, the step of determining the first loss value and the fourth loss value according to the first mask image and the label of the first image may further include:

[0124] Determining a first difference image between the first mask image and the third mask image;

[0125] respectively determining edge information of the second mask image and edge information of the first difference image;

[0126] The fourth loss value is determined according to a difference between edge information of the second mask image and edge information of the first difference image.

[0127] For example, the segmentation effect of the hair edge region (such as hair strands) in the related art is not good, so the embodiment of the present disclosure removes the hair edge region from the second mask image when generating the label, and obtains a third mask image with higher accuracy. The third mask image is represented as a mask image including the face and hair (such as the hair trunk) of the target object. In training, the difference in edge information between the first mask image output by the model and the label can be considered.

[0128] In some possible implementations, the difference between the first mask image and the third mask image may be calculated to obtain a first difference image. In this way, the first difference image is a mask image including the hair edge region. Furthermore, the edge information of the second mask image and the edge information of the first difference image may be determined respectively by a preset edge detection operator, and the edge information may be an edge image. The edge detection operator is, for example, a Sobel operator, and the present disclosure does not limit the specific type of the edge detection operator.

[0129] In some possible implementations, the fourth loss value may be determined based on the edge information of the second mask image and the edge information of the first difference image. The fourth loss value L4 may be calculated by a mean square error loss function mse, which is expressed as:

[0130] L4 = mse (Sobel(label1), Sobel(net(x)-label2) ) (4)

[0131] In formula (4), x represents the first image, net(x) represents the first mask image output by the image processing model, label1 represents the second mask image, label2 represents the third mask image, and Sobel() represents the first-order differential of the Sobel operator, which is used to generate the edge image of the image, that is, the edge information. Because there may be impurity black spots in the first mask image output by the image processing model, the first-order differential of the Sobel operator is used to calculate the approximate value of the brightness of the image, and by comparing the brightness of the edge of the image, the specific pixel points in the area that exceed the threshold are recorded as edge points and removed, which can make the brightness information in the image more obvious.

[0132] By introducing the fourth loss value, the network can better identify the hair edge area in the portrait during training, improve the segmentation effect of the hair edge area, and thus improve the overall network training effect.

[0133] In some possible implementations, the overall loss value L of the image processing model may be determined according to the weighted sum of the first loss value, the second loss value, the third loss value, and the fourth loss value. 总 , expressed as:

[0134] L 总 =a1L1+a2L2+a3L3+a4L4 (5)

[0135] In formula (5), a1, a2, a3, and a4 represent the weights of the first loss value, the second loss value, the third loss value, and the fourth loss value, respectively, and are, for example, 3, 3, 5, and 2. It should be understood that those skilled in the art may set the weights of the various loss values ​​according to actual conditions, and the present disclosure does not limit this.

[0136] In some possible implementations, after obtaining the overall loss value, the network parameters of the image processing model can be reversely adjusted according to the overall loss value to complete an iterative training; after multiple rounds of iterative training, when the training conditions (such as network convergence) are met, a trained image processing model can be obtained.

[0137] In this way, an image processing model with higher segmentation accuracy can be obtained.

[0138] Figure 6 A flowchart of an image processing method provided by an embodiment of the present disclosure.

[0139] According to an embodiment of the present disclosure, there is also provided an image processing method, such as Figure 6 As shown, the method includes:

[0140] Step S61, processing the fifth image by using an image processing model to obtain a mask image of the fifth image;

[0141] Step S62, performing image segmentation on the fifth image according to the mask image of the fifth image to obtain a sixth image of the target object in the fifth image;

[0142] Wherein, the image processing model is trained according to the above-mentioned training method.

[0143] For example, after the training of the image processing model is completed, the image processing model can be deployed to actual application scenarios, such as video call scenarios, video conferencing scenarios, etc. In the deployment stage, the image processing model may only include an encoding network and a decoding network.

[0144] In some possible implementations, in step S61, the image processing model may be used to perform image segmentation processing on the fifth image to be processed, and a mask image of the fifth image may be output; and then in step S62, the fifth image may be segmented according to the mask image of the fifth image. The mask image of the fifth image may be multiplied with the fifth image to remove the background area in the fifth image, retain the portrait area in the fifth image, and segment the sixth image of the target object from the fifth image.

[0145] In some possible implementations, the sixth image may be applied to actual application scenarios, such as video call scenarios, video conference scenarios, etc., so that the portrait of the target object in the scene is displayed more naturally and beautifully.

[0146] According to the image processing method of the embodiment of the present disclosure, high-precision image segmentation can be achieved, achieving better display effects in the scene.

[0147] According to the training method of the image processing model of the embodiment of the present disclosure, it is possible to generate a second image with the same object as the first image based on the latent coding features, thereby assisting the supervised training of the image processing model; during the training, an attention layer is introduced to strengthen the attention on the label of the first image, so that the image processing model is more sensitive to the data distribution of the segmented edge area, thereby improving the model's edge segmentation accuracy.

[0148] According to the training method of the image processing model of the embodiment of the present disclosure, when calculating the network loss value, in addition to the first loss value representing the difference between the first mask image and the second mask image output by the network, multiple loss values ​​are introduced:

[0149] By introducing a second loss value that characterizes the difference in object identity between the first image and the generated second image, the face of the generated image can be made closer to the face of the original image, and unnatural parts in the generated image can be reduced, making the generated image more beautiful and natural.

[0150] By introducing a third loss value that represents the difference between the first mask image output by the model and the generated second image, and expanding the face frame during calculation, the area involved in the comparison can be expanded to the edge pixels of the portrait, thereby improving the network's segmentation accuracy for edge pixels during training.

[0151] By introducing the fourth loss value that represents the difference in edge information between the first mask image output by the model and the label, the network can better identify the segmentation edge area in the portrait during training, improve the segmentation effect of the segmentation edge area, and thus improve the overall network training effect.

[0152] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0153] In addition, the present disclosure also provides a training device for an image processing model, an image processing device, an electronic device, a computer-readable storage medium, and a computer program product, all of which can be used to implement any image processing model training method or image processing method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.

[0154] Figure 7 A block diagram of a training device for an image processing model provided in an embodiment of the present disclosure.

[0155] Reference Figure 7 , the present disclosure provides a training device for an image processing model, the device comprising:

[0156] A downsampling module 71, configured to perform feature extraction and downsampling on the first image to obtain latent coding features of the first image;

[0157] An upsampling module 72, configured to upsample the latent coding features to obtain a first mask image of the first image;

[0158] An image generation module 73, configured to perform image generation processing on the latent coding feature to obtain a second image;

[0159] The training module 74 is used to train the image processing model according to the first image and the label of the first image, the first mask image and the second image.

[0160] In some possible implementations, the training module 74 is used to: determine a first loss value and a fourth loss value based on the first mask map and the label of the first image; determine a second loss value based on the first image and the second image; determine a third loss value based on the first mask map and the second image; and train the image processing model based on the first loss value, the second loss value, the third loss value, and the fourth loss value.

[0161] In some possible implementations, the training module 74 is used to: perform face recognition on the first image to obtain a third image; and determine the second loss value according to the distance between the third image and the second image.

[0162] In some possible implementations, the training module 74 is used to: determine a fourth image of the target object based on the first mask image and the first image; perform face detection on the fourth image and the second image respectively to determine a first face area of ​​the fourth image and a second face area of ​​the second image; expand the first face area in the fourth image to obtain a first expanded area; and expand the second face area in the second image to obtain a second expanded area; and determine the third loss value based on the difference between the first expanded area and the second expanded area.

[0163] In some possible implementations, the label of the first image includes a second mask image and a third mask image;

[0164] Among them, the training module 74 is used to: determine the first loss value according to the difference between the first mask image and the second mask image; determine the first difference image between the first mask image and the third mask image; respectively determine the edge information of the first mask image and the edge information of the first difference image; determine the fourth loss value according to the difference between the edge information of the first mask image and the edge information of the first difference image.

[0165] In some possible implementations, the image processing model further includes an attention layer, and the apparatus further includes:

[0166] an attention enhancement module, configured to enhance the attention of the label of the first image through the attention layer to obtain attention weight information;

[0167] Among them, the upsampling module 72 is used to: upsample the latent coding features of the first image to obtain a first sampling feature; upsample the first sampling feature based on the attention weight information to obtain a second sampling feature; upsample the second sampling feature to obtain the first mask image.

[0168] In some possible implementations, the label of the first image includes a second mask image, and the attention enhancement module is configured to: determine a second difference image between the second mask image and the first mask image; and normalize the second mask image according to the second difference image to obtain a fourth mask image;

[0169] The fourth mask image is input into the attention layer for processing to obtain the attention weight information.

[0170] Figure 8 A block diagram of an image processing device provided in an embodiment of the present disclosure.

[0171] Reference Figure 8 , an embodiment of the present disclosure provides an image processing device, the device comprising:

[0172] A processing module 81 is used to process the fifth image through an image processing model to obtain a mask image of the fifth image;

[0173] The segmentation module 82 is used to perform image segmentation on the fifth image according to the mask image of the fifth image to obtain a sixth image of the target object in the fifth image; wherein the image processing model is trained according to the above-mentioned training method.

[0174] The training device of the above-mentioned image processing model and each module in the image processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0175] Fig. 9 A block diagram of an electronic device provided in an embodiment of the present disclosure.

[0176] Reference Fig. 9 An embodiment of the present disclosure provides an electronic device, which includes: at least one processor 701; at least one memory 702, and one or more I / O interfaces 703, connected between the processor 701 and the memory 702; wherein the memory 702 stores one or more computer programs that can be executed by the at least one processor 701, and the one or more computer programs are executed by the at least one processor 701 so that the at least one processor 701 can execute the above-mentioned image processing model training method or image processing method.

[0177] Each module in the above electronic device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0178] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned image processing model training method or image processing method when executed by a processor / processing core. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0179] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned image processing model training method or image processing method.

[0180] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0181] As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0182] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0183] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0184] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0185] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0186] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0187] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0188] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0189] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A training method for an image processing model, characterized in that: include: Performing feature extraction and downsampling on the first image to obtain latent coding features of the first image; Upsampling the latent coding features to obtain a first mask image of the first image; Performing image generation processing on the latent coding features to obtain a second image; The image processing model is trained according to the first image and the label of the first image, the first mask image and the second image.

2. The method according to claim 1, characterized in that The step of training the image processing model according to the first image and the label of the first image, the first mask image and the second image comprises: Determine a first loss value and a fourth loss value according to the first mask image and the label of the first image; determining a second loss value according to the first image and the second image; Determine a third loss value according to the first mask image and the second image; The image processing model is trained according to the first loss value, the second loss value, the third loss value and the fourth loss value.

3. The method according to claim 2, characterized in that The determining a second loss value according to the first image and the second image includes: Performing face recognition on the first image to obtain a third image; The second loss value is determined according to a distance between the third image and the second image.

4. The method according to claim 2, characterized in that: The first image includes a target object, and determining a third loss value according to the first mask image and the second image includes: Determining a fourth image of the target object according to the first mask image and the first image; Performing face detection on the fourth image and the second image respectively to determine a first face region of the fourth image and a second face region of the second image; Expanding the first face region in the fourth image to obtain a first expanded region; and expanding the second face region in the second image to obtain a second expanded region; The third loss value is determined according to a difference between the first extension area and the second extension area.

5. The method according to claim 2, characterized in that: The label of the first image includes a second mask image and a third mask image; Wherein, determining the first loss value and the fourth loss value of the image processing model according to the label of the first image and the first mask image includes: Determining the first loss value according to a difference between the first mask image and the second mask image; Determining a first difference image between the first mask image and the third mask image; respectively determining edge information of the first mask image and edge information of the first difference image; The fourth loss value is determined according to a difference between edge information of the first mask image and edge information of the first difference image.

6. The method according to claim 1, characterized in that The image processing model further includes an attention layer, and the method further includes: Attention is enhanced on the label of the first image by the attention layer to obtain attention weight information; The step of upsampling the latent coding feature to obtain a first mask image of the first image includes: Upsampling the latent coding features of the first image to obtain first sampling features; Upsampling the first sampling feature based on the attention weight information to obtain a second sampling feature; The second sampled features are upsampled to obtain the first mask image.

7. The method according to claim 6, characterized in that The label of the first image includes a second mask image, and the attention weight information is obtained by performing attention enhancement on the label of the first image through the attention layer, including: determining a second difference image between the second mask image and the first mask image; normalizing the second mask image according to the second difference image to obtain a fourth mask image; The fourth mask image is input into the attention layer for processing to obtain the attention weight information.

8. An image processing method, characterized in that: include: Processing the fifth image by using the image processing model to obtain a mask image of the fifth image; performing image segmentation on the fifth image according to the mask image of the fifth image to obtain a sixth image of the target object in the fifth image; Wherein, the image processing model is trained according to any one of the training methods according to claims 1-7.

9. A training device for an image processing model, characterized in that: include: A downsampling module, used for performing feature extraction and downsampling on the first image to obtain latent coding features of the first image; an upsampling module, configured to upsample the latent coding features to obtain a first mask image of the first image; An image generation module, used for performing image generation processing on the latent coding feature to obtain a second image; A training module is used to train the image processing model according to the first image and the label of the first image, the first mask image and the second image.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor so that the at least one processor can execute the training method of the image processing model described in any one of claims 1 to 7 or the image processing method described in claim 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the training method of the image processing model as described in any one of claims 1 to 7 or the image processing method as described in claim 8.

12. A computer program product, characterized in that It includes a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the training method of the image processing model as described in any one of claims 1 to 7 or the image processing method as described in claim 8.

Images