Image processing model training method, image processing method and related equipment

Abstract

The present disclosure relates to an image processing model training method, an image processing method, and related equipment. The method inputs a sample image and a sample reference image into a feature extraction network and a color mapping table generation network of an image processing model to be trained, respectively, to obtain a sample feature map and a generated color mapping table. The sample feature map is then input into the light scene recognition network of the model to obtain a light scene recognition result. Based on the light scene recognition result, the generated color mapping table is fused in the light scene dimension to obtain a target color mapping table. Based on the target color mapping table, a color mapping process is performed on the sample image to obtain a target processed image. A target loss is determined based on the target processed image, the sample image, and the sample reference image. The target loss is then used to train the image processing model to be trained to obtain an image processing model. The present disclosure improves image processing effects and can achieve skin color editing with any desired effect, thereby improving image processing effects.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an image processing model training method, an image processing method, and related equipment. Background Art

[0002] With the development of artificial intelligence (AI) technology, skin tone editing based on image processing has become widely used in various applications. For example, traditional portrait enhancement techniques can be used to edit skin tones to specific skin tones, such as fair or wheatish. However, the skin tone editing effects of related technologies are still poor, and it is impossible to achieve any desired skin tone editing effect, resulting in poor image processing results. Summary of the Invention

[0003] The present disclosure provides an image processing model training method, an image processing method, and related devices to at least address the problem in related technologies of poor image skin color editing effects and the inability to achieve any desired skin color editing effect, resulting in poor image processing effects. The technical solutions of the present disclosure are as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing model training method, comprising:

[0005] Acquire a sample image and a sample reference image corresponding to the sample image, wherein the sample image includes a skin area of ​​the subject, and the sample reference image includes a reference skin area;

[0006] Inputting the sample image and the sample reference image into a feature extraction network and a color mapping table generation network of the image processing model to be trained, respectively, to obtain a sample feature map output by the feature extraction network and a generated color mapping table output by the color mapping table generation network; the generated color mapping table includes color mapping sub-tables corresponding to a plurality of preset light scenes;

[0007] Inputting the sample feature map into the light scene recognition network of the image processing model to be trained to obtain a light scene recognition result; the light scene recognition result represents the probability of belonging to each of the preset light scenes;

[0008] Based on the light scene recognition result, the generated color mapping table is fused in the light scene dimension to obtain a target color mapping table;

[0009] Performing color mapping processing on the sample image based on the target color mapping table to obtain a target processed image;

[0010] A target loss is determined based on the target processing image, the sample image, and the sample reference image. Minimizing the target loss is used as a training goal, and the model parameters of the image processing model to be trained are adjusted until a preset training end condition is met, thereby obtaining a trained image processing model; wherein the target loss characterizes the background difference between the target processing image and the sample image, as well as the skin area difference and skin color difference between the target processing image and the sample reference image.

[0011] In some exemplary embodiments, the method further comprises:

[0012] Inputting the sample feature map into the spatial structure capture network of the image processing model to be trained to obtain a spatial structure map output by the spatial structure capture network; the spatial structure map represents the spatial structure features of the sample image;

[0013] The performing color mapping processing on the sample image based on the target color mapping table to obtain a target processed image includes:

[0014] Performing color mapping processing on the sample image based on the target color mapping table to obtain an intermediate processed image;

[0015] The intermediate processed image and the spatial structure map are fused to obtain a target processed image; wherein the target loss further represents the difference in object contours between the target processed image and the sample image.

[0016] In some exemplary embodiments, the spatial structure graph includes a plurality of spatial structure subgraphs of preset channels, and each of the color mapping subtables includes color mapping information corresponding to each of the preset channels;

[0017] The step of fusing the generated color mapping table in a light scene dimension based on the light scene recognition result to obtain a target color mapping table includes:

[0018] For each of the preset channels, weighted summing is performed on the color mapping information of each color mapping sub-table for the preset channel based on the light scene recognition result to obtain a target color mapping sub-table for each of the preset channels; wherein the target color mapping table includes the target color mapping sub-tables for each of the preset channels;

[0019] The performing color mapping processing on the sample image based on the target color mapping table to obtain an intermediate processed image includes:

[0020] Color mapping processing is performed on the sample images based on the target color mapping sub-tables of the preset channels to obtain intermediate processed sub-images of the preset channels; the intermediate processed images include the intermediate processed sub-images of the preset channels.

[0021] In some exemplary embodiments, fusing the intermediate processed image and the spatial structure map to obtain a target processed image includes:

[0022] Fusing the intermediate processed sub-image of each preset channel with the spatial structure sub-image of the corresponding preset channel to obtain the target processed sub-image of each preset channel;

[0023] The target processed sub-images of the preset channels are spliced ​​together to obtain the target processed image.

[0024] In some exemplary embodiments, determining the target loss based on the target processed image, the sample image, and the sample reference image includes:

[0025] Determining a background consistency loss based on a difference between a background region in the target processed image and a background region in the sample image; the background region refers to an image region excluding the object;

[0026] determining a structural consistency loss based on a difference between contour information of the object in the target processed image and contour information of the object in the sample image;

[0027] Determining a first feature map corresponding to a skin region of the subject in the target processed image and a second feature map corresponding to the reference skin region in the sample reference image; and determining a perceptual loss based on a distance between the first feature map and the second feature map;

[0028] Migrating the skin color of the reference skin area in the sample reference image to the skin area of ​​the subject in the sample image to obtain a migrated sample image; determining a color migration loss based on a pixel difference between the target processed image and the migrated sample image;

[0029] The background consistency loss, the structure consistency loss, the perception loss and the color migration loss are linearly fused to obtain the target loss.

[0030] In some exemplary embodiments, migrating the skin color of the reference skin area in the sample reference image to the skin area of ​​the subject in the sample image to obtain the migrated sample image includes:

[0031] determining a first color histogram of the reference skin area in the sample reference image and a second color histogram of the skin area of ​​the subject in the sample image;

[0032] A histogram matching algorithm is used to perform color migration on the pixel value of each pixel in the skin area of ​​the object in the sample image based on the first color histogram and the second color histogram to obtain a migrated sample image.

[0033] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing method, including:

[0034] Acquire an image to be processed and a reference image; the image to be processed includes a skin area of ​​a target object, and the reference image includes a reference skin area;

[0035] Inputting the image to be processed and the reference image into an image processing model for image processing, respectively, to obtain a processed target image; wherein the color of the skin area of ​​the target object in the target image is consistent with the color of the reference skin area in the reference image;

[0036] The image processing model is trained based on the image processing model training method of the first aspect mentioned above.

[0037] According to a third aspect of an embodiment of the present disclosure, there is provided an image processing model training device, comprising:

[0038] a sample image acquisition unit configured to acquire a sample image and a sample reference image corresponding to the sample image, wherein the sample image includes a skin area of ​​the subject, and the sample reference image includes a reference skin area;

[0039] a first processing unit configured to input the sample image and the sample reference image into a feature extraction network and a color mapping table generation network of a to-be-trained image processing model, respectively, to obtain a sample feature map output by the feature extraction network and a generated color mapping table output by the color mapping table generation network; the generated color mapping table includes color mapping sub-tables corresponding to a plurality of preset light scenes;

[0040] a light scene recognition unit configured to input the sample feature map into the light scene recognition network of the image processing model to be trained to obtain a light scene recognition result; the light scene recognition result represents the probability of belonging to each of the preset light scenes;

[0041] A first fusion unit is configured to perform fusion of the generated color mapping table in a light scene dimension based on the light scene recognition result to obtain a target color mapping table;

[0042] A color mapping unit is configured to perform color mapping processing on the sample image based on the target color mapping table to obtain a target processed image;

[0043] A training unit is configured to determine a target loss based on the target processing image, the sample image and the sample reference image, and to adjust the model parameters of the image processing model to be trained with minimizing the target loss as the training goal until a preset training end condition is met, thereby obtaining a trained image processing model; wherein the target loss represents the background difference between the target processing image and the sample image, and the skin area difference and skin color difference between the target processing image and the sample reference image.

[0044] In some exemplary embodiments, the apparatus further comprises:

[0045] a spatial structure capture unit configured to input the sample feature map into the spatial structure capture network of the image processing model to be trained, and obtain a spatial structure map output by the spatial structure capture network; the spatial structure map represents the spatial structure features of the sample image;

[0046] The color mapping unit includes:

[0047] an intermediate color mapping unit, configured to perform color mapping processing on the sample image based on the target color mapping table to obtain an intermediate processed image;

[0048] The second fusion unit is configured to perform fusion of the intermediate processed image and the spatial structure map to obtain a target processed image; wherein the target loss also represents the difference in object contour between the target processed image and the sample image.

[0049] In some exemplary embodiments, the spatial structure graph includes a plurality of spatial structure subgraphs of preset channels, and each of the color mapping subtables includes color mapping information corresponding to each of the preset channels;

[0050] The first fusion unit is specifically configured to perform, for each preset channel, a weighted summation of the color mapping information of each color mapping sub-table in the preset channel based on the light scene recognition result, to obtain a target color mapping sub-table for each preset channel; wherein the target color mapping table includes the target color mapping sub-table of each preset channel;

[0051] The intermediate color mapping unit is specifically configured to perform color mapping processing on the sample image based on the target color mapping sub-table of each preset channel to obtain an intermediate processed sub-image of each preset channel; the intermediate processed image includes the intermediate processed sub-image of each preset channel.

[0052] In some exemplary embodiments, the second fusion unit is specifically configured to perform fusion of the intermediate processing sub-image of each of the preset channels with the spatial structure sub-image of the corresponding preset channel to obtain the target processing sub-image of each of the preset channels; and splice the target processing sub-images of each of the preset channels to obtain the target processing image.

[0053] In some exemplary embodiments, the training unit includes:

[0054] A background consistency loss determining unit is configured to determine a background consistency loss based on a difference between a background area in the target processing image and a background area in the sample image; the background area refers to an image area excluding the object;

[0055] a structure consistency loss determining unit configured to determine a structure consistency loss based on a difference between contour information of the object in the target processing image and contour information of the object in the sample image;

[0056] a perceptual loss determining unit configured to determine a first feature map corresponding to a skin region of the object in the target processed image and a second feature map corresponding to the reference skin region in the sample reference image; and determine a perceptual loss based on a distance between the first feature map and the second feature map;

[0057] a color migration loss determining unit configured to migrate the skin color of the reference skin area in the sample reference image to the skin area of ​​the subject in the sample image to obtain a migrated sample image; and determine the color migration loss based on the pixel difference between the target processed image and the migrated sample image;

[0058] The target loss determination unit is configured to perform linear fusion of the background consistency loss, the structure consistency loss, the perception loss and the color migration loss to obtain the target loss.

[0059] In some exemplary embodiments, the color migration loss determining unit includes:

[0060] a color histogram determining unit configured to determine a first color histogram of the reference skin area in the sample reference image and a second color histogram of the skin area of ​​the subject in the sample image;

[0061] The color migration unit is configured to perform color migration on the pixel value of each pixel in the skin area of ​​the object in the sample image based on the first color histogram and the second color histogram using a histogram matching algorithm to obtain a migrated sample image.

[0062] According to a fourth aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, including:

[0063] An image acquisition unit is configured to acquire an image to be processed and a reference image; the image to be processed includes a skin area of ​​a target object, and the reference image includes a reference skin area;

[0064] a second processing unit configured to input the to-be-processed image and the reference image into an image processing model for image processing, respectively, to obtain a processed target image; wherein a color of a skin region of the target object in the target image is consistent with a color of a reference skin region in the reference image;

[0065] The image processing model is trained based on the image processing model training method of the first aspect mentioned above.

[0066] According to a fifth aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0067] processor;

[0068] a memory for storing instructions executable by the processor;

[0069] In which, the processor is configured to execute the instructions to implement the image processing model training method of the first aspect mentioned above, or the image processing method of the second aspect mentioned above.

[0070] According to the sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing model training method of the first aspect above, or the image processing method of the second aspect above.

[0071] According to the seventh aspect of the embodiments of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the image processing model training method of the first aspect or the image processing method of the second aspect.

[0072] The embodiment of the present disclosure obtains a sample image and a sample reference image corresponding to the sample image, and inputs the sample image and the sample reference image into a feature extraction network and a color mapping table generation network of an image processing model to be trained, respectively, to obtain a sample feature map and a generated color mapping table. The generated color mapping table includes color mapping sub-tables corresponding to a plurality of preset light scenes, and then inputs the sample feature map into a light scene recognition network of the image processing model to be trained to obtain a light scene recognition result. The light scene recognition result represents the probability of belonging to each preset light scene, and based on the light scene recognition result, the generated color mapping table is fused in the light scene dimension to obtain a target color mapping table. Based on the target color mapping table, the sample image is color mapped to obtain a target processed image. , based on the target processing image, the sample image and the sample reference image, the target loss is determined, and the model parameters of the image processing model to be trained are adjusted with minimizing the target loss as the training target until the preset training end conditions are met to obtain a trained image processing model, wherein the target loss represents the background difference between the target processing image and the sample image, as well as the skin area difference and skin color difference between the target processing image and the sample reference image, so that the trained image processing model can make people with different skin colors reach the skin color of the reference image, and ensure that it only acts on the skin area without affecting the background. At the same time, it has light adaptation, such as no whitening under over-exposure light, which improves the image processing effect, and can provide users with sufficient freedom to achieve any desired skin color editing effect.

[0073] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0075] Figure 1 is a schematic diagram showing an application environment according to an exemplary embodiment;

[0076] Figure 2 is a flowchart illustrating a method for training an image processing model according to an exemplary embodiment;

[0077] Figure 3 is a schematic structural diagram of an image processing model to be trained according to an exemplary embodiment;

[0078] Figure 4 is a schematic structural diagram of another image processing model to be trained according to an exemplary embodiment;

[0079] Figure 5 is a flowchart illustrating another image processing model training method according to an exemplary embodiment;

[0080] Figure 6 is a flowchart illustrating another image processing model training method according to an exemplary embodiment;

[0081] Figure 7 is a flowchart illustrating another image processing model training method according to an exemplary embodiment;

[0082] Figure 8 is a structural block diagram of an image processing model training device according to an exemplary embodiment;

[0083] Figure 9 is a structural block diagram of an image processing apparatus according to an exemplary embodiment;

[0084] Figure 10 The figure is a structural block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0085] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0086] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0087] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0088] See also Figure 1 , which is a schematic diagram of an application environment according to an exemplary embodiment. The application environment may include a terminal 110 and a server 120. The terminal 110 and the server 120 may be connected via a wired network or a wireless network.

[0089] Terminal 110 may be, but is not limited to, a smartphone, tablet computer, laptop computer, or desktop computer. Terminal 110 may be installed with client software, such as an application (App), that provides image processing functionality, including skin color editing. The application may be a standalone application or a subroutine within the application. Exemplary applications may include video applications and live streaming applications. Users of terminal 110 may log in to the application using pre-registered user information, which may include an account number and password.

[0090] Server 120 may be a server that provides background services for applications in terminal 110, and the background services may be image processing services. Server 120 may store an image processing model, and server 120 may invoke the image processing model to perform image processing on the image to be processed. The image processing includes adjusting the color of the skin area of ​​the target object in the image to be processed to be consistent with the color of the reference skin area in the reference image.

[0091] In a specific application scenario, the terminal 110 may send the image to be processed and the reference image to the server 120, and the server 120 may call the above-mentioned image processing model to perform image processing on the image to be processed to obtain a target image, and then return the target image to the terminal 110. In another specific application scenario, the terminal 110 may download the image processing model from the server 120 in advance and store it locally. Then, during image processing, the locally stored image processing model may be called to process the image to be processed based on the obtained reference image to obtain the target image.

[0092] In the embodiment of the present disclosure, the server 120 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, an edge computing node, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0093] The following combination Figures 2 to 7 To introduce the training process of the image processing model of the embodiment of the present disclosure.

[0094] See also Figure 2 , which is a flowchart of an image processing model training method according to an exemplary embodiment. Figure 2 As shown, the following steps are included:

[0095] In step S201, a sample image and a sample reference image corresponding to the sample image are acquired, wherein the sample image includes a skin area of ​​the object, and the sample reference image includes a reference skin area.

[0096] The object may be a complete human body or a part of a human body, such as a face. The sample reference image may be any image including a skin area.

[0097] In the embodiment of the present disclosure, a sample image and its corresponding sample reference image are regarded as an image pair, and the image pair can be any image pair in a set of image pairs used to train an image processing model.

[0098] In step S203, the sample image and the sample reference image are respectively input into the feature extraction network and the color mapping table generation network of the image processing model to be trained to obtain the sample feature map output by the feature extraction network and the generated color mapping table output by the color mapping table generation network.

[0099] The color mapping table generated by the color mapping table generation network includes color mapping sub-tables corresponding to multiple preset light scenes, each color mapping sub-table is used to represent the mapping relationship between input pixel information and output pixel information. The multiple preset light scenes have different light intensities.

[0100] In step S205, the sample feature map is input into the light scene recognition network of the image processing model to be trained to obtain a light scene recognition result.

[0101] The light scene recognition result represents the probability of belonging to each of the preset light scenes.

[0102] like Figure 3 The figure shows a structural diagram of an image processing model to be trained provided by an embodiment of the present disclosure, and the image processing model to be trained may include a feature extraction network, a light scene recognition network and a color mapping table generation network. Among them, the feature extraction network is used to extract features from the sample image to obtain a sample feature map, and the sample feature map can be used as the input of the light scene recognition network to identify the probability that the current light scene belongs to each preset light scene to obtain a light scene recognition result. The color mapping table generation network takes the sample reference image as input, and is used to generate a color mapping table using the sample reference image output. The generated color mapping table includes a color mapping sub-table corresponding to each preset light scene. For example, if there are T different preset light scenes, the light scene recognition result can be expressed as {ω0,ω1,…,ω T-1}, the generated color map output by the color map generation network can be expressed as {V0,V1,…,V T-1}.

[0103] The feature extraction network and color map generation network in the image processing model to be trained can both be untrained or incompletely trained artificial neural networks, for example, both can be machine learning models based on convolutional neural network technology. The light scene recognition network in the image processing model to be trained can be a neural network layer with classification capabilities, for example, an Affine transformation layer. T different preset light scenes can serve as hyperparameters of the neural network.

[0104] In step S207, the generated color mapping table is fused in the light scene dimension based on the light scene recognition result to obtain a target color mapping table.

[0105] Specifically, the light scene recognition result can be used as a fusion coefficient for generating color mapping sub-tables corresponding to preset light scenes in the color mapping table, and weighted summation is performed on each color mapping sub-table to obtain a target color mapping table.

[0106] In step S209, color mapping processing is performed on the sample image based on the target color mapping table to obtain a target processed image.

[0107] Specifically, for the pixel information of any pixel in the sample image, the pixel information is used as the input pixel information of the target color mapping table, and the corresponding target pixel information is searched as the output pixel information through the target color mapping table with the input pixel information as the index, and the pixel information of the pixel in the sample image is replaced with the output pixel information, so that the target processed image can be obtained after performing the above-mentioned color mapping processing on each pixel of the sample image.

[0108] In step S211, the target loss is determined based on the target processing image, the sample image and the sample reference image, and the model parameters of the image processing model to be trained are adjusted with minimizing the target loss as the training goal until the preset training end conditions are met to obtain a trained image processing model.

[0109] The target loss represents the background difference between the target processed image and the sample image, and the perceptual difference and skin color difference between the target processed image and the sample reference image.

[0110] The target loss can be obtained based on a background consistency loss for measuring the background difference between the target processed image and the sample image, a perceptual loss for measuring the perceptual difference between the target processed image and the sample reference image, and a color histogram loss for measuring the skin color difference between the target processed image and the sample reference image. For example, the target loss can be obtained by performing a weighted summation on the above losses. Exemplarily, when calculating the color histogram loss, the skin color of the reference skin area in the sample reference image can be first migrated to the skin area of ​​the object in the sample image to obtain a migrated sample image, and then the color histogram loss is determined based on the pixel difference between the target processed image and the migrated sample image.

[0111] Specifically, the training objective is to minimize the target loss, adjust the model parameters of the image processing model to be trained, and iterate training based on the adjusted model parameters until a preset training termination condition is met. The image processing model corresponding to the model parameters at the end of training is used as the final trained image processing model. The adjusted model parameters include those of the feature extraction network, the color map generation network, and the light scene recognition network.

[0112] The preset training end condition may be that the target loss reaches a preset loss threshold, or the number of iterations reaches a preset number of iterations, or the difference between the target losses of two adjacent iterations reaches a preset loss difference.

[0113] The image processing model trained through the above-mentioned implementation method can have the ability to perceive the skin color and light scene of the image to be processed, and has the ability to generate a target color mapping table based on the reference image provided by the user. The target color mapping table is different from a simple three-dimensional color mapping table. In actual tasks, it can be understood as a set of feature three-dimensional color mapping tables with adaptive capabilities and their combined weight matrices, so that any skin color migration can be achieved without changing the background color of the image to be processed. In turn, the trained image processing model can enable people with different skin colors to reach the skin color of the reference image, and ensure that it only acts on the skin area without affecting the background. At the same time, it has light adaptation, such as no whitening under overexposed light, which improves the image processing effect, and can provide users with sufficient freedom to achieve any desired skin color editing effect.

[0114] In some exemplary embodiments, in order to further improve the image processing effect of the trained image processing model and avoid the problem of lack of three-dimensionality and layering in the processed image, as shown in FIG. Figure 4A structural schematic diagram of another image processing model to be trained is provided, which may also include a spatial structure capture network (decoder), which takes a sample feature map as input and a spatial structure map representing the spatial structure features of the sample image as output, and the resolution of the spatial structure map is consistent with the resolution of the sample image.

[0115] Based on this, Figure 5 A flowchart of another image processing model training method is provided, the method further comprising:

[0116] In step S501, the sample feature map is input into the spatial structure capture network of the image processing model to be trained to obtain a spatial structure map output by the spatial structure capture network.

[0117] The spatial structure graph represents the spatial structure features of the sample image.

[0118] Then, the aforementioned step S209, when performing color mapping processing on the sample image based on the target color mapping table to obtain the target processed image, can be specifically implemented as the following steps S503 to S505:

[0119] In step S503, color mapping processing is performed on the sample image based on the target color mapping table to obtain an intermediate processed image.

[0120] In step S505, the intermediate processed image and the spatial structure map are fused to obtain a target processed image; wherein the target loss also represents the difference in object contour between the target processed image and the sample image.

[0121] In a specific implementation, the intermediate processed image and the spatial structure map may be linearly added to achieve fusion of the two to obtain a target processed image.

[0122] In some exemplary embodiments, the spatial structure graph includes a plurality of spatial structure subgraphs of preset channels, the resolution of each spatial structure subgraph is consistent with the resolution of the sample image, and each of the color mapping subtables includes color mapping information corresponding to each of the preset channels. Among them, the plurality of preset channels can be represented by the spatial positions of pixels in the image. Taking the spatial positions of pixels in the image as an example, which include three categories, namely, those located in the background area, those located in the skin area, and those located in the three-dimensional area related to the stereoscopic sense (such as the nose area and forehead area of ​​the human face), the spatial structure graph output by the spatial structure capture network of the image processing model to be trained may include the spatial structure subgraphs of the three preset channels corresponding to the above three categories. Based on this, if Figure 6As shown, the aforementioned step S207, when fusing the generated color mapping table in the light scene dimension based on the light scene recognition result to obtain the target color mapping table, can be specifically implemented as the following step S601:

[0123] In step S601, for each of the preset channels, the color mapping information of each color mapping sub-table in the preset channel is weightedly summed based on the light scene recognition result to obtain a target color mapping sub-table for each of the preset channels; wherein the target color mapping table includes the target color mapping sub-table of each of the preset channels.

[0124] Then, the aforementioned step S503, when performing color mapping processing on the sample image based on the target color mapping table to obtain an intermediate processed image, can be specifically implemented as the following step S603:

[0125] In step S603, color mapping processing is performed on the sample image based on the target color mapping sub-table of each preset channel to obtain an intermediate processed sub-image of each preset channel; the intermediate processed image includes the intermediate processed sub-image of each preset channel.

[0126] In a specific implementation, the spatial structure diagram can be expressed as (H, W, M), where H represents the image height, W represents the image width, and M represents the number of multiple preset channels. Accordingly, the generated color mapping table output by the color mapping table generation network can be expressed as a TxM matrix, where each row in the T rows of the matrix represents a color mapping sub-table, and each column in the M columns is the color mapping information of each color mapping sub-table in a preset channel. The color mapping information represents the mapping relationship between the input pixel value and the output pixel value. Figure 4 As shown, for each column of the TxM matrix, based on the light scene recognition results {ω0,ω1,…,ω T-1 Perform a weighted summation on the color mapping information of the column to obtain M target color mapping sub-tables. Then apply these M target color mapping sub-tables to the sample image to obtain M intermediate processed sub-images.

[0127] In the above embodiment, the spatial structure graph includes a plurality of spatial structure subgraphs of preset channels, and each color mapping subtable includes color mapping information corresponding to each preset channel, so that the target color mapping subtable of each preset channel is determined when determining the target color mapping table, and the target color mapping subtable of each preset channel is obtained by weighted summing the color mapping information of each color mapping subtable in the preset channel based on the light scene recognition result. Then, when the target color mapping table is applied to the sample image, the target color mapping subtable of each preset channel can be applied to the sample image respectively to obtain the intermediate processed subimage of each preset channel, thereby realizing color mapping based on the spatial structure characteristics of the sample image, which is beneficial for realizing the skin color migration of the reference skin area without changing the three-dimensional sense and background color of the image.

[0128] Based on this, see Figure 6 The aforementioned step S505 can be specifically implemented as the following steps S605 to S607 when fusing the intermediate processed image and the spatial structure diagram to obtain the target processed image:

[0129] In step S605 , the intermediate processed sub-image of each preset channel is fused with the spatial structure sub-image of the corresponding preset channel to obtain the target processed sub-image of each preset channel.

[0130] In a specific implementation, corresponding weights can be assigned to the intermediate processing sub-image and the spatial structure sub-image, and then when the images of each preset channel are fused, the weighted summation of the intermediate processing sub-image and the spatial structure sub-image of the preset channel is performed based on the corresponding weights to obtain the target processing sub-image of the preset channel.

[0131] In step S607 , the target processed sub-images of the preset channels are spliced ​​together to obtain the target processed image.

[0132] In the above embodiment, the intermediate processing sub-image and the spatial structure sub-image are fused on each preset channel to obtain the target processing sub-image, and then the target processing sub-images of each preset channel are spliced ​​to obtain the target processing image, which is beneficial for achieving the skin color migration of the reference skin area while not changing the three-dimensional effect and background color of the image.

[0133] In some exemplary embodiments, Figure 7 As shown, the aforementioned step S211, when determining the target loss based on the target processed image, the sample image and the sample reference image, may include:

[0134] In step S701 , a background consistency loss is determined based on a difference between a background region in the target processing image and a background region in the sample image.

[0135] The background area refers to the image area excluding the object.

[0136] In a specific implementation, the background consistency loss can be expressed as Loss1 = ||P1-P2||2, where ||*||2 represents the 2-norm, P1 represents the background area in the target processed image, and P2 represents the background area in the sample image.

[0137] In step S703 , a structural consistency loss is determined based on a difference between the contour information of the object in the target processing image and the contour information of the object in the sample image.

[0138] In a specific implementation, the structural consistency loss can be expressed as Loss2=||O1-O2||2, where ||*||2 represents the 2-norm, O1 represents the contour area of ​​the object in the target processing image, and O2 represents the contour area of ​​the object in the sample image.

[0139] In step S705, a first feature map corresponding to the skin area of ​​the object in the target processed image and a second feature map corresponding to the reference skin area in the sample reference image are determined; and a perceptual loss is determined based on a distance between the first feature map and the second feature map.

[0140] Among them, the perceptual loss can be expressed as Loss3==||F1-F3||2, where ||*||2 represents the 2-norm, F1 represents the first feature map, and F3 represents the second feature map.

[0141] In step S707, the skin color of the reference skin area in the sample reference image is transferred to the skin area of ​​the subject in the sample image to obtain a transferred sample image; and a color transfer loss is determined based on the pixel difference between the target processed image and the transferred sample image;

[0142] Specifically, the color migration loss can be expressed as Where G represents the target processing image, Y represents the sample image after migration, N represents the total number of pixels in the target processing image (the sample image after migration has the same resolution as the target processing image), G i Represents the pixel value of the i-th pixel in the target processing image G, Y i Represents the pixel value of the i-th pixel in the sample image Y after migration.

[0143] Exemplarily, migrating the skin color of the reference skin area in the sample reference image to the skin area of ​​the object in the sample image to obtain the migrated sample image may include:

[0144] determining a first color histogram of the reference skin area in the sample reference image and a second color histogram of the skin area of ​​the subject in the sample image;

[0145] A histogram matching algorithm is used to perform color migration on the pixel value of each pixel in the skin area of ​​the object in the sample image based on the first color histogram and the second color histogram to obtain a migrated sample image.

[0146] In the above embodiment, the histogram matching algorithm is used to achieve the migration of the skin color of the reference skin area to the skin area of ​​the object in the sample image, which is conducive to improving the efficiency of skin color migration.

[0147] In a specific implementation, when skin color migration is performed based on the above-mentioned histogram matching algorithm, the above-mentioned color migration loss can also be referred to as color histogram loss.

[0148] In step S709 , the background consistency loss, the structure consistency loss, the perception loss, and the color histogram loss are linearly fused to obtain the target loss.

[0149] In a specific implementation, the background consistency loss, the structural consistency loss, the perceptual loss, and the color histogram loss may be weighted and summed to obtain the target loss, wherein the weights of each may be set based on actual experience. For example, the target loss L may be calculated using the following formula:

[0150] L=C1*Loss1+C2*Loss2+C3*Loss3+C4*Loss4

[0151] Among them, C1 to C4 represent the weight coefficients of each loss, and C1+C2+C3+C4=1.

[0152] The target loss in the above-mentioned implementation simultaneously models the trained image processing model's ability to perceive and adapt to skin color and lighting scenarios, as well as its ability to capture spatial structure. This allows the trained image processing model to edit different skin tones while only affecting the skin area, leaving the image background color unchanged and preserving the object's three-dimensionality and layering. Furthermore, it can achieve the same target skin color for different skin tones and adapt to various lighting scenarios. Furthermore, it provides users with full freedom, freeing them from being restricted by "official functions" and allowing them to achieve any desired skin tone editing effect.

[0153] The present disclosure also provides an image processing method, which specifically includes:

[0154] Acquire an image to be processed and a reference image; the image to be processed includes a skin area of ​​a target object, and the reference image includes a reference skin area;

[0155] Inputting the image to be processed and the reference image into an image processing model for image processing, respectively, to obtain a processed target image; wherein the color of the skin area of ​​the target object in the target image is consistent with the color of the reference skin area in the reference image;

[0156] The image processing model is trained based on any one of the image processing model training methods implemented in the present disclosure.

[0157] Specifically, the image to be processed is input into the feature extraction network of the image processing model to obtain a feature map corresponding to the image to be processed. The reference image is input into the color mapping table generation network of the image processing model to obtain an output color mapping table, which includes color mapping sub-tables corresponding to multiple preset lighting scenarios. Next, the feature map corresponding to the image to be processed is input into the lighting scene recognition network of the image processing model to obtain a lighting scene recognition result. The lighting scene recognition result represents the probability of belonging to each preset lighting scenario. Based on the lighting scene recognition result, the color mapping table is then fused in the lighting scene dimension to obtain a target color mapping table. Based on this target color mapping table, color mapping processing is performed on the image to be processed to obtain a processed image corresponding to the image to be processed, namely the target image.

[0158] Exemplarily, the feature map can also be input into the spatial structure capture network of the image processing model to obtain a spatial structure map corresponding to the image to be processed. Then, when the image to be processed is color mapped based on the target color mapping table to obtain the target image, the image to be processed can be color mapped based on the target color mapping table to obtain an intermediate image to be processed, and the intermediate image to be processed and the spatial structure map corresponding to the image to be processed are fused to obtain the target image.

[0159] Exemplarily, the spatial structure graph corresponding to the image to be processed includes spatial structure subgraphs of multiple preset channels, and each color mapping subtable includes color mapping information corresponding to each preset channel. Then, when the color mapping tables are fused in the light scene dimension based on the light scene recognition result, the color mapping information of each color mapping subtable in the preset channel can be weightedly summed for each preset channel based on the light scene recognition result to obtain a target color mapping subtable for each preset channel. Furthermore, when the image to be processed is color mapped based on the target color mapping table to obtain an intermediate image to be processed, the image to be processed can be color mapped based on the target color mapping subtable of each preset channel to obtain an intermediate sub-image to be processed for each preset channel. Then, when the intermediate image to be processed and the spatial structure graph corresponding to the image to be processed are fused, the intermediate sub-graphs to be processed for each preset channel can be fused with the spatial structure subgraph of the corresponding preset channel to obtain the target sub-image of each preset channel. Then, the target sub-images of each preset channel can be spliced ​​to obtain the processed image corresponding to the image to be processed, i.e., the target image.

[0160] The above-mentioned embodiment inputs the image to be processed and the reference image into a trained image processing model. The image processing performed by the image processing model can only apply the skin color of the reference image to the skin area of ​​the image to be processed, and the background color of the image remains unchanged, while retaining the three-dimensionality and layering of the target object, and can adapt to various lighting scenes, thereby improving the image processing effect; in addition, for images to be processed with different skin color degrees, the same target skin color (that is, the color of the reference skin area in the reference image) can be achieved, which increases the user's freedom in editing the skin color in the image and can achieve any desired skin color editing effect.

[0161] Figure 8 1 is a structural block diagram of an image processing model training device according to an exemplary embodiment. Figure 8 , the image processing model training device 800 includes:

[0162] The sample image acquisition unit 810 is configured to acquire a sample image and a sample reference image corresponding to the sample image, wherein the sample image includes a skin area of ​​the subject, and the sample reference image includes a reference skin area;

[0163] The first processing unit 820 is configured to input the sample image and the sample reference image into a feature extraction network and a color mapping table generation network of the image processing model to be trained, respectively, to obtain a sample feature map output by the feature extraction network and a generated color mapping table output by the color mapping table generation network; the generated color mapping table includes color mapping sub-tables corresponding to multiple preset light scenes;

[0164] The light scene recognition unit 830 is configured to input the sample feature map into the light scene recognition network of the image processing model to be trained to obtain a light scene recognition result; the light scene recognition result represents the probability of belonging to each of the preset light scenes;

[0165] A first fusion unit 840 is configured to perform fusion of the generated color mapping table in a light scene dimension based on the light scene recognition result to obtain a target color mapping table;

[0166] The color mapping unit 850 is configured to perform color mapping processing on the sample image based on the target color mapping table to obtain a target processed image;

[0167] The training unit 860 is configured to determine a target loss based on the target processing image, the sample image and the sample reference image, and to adjust the model parameters of the image processing model to be trained with minimizing the target loss as the training goal until a preset training end condition is met, thereby obtaining a trained image processing model; wherein the target loss characterizes the background difference between the target processing image and the sample image, as well as the perceptual difference and skin color difference between the target processing image and the sample reference image.

[0168] In some exemplary embodiments, the apparatus 800 further includes:

[0169] a spatial structure capture unit configured to input the sample feature map into the spatial structure capture network of the image processing model to be trained, and obtain a spatial structure map output by the spatial structure capture network; the spatial structure map represents the spatial structure features of the sample image;

[0170] The color mapping unit 850 includes:

[0171] an intermediate color mapping unit, configured to perform color mapping processing on the sample image based on the target color mapping table to obtain an intermediate processed image;

[0172] The second fusion unit is configured to perform fusion of the intermediate processed image and the spatial structure map to obtain a target processed image; wherein the target loss also represents the difference in object contour between the target processed image and the sample image.

[0173] In some exemplary embodiments, the spatial structure graph includes a plurality of spatial structure subgraphs of preset channels, and each of the color mapping subtables includes color mapping information corresponding to each of the preset channels;

[0174] The first fusion unit 840 is specifically configured to perform, for each preset channel, a weighted summation of the color mapping information of each color mapping sub-table in the preset channel based on the light scene recognition result, to obtain a target color mapping sub-table for each preset channel; wherein the target color mapping table includes the target color mapping sub-table of each preset channel;

[0175] The intermediate color mapping unit is specifically configured to perform color mapping processing on the sample image based on the target color mapping sub-table of each preset channel to obtain an intermediate processed sub-image of each preset channel; the intermediate processed image includes the intermediate processed sub-image of each preset channel.

[0176] In some exemplary embodiments, the second fusion unit is specifically configured to perform fusion of the intermediate processing sub-image of each of the preset channels with the spatial structure sub-image of the corresponding preset channel to obtain the target processing sub-image of each of the preset channels; and splice the target processing sub-images of each of the preset channels to obtain the target processing image.

[0177] In some exemplary embodiments, the training unit 860 includes:

[0178] A background consistency loss determining unit is configured to determine a background consistency loss based on a difference between a background area in the target processing image and a background area in the sample image; the background area refers to an image area excluding the object;

[0179] a structure consistency loss determining unit configured to determine a structure consistency loss based on a difference between contour information of the object in the target processing image and contour information of the object in the sample image;

[0180] a perceptual loss determining unit configured to determine a first feature map corresponding to a skin region of the object in the target processed image and a second feature map corresponding to the reference skin region in the sample reference image; and determine a perceptual loss based on a distance between the first feature map and the second feature map;

[0181] a color migration loss determining unit configured to migrate the skin color of the reference skin area in the sample reference image to the skin area of ​​the subject in the sample image to obtain a migrated sample image; and determine the color migration loss based on the pixel difference between the target processed image and the migrated sample image;

[0182] The target loss determination unit is configured to perform linear fusion of the background consistency loss, the structure consistency loss, the perception loss and the color migration loss to obtain the target loss.

[0183] In some exemplary embodiments, the color migration loss determining unit includes:

[0184] a color histogram determining unit configured to determine a first color histogram of the reference skin area in the sample reference image and a second color histogram of the skin area of ​​the subject in the sample image;

[0185] The color migration unit is configured to perform color migration on the pixel value of each pixel in the skin area of ​​the object in the sample image based on the first color histogram and the second color histogram using a histogram matching algorithm to obtain a migrated sample image.

[0186] Figure 9 FIG. 1 is a structural block diagram of an image processing apparatus according to an exemplary embodiment. Figure 9 , the image processing apparatus 900 includes:

[0187] The image acquisition unit 910 is configured to acquire an image to be processed and a reference image; the image to be processed includes a skin area of ​​a target object, and the reference image includes a reference skin area;

[0188] The second processing unit 920 is configured to input the image to be processed and the reference image into an image processing model for image processing, respectively, to obtain a processed target image; wherein the color of the skin area of ​​the target object in the target image is consistent with the color of the reference skin area in the reference image;

[0189] The image processing model is trained based on any one of the image processing model training methods of the embodiments of the present disclosure.

[0190] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0191] In an exemplary embodiment, an electronic device is also provided, including a processor; a memory for storing instructions executable by the processor; wherein, when the processor is configured to execute the instructions stored on the memory, it implements the image processing model training method or image processing method provided in any embodiment of the present disclosure.

[0192] The electronic device may be a terminal, a server or a similar computing device. For example, the electronic device is a server. Figure 10 is a block diagram of an electronic device for running an image processing model training method or an image processing method according to an exemplary embodiment. Figure 10 As shown, the server 1000 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 1010 (the processor 1010 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1030 for storing data, and one or more storage media 1020 (such as one or more mass storage devices) for storing application programs 1023 or data 1022. Among them, the memory 1030 and the storage medium 1020 can be temporary storage or permanent storage. The program stored in the storage medium 1020 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1010 can be configured to communicate with the storage medium 720 to execute a series of instruction operations in the storage medium 1020 on the server 1000. The server 1000 may also include one or more power supplies 1060, one or more wired or wireless network interfaces 750, one or more input and output interfaces 1040, and / or one or more operating systems 1021, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0193] The input / output interface 1040 can be used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by the communication provider of the server 1000. In one embodiment, the input / output interface 740 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one embodiment, the input / output interface 1040 can be a radio frequency (RF) module for wirelessly communicating with the Internet.

[0194] It can be understood by those skilled in the art that Figure 10 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 10 More or fewer components than shown, or with Figure 10 Different configurations shown.

[0195] In an exemplary embodiment, a computer-readable storage medium is further provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the speech processing method provided in any of the above embodiments.

[0196] In an exemplary embodiment, a computer program product is further provided, including a computer program, wherein when the computer program is executed by a processor, the speech processing method provided in any of the above embodiments is implemented.

[0197] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0198] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for training an image processing model, characterized in that: include: Acquire a sample image and a sample reference image corresponding to the sample image, wherein the sample image includes a skin area of ​​the subject, and the sample reference image includes a reference skin area; Inputting the sample image and the sample reference image into a feature extraction network and a color mapping table generation network of the image processing model to be trained, respectively, to obtain a sample feature map output by the feature extraction network and a generated color mapping table output by the color mapping table generation network; the generated color mapping table includes color mapping sub-tables corresponding to a plurality of preset light scenes; Inputting the sample feature map into the light scene recognition network of the image processing model to be trained to obtain a light scene recognition result; The light scene recognition result represents the probability of belonging to each of the preset light scenes; Inputting the sample feature map into the spatial structure capture network of the image processing model to be trained to obtain a spatial structure map output by the spatial structure capture network; The spatial structure graph represents the spatial structure features of the sample image; Based on the light scene recognition result, the generated color mapping table is fused in the light scene dimension to obtain a target color mapping table; Performing color mapping processing on the sample image based on the target color mapping table to obtain an intermediate processed image; fusing the intermediate processed image with the spatial structure map to obtain a target processed image; A target loss is determined based on the target processing image, the sample image, and the sample reference image. Minimizing the target loss is used as a training goal, and the model parameters of the image processing model to be trained are adjusted until a preset training end condition is met, thereby obtaining a trained image processing model; wherein the target loss characterizes the background difference and object contour difference between the target processing image and the sample image, as well as the perceptual difference and skin color difference between the target processing image and the sample reference image.

2. The method according to claim 1, characterized in that The spatial structure graph includes a plurality of spatial structure subgraphs of preset channels, and each of the color mapping subtables includes color mapping information corresponding to each of the preset channels; The step of fusing the generated color mapping table in a light scene dimension based on the light scene recognition result to obtain a target color mapping table includes: For each of the preset channels, weighted summing is performed on the color mapping information of each color mapping sub-table for the preset channel based on the light scene recognition result to obtain a target color mapping sub-table for each of the preset channels; wherein the target color mapping table includes the target color mapping sub-tables for each of the preset channels; The performing color mapping processing on the sample image based on the target color mapping table to obtain an intermediate processed image includes: Color mapping processing is performed on the sample images based on the target color mapping sub-tables of the preset channels to obtain intermediate processed sub-images of the preset channels; the intermediate processed images include the intermediate processed sub-images of the preset channels.

3. The method according to claim 2, characterized in that The fusing the intermediate processed image and the spatial structure diagram to obtain a target processed image includes: Fusing the intermediate processed sub-image of each preset channel with the spatial structure sub-image of the corresponding preset channel to obtain the target processed sub-image of each preset channel; The target processed sub-images of the preset channels are spliced ​​together to obtain the target processed image.

4. The method according to claim 1, wherein The determining of the target loss based on the target processed image, the sample image and the sample reference image comprises: Determining a background consistency loss based on a difference between a background region in the target processed image and a background region in the sample image; the background region refers to an image region excluding the object; determining a structural consistency loss based on a difference between contour information of the object in the target processed image and contour information of the object in the sample image; Determining a first feature map corresponding to a skin region of the subject in the target processed image and a second feature map corresponding to the reference skin region in the sample reference image; and determining a perceptual loss based on a distance between the first feature map and the second feature map; Migrating the skin color of the reference skin area in the sample reference image to the skin area of ​​the subject in the sample image to obtain a migrated sample image; determining a color migration loss based on a pixel difference between the target processed image and the migrated sample image; The background consistency loss, the structure consistency loss, the perception loss and the color migration loss are linearly fused to obtain the target loss.

5. The method according to claim 4, characterized in that The step of migrating the skin color of the reference skin area in the sample reference image to the skin area of ​​the object in the sample image to obtain the migrated sample image comprises: determining a first color histogram of the reference skin area in the sample reference image and a second color histogram of the skin area of ​​the subject in the sample image; A histogram matching algorithm is used to perform color migration on the pixel value of each pixel in the skin area of ​​the object in the sample image based on the first color histogram and the second color histogram to obtain a migrated sample image.

6. An image processing method, characterized in that: include: Obtaining an image to be processed and a reference image; The image to be processed includes a skin area of ​​a target object, and the reference image includes a reference skin area; Inputting the image to be processed and the reference image into an image processing model for image processing, respectively, to obtain a processed target image; wherein the color of the skin area of ​​the target object in the target image is consistent with the color of the reference skin area in the reference image; The image processing model is trained based on the image processing model training method described in any one of claims 1 to 5.

7. An image processing model training device, characterized in that: include: a sample image acquisition unit configured to acquire a sample image and a sample reference image corresponding to the sample image, wherein the sample image includes a skin area of ​​the subject, and the sample reference image includes a reference skin area; a first processing unit configured to input the sample image and the sample reference image into a feature extraction network and a color mapping table generation network of a to-be-trained image processing model, respectively, to obtain a sample feature map output by the feature extraction network and a generated color mapping table output by the color mapping table generation network; the generated color mapping table includes color mapping sub-tables corresponding to a plurality of preset light scenes; a light scene recognition unit configured to input the sample feature map into the light scene recognition network of the image processing model to be trained to obtain a light scene recognition result; The light scene recognition result represents the probability of belonging to each of the preset light scenes; A spatial structure capture unit is configured to input the sample feature map into the spatial structure capture network of the image processing model to be trained, and obtain a spatial structure map output by the spatial structure capture network; The spatial structure graph represents the spatial structure features of the sample image; A first fusion unit is configured to perform fusion of the generated color mapping table in a light scene dimension based on the light scene recognition result to obtain a target color mapping table; A color mapping unit is configured to perform color mapping processing on the sample image based on the target color mapping table to obtain an intermediate processed image; and fuse the intermediate processed image with the spatial structure map to obtain a target processed image; A training unit is configured to determine a target loss based on the target processing image, the sample image and the sample reference image, and to adjust the model parameters of the image processing model to be trained with minimizing the target loss as the training goal until a preset training end condition is met, thereby obtaining a trained image processing model; wherein the target loss characterizes the background difference and object contour difference between the target processing image and the sample image, as well as the perceptual difference and skin color difference between the target processing image and the sample reference image.

8. An image processing device, characterized in that: include: An image acquisition unit, configured to acquire an image to be processed and a reference image; The image to be processed includes a skin area of ​​a target object, and the reference image includes a reference skin area; a second processing unit configured to input the to-be-processed image and the reference image into an image processing model for image processing, respectively, to obtain a processed target image; wherein a color of a skin region of the target object in the target image is consistent with a color of a reference skin region in the reference image; The image processing model is trained based on the image processing model training method described in any one of claims 1 to 5.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image processing model training method as described in any one of claims 1 to 5, or the image processing method described in claim 6.

10. A computer-readable storage medium, characterized in that When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing model training method as described in any one of claims 1 to 5, or the image processing method described in claim 6.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the image processing model training method as described in any one of claims 1 to 5, or the image processing method described in claim 6.

Images