Image processing method and device, camera equipment and storage medium
The image is segmented through the semantic segmentation model and the white balance correction coefficient is calculated, which solves the problem of poor accuracy caused by relying on preset thresholds in the prior art, and achieves a better white balance effect.
Patent Information
- Application Number
- CN202310580069.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-22
- Publication Date
- 2024-11-22
AI Technical Summary
In the image white balance processing, the pixel points of skin color are judged by preset thresholds, and are easily affected by light and skin color, resulting in poor accuracy and poor white balance effect.
The semantic segmentation model is used to segment the image object, determine the object pixel points with constant color, calculate the white balance correction coefficient, and perform white balance correction based on this coefficient to improve accuracy and effect.
Through the use of object segmentation and semantic segmentation models, the accuracy is improved, and the white balance correction effect is better, which reduces the influence of lighting and skin tone, and improves the image display effect.
Smart Images

Figure CN119012022A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, camera equipment and storage medium. Background Art
[0002] When an electronic device photographs an object under different lighting conditions, the captured image shows different colors of the object. Therefore, it is necessary to perform white balance correction on the captured image to solve the problem of color distortion of the photographed object. White balance is an indicator that describes the accuracy of white generated by mixing the three primary colors of red, green and blue in the display.
[0003] In the related art, when performing white balance processing on a portrait in an image, it is usually determined whether a pixel is a skin color pixel by using a preset threshold and the RGB (red green blue) parameter or HSV (Hue Saturation Value) parameter of the pixel, and then the parameters used for white balance processing are determined based on the determined skin color pixel. However, this method of determining based on a preset threshold is easily affected by lighting or the skin color of the portrait itself, resulting in poor accuracy of the determined skin color pixel, and thus poor white balance effect. Summary of the invention
[0004] In order to overcome the problems existing in the related art, the present disclosure provides an image processing method, device, electronic device and storage medium.
[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method.
[0006] Performing object segmentation on the first image to obtain a second image, wherein the second image includes a plurality of first pixel points and a plurality of second pixel points, and the object is an object with color constancy;
[0007] Determining a white balance correction coefficient according to pixel values corresponding to the plurality of first pixel points;
[0008] Based on the white balance correction coefficient, white balance correction is performed on the first image to obtain a third image.
[0009] In some embodiments, performing object segmentation on the first image to obtain the second image includes:
[0010] A semantic segmentation model is called to perform object segmentation on the first image to obtain the second image, wherein the semantic segmentation model is used to perform object segmentation on the image.
[0011] In some embodiments, the semantic segmentation model includes an encoder and a decoder, and calling the semantic segmentation model to perform portrait skin color segmentation on the first image to obtain the second image includes:
[0012] Calling the encoder to perform feature extraction on the first image to obtain image features;
[0013] The decoder is called to decode the first image feature to obtain the second image.
[0014] In some embodiments, the encoder includes a plurality of downsampling networks connected in sequence, and calling the encoder to extract features from the first image to obtain image features includes:
[0015] Calling the first downsampling network to downsample the first image to obtain a first downsampled feature;
[0016] For the downsampling networks other than the first downsampling network, the downsampling network is called to downsample the downsampling features output by the previous downsampling network to obtain the next downsampling feature, until the last downsampling feature output by the last downsampling network among the multiple downsampling networks is obtained, and the last downsampling feature is used as the image feature.
[0017] In some embodiments, the downsampling network includes a first convolutional layer and a pooling layer; the first convolutional layer is used to perform convolution processing on the input first image or downsampling features to obtain convolution features; the pooling layer is used to process the convolution features to obtain the downsampling features.
[0018] In some embodiments, the decoder includes a plurality of upsampling networks and a second convolutional layer connected in sequence, and calling the decoder to decode the first image feature to obtain the second image includes:
[0019] Calling the first upsampling network to upsample the image features and the convolution features outputted from the last downsampling network to obtain a first upsampling feature;
[0020] For the upsampling networks other than the first upsampling network, calling the upsampling network, upsampling the upsampling features output by the last upsampling network and the convolution features output by the corresponding downsampling network, and obtaining the next upsampling feature, until the last upsampling feature output by the last upsampling network in the multiple upsampling networks is obtained; wherein the convolution features output by the downsampling network corresponding to the upsampling network and the upsampling features output by the last upsampling network have the same number of image channels;
[0021] The second convolutional layer is called to perform convolution processing on the last up-sampled feature to obtain the second image.
[0022] In some embodiments, the upsampling network includes a deconvolution layer, a splicing layer and a third convolution layer; the deconvolution layer is used to perform deconvolution processing on the input image features or upsampling features to obtain deconvolution features; the splicing layer is used to splice the deconvolution features with the corresponding convolution features output by the downsampling network to obtain splicing features; the third convolution layer is used to perform convolution processing on the splicing features to obtain the upsampling features.
[0023] In some embodiments, the object is a human skin color, the first pixel is a skin color pixel, and the second pixel is a non-skin color pixel.
[0024] In some embodiments, the pixel value corresponding to the skin color pixel includes a first parameter of a first channel, a second parameter of a second channel, and a third parameter of a third channel, and determining the white balance correction coefficient according to the pixel values corresponding to the plurality of first pixels includes:
[0025] Determine, according to the pixel values corresponding to the plurality of skin color pixels, a first average value of a plurality of the first parameters, a second average value of a plurality of the second parameters, and a third average value of a plurality of the third parameters;
[0026] determining a first ratio between the first average and the second average, and a second ratio between the third average and the second average;
[0027] A first correction coefficient is determined based on the first ratio, and a second correction coefficient is determined based on the second ratio.
[0028] In some embodiments, performing white balance correction on the first image based on the white balance correction coefficient to obtain a third image includes:
[0029] Based on the first correction coefficient, white balance correction is performed on the first channel in the first image, and based on the second correction coefficient, white balance correction is performed on the third channel in the first image to obtain the third image.
[0030] In some embodiments, determining a first correction coefficient based on the first ratio and determining a second correction coefficient based on the second ratio includes:
[0031] Based on the category to which the portrait in the first image belongs, obtaining a first color temperature curve corresponding to the category, wherein the first color temperature curve is located in a coordinate system, the abscissa of the coordinate system is a ratio between the first parameter and the third parameter, and the ordinate is a ratio between the second parameter and the third parameter, and the first color temperature curve includes first coordinate points corresponding to different light sources;
[0032] Acquire a second color temperature curve, where the second color temperature curve is in the coordinate system and includes second coordinate points corresponding to different light sources;
[0033] Based on a first position point in the coordinate system, two first coordinate points in the first color temperature curve, and two second coordinate points in the second color temperature curve, a triangular mapping method is used to determine a second position point corresponding to the first position point, the first position point is a point formed by the first ratio and the second ratio, a distance between the two first coordinate points and the first position point is greater than a distance between other first coordinate points and the first position point, and the light sources corresponding to the two first coordinate points and the two second coordinate points are the same;
[0034] The first correction coefficient is determined based on the horizontal coordinate of the second position point, and the second correction coefficient is determined based on the vertical coordinate of the second position point.
[0035] In some embodiments, determining the first correction coefficient based on the abscissa of the second position point, and determining the second correction coefficient based on the ordinate of the second position point includes:
[0036] determining the reciprocal of the horizontal coordinate of the second position point as the first correction coefficient;
[0037] The reciprocal of the ordinate of the second position point is determined as the second correction coefficient.
[0038] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, the apparatus comprising:
[0039] a segmentation module configured to perform object segmentation on the first image to obtain a second image, wherein the second image includes a plurality of first pixels and a plurality of second pixels, and the object is an object with color constancy;
[0040] A correction coefficient determination module, configured to determine a white balance correction coefficient according to pixel values corresponding to the plurality of first pixel points;
[0041] The white balance correction module is configured to perform white balance correction on the first image based on the white balance correction coefficient to obtain a third image.
[0042] In some embodiments, the segmentation module is configured to:
[0043] A semantic segmentation model is called to perform object segmentation on the first image to obtain the second image, wherein the semantic segmentation model is used to perform object segmentation on the image.
[0044] In some embodiments, the semantic segmentation model includes an encoder and a decoder, and the segmentation module is configured to:
[0045] Calling the encoder to perform feature extraction on the first image to obtain image features;
[0046] The decoder is called to decode the first image feature to obtain the second image.
[0047] In some embodiments, the encoder includes a plurality of downsampling networks connected in sequence, and the segmentation module is configured as follows:
[0048] Calling the first downsampling network to downsample the first image to obtain a first downsampled feature;
[0049] For the downsampling networks other than the first downsampling network, the downsampling network is called to downsample the downsampling features output by the previous downsampling network to obtain the next downsampling feature, until the last downsampling feature output by the last downsampling network among the multiple downsampling networks is obtained, and the last downsampling feature is used as the image feature.
[0050] In some embodiments, the downsampling network includes a first convolutional layer and a pooling layer; the first convolutional layer is used to perform convolution processing on the input first image or downsampling features to obtain convolution features; the pooling layer is used to process the convolution features to obtain the downsampling features.
[0051] In some embodiments, the decoder includes a plurality of upsampling networks and a second convolutional layer connected in sequence, and the segmentation module is configured as follows:
[0052] Calling the first upsampling network to upsample the image features and the convolution features outputted from the last downsampling network to obtain a first upsampling feature;
[0053] For the upsampling networks other than the first upsampling network, calling the upsampling network, upsampling the upsampling features output by the last upsampling network and the convolution features output by the corresponding downsampling network, and obtaining the next upsampling feature, until the last upsampling feature output by the last upsampling network in the multiple upsampling networks is obtained; wherein the convolution features output by the downsampling network corresponding to the upsampling network and the upsampling features output by the last upsampling network have the same number of image channels;
[0054] The second convolutional layer is called to perform convolution processing on the last up-sampled feature to obtain the second image.
[0055] In some embodiments, the upsampling network includes a deconvolution layer, a splicing layer and a third convolution layer; the deconvolution layer is used to perform deconvolution processing on the input image features or upsampling features to obtain deconvolution features; the splicing layer is used to splice the deconvolution features with the corresponding convolution features output by the downsampling network to obtain splicing features; the third convolution layer is used to perform convolution processing on the splicing features to obtain the upsampling features.
[0056] In some embodiments, the object is a human skin color, the first pixel is a skin color pixel, and the second pixel is a non-skin color pixel.
[0057] In some embodiments, the pixel value corresponding to the skin color pixel includes a first parameter of a first channel, a second parameter of a second channel, and a third parameter of a third channel, and the correction coefficient determination module is configured as follows:
[0058] Determine, according to the pixel values corresponding to the plurality of skin color pixels, a first average value of a plurality of the first parameters, a second average value of a plurality of the second parameters, and a third average value of a plurality of the third parameters;
[0059] determining a first ratio between the first average and the second average, and a second ratio between the third average and the second average;
[0060] A first correction coefficient is determined based on the first ratio, and a second correction coefficient is determined based on the second ratio.
[0061] In some embodiments, the white balance correction module is configured to:
[0062] Based on the first correction coefficient, white balance correction is performed on the first channel in the first image, and based on the second correction coefficient, white balance correction is performed on the third channel in the first image to obtain the third image.
[0063] In some embodiments, the correction coefficient determination module is configured to:
[0064] Based on the category to which the portrait in the first image belongs, obtaining a first color temperature curve corresponding to the category, wherein the first color temperature curve is located in a coordinate system, the abscissa of the coordinate system is a ratio between the first parameter and the third parameter, and the ordinate is a ratio between the second parameter and the third parameter, and the first color temperature curve includes first coordinate points corresponding to different light sources;
[0065] Acquire a second color temperature curve, where the second color temperature curve is in the coordinate system and includes second coordinate points corresponding to different light sources;
[0066] Based on a first position point in the coordinate system, two first coordinate points in the first color temperature curve, and two second coordinate points in the second color temperature curve, a triangular mapping method is used to determine a second position point corresponding to the first position point, the first position point is a point formed by the first ratio and the second ratio, a distance between the two first coordinate points and the first position point is greater than a distance between other first coordinate points and the first position point, and the light sources corresponding to the two first coordinate points and the two second coordinate points are the same;
[0067] The first correction coefficient is determined based on the horizontal coordinate of the second position point, and the second correction coefficient is determined based on the vertical coordinate of the second position point.
[0068] In some embodiments, the correction coefficient determination module is configured to:
[0069] determining the reciprocal of the horizontal coordinate of the second position point as the first correction coefficient;
[0070] The reciprocal of the ordinate of the second position point is determined as the second correction coefficient.
[0071] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:
[0072] processor;
[0073] a memory for storing processor-executable instructions;
[0074] Wherein, the processor is configured to execute the image processing method as described in the first aspect of the embodiment of the present disclosure.
[0075] According to the fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can perform the image processing method as described in the first aspect of the embodiment of the present disclosure.
[0076] The above method of the present disclosure has the following beneficial effects:
[0077] The method provided by the embodiment of the present disclosure performs object segmentation on a first image to obtain a second image, wherein the second image includes multiple first pixel points and multiple second pixel points, wherein the object is an object with color constancy. Compared with the method of judging whether a pixel point is a first pixel point based on a preset threshold in the related art, the first pixel point determined by object segmentation is more accurate, and thus, the white balance correction coefficient determined according to the pixel value corresponding to the first pixel point is more accurate, and then white balance correction is performed on the first image based on the white balance correction coefficient to obtain a third image, and the white balance effect of the third image is better.
[0078] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0079] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0080] Figure 1 is a flow chart of an image processing method according to an exemplary embodiment;
[0081] Figure 2 is a flowchart of an image processing method according to another exemplary embodiment;
[0082] Figure 3 is a schematic diagram of a semantic segmentation model according to an exemplary embodiment;
[0083] Figure 4 is a schematic diagram of an image according to an exemplary embodiment;
[0084] Figure 5 is a schematic diagram of an image according to another exemplary embodiment;
[0085] Figure 6 is a schematic diagram of a color temperature curve according to an exemplary embodiment;
[0086] Figure 7 is a schematic diagram showing a position point and a color temperature curve according to an exemplary embodiment;
[0087] Figure 8 is a schematic diagram showing a location point mapping according to an exemplary embodiment;
[0088] Fig. 9is a device block diagram of an image processing device according to an exemplary embodiment;
[0089] Fig.10 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION
[0090] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.
[0091] The method provided in the embodiment of the present disclosure is executed by an electronic device, which is a device with image processing function, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, a smart home device, a wearable device, etc.
[0092] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment, which is executed by an electronic device, see Figure 1 , the method comprises the following steps:
[0093] Step S101, performing object segmentation on the first image to obtain a second image, where the second image includes a plurality of first pixel points and a plurality of second pixel points, and the object is an object with color constancy.
[0094] In the disclosed embodiment, in order to reduce the influence of the illumination environment on the objects with color constancy in the first image when shooting the first image, white balance correction is performed on the first image to improve the display effect of the objects with color constancy in the first image. Therefore, it is necessary to determine the white balance correction coefficient according to the first pixel in the first image. Then, it is first necessary to determine which pixels in the first image are the first pixels and which pixels are the second pixels. Among them, the objects with color constancy are objects with the perceptual characteristics that when the color light irradiating the surface of the object changes, people's perception of the color of the surface of the object remains unchanged, such as the skin color of a portrait, the color of an object, etc.
[0095] The first image is an image to be processed, and the first image includes an object with color constancy. Color constancy means that when the color of the light irradiating the surface of a person or an object changes, people's perception of the color of the surface of the person or the object remains unchanged. Object segmentation of the first image refers to: detecting a first pixel and a second pixel in the first image. The first pixel refers to a pixel in an area with color constancy, and the second pixel refers to other pixels in the first image except the first pixel. For example, when white balance correction is required for the skin color of a portrait in the first image, the first pixel is a skin color pixel in the portrait, and the second pixel is a non-skin color pixel in the portrait.
[0096] Step S102: determining a white balance correction coefficient according to pixel values corresponding to a plurality of first pixel points.
[0097] In order to improve the display effect of the object in the first image, a white balance correction coefficient is determined according to the pixel values corresponding to the plurality of first pixel points, and the white balance correction coefficient is used to correct the white balance of the first image. The pixel value corresponding to the first pixel point refers to the pixel value of the pixel point located at the same position as the first pixel point in the first image.
[0098] Step S103: performing white balance correction on the first image based on the white balance correction coefficient to obtain a third image.
[0099] After obtaining the white balance correction coefficient, the first image is directly subjected to white balance correction based on the white balance correction coefficient to obtain a third image. The third image is the image of the first image after white balance correction. The third image has a better display effect than the first image.
[0100] The method provided by the embodiment of the present disclosure performs object segmentation on the first image to obtain a second image, wherein the second image includes multiple first pixel points and multiple second pixel points. Compared with the method of judging whether a pixel point is a first pixel point based on a preset threshold in the related art, the first pixel point determined by object segmentation is more accurate. Therefore, the white balance correction coefficient determined according to the pixel value corresponding to the first pixel point is more accurate, and then the white balance correction is performed on the first image based on the white balance correction coefficient to obtain a third image, and the white balance effect of the third image is better.
[0101] Below through Figure 2 The illustrated embodiment describes in detail the case where the object is the skin color of a portrait.
[0102] Figure 2 is a flowchart of another image processing method according to an exemplary embodiment, which is executed by an electronic device, see Figure 2 , the method comprises the following steps:
[0103] Step S201, calling a semantic segmentation model to perform skin color segmentation on a first image to obtain a second image, wherein the second image includes a plurality of skin color pixels and a plurality of non-skin color pixels.
[0104] Among them, the semantic segmentation model is used to perform portrait skin color segmentation on the image, and the semantic segmentation model is a trained model.
[0105] In some embodiments, the semantic segmentation model includes an encoder and a decoder, the encoder and the decoder are connected, the encoder is used to encode a high-resolution first image to obtain image features, the image features represent a low-resolution image, and the decoder is used to decode the image features to obtain a high-resolution second image. Calling the semantic segmentation model to perform portrait skin color segmentation on the first image to obtain a second image includes: calling the encoder to extract features from the first image to obtain image features; calling the decoder to decode the first image features to obtain the second image. Since the semantic segmentation model is a trained model for performing portrait skin color segmentation on images, the second image obtained by calling the encoder and the decoder is an image that has completed portrait skin color segmentation.
[0106] In some embodiments, the encoder includes a plurality of downsampling networks connected in sequence. Calling the encoder to extract features from the first image to obtain image features includes: calling the first downsampling network to downsample the first image to obtain the first downsampled feature; for downsampling networks other than the first downsampling network, calling the downsampling network to downsample the downsampled feature output by the previous downsampling network to obtain the next downsampled feature, until the last downsampled feature output by the last downsampling network in the plurality of downsampling networks is obtained, and the last downsampled feature is used as the image feature. Each downsampled feature is used to describe the first image, but the difference is that different downsampled features represent different image resolutions, and the number of channels of the first image gradually increases during the downsampling process, and the length and width of the first image gradually decrease. For example, if the encoder includes three downsampling networks, then the first downsampled network is called to downsample the first image to obtain the first downsampled feature, and then the second downsampled network is called to downsample the first downsampled feature to obtain the second downsampled feature, and finally the third downsampled network is called to downsample the second downsampled feature to obtain the image feature.
[0107] In some embodiments, the downsampling network includes a first convolution layer and a pooling layer, and the first convolution layer and the pooling layer are connected; the first convolution layer is used to perform convolution processing on the input first image or downsampled features to obtain convolution features; the pooling layer is used to process the convolution features to obtain downsampled features. Optionally, the downsampling network includes two first convolution layers, one first convolution layer is a BN layer (normalization layer), and the other first convolution layer is a ReLU layer (activation function layer), for example, the two first convolution layers are 3*3 convolution layers; the pooling layer can be a maximum pooling layer (Maximum Pooling) or an average pooling layer (Average Pooling).
[0108] For example, see Figure 3 The encoder includes two downsampling networks. The first input image is a 256*192*3 RGB image. After two 3*3 convolutional layers (the first convolutional layer), the number of channels changes from 3 to 64. Then, after the pooling layer, the length and width of the first image are downsampled by half, that is, the length becomes 128 and the width becomes 96; then after two 3*3 convolutional layers, the number of channels becomes 128, and then after the pooling layer, the image that has been downsampled once is downsampled again by half, that is, the length becomes 64 and the width becomes 48.
[0109] In some embodiments, there is also a fourth convolution layer between the encoder and the decoder, and the fourth convolution layer is used to perform convolution processing on the image feature roots to obtain the image features after convolution processing. For example, see Figure 3 , the fourth convolutional layer is a 3*3 convolutional layer. After passing the fourth convolutional layer, the number of channels becomes 256.
[0110] In some embodiments, the decoder includes a plurality of upsampling networks and a second convolutional layer connected in sequence, and the plurality of upsampling networks are connected in sequence, and the last upsampling network is connected to the second convolutional layer. Calling the decoder to decode the first image feature to obtain the second image includes: calling the first upsampling network to upsample the image feature and the convolutional feature output from the last downsampling network to obtain the first upsampling feature; for upsampling networks other than the first upsampling network, calling the upsampling network to upsample the upsampling feature output from the last upsampling network and the convolutional feature output from the corresponding downsampling network to obtain the next upsampling feature, until the last upsampling feature output from the last upsampling network among the plurality of upsampling networks is obtained; calling the second convolutional layer to convolute the last upsampling feature to obtain the second image. The convolutional feature output from the downsampling network corresponding to the upsampling network has the same number of image channels as the upsampling feature output from the last upsampling network. In the upsampling process, the number of channels gradually decreases, and the length and width gradually increase until a second image of 256*192*2 is obtained. The second image includes two channels, one channel for skin color pixels and the other channel for non-skin color pixels. That is, by encoding and then decoding the first image, the skin color segmentation of the portrait in the first image is achieved to obtain the second image. For example, see Figure 4 A schematic diagram of the image shown, Figure 4 The image on the left is the first image, and the image on the right is the second image. Figure 4 The first image and the second image are grayscale images. In fact, the input first image is an RGB image, and the output second image is a two-channel image. The skin color area in the second image is represented by one color, and the skin color area is marked in red. Other areas except the skin color area can be represented as grayscale areas, that is, different colors are used to represent the skin color area and the non-skin color area in the second image.
[0111] In some embodiments, the upsampling network includes a deconvolution layer, a splicing layer, and a third convolution layer; the deconvolution layer is used to perform deconvolution processing on the input image features or upsampling features to obtain deconvolution features; the splicing layer is used to splice the deconvolution features with the convolution features output by the corresponding downsampling network to obtain splicing features; the third convolution layer is used to perform convolution processing on the splicing features to obtain upsampling features. Optionally, the upsampling network includes two third convolution layers, one third convolution layer is a BN layer, and the other third convolution layer is a ReLU layer, for example, the two third convolution layers are 3*3 convolution layers.
[0112] For example, see Figure 3The decoder includes two upsampling networks. The first image with a channel number of 256 is input as a 256*192*3 RGB image. After two 3*3 convolutional layers (the first convolutional layer), the number of channels changes from 3 to 64. Then, after the pooling layer, the length and width of the first image are downsampled by half, that is, the length becomes 128 and the width becomes 86; then after two 3*3 convolutional layers, the number of channels changes to 128, and then after the pooling layer, the image that has been downsampled once is downsampled again by half, that is, the length becomes 64 and the width becomes 43.
[0113] In some embodiments, in order to reduce the amount of calculation, the resolution of the first image is set to a preset resolution, for example, the preset resolution is 64*48. When the resolution of the second image is greater than the preset resolution, the second image is downsampled so that the resolution of the second image is equal to the preset resolution. When the resolution of the second image is less than or equal to the preset resolution, the second image does not need to be downsampled. For example, see Figure 5 , the preset resolution is 64*48, and the resolution of the second image is 256*192 (the image on the left), then the second image is downsampled to obtain a second image with a resolution of 64*48 (the image on the right), and the black area in the image on the right is the non-skin color area. Optionally, the resolution of the first image is set to the preset resolution through the ISP (Image Signal Processor) module. It should be noted that even if the second image is downsampled, the skin color area in the downsampled second image remains unchanged.
[0114] It should be noted that step S201 only introduces the process of calling the semantic segmentation model, performing portrait skin color segmentation on the first image, and obtaining the second image, and does not limit the training process of the semantic segmentation model. In some embodiments, multiple first sample images and second sample images corresponding to each first sample image are obtained, the first sample image is an image to be subjected to portrait skin color segmentation, and the second sample image corresponding to the first sample image is an image after portrait skin color segmentation is performed, the semantic segmentation model to be trained is called to process the first sample image to obtain a predicted image corresponding to the first sample image, and the model parameters of the semantic segmentation model are adjusted according to the difference between the predicted image corresponding to the first sample image and the corresponding second sample image, until a trained semantic segmentation model is obtained.
[0115] It should be noted that, in the above implementation, only the semantic segmentation model is used to perform portrait skin color segmentation on an image as an example. In another embodiment, when the object is an object other than portrait skin color, other semantic segmentation models with the same structure as the above semantic segmentation model can be called to perform object segmentation. The difference from the semantic segmentation model in the above implementation is that the sample images used in the training process of other semantic segmentation models are different. For example, if the object is the color of a certain object, it is necessary to use a third sample image containing the object and a fourth sample image corresponding to the third sample image to train the other semantic segmentation model. The fourth sample image corresponding to the third sample image is the image after the object color segmentation is performed.
[0116] Step S202, determining a first average value of multiple first parameters, a second average value of multiple second parameters, and a third average value of multiple third parameters based on the pixel values corresponding to the multiple skin color pixels; determining a first ratio between the first average value and the second average value, and a second ratio between the third average value and the second average value.
[0117] The pixel value corresponding to the skin color pixel refers to the pixel value of the pixel located at the same position as the skin color pixel in the first image. The pixel value corresponding to the skin color pixel includes a first parameter of a first channel, a second parameter of a second channel, and a third parameter of a third channel, and the first channel, the second channel, and the third channel are different channels, for example, the first channel is an R channel, the second channel is a G channel, and the third channel is a B channel.
[0118] In the disclosed embodiment, when determining the white balance correction coefficient, a unified white balance correction coefficient is determined for multiple pixel points in the first image, and the white balance correction of the portrait is mainly considered. Therefore, it is determined based on the pixel values corresponding to multiple skin color pixel points.
[0119] In some embodiments, the second image includes multiple skin color pixels. Since the second image is obtained by performing portrait skin color segmentation on the first image, each pixel in the second image corresponds one-to-one to each pixel in the first image. After determining the multiple skin color pixels in the second image, it is possible to determine which pixels in the first image are skin color pixels based on the correspondence between the pixels in the second image and the pixels in the first image, thereby determining the white balance correction coefficient based on the pixel values of the skin color pixels in the first image.
[0120] It should be noted that the embodiment of the present disclosure is only described by first determining the first average value, the second average value, and the third average value. In another embodiment, for each skin color pixel point, a third ratio between the first parameter and the second parameter of the skin color pixel point, and a fourth ratio between the third parameter and the second parameter of the skin color pixel point can be obtained, and then for multiple skin color pixels, the average value of multiple third ratios corresponding to the multiple skin color pixels is determined as the first ratio, and the average value of multiple fourth ratios corresponding to the multiple skin color pixels is determined as the second ratio.
[0121] Step S203: determining a first correction coefficient based on the first ratio, and determining a second correction coefficient based on the second ratio.
[0122] In the embodiment of the present disclosure, for different categories, a color temperature curve corresponding to each category is set. The categories are divided according to colors. For example, see Figure 6 , the horizontal axis of the coordinate system is the ratio between the first parameter and the third parameter, and the vertical axis is the ratio between the second parameter and the third parameter, Figure 6 The first parameter is the pixel value of the R channel, the second parameter is the pixel value of the G channel, and the third parameter is the pixel value of the B channel. Curve 1, curve 2, and curve 3 are color temperature curves corresponding to different categories, respectively. Curve 0 is the color temperature curve corresponding to the preset gray card, such as the preset gray card can be an 18% gray card. Figure 6 In the color temperature curve corresponding to each category, the R / G and B / G position points of the object belonging to the category under the light source (D75, D65, D50, TL84A, H) are calibrated. Similarly, the color temperature curve corresponding to the preset gray card also calibrates the R / G and B / G position points of the object belonging to the preset gray card under the light source (D75, D65, D50, TL84A, H). In the embodiment of the present disclosure, there is no limitation on the method of obtaining the color temperature curve.
[0123] In some embodiments, based on the category to which the portrait in the first image belongs, a first color temperature curve corresponding to the category is obtained, the first color temperature curve is located in a coordinate system, and the first color temperature curve includes first coordinate points corresponding to different light sources; a second color temperature curve is obtained, the second color temperature curve is located in a coordinate system, and the second color temperature curve includes second coordinate points corresponding to different light sources; based on the first position point in the coordinate system, two first coordinate points in the first color temperature curve, and two second coordinate points in the second color temperature curve, a triangular mapping method is used to determine the second position point corresponding to the first position point. Among them, the first position point is a point composed of a first ratio and a second ratio, the distance between the two first coordinate points and the first position point is greater than the distance between other first coordinate points and the first position point, and the light sources corresponding to the two first coordinate points and the two second coordinate points are the same; based on the horizontal coordinate of the second position point, a first correction coefficient is determined, and based on the vertical coordinate of the second position point, a second correction coefficient is determined.
[0124] For example, see Figure 7 , the first color temperature curve corresponding to the category to which the portrait in the first image belongs is curve 1, the second color temperature curve is curve 0, and the first position point is B (R / G, B / G). Then, see Figure 8 , using the triangular mapping method, determine the second position point corresponding to the first position point, the second position point is F (R / G, B / G). Among them, the triangle formed by the first position point B and the coordinate point corresponding to D50 in curve 1 and the coordinate point corresponding to TL84, and the triangle formed by the second position point F and the coordinate point corresponding to D50 in curve 0 and the coordinate point corresponding to TL84 are similar triangles.
[0125] In some embodiments, the second channel is used as a reference, and the parameters of the first channel and the third channel are adjusted based on the second channel. The reciprocal of the horizontal coordinate of the second position point is determined as the first correction coefficient, and the first correction coefficient is used to correct the first channel in the first image; the reciprocal of the vertical coordinate of the second position point is determined as the second correction coefficient, and the second correction coefficient is used to correct the third channel in the first image. The white balance correction coefficient includes the first correction coefficient and the second correction coefficient.
[0126] In addition, it should be noted that when determining the category to which the portrait in the first image belongs, if the first image includes one portrait, the category to which the portrait belongs is directly determined; if the first image includes multiple portraits, the target portrait that occupies the largest area among the multiple portraits is determined, and the category to which the target portrait belongs is determined as the category to which the portrait in the first image belongs.
[0127] Step S204: performing white balance correction on the first channel in the first image based on the first correction coefficient, and performing white balance correction on the third channel in the first image based on the second correction coefficient, to obtain a third image.
[0128] Based on the first correction coefficient, white balance correction is performed on the first channel in the first image, including: multiplying the parameter of the first channel of each pixel in the first image by the first correction coefficient; based on the second correction coefficient, white balance correction is performed on the third channel in the first image, including: multiplying the parameter of the third channel of each pixel in the first image by the second correction coefficient.
[0129] The method provided by the embodiment of the present disclosure performs portrait skin color segmentation on a first image to obtain a second image, wherein the second image includes multiple skin color pixels and multiple non-skin color pixels. Compared with the method of judging whether a pixel is a skin color pixel based on a preset threshold in the related art, the skin color pixels determined by portrait skin color segmentation are more accurate. Thus, the white balance correction coefficient determined according to the pixel value corresponding to the skin color pixel is more accurate. Then, white balance correction is performed on the first image based on the white balance correction coefficient to obtain a third image, and the white balance effect of the third image is better.
[0130] Furthermore, when determining the white balance correction coefficient, the corresponding color temperature curve is determined according to the category to which the portrait in the first image belongs, and different categories use different color temperature curves to determine the corresponding white balance correction coefficient. That is, when determining the white balance correction coefficient, the color of the portrait in the first image is taken into consideration, so that the determined white balance correction coefficient can better correct the portrait in the first image, thereby improving the white balance effect.
[0131] Moreover, in the disclosed embodiment, the portrait skin color segmentation method is adopted to directly remove the influence of the mask color, the eyes, eyebrows and other parts of the face, thereby reducing the probability of misjudgment of skin color and improving the accuracy of the determined skin color pixels.
[0132] In another embodiment, the object is the color of a certain object, such as the color of clothes, the color of furniture, etc. Taking the color of clothes as an example, if the clothes are photographed under some special lighting, the color of the clothes in the first image may be greatly different from the real color. In order to restore the real color of the clothes as much as possible, for this situation, the semantic segmentation model can be called to perform clothing color segmentation on the first image to obtain a second image, which includes multiple clothing pixels and multiple non-clothing pixels. The semantic segmentation model is similar to the above Figure 2 The semantic segmentation model structure in the embodiment shown is similar, except that the semantic segmentation model is used to perform clothing color segmentation on the image, and then determine the white balance correction coefficient according to the pixel values corresponding to the plurality of clothing pixels in the second image, and perform white balance correction on the first image based on the white balance correction coefficient to obtain the third image. There is no limitation on the implementation method of determining the white balance correction coefficient after determining the plurality of first pixels, for example, it can be the same as the above Figure 2 The embodiment shown is similar, except that the first correction coefficient and the second correction coefficient do not need to be determined according to the color temperature curve, but the reciprocal of the first ratio is directly used as the first correction coefficient, and the reciprocal of the second ratio is directly used as the second correction coefficient.
[0133] Of course, in other embodiments, for other objects with color constancy, white balance correction can be performed in a manner similar to the above embodiment to obtain an image with better display effect, which will not be described in detail in the embodiments of the present disclosure.
[0134] Fig. 9 is a device block diagram of an image processing device according to an exemplary embodiment. Fig. 9 , the device comprises:
[0135] The segmentation module 901 is configured to perform object segmentation on the first image to obtain a second image, where the second image includes a plurality of first pixels and a plurality of second pixels, and the object is an object with color constancy;
[0136] The correction coefficient determination module 902 is configured to determine a white balance correction coefficient according to pixel values corresponding to a plurality of first pixel points;
[0137] The white balance correction module 903 is configured to perform white balance correction on the first image based on the white balance correction coefficient to obtain a third image.
[0138] The device provided by the embodiment of the present disclosure performs object segmentation on a first image to obtain a second image, wherein the second image includes multiple first pixel points and multiple second pixel points, wherein the object is an object with color constancy. Compared with the method of judging whether a pixel point is a first pixel point based on a preset threshold in the related art, the first pixel point determined by object segmentation is more accurate, and thus, the white balance correction coefficient determined according to the pixel value corresponding to the skin color pixel point is more accurate, and then the white balance correction is performed on the first image based on the white balance correction coefficient to obtain a third image, and the white balance effect of the third image is better.
[0139] In some embodiments, the segmentation module 901 is configured to:
[0140] The semantic segmentation model is called to perform object segmentation on the first image to obtain a second image, and the semantic segmentation model is used to perform object segmentation on the image.
[0141] In some embodiments, the semantic segmentation model includes an encoder and a decoder, and the segmentation module 901 is configured to:
[0142] Calling the encoder to perform feature extraction on the first image to obtain image features;
[0143] The decoder is called to decode the features of the first image to obtain the second image.
[0144] In some embodiments, the encoder includes a plurality of downsampling networks connected in sequence, and the segmentation module 901 is configured to:
[0145] Call the first downsampling network to downsample the first image to obtain the first downsampled feature;
[0146] For the downsampling networks other than the first downsampling network, the downsampling network is called to downsample the downsampling features output by the previous downsampling network to obtain the next downsampling feature, until the last downsampling feature output by the last downsampling network in the multiple downsampling networks is obtained, and the last downsampling feature is used as the image feature.
[0147] In some embodiments, the downsampling network includes a first convolutional layer and a pooling layer; the first convolutional layer is used to perform convolution processing on the input first image or downsampled features to obtain convolutional features; the pooling layer is used to downsample the convolutional features to obtain downsampled features.
[0148] In some embodiments, the decoder includes a plurality of upsampling networks and a second convolutional layer connected in sequence, and the segmentation module 901 is configured to:
[0149] Call the first upsampling network to upsample the image features and the convolution features output from the last downsampling network to obtain the first upsampling features;
[0150] For upsampling networks other than the first upsampling network, the upsampling network is called to upsample the upsampling features output by the previous upsampling network and the convolution features output by the corresponding downsampling network to obtain the next upsampling feature, until the last upsampling feature output by the last upsampling network in the multiple upsampling networks is obtained; wherein the convolution features output by the downsampling network corresponding to the upsampling network and the upsampling features output by the previous upsampling network have the same number of image channels;
[0151] The second convolutional layer is called to perform convolution on the last upsampled feature to obtain the second image.
[0152] In some embodiments, the upsampling network includes a deconvolution layer, a splicing layer and a third convolution layer; the deconvolution layer is used to perform deconvolution processing on the input image features or upsampling features to obtain deconvolution features; the splicing layer is used to splice the deconvolution features with the convolution features output by the corresponding downsampling network to obtain splicing features; the third convolution layer is used to perform convolution processing on the splicing features to obtain upsampling features.
[0153] In some embodiments, the object is a skin color of a portrait, the first pixel is a skin color pixel, and the second pixel is a non-skin color pixel.
[0154] In some embodiments, the pixel value corresponding to the skin color pixel includes a first parameter of a first channel, a second parameter of a second channel, and a third parameter of a third channel, and the correction coefficient determination module 902 is configured as follows:
[0155] Determining, according to the pixel values corresponding to the plurality of skin color pixels, a first average value of the plurality of first parameters, a second average value of the plurality of second parameters, and a third average value of the plurality of third parameters;
[0156] determining a first ratio between the first average and the second average, and a second ratio between the third average and the second average;
[0157] Based on the first ratio, a first correction coefficient is determined, and based on the second ratio, a second correction coefficient is determined.
[0158] In some embodiments, the white balance correction module 903 is configured to:
[0159] Based on the first correction coefficient, white balance correction is performed on the first channel in the first image, and based on the second correction coefficient, white balance correction is performed on the third channel in the first image to obtain a third image.
[0160] In some embodiments, the correction coefficient determination module 902 is configured to:
[0161] Based on the category to which the portrait in the first image belongs, a first color temperature curve corresponding to the category is obtained, the first color temperature curve is located in a coordinate system, the abscissa of the coordinate system is a ratio between the first parameter and the third parameter, the ordinate is a ratio between the second parameter and the third parameter, and the first color temperature curve includes first coordinate points corresponding to different light sources;
[0162] Acquire a second color temperature curve, where the second color temperature curve is located in a coordinate system, and the second color temperature curve includes second coordinate points corresponding to different light sources;
[0163] Based on a first position point in the coordinate system, two first coordinate points in the first color temperature curve, and two second coordinate points in the second color temperature curve, a triangular mapping method is used to determine a second position point corresponding to the first position point, the first position point is a point formed by a first ratio and a second ratio, a distance between the two first coordinate points and the first position point is greater than a distance between other first coordinate points and the first position point, and the light sources corresponding to the two first coordinate points and the two second coordinate points are the same;
[0164] Based on the abscissa of the second position point, a first correction coefficient is determined, and based on the ordinate of the second position point, a second correction coefficient is determined.
[0165] In some embodiments, the correction coefficient determination module 902 is configured to:
[0166] The reciprocal of the horizontal coordinate of the second position point is determined as the first correction coefficient;
[0167] The reciprocal of the ordinate of the second position point is determined as the second correction coefficient.
[0168] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0169] An embodiment of the present disclosure further provides an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the image processing method in the above embodiment.
[0170] Fig.10 is a block diagram of an electronic device 1000 according to an exemplary embodiment.
[0171] Reference Fig.10 , the electronic device 1000 may include one or more of the following components: a processing component 1002 , a memory 1004 , a power component 1006 , a multimedia component 1008 , an audio component 1010 , an input / output (I / O) interface 1012 , a sensor component 1014 , and a communication component 1016 .
[0172] The processing component 1002 generally controls the overall operation of the electronic device 1000, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1002 may include one or more processors 1020 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 1002 may include one or more modules to facilitate the interaction between the processing component 1002 and other components. For example, the processing component 1002 may include a multimedia module to facilitate the interaction between the multimedia component 1008 and the processing component 1002.
[0173] The memory 1004 is configured to store various types of data to support operations on the electronic device 1000. Examples of such data include instructions for any application or method operating on the electronic device 1000, contact data, phone book data, messages, pictures, videos, etc. The memory 1004 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0174] The power supply component 1006 provides power to various components of the electronic device 1000. The power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 1000.
[0175] The multimedia component 1008 includes a screen that provides an output interface between the electronic device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1008 includes a front camera and / or a rear camera. When the electronic device 1000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0176] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC), and when the electronic device 1000 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1004 or sent via the communication component 1016. In some embodiments, the audio component 1010 also includes a speaker for outputting audio signals.
[0177] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0178] The sensor assembly 1014 includes one or more sensors for providing various aspects of status assessment for the electronic device 1000. For example, the sensor assembly 1014 can detect the open / closed state of the electronic device 1000, the relative positioning of components, such as the display and keypad of the electronic device 1000, and the sensor assembly 1014 can also detect the position change of the electronic device 1000 or a component of the electronic device 1000, the presence or absence of user contact with the electronic device 1000, the orientation or acceleration / deceleration of the electronic device 1000, and the temperature change of the electronic device 1000. The sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1014 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1014 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0179] The communication component 1016 is configured to facilitate wired or wireless communication between the electronic device 1000 and other devices. The electronic device 1000 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1016 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0180] In an exemplary embodiment, the electronic device 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.
[0181] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1004 including instructions, and the instructions can be executed by the processor 1020 of the electronic device 1000 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0182] The embodiments of the present disclosure also provide a non-temporary computer-readable storage medium. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the image processing method in the above embodiments.
[0183] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.
[0184] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.
Claims
1. An image processing method, It is characterized in that The method comprises: Performing object segmentation on the first image to obtain a second image, wherein the second image includes a plurality of first pixel points and a plurality of second pixel points, and the object is an object with color constancy; Determining a white balance correction coefficient according to pixel values corresponding to the plurality of first pixel points; Based on the white balance correction coefficient, white balance correction is performed on the first image to obtain a third image.
2. The image processing method according to claim 1, It is characterized in that The step of performing object segmentation on the first image to obtain the second image comprises: A semantic segmentation model is called to perform object segmentation on the first image to obtain the second image, wherein the semantic segmentation model is used to perform object segmentation on the image.
3. The image processing method according to claim 2, It is characterized in that The semantic segmentation model includes an encoder and a decoder, and calling the semantic segmentation model to perform object segmentation on the first image to obtain the second image includes: Calling the encoder to perform feature extraction on the first image to obtain image features; The decoder is called to decode the first image feature to obtain the second image.
4. The image processing method according to claim 3, It is characterized in that The encoder includes a plurality of downsampling networks connected in sequence, and calling the encoder to extract features from the first image to obtain image features includes: Calling the first downsampling network to downsample the first image to obtain a first downsampled feature; For the downsampling networks other than the first downsampling network, the downsampling network is called to downsample the downsampling features output by the previous downsampling network to obtain the next downsampling feature, until the last downsampling feature output by the last downsampling network among the multiple downsampling networks is obtained, and the last downsampling feature is used as the image feature.
5. The image processing method according to claim 4, It is characterized in that The downsampling network includes a first convolution layer and a pooling layer; the first convolution layer is used to perform convolution processing on the input first image or downsampling features to obtain convolution features; the pooling layer is used to process the convolution features to obtain the downsampling features.
6. The image processing method according to claim 5, It is characterized in that The decoder includes a plurality of upsampling networks and a second convolutional layer connected in sequence, and calling the decoder to decode the first image feature to obtain the second image includes: Calling the first upsampling network to upsample the image features and the convolution features outputted from the last downsampling network to obtain a first upsampling feature; For the upsampling networks other than the first upsampling network, calling the upsampling network, upsampling the upsampling features output by the last upsampling network and the convolution features output by the corresponding downsampling network, and obtaining the next upsampling feature, until the last upsampling feature output by the last upsampling network in the multiple upsampling networks is obtained; wherein the convolution features output by the downsampling network corresponding to the upsampling network and the upsampling features output by the last upsampling network have the same number of image channels; The second convolutional layer is called to perform convolution processing on the last up-sampled feature to obtain the second image.
7. The image processing method according to claim 6, It is characterized in that The upsampling network includes a deconvolution layer, a splicing layer and a third convolution layer; the deconvolution layer is used to perform deconvolution processing on the input image features or upsampling features to obtain deconvolution features; the splicing layer is used to splice the deconvolution features with the corresponding convolution features output by the downsampling network to obtain splicing features; the third convolution layer is used to perform convolution processing on the splicing features to obtain the upsampling features.
8. The image processing method according to claim 1, It is characterized in that The object is a human skin color, the first pixel is a skin color pixel, and the second pixel is a non-skin color pixel.
9. The image processing method according to claim 8, It is characterized in that The pixel value corresponding to the skin color pixel point includes a first parameter of a first channel, a second parameter of a second channel, and a third parameter of a third channel, and determining a white balance correction coefficient according to the pixel values corresponding to the plurality of first pixel points includes: Determine, according to the pixel values corresponding to the plurality of skin color pixels, a first average value of a plurality of the first parameters, a second average value of a plurality of the second parameters, and a third average value of a plurality of the third parameters; determining a first ratio between the first average and the second average, and a second ratio between the third average and the second average; A first correction coefficient is determined based on the first ratio, and a second correction coefficient is determined based on the second ratio.
10. The image processing method according to claim 9, It is characterized in that The step of performing white balance correction on the first image based on the white balance correction coefficient to obtain a third image includes: Based on the first correction coefficient, white balance correction is performed on the first channel in the first image, and based on the second correction coefficient, white balance correction is performed on the third channel in the first image to obtain the third image.
11. The image processing method according to claim 9, It is characterized in that The determining a first correction coefficient based on the first ratio and determining a second correction coefficient based on the second ratio comprises: Based on the category to which the portrait in the first image belongs, obtaining a first color temperature curve corresponding to the category, wherein the first color temperature curve is located in a coordinate system, the abscissa of the coordinate system is a ratio between the first parameter and the third parameter, and the ordinate is a ratio between the second parameter and the third parameter, and the first color temperature curve includes first coordinate points corresponding to different light sources; Acquire a second color temperature curve, where the second color temperature curve is in the coordinate system and includes second coordinate points corresponding to different light sources; Based on a first position point in the coordinate system, two first coordinate points in the first color temperature curve, and two second coordinate points in the second color temperature curve, a triangular mapping method is used to determine a second position point corresponding to the first position point, the first position point is a point formed by the first ratio and the second ratio, a distance between the two first coordinate points and the first position point is greater than a distance between other first coordinate points and the first position point, and the light sources corresponding to the two first coordinate points and the two second coordinate points are the same; The first correction coefficient is determined based on the horizontal coordinate of the second position point, and the second correction coefficient is determined based on the vertical coordinate of the second position point.
12. The image processing method according to claim 11, It is characterized in that The determining the first correction coefficient based on the abscissa of the second position point, and determining the second correction coefficient based on the ordinate of the second position point, comprises: determining the reciprocal of the horizontal coordinate of the second position point as the first correction coefficient; The reciprocal of the ordinate of the second position point is determined as the second correction coefficient.
13. An image processing device, It is characterized in that The device comprises: a segmentation module configured to perform object segmentation on the first image to obtain a second image, wherein the second image includes a plurality of first pixels and a plurality of second pixels, and the object is an object with color constancy; A correction coefficient determination module, configured to determine a white balance correction coefficient according to pixel values corresponding to the plurality of first pixel points; The white balance correction module is configured to perform white balance correction on the first image based on the white balance correction coefficient to obtain a third image.
14. An electronic device, It is characterized in that include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the image processing method as described in any one of claims 1-12.
15. A non-transitory computer-readable storage medium, It is characterized in that When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing method as described in any one of claims 1 to 12.
