Image color detection method, device, equipment, storage medium and program product

Abstract

The present disclosure relates to an image color detection method, device, equipment, storage medium and program product, the method comprising: obtaining an image to be detected and a plurality of pre-trained color lookup tables, each of the color lookup tables being used to characterize the correspondence between a search color value and a mapping color value; inputting the image to be detected into a fusion coefficient determination model to obtain a plurality of fusion coefficients, the number of the fusion coefficients being consistent with the total number of the plurality of color lookup tables; performing weighted summation on the plurality of color lookup tables according to the plurality of fusion coefficients to obtain a fused color lookup table; taking the color value of each pixel in the image to be detected as the search color value, performing a search in the fused color lookup table to obtain a color detection result of the image to be detected. The present disclosure obtains a fused color lookup table applicable to the image through an image, a pre-trained color lookup table and a fusion coefficient determination model, thereby being able to accurately and real-time detect the color of the image.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image color detection method, device, equipment, storage medium and program product. Background Art

[0002] In the field of image processing technology, it is usually necessary to identify the color to be detected contained in the image, and then perform subsequent processing on the image area where the color to be detected is located. For example, when beautifying a portrait image, it is usually necessary to identify the skin color in the portrait, and then determine the skin area in the portrait image, and then perform fine beautification on the portrait skin.

[0003] In the related technologies, the accuracy and speed of identifying the color to be detected contained in the image are often contradictory. For example, the skin color ellipse model, threshold segmentation model, Gaussian mixture segmentation model, etc. are used for identification, which is fast but not accurate; the model with a large number of parameters is used to identify the color of the image, which is accurate but not fast. Therefore, how to quickly and accurately identify the color to be detected contained in the image is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides an image color detection method, device, equipment, storage medium and program product. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a method for detecting image color is provided, the method comprising:

[0006] Acquire an image to be detected and a plurality of pre-trained color lookup tables, each of which is used to represent a corresponding relationship between a lookup color value and a mapping color value;

[0007] Inputting the image to be detected into a fusion coefficient determination model to obtain a plurality of fusion coefficients, wherein the number of the fusion coefficients is consistent with the total number of the plurality of color lookup tables;

[0008] According to the multiple fusion coefficients, weighted summation is performed on the multiple color lookup tables to obtain a fused color lookup table;

[0009] The color value of each pixel in the image to be detected is used as the search color value, and a search is performed in the fused color lookup table to obtain a color detection result of the image to be detected.

[0010] Optionally, the plurality of pre-trained color lookup tables and the fusion coefficient determination model are trained by the following steps:

[0011] Acquire a plurality of image samples under different scenes and a mask image of a color to be detected in the plurality of image samples;

[0012] Obtaining a plurality of color lookup tables to be trained;

[0013] Inputting the plurality of image samples into the fusion coefficient determination model to be trained to obtain a plurality of fusion coefficients for training;

[0014] Obtaining color prediction results of each of the plurality of image samples according to the plurality of fusion coefficients for training, the plurality of color lookup tables to be trained, and the plurality of image samples;

[0015] Establishing a loss function according to the difference between the color prediction results of each of the plurality of image samples and the mask image of the color to be detected;

[0016] Based on the loss function, the model parameters of the fusion coefficient determination model to be trained are updated to obtain the fusion coefficient determination model, and the corresponding relationships contained in the multiple color lookup tables to be trained are updated to obtain the multiple pre-trained color lookup tables.

[0017] Optionally, obtaining color prediction results of each of the plurality of image samples according to the plurality of fusion coefficients for training, the plurality of color lookup tables to be trained, and the plurality of image samples comprises:

[0018] According to the multiple fusion coefficients for training, weighted summation is performed on the multiple color lookup tables to be trained to obtain the fusion color lookup tables to be trained corresponding to the multiple image samples respectively;

[0019] The color value of each pixel in the multiple image samples is used as the search color value, and the search is performed in the fusion color lookup table to be trained to obtain the color prediction results of each of the multiple image samples.

[0020] Optionally, the fusion coefficient determination model includes a downsampling module; the step of inputting the plurality of image samples into the fusion coefficient determination model to be trained to obtain a plurality of the fusion coefficients for training includes:

[0021] The multiple image samples are input into the fusion coefficient determination model to be trained, and the downsampling module downsamples the multiple image samples to obtain the multiple image samples after downsampling;

[0022] The multiple downsampled image samples are calculated layer by layer to obtain the multiple fusion coefficients used for training.

[0023] Optionally, the image to be detected is a portrait image;

[0024] The method of using the color value of each pixel in the image to be detected as the search color value, searching in the fused color lookup table, and obtaining the color detection result of the image to be detected includes:

[0025] Taking the color value of each pixel in the portrait image as the search color value, searching in the fused color lookup table to obtain the skin color in the portrait image;

[0026] A skin area in the portrait image is determined according to the skin color in the portrait image.

[0027] According to a second aspect of an embodiment of the present disclosure, there is provided an image color detection device, the device comprising:

[0028] An image and lookup table acquisition module is configured to acquire an image to be detected and a plurality of pre-trained color lookup tables, each of which is used to represent a corresponding relationship between a lookup color value and a mapping color value;

[0029] a fusion coefficient determination module, configured to input the image to be detected into a fusion coefficient determination model to obtain a plurality of fusion coefficients, wherein the number of the fusion coefficients is consistent with the total number of the plurality of color lookup tables;

[0030] A weighted summation module is configured to perform weighted summation on the plurality of color lookup tables according to the plurality of fusion coefficients to obtain a fused color lookup table;

[0031] The color search module is configured to use the color value of each pixel in the image to be detected as the search color value, search in the fused color search table, and obtain the color detection result of the image to be detected.

[0032] Optionally, the plurality of pre-trained color lookup tables and the fusion coefficient determination model are trained by the following steps:

[0033] Acquire a plurality of image samples under different scenes and a mask image of a color to be detected in the plurality of image samples;

[0034] Obtaining a plurality of color lookup tables to be trained;

[0035] Inputting the plurality of image samples into the fusion coefficient determination model to be trained to obtain a plurality of fusion coefficients for training;

[0036] Obtaining color prediction results of each of the plurality of image samples according to the plurality of fusion coefficients for training, the plurality of color lookup tables to be trained, and the plurality of image samples;

[0037] Establishing a loss function according to the difference between the color prediction results of each of the plurality of image samples and the mask image of the color to be detected;

[0038] Based on the loss function, the model parameters of the fusion coefficient determination model to be trained are updated to obtain the fusion coefficient determination model, and the corresponding relationships contained in the multiple color lookup tables to be trained are updated to obtain the multiple pre-trained color lookup tables.

[0039] Optionally, obtaining color prediction results of each of the plurality of image samples according to the plurality of fusion coefficients for training, the plurality of color lookup tables to be trained, and the plurality of image samples comprises:

[0040] According to the multiple fusion coefficients for training, weighted summation is performed on the multiple color lookup tables to be trained to obtain the fusion color lookup tables to be trained corresponding to the multiple image samples respectively;

[0041] The color value of each pixel in the multiple image samples is used as the search color value, and the search is performed in the fusion color lookup table to be trained to obtain the color prediction results of each of the multiple image samples.

[0042] Optionally, the fusion coefficient determination model includes a downsampling module; the step of inputting the plurality of image samples into the fusion coefficient determination model to be trained to obtain a plurality of the fusion coefficients for training includes:

[0043] The multiple image samples are input into the fusion coefficient determination model to be trained, and the downsampling module downsamples the multiple image samples to obtain the multiple image samples after downsampling;

[0044] The multiple downsampled image samples are calculated layer by layer to obtain the multiple fusion coefficients used for training.

[0045] Optionally, the image to be detected is a portrait image;

[0046] The color search module includes:

[0047] a skin color search unit configured to use the color value of each pixel in the portrait image as the search color value, search in the fused color lookup table, and obtain the skin color in the portrait image;

[0048] The skin area determination unit is configured to determine the skin area in the portrait image according to the skin color in the portrait image.

[0049] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the image color detection method as described in the first aspect.

[0050] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image color detection method as described in the first aspect.

[0051] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the image color detection method as described in the first aspect is implemented.

[0052] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0053] In the present disclosure, the fused color lookup table is obtained by weighted summing of multiple pre-trained color lookup tables, wherein the fusion coefficients of the multiple pre-trained color lookup tables are determined according to the image to be detected, and therefore the obtained fused color lookup table is applicable to the image to be detected. By searching in the fused color lookup table, complex real-time calculations are replaced by simple search operations, and the complicated calculation process is omitted, the color detection result of the image to be detected can be obtained quickly and accurately.

[0054] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0056] Figure 1 is a flowchart of a method for detecting image color according to an exemplary embodiment;

[0057] Figure 2 is a structural schematic diagram of the image color detection model in the present disclosure;

[0058] Figure 3 is a block diagram of an image color detection device according to an exemplary embodiment;

[0059] Figure 4 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0060] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0061] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0062] Figure 1 is a flowchart of a method for detecting image color according to an exemplary embodiment. Figure 1 As shown, the image color detection method can be used in electronic devices such as computers, mobile phones, and tablet computers, and includes the following steps.

[0063] In step S11, an image to be detected and a plurality of pre-trained color lookup tables are obtained, each of the color lookup tables being used to characterize a corresponding relationship between a lookup color value and a mapping color value.

[0064] The color lookup table includes various colors, and the color values ​​of various colors included in the color lookup table are used as the lookup color values. Each color / lookup color value corresponds to a mapping color value (RGB value). When using the color lookup table, the color value to be searched is first used as the lookup color value, and then the mapping color value corresponding to the lookup color value is queried in the color lookup table.

[0065] In the color lookup table to be trained, each lookup color value is the same as its corresponding mapping color value. For example, if the color value of white is known to be (255,255,255), then in the color lookup table to be trained, the mapping color value corresponding to white is (255,255,255). However, the correspondence between the lookup color value and the mapping color value in the pre-trained color lookup table may be different from the correspondence between the lookup color value and the mapping color value in the pre-trained color lookup table. For example, if the color value (lookup color value) of white is known to be (255,255,255), the mapping color value corresponding to white in the pre-trained color lookup table may not be (255,255,255).

[0066] Searching in a pre-trained color lookup table can be used to search for specific colors. For example, if you want to search for a red area in an image, you can obtain a pre-trained color lookup table in which the mapping color value corresponding to the red search color value is white, and the mapping color values ​​corresponding to other non-red search color values ​​are all black; when the color value of each pixel of the image is used as the search color value to search in the pre-trained color lookup table, the mapping color value corresponding to each pixel can be obtained, and thus a mapping image of the image is generated according to the mapping color value corresponding to each pixel; the image area in the image corresponding to the white area of ​​the mapping image is the red area in the image, and the image area in the image corresponding to the black area of ​​the mapping image is the red area in the image.

[0067] Optionally, the color lookup table may have a total of 8×8 grids, each grid corresponds to a blue value of a blue channel, and the blue values ​​corresponding to the grids form an arithmetic progression from 0 to 255; each grid contains 64×64 pixels, and the red values ​​of each pixel in each horizontal row in each grid are an arithmetic progression from 0 to 255, and the green values ​​of each pixel in each vertical row in each grid are an arithmetic progression from 0 to 255. When the color value to be searched is (153, 101, 213), the blue value 213 of the color value can be first searched to determine that it belongs to the 54th grid, and then in the 54th grid, according to the coordinates (153, 101), it can be determined that it belongs to the 39th pixel in the horizontal direction and the 26th pixel in the vertical direction; after the pixel in the color lookup table is determined, the mapped color value corresponding to the pixel can be used as the mapped color value corresponding to the search color value. Optionally, the accuracy of the color lookup table may also be changed, for example, it may include 15×15 grids corresponding to blue values, and each grid may include more pixels.

[0068] Optionally, the pre-trained color lookup table is obtained by adjusting the correspondence between the lookup color value and the mapping color value in the color lookup table to be trained. The method for adjusting the correspondence between the lookup color value and the mapping color value in the color lookup table to be trained can be referred to below, or a deep learning method can be used to continuously adjust the stored correspondence in the color lookup table to be trained. The correspondence between the lookup color value and the mapping color value in each pre-trained color lookup table can be different. For example, the mapping color value corresponding to the lookup color value of white in one pre-trained color lookup table can be (123, 211, 233), and the mapping color value corresponding to the lookup color value of white in another pre-trained color lookup table can be (121, 201, 213).

[0069] In step S12, the image to be detected is input into a fusion coefficient determination model to obtain a plurality of fusion coefficients, and the number of the fusion coefficients is consistent with the total number of the plurality of color lookup tables.

[0070] In step S13, a weighted sum is performed on the plurality of color lookup tables according to the plurality of fusion coefficients to obtain a fused color lookup table.

[0071] The image to be detected is input into the fusion coefficient determination model, and the fusion coefficient determination model extracts features of the image to be detected to obtain multiple fusion coefficients, and the number of the fusion coefficients is consistent with the total number of the multiple color lookup tables.

[0072] Multiple average pooling layers can be used to extract features from the image to be detected, and then pass through the fully connected layer to finally obtain a one-dimensional N×1 weight vector, where N is a non-0 positive integer. The weight vector contains N elements as N fusion coefficients.

[0073] After obtaining N fusion coefficients, each fusion coefficient can be used to weight each pre-trained color lookup table, and then the N weighted pre-trained color lookup tables are averaged to obtain a fused color lookup table. For example, assuming there are 3 pre-trained color lookup tables, the mapped color values ​​corresponding to the pixels at the upper left corner vertex of the 3 pre-trained color lookup tables are (10, 10, 10), (20, 20, 20), and (30, 30, 30), respectively, and the fusion coefficients of the 3 pre-trained color lookup tables are 0.2, 0.3, and 0.5, respectively. Then, the mapped color value corresponding to the pixel at the upper left corner vertex of the obtained fused color lookup table should be (10, 10, 10) × 0.2 + (20, 20, 20) × 0.3 + (30, 30, 30) × 0.5 = (23, 23, 23).

[0074] Multiple pre-trained color lookup tables and fusion coefficient determination models are used in pairs. There is a sequential relationship between the multiple pre-trained color lookup tables. The fusion coefficients output by the fusion coefficient determination model also have a sequential relationship and correspond one-to-one with the multiple pre-trained color lookup tables.

[0075] In this way, a fused color lookup table is obtained through multiple pre-trained color lookup tables, which can avoid the problem of inaccurate color detection results caused by errors in one pre-trained color lookup table, that is, improve the accuracy.

[0076] Optionally, other image feature extraction methods in the related art may be used to extract features from the image to be detected, which will not be described in detail here.

[0077] In step S14, the color value of each pixel in the image to be detected is used as the search color value, and a search is performed in the fused color lookup table to obtain a color detection result of the image to be detected.

[0078] The color value of each pixel in the image to be detected is used as the search color value, and the fused color lookup table is searched to obtain the mapped color value of each pixel. A mapped image of the image to be detected is generated according to the mapped color value of each pixel, and the mapped image can be used as the color detection result of the image to be detected. The color detection result of the image to be detected determines whether the color to be detected exists in the image to be detected, and can also determine the image area where the color to be detected is located in the image to be detected.

[0079] Optionally, the color detection result of the image to be detected is the display result of one or more colors to be detected in the image to be detected, which may be to display one or more colors to be detected as white, and display other colors except the colors to be detected as white.

[0080] Optionally, the image to be detected can be divided into multiple units, the color value of each unit is obtained, and the color value of each unit is used as the search color value to search in the fusion color lookup table, so as to obtain the color detection result of the image to be detected. When the number of divided units is more, the color detection result is more accurate, and when the number of divided units is less, the speed is faster. However, because the present disclosure uses a simple search operation instead of a complex calculation process, even if the accuracy is improved, the speed is still very fast.

[0081] The color detection result (mapped image) of the image to be detected is consistent with the size of the image to be detected, so it is beneficial to perform subsequent processing according to the color detection result of the image to be detected without adjusting the size of the color detection result. In addition, when searching in pixels, it can also ensure that the obtained color detection result is consistent with the resolution of the image to be detected, which is more conducive to subsequent processing.

[0082] The color to be detected is determined in the process of training the fusion coefficient determination model and the color lookup table. When the mask map of the target color is used, the target color is the color to be detected. Therefore, for the paired "multiple pre-trained color lookup tables and fusion coefficient determination models", for different images, the fusion color lookup table corresponding to the image can be obtained, and then the color detection result of the color to be detected in the image can be obtained. The color to be detected is determined by the mask map used when training the fusion coefficient determination model and the color lookup table.

[0083] In the image color detection method provided by the present exemplary embodiment, the fused color lookup table is obtained by weighted summing of multiple pre-trained color lookup tables, wherein the fusion coefficients of the multiple pre-trained color lookup tables are determined according to the image to be detected, so the obtained fused color lookup table is suitable for the image to be detected. Searching in the fused color lookup table replaces complex real-time calculations with simple search operations, omitting the complicated calculation process, and can quickly and accurately obtain the color detection result of the image to be detected.

[0084] Optionally, when a skin color mask image is used in the process of training the fusion coefficient determination model and the color lookup table, the color to be detected is skin color. When the image to be detected is a portrait image, the skin color in the portrait image can be detected by the method of the above embodiment. Further, the skin area in the portrait image can be determined according to the color detection result of the skin color in the portrait image.

[0085] Determining the skin area in the portrait image is beneficial to fine-tuning the portrait skin, for example, performing skin smoothing only on the skin while retaining the texture of other non-skin areas.

[0086] On the basis of the above technical scheme, multiple pre-trained color lookup tables and the fusion coefficient determination model can be trained through the following steps: obtaining multiple image samples in different scenes and mask images of the colors to be detected in the multiple image samples; obtaining multiple color lookup tables to be trained; inputting the multiple image samples into the fusion coefficient determination model to be trained to obtain multiple fusion coefficients for training; obtaining color prediction results of each of the multiple image samples according to the multiple fusion coefficients for training, the multiple color lookup tables to be trained and the multiple image samples; establishing a loss function according to the difference between the color prediction results of each of the multiple image samples and the mask image of the color to be detected; based on the loss function, updating the model parameters of the fusion coefficient determination model to be trained to obtain the fusion coefficient determination model, and updating the corresponding relationships contained in each of the multiple color lookup tables to be trained to obtain the multiple pre-trained color lookup tables.

[0087] In order to enable the image color detection method in the present disclosure to accurately obtain the color detection results of the images to be detected in various scenes, the image samples used may be image samples in different scenes, such as dark light scenes, backlight scenes, indoor scenes, outdoor scenes, single-person scenes, multi-person scenes, etc. In order to improve the processing speed of the fusion coefficient determination model, the fusion coefficient determination model may be a lightweight neural network model.

[0088] The fusion coefficient determination model to be trained extracts features from the input image sample, and then obtains multiple sets of fusion coefficients for training. Among them, a set of fusion coefficients for training is obtained based on an image sample, and the number of fusion coefficients in a set of fusion coefficients for training is consistent with the number of multiple color lookup tables to be trained. The method for obtaining the fusion coefficients for training can refer to the method for obtaining the fusion coefficients through the fusion coefficient determination model mentioned above.

[0089] A plurality of color lookup tables to be trained are obtained, wherein the greater the total number of color lookup tables, the more accurate the color detection result obtained, but too many color lookup table samples will hinder the speed of obtaining the color detection result.

[0090] Each image sample is searched in a fusion color lookup table for training obtained by using a fusion coefficient obtained according to the image sample, so as to obtain a color prediction result of the image sample.

[0091] The color value of each pixel in the multiple image samples is used as the search color value, and the search is performed in the fusion color lookup table to be trained to obtain the color prediction results of each of the multiple image samples.

[0092] A loss function is established based on the difference between the color prediction result of each image sample and the mask map of the color to be detected. Alternatively, a loss function can be established based on the value of the color prediction result of each image sample and the corresponding position in the mask map of the color to be detected, or the square value of the corresponding position after subtraction.

[0093] Based on the loss function, the model parameters of the fusion coefficient determination model to be trained and the corresponding relationships contained in the multiple color lookup tables to be trained are updated simultaneously to obtain the fusion coefficient determination model and the multiple pre-trained color lookup tables respectively.

[0094] In this way, the fusion coefficient determination model to be trained and the color lookup table to be trained can be trained at the same time to obtain a fusion coefficient determination model and multiple pre-trained color lookup tables used in pairs. For the paired "multiple pre-trained color lookup tables and fusion coefficient determination models", for different images, the fusion color lookup table corresponding to the image can be quickly and accurately obtained, and then the color detection result of the color to be detected in the image can be obtained.

[0095] Optionally, based on the above technical solution, according to the multiple fusion coefficients for training, the multiple color lookup tables to be trained and the multiple image samples, the color prediction results of each of the multiple image samples are obtained, including: performing weighted summation on the multiple color lookup tables to be trained according to each group of fusion coefficients to obtain multiple fused color lookup tables to be trained. The training of the fused color lookup table to be trained is completed by training each color lookup table to be trained.

[0096] After obtaining the fused color lookup table to be trained, the color prediction results of each image sample can be obtained. For each image sample, the color value of each pixel in the image sample is used as the lookup color value and searched in the fused color lookup table, thereby obtaining the color prediction results of multiple image samples.

[0097] In this way, the color prediction result of the image sample obtained has the same size and resolution as the mask map of the color to be detected in the image sample, so it is convenient to calculate the difference based on the color prediction result of the image sample and the mask map of the color to be detected later.

[0098] Optionally, the fusion coefficient determination model may include a downsampling module, and after the image sample is input into the fusion coefficient determination model to be trained, the downsampling module downsamples the image sample to obtain a downsampled image sample, and each downsampled image sample has the same resolution. Then, the downsampled image sample is passed through an average pooling layer, a fully connected layer, etc., and is calculated layer by layer to obtain a fusion coefficient for training.

[0099] Optionally, the downsampling module may be a module outside the fusion coefficient determination model and is a pre-module of the fusion coefficient determination model. When the downsampling module is a pre-module of the fusion coefficient determination model, the image sample is first input into the downsampling module to obtain the down-sampled image sample, and then the down-sampled image sample is input into the fusion coefficient determination model.

[0100] In this way, downsampling the image samples is beneficial to reducing the subsequent amount of calculation; and the downsampled image samples have the same resolution, which is beneficial for the fusion coefficient determination model to perform subsequent processing on the downsampled image samples.

[0101] Optionally, in order to ensure that the fusion coefficient determination model and multiple pre-trained color lookup tables can be used in pairs, an image color detection model can be generated based on the fusion coefficient determination model and multiple pre-trained color lookup tables, and the image to be detected is input into the image color detection model to obtain the color detection result of the image to be detected. Figure 2A structural schematic diagram of an image color detection model is shown, and the image color detection model may include a downsampling module, a fusion coefficient generation module, a lookup table storage module, a weighted processing module, and a detection result generation module. Among them, the downsampling module is used to downsample the image; the fusion coefficient generation module is used to generate the fusion coefficient; the lookup table storage module is used to store multiple pre-trained color lookup tables; the weighted processing module is used to perform weighted summation processing on multiple pre-trained color lookup table samples according to the fusion coefficient to generate a fused color lookup table; the detection result generation module is used to generate the color detection result of the image according to the image and the fused color lookup table.

[0102] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

[0103] Figure 3 is a block diagram of an image color detection device according to an exemplary embodiment. Figure 3 The device includes an image and lookup table acquisition module 31, a fusion coefficient determination module 32, a weighted summation module 33 and a color lookup module 34.

[0104] The image and lookup table acquisition module 31 is configured to acquire an image to be detected and a plurality of pre-trained color lookup tables, each of which is used to represent a correspondence between a lookup color value and a mapping color value;

[0105] The fusion coefficient determination module 32 is configured to input the image to be detected into a fusion coefficient determination model to obtain a plurality of fusion coefficients, wherein the number of the fusion coefficients is consistent with the total number of the plurality of color lookup tables;

[0106] The weighted sum module 33 is configured to perform weighted summation on the plurality of color lookup tables according to the plurality of fusion coefficients to obtain a fusion color lookup table;

[0107] The color search module 34 is configured to use the color value of each pixel in the image to be detected as the search color value, search in the fused color search table, and obtain the color detection result of the image to be detected.

[0108] Optionally, the plurality of pre-trained color lookup tables and the fusion coefficient determination model are trained by the following steps:

[0109] Acquire a plurality of image samples under different scenes and a mask image of the color to be detected in the plurality of image samples;

[0110] Obtaining a plurality of color lookup tables to be trained;

[0111] Inputting the plurality of image samples into the fusion coefficient determination model to be trained to obtain a plurality of fusion coefficients for training;

[0112] Obtaining color prediction results of each of the plurality of image samples according to the plurality of fusion coefficients for training, the plurality of color lookup tables to be trained, and the plurality of image samples;

[0113] Establishing a loss function according to the difference between the color prediction results of each of the plurality of image samples and the mask image of the color to be detected;

[0114] Based on the loss function, the model parameters of the fusion coefficient determination model to be trained are updated to obtain the fusion coefficient determination model, and the corresponding relationships contained in the multiple color lookup tables to be trained are updated to obtain the multiple pre-trained color lookup tables.

[0115] Optionally, obtaining color prediction results of each of the plurality of image samples according to the plurality of fusion coefficients for training, the plurality of color lookup tables to be trained, and the plurality of image samples comprises:

[0116] According to the multiple fusion coefficients for training, weighted summation is performed on the multiple color lookup tables to be trained to obtain the fusion color lookup tables to be trained corresponding to the multiple image samples respectively;

[0117] The color value of each pixel in the multiple image samples is used as the search color value, and the search is performed in the fusion color lookup table to be trained to obtain the color prediction results of each of the multiple image samples.

[0118] Optionally, the fusion coefficient determination model includes a downsampling module; the step of inputting the plurality of image samples into the fusion coefficient determination model to be trained to obtain a plurality of the fusion coefficients for training includes:

[0119] The multiple image samples are input into the fusion coefficient determination model to be trained, and the downsampling module downsamples the multiple image samples to obtain the multiple image samples after downsampling;

[0120] The multiple downsampled image samples are calculated layer by layer to obtain the multiple fusion coefficients used for training.

[0121] Optionally, the image to be detected is a portrait image;

[0122] The color search module 34 includes:

[0123] a skin color search unit configured to use the color value of each pixel in the portrait image as the search color value, search in the fused color lookup table, and obtain the skin color in the portrait image;

[0124] The skin area determination unit is configured to determine the skin area in the portrait image according to the skin color in the portrait image.

[0125] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0126] Figure 4 4 is a block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0127] Reference Figure 4 , the electronic device 400 may include one or more of the following components: a processing component 402 , a memory 404 , a power component 406 , a multimedia component 408 , an audio component 410 , an input / output (I / O) interface 412 , a sensor component 414 , and a communication component 416 .

[0128] The processing component 402 generally controls the overall operation of the electronic device 400, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 402 may include one or more modules to facilitate the interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate the interaction between the multimedia component 408 and the processing component 402.

[0129] The memory 404 is configured to store various types of data to support operations on the electronic device 400. Examples of such data include instructions for any application or method operating on the electronic device 400, contact data, phone book data, messages, pictures, videos, etc. The memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0130] The power supply component 406 provides power to the various components of the electronic device 400. The power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 400.

[0131] The multimedia component 408 includes a screen that provides an output interface between the electronic device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 408 includes a front camera and / or a rear camera. When the electronic device 400 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0132] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC), and when the electronic device 400 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 404 or sent via the communication component 416. In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.

[0133] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0134] The sensor assembly 414 includes one or more sensors for providing various aspects of status assessment for the electronic device 400. For example, the sensor assembly 414 can detect the open / closed state of the electronic device 400, the relative positioning of the components, such as the display and keypad of the electronic device 400, and the sensor assembly 414 can also detect the position change of the electronic device 400 or a component of the electronic device 400, the presence or absence of contact between the user and the electronic device 400, the orientation or acceleration / deceleration of the electronic device 400, and the temperature change of the electronic device 400. The sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 414 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0135] The communication component 416 is configured to facilitate wired or wireless communication between the electronic device 400 and other devices. The electronic device 400 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0136] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned image color detection method.

[0137] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, and the above instructions can be executed by the processor 420 of the electronic device 400 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0138] In an exemplary embodiment, a computer program product is also provided, the computer program product includes a readable program code, the readable program code can be executed by the processor 420 of the electronic device 400 to complete the above-mentioned image color detection method. Optionally, the program code can be stored in a storage medium of the electronic device 400, and the storage medium can be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0139] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0140] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for detecting image color, characterized in that: The method comprises: Acquire an image to be detected and a plurality of pre-trained color lookup tables, each of which is used to represent a corresponding relationship between a lookup color value and a mapping color value; Inputting the image to be detected into a fusion coefficient determination model to obtain a plurality of fusion coefficients, wherein the number of the fusion coefficients is consistent with the total number of the plurality of color lookup tables; According to the multiple fusion coefficients, weighted summation is performed on the multiple color lookup tables to obtain a fused color lookup table; Taking the color value of each pixel in the image to be detected as the search color value, searching in the fused color lookup table to obtain the color detection result of the image to be detected; The plurality of pre-trained color lookup tables and the fusion coefficient determination model are trained by the following steps: Acquire a plurality of image samples under different scenes and a mask image of a color to be detected in the plurality of image samples; Obtaining a plurality of color lookup tables to be trained; Inputting the plurality of image samples into the fusion coefficient determination model to be trained to obtain a plurality of fusion coefficients for training; Obtaining color prediction results of each of the plurality of image samples according to the plurality of fusion coefficients for training, the plurality of color lookup tables to be trained, and the plurality of image samples; Establishing a loss function according to the difference between the color prediction results of each of the plurality of image samples and the mask image of the color to be detected; Based on the loss function, the model parameters of the fusion coefficient determination model to be trained are updated to obtain the fusion coefficient determination model, and the corresponding relationships contained in the multiple color lookup tables to be trained are updated to obtain the multiple pre-trained color lookup tables.

2. The method according to claim 1, characterized in that The obtaining color prediction results of each of the plurality of image samples according to the plurality of fusion coefficients for training, the plurality of color lookup tables to be trained, and the plurality of image samples comprises: According to the multiple fusion coefficients for training, weighted summation is performed on the multiple color lookup tables to be trained to obtain the fusion color lookup tables to be trained corresponding to the multiple image samples respectively; The color value of each pixel in the multiple image samples is used as the search color value, and the search is performed in the fusion color lookup table to be trained to obtain the color prediction results of each of the multiple image samples.

3. The method according to claim 1, characterized in that The fusion coefficient determination model includes a downsampling module; the inputting the plurality of image samples into the fusion coefficient determination model to be trained to obtain a plurality of the fusion coefficients for training includes: The multiple image samples are input into the fusion coefficient determination model to be trained, and the downsampling module downsamples the multiple image samples to obtain the multiple image samples after downsampling; The multiple downsampled image samples are calculated layer by layer to obtain the multiple fusion coefficients used for training.

4. The method according to any one of claims 1 to 3, characterized in that: The image to be detected is a portrait image; The method of using the color value of each pixel in the image to be detected as the search color value, searching in the fused color lookup table, and obtaining the color detection result of the image to be detected includes: Taking the color value of each pixel in the portrait image as the search color value, searching in the fused color lookup table to obtain the skin color in the portrait image; A skin area in the portrait image is determined according to the skin color in the portrait image.

5. An image color detection device, characterized in that: The device comprises: An image and lookup table acquisition module is configured to acquire an image to be detected and a plurality of pre-trained color lookup tables, each of which is used to represent a corresponding relationship between a lookup color value and a mapping color value; a fusion coefficient determination module, configured to input the image to be detected into a fusion coefficient determination model to obtain a plurality of fusion coefficients, wherein the number of the fusion coefficients is consistent with the total number of the plurality of color lookup tables; A weighted summation module is configured to perform weighted summation on the plurality of color lookup tables according to the plurality of fusion coefficients to obtain a fused color lookup table; A color search module is configured to use the color value of each pixel in the image to be detected as the search color value, search in the fused color search table, and obtain a color detection result of the image to be detected; The plurality of pre-trained color lookup tables and the fusion coefficient determination model are trained by the following steps: Acquire a plurality of image samples under different scenes and a mask image of a color to be detected in the plurality of image samples; Obtaining a plurality of color lookup tables to be trained; Inputting the plurality of image samples into the fusion coefficient determination model to be trained to obtain a plurality of fusion coefficients for training; Obtaining color prediction results of each of the plurality of image samples according to the plurality of fusion coefficients for training, the plurality of color lookup tables to be trained, and the plurality of image samples; Establishing a loss function according to the difference between the color prediction results of each of the plurality of image samples and the mask image of the color to be detected; Based on the loss function, the model parameters of the fusion coefficient determination model to be trained are updated to obtain the fusion coefficient determination model, and the corresponding relationships contained in the multiple color lookup tables to be trained are updated to obtain the multiple pre-trained color lookup tables.

6. The device according to claim 5, characterized in that The obtaining color prediction results of each of the plurality of image samples according to the plurality of fusion coefficients for training, the plurality of color lookup tables to be trained, and the plurality of image samples comprises: According to the multiple fusion coefficients for training, weighted summation is performed on the multiple color lookup tables to be trained to obtain the fusion color lookup tables to be trained corresponding to the multiple image samples respectively; The color value of each pixel in the multiple image samples is used as the search color value, and the search is performed in the fusion color lookup table to be trained to obtain the color prediction results of each of the multiple image samples.

7. The device according to claim 5, characterized in that The fusion coefficient determination model includes a downsampling module; the inputting the plurality of image samples into the fusion coefficient determination model to be trained to obtain a plurality of the fusion coefficients for training includes: The multiple image samples are input into the fusion coefficient determination model to be trained, and the downsampling module downsamples the multiple image samples to obtain the multiple image samples after downsampling; The multiple downsampled image samples are calculated layer by layer to obtain the multiple fusion coefficients used for training.

8. The device according to any one of claims 5 to 7, characterized in that: The image to be detected is a portrait image; The color search module includes: a skin color search unit configured to use the color value of each pixel in the portrait image as the search color value, search in the fused color lookup table, and obtain the skin color in the portrait image; The skin area determination unit is configured to determine the skin area in the portrait image according to the skin color in the portrait image.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image color detection method according to any one of claims 1 to 4. 10 . A computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image color detection method according to any one of claims 1 to 4.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image color detection method according to any one of claims 1 to 4 is implemented.

Images