A method for extracting a color palette of a color matching style combined with visual saliency

Abstract

The present invention relates to a method for extracting a color palette of a color matching style map combined with visual saliency. After inputting an initial color matching style map, preprocessing is performed to obtain a color matching style map; the significant regions are detected for the preprocessed color matching style map to obtain significant regions; the colors of the detected significant regions are extracted; non-significant regions are obtained based on the significant regions, and the colors of the non-significant regions are extracted; the color palette of the significant regions and the colors extracted from the non-significant regions are combined and adjusted; a color matching palette is generated. The beneficial effect of the present invention lies in that the color extraction of the color matching style map is performed by combining saliency detection, overcoming the problem of the loss of small-area significant colors that easily occurs when directly using a clustering algorithm, enhancing the color palette of the significant regions by using weights, increasing the proportion of significant colors in the final color matching palette, and being beneficial to improving the subsequent intelligent color matching effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of general image data processing or the sub-class index of general image data processing, and particularly relates to a method for extracting a color matching style map color palette combining visual saliency in the application of deep learning and machine learning technologies in the fabric pattern design industry. Background Art

[0002] With the development of social economy, people's demand for fabric patterns tends to be personalized, and the competition in the fabric pattern design industry is becoming increasingly fierce. The intelligent color matching technology has a great impact on the efficiency and benefits of pattern design and has become the core technology of this industry. The automatic extraction effect of the color matching style map color palette will directly affect the efficiency and final result of intelligent color matching and is a key link in intelligent color matching.

[0003] The existing image color extraction methods mainly include color extraction methods based on quantization and color extraction methods based on feature modeling.

[0004] The color extraction methods based on quantization mainly focus on the color information of image pixels. Among them, the more commonly used methods include median cut method, histogram analysis method, and clustering-based method. The median cut method performs cutting in the RGB color space, and the cutting position is selected as the median of the longest spatial axis, and repeated cutting is performed until the requirements are met. When the color space is large but the colors are relatively sparse, large errors will occur. The histogram analysis method is often used together with other methods to extract colors by finding important hue, saturation, gray value, etc. according to the statistical results, but this method usually produces redundant results. The clustering-based method requires specifying the number of initial clustering centers as input, and through repeated iteration, the sum of the distances from the data samples in each category to the center of that category is minimized. However, when the color data distribution is uneven, this method is prone to the situation of partial color loss.

[0005] The color extraction methods based on feature modeling mainly focus on the features of the image and extract colors by modeling these features. Among them, the more commonly used method is Gaussian mixture modeling method. This method regards the color distribution of an image as a mixture of Gaussian distributions of multiple main colors, takes the image pixels as samples and inputs them into the model for parameter iteration. Usually, the expectation maximization algorithm (EM algorithm) is used, and finally the distribution models of multiple main colors are obtained. However, this method has the problem of slow processing speed when implemented.

[0006] In the prior art, a patent with the application number 201910598501.7 discloses a method for extracting theme colors based on human vision. The method constructs a color map model in the RGB color space, establishes the connection relationship between pixel colors in the color map model, uses the Louvain community discovery algorithm to obtain the initial theme colors, and finally obtains the sorted theme colors similar to human vision in a data-driven manner. Although this method obtains a color extraction effect similar to that of the human eye by constructing a linear equation to fit the law of color extraction by the human eye, it is difficult to obtain an annotation dataset that conforms to the subjective feelings of designers in practical application scenarios such as flower pattern color matching, and the application scenarios are limited. Summary of the Invention

[0007] In order to solve the problems existing in the prior art, the present invention provides a method for extracting a color palette of a color matching style map combined with visual saliency, which can quickly extract the main colors and visually salient colors of the color matching style map, generate a color palette map, and provide it for use by an intelligent color matching algorithm.

[0008] The technical solution adopted by the present invention is a method for extracting a color palette of a color matching style map combined with visual saliency, and the method includes the following steps:

[0009] Step 1: Input an initial color matching style map P orin , generally, it is an RGB pattern, and after preprocessing, a color matching style map is obtained P scale ;

[0010] Step 2: Detect the salient regions of the preprocessed color matching style map to obtain the salient regions P roi ;

[0011] Step 3: Extract the colors of the detected salient regions P roi ;

[0012] Step 4: Obtain the non-salient regions based on the salient regions P roi , and extract the colors of the non-salient regions P nroi , and extract the colors of the non-salient regions P nroi ;

[0013] Step 5: Combine and adjust the color palette of the salient regions and the colors extracted from the non-salient regions;

[0014] Step 6: Generate a color matching color palette.

[0015] Preferably, in the step 1, the color matching style map is P orinThe resolution is scaled to W × H to obtain the scaled color matching style map P scale ; Generally speaking, W The value range of is [250, 1000], H The value range of is [250, 1000].

[0016] Preferably, step 2 includes the following steps:

[0017] Step 2-1: Use the pre-trained saliency detection network model to perform saliency detection on the color matching style map P scale to obtain the attention map M 0; "Saliency" means that the output result of the network model segments the significant regions in the image, that is, the regions that are more obvious to the human eye and can attract attention;

[0018] Step 2-2: Process the attention map M 0 with Otsu's method to obtain the significant region mask M 1;

[0019] Step 2-3: Convert the color matching style map P scale to the HSV space to obtain the color matching style map P hsv ;

[0020] Step 2-4: Use the significant region mask M 1 to perform region segmentation on the color matching style map P hsv to obtain the significant region in the color matching style map P roi .

[0021] Preferably, step 3 includes the following steps:

[0022] Step 3-1: Use the K-means algorithm to perform color clustering on all pixels of the significant region P roi , set K 1 clustering centers, K 1 is a positive integer; Generally speaking, K The value range of 1 is [5, 20];

[0023] Step 3-2: Convert the pixel values of the K 1 clustering centers obtained by clustering from the HSV space to the RGB space to obtain the color palette color list P roi of the significant region P1 and the corresponding list of the number of color swatch pixels C 1, where P 1 includes K the pixel values of 1 clustering center, C 1 includes K the number of pixels corresponding to 1 clustering center, P the pixel values in 1 and C the number of pixels in 1 correspond, that is, they belong to the same clustering result. A certain pixel value corresponds to the number of pixels included in the class with this pixel as the clustering center.

[0024] Preferably, step 4 includes the following steps:

[0025] Step 4-1: Invert the significant region mask obtained in step 2-2 M 1 to obtain a non-significant region mask M 2, and use the non-significant region mask M 2 to perform region segmentation on the color matching style map P hsv to obtain a non-significant region P nroi , that is, the part of the color matching style map P hsv excluding the significant region P roi outside.

[0026] Step 4-2: Cluster all the pixels in the non-significant region P nroi using the K-means algorithm, and set K 2 clustering centers, K 2 is a positive integer; generally speaking, K the value range of 2 is [8, 16];

[0027] Step 4-3: Convert the pixel values of the 2 clustering centers obtained by clustering from the HSV space to the RGB space to obtain the color swatch color list K of the non-significant region P nroi and the corresponding list of the number of color swatch pixels P 2 and the corresponding list of the number of color swatch pixels C 2, where P 2 includes K the pixel values of 2 clustering centers, C 2 includes K the number of pixels corresponding to 2 clustering centers, P the pixel values in 2 and C the number of pixels in 2 correspond, that is, they belong to the same clustering result, P the central pixel values in 2 are the class central pixel values, CThe number of pixels in 2 is the number of pixels in the corresponding category.

[0028] Preferably, in step 5, the significant region is extracted P roi of the color to obtain a color palette color list P 1; Extract the non-significant region P nroi of the color to obtain a color palette color list P 2; Based on the color palette color list P 1 and the color palette color list P 2, combine and adjust the colors extracted from the significant region color palette and the non-significant region.

[0029] Preferably, step 5 includes the following steps:

[0030] Step 5-1: Measure the closeness of colors by the Euclidean distance of the RGB vectors of two pixel values, and set a distance threshold DIS , DIS greater than 0; Generally, DIS ranges from [10, 30];

[0031] Step 5-2: Traverse the color palette color list P 1, and find in the color palette color list P 2 the pixel values whose vector Euclidean distance from any color in the color palette color list P 1 is less than DIS . If there are any, delete this color from P 1, and merge the number of pixels of this class of colors in C 1 into the number of pixels of the corresponding color in C 2;

[0032] Step 5-3: Multiply the significant region color palette pixel point number list C 1 by the weight W c , W c is positive. Generally, W c ranges from [1, 10], and then merge it with the non-significant region color palette pixel point number list C 2 to obtain the overall color palette pixel point number list C 3 of the color matching style map; Merge the significant region color palette color list P 1 and the non-significant region color palette color list P 2 to obtain the overall color palette color list P 3 of the color matching style map. After merging, P the pixel values in 3 and CThe number of pixels in 3 corresponds, that is, they belong to the same color category.

[0033] Preferably, in the step 5-2, from P The colors deleted from 1 exist in the non-significant area P nroi and are classified as non-significant colors.

[0034] Preferably, the step 6 includes the following steps:

[0035] Step 6-1: Sort the list of the number of pixels of the color swatch in descending order according to the number of pixels in 3, and adjust the color order of the color swatch in the color list C correspondingly according to the sorting result; P in 3;

[0036] Step 6-2: Fill the colors in 3 into the P color swatch canvas of size × W out × H out in strips in order. Generally, they are arranged from left to right in vertical strips. The height of each color strip is the same as that of the canvas, and the proportion of the width of the color strip in the total width of the canvas is set according to C the proportion of the number of pixels of each color category in 3; For the convenience of subsequent intelligent color matching, W out the value range of is [512, 1024], H out the value range of is [512, 1024];

[0037] Step 6-3: Output the color swatch canvas filled with colors as a color matching color swatch.

[0038] The present invention relates to a method for extracting a color matching style map color swatch combining visual saliency. After inputting an initial color matching style map P orin , preprocessing is performed to obtain a color matching style map P scale ; the saliency region of the preprocessed color matching style map is detected to obtain a saliency region P roi ; the colors of the detected saliency region P roi are extracted; based on the saliency region P roi the non-saliency region P nroi is obtained, and the colors of the non-saliency region P nroi are extracted; the colors of the saliency region color swatch and the non-saliency region are combined and adjusted; a color matching color swatch is generated.

[0039] The technical concept of the present invention is as follows: First, the input color matching style map is scaled and then sent into a pre-trained saliency detection network model to obtain an attention map, and the Otsu method is used to process the attention map to obtain a saliency region mask; Second, the scaled color matching style map is converted to the HSV space, and the scaled color matching style map is segmented with the help of the saliency region mask to obtain the saliency region in the color matching style map, and a clustering algorithm is used to perform color clustering on the saliency region to obtain a saliency region color palette and a corresponding color palette pixel point quantity table; Then, after inverting the saliency region mask, the scaled color matching style map is segmented to obtain the non-saliency region in the color matching style map, and the same clustering algorithm is used to perform color clustering on the non-saliency region to obtain a non-saliency region color palette and a corresponding color palette pixel point quantity table; Finally, the saliency region color palette and the non-saliency region color palette are combined, colors in the saliency region color palette that are similar to the content of the non-saliency region color palette are removed, and the proportion of the saliency region color palette is adjusted by setting weights to obtain the final color matching color palette.

[0040] The beneficial effect of the present invention is that it combines saliency detection to extract colors from the color matching style map, overcomes the problem of small-area significant color loss that easily occurs when directly using the clustering algorithm, enhances the saliency region color palette by using weights, increases the proportion of significant colors in the final color matching color palette, and is beneficial to improving the subsequent intelligent color matching effect. Description of the Drawings

[0041] Figure 1 is the flow block diagram of the present invention;

[0042] Figure 2 is the color matching style map to be processed in the present invention;

[0043] Figure 3 is the saliency region mask map of the color matching style map in the present invention;

[0044] Figure 4 is the effect diagram after color palette extraction for Figure 2 using the method of the present invention;

[0045] Figure 2 and Figure 4 In, the RGB values of the current color block are marked with a lead wire. Detailed Embodiment

[0046] The following further describes the present invention in detail with reference to the embodiments, but the protection scope of the present invention is not limited thereto.

[0047] The implementation object of the color palette extraction of the present invention is various common color matching style images for intelligent color matching. The selected processing platform is a combination of Intel i9-11980HK, NVIDIA Geforce RTX 3080 Laptop, and 16G RAM, and the operating system is Windows 11. The method of the present invention is implemented based on the OpenCV image processing function library, including functions such as resize, threshold, cvtColor, kmeans, and sort.

[0048] As Figure 1 shown, the present invention relates to a method for extracting a color palette of a color matching style map combined with visual saliency, which is carried out according to the following steps:

[0049] (1) Input the initial color matching style map P orin , and obtain the color matching style map P scale after preprocessing;

[0050] (2) Detect the significant region of the preprocessed color matching style map to obtain the significant region P roi ;

[0051] (3) Extract the colors of the detected significant region P roi ;

[0052] (4) Based on the significant region P roi obtain the non-significant region P nroi , and extract the colors of the non-significant region P nroi ;

[0053] (5) Combine and adjust the color palette of the significant region and the colors extracted from the non-significant region;

[0054] (6) Generate a color matching palette.

[0055] Step (1) specifically includes:

[0056] Input a color matching style map in the RGB pixel space P orin , as Figure 2 shown. Call the resize function to scale the resolution of the input color matching style map to 600×400, and use bilinear interpolation for the interpolation method to obtain the scaled color matching style map P scale .

[0057] Step (2) specifically includes:

[0058] (2-1) First, send the color matching style map obtained in step 1 P scale into the existing pre-trained U2-Net saliency detection network model for saliency detection to obtain an attention map M 0; As a common neural network, the saliency detection network, the optional model is not limited to U2-Net, and those skilled in the art can select it based on their needs;

[0059] (2-2) Call the threshold function to perform binarization on the attention map M 0 using the Otsu method to obtain a significant region mask M 1, as shown in Figure 3 ;

[0060] (2-3) Call the cvtColor function to convert all pixels of the scaled color matching style map obtained in step 1 P scale from the RGB space to the HSV space to obtain the color matching style map P hsv ;

[0061] (2-4) Use the significant region mask M 1 obtained in step (2-2) to perform region segmentation on the color matching style map P hsv to obtain its significant region P roi .

[0062] Step (3) specifically includes:

[0063] (3-1) Call the kmeans function to perform clustering on all pixels of the significant region P roi using the K-means algorithm, set 8 clustering centers, set the iteration stop condition to an error less than 0.01 or the number of iterations exceeding 100 times, and randomly select the initial clustering centers;

[0064] (3-2) Call the cvtColor function to convert the 8 clustering centers obtained by clustering from the HSV space back to the RGB space to obtain the color palette color list of the significant region of the color matching style map P 1 and the corresponding color palette pixel point quantity list C 1, where P 1 contains the pixel values of 8 clustering centers, C 1 contains the pixel point quantities corresponding to 8 clustering centers, P the pixel values in C 1 and

[0065] Step (4) specifically includes:

[0066] (4-1) Invert the significant region mask obtained in step (2-2) to obtain a non-significant region mask M 1, and use the non-significant region mask M 2 to perform region segmentation on the color matching style map obtained in step (2-3) to obtain non-significant regions M 2; P hsv ; P nroi ;

[0067] (4-2) Call the kmeans function to cluster all pixels of the non-significant region using the K-means algorithm, set 8 cluster centers, set the iteration stop condition to an error less than 0.01 or the number of iterations exceeding 100 times, and randomly select the initial cluster centers; P nroi ;

[0068] (4-3) Call the cvtColor function to convert the 8 cluster centers obtained by clustering from the HSV space back to the RGB space to obtain the color palette color list of the non-significant region of the color matching style map P 2 and the corresponding color palette pixel point number list C 2, where P 2 contains the pixel values of 8 cluster centers, C 2 contains the pixel point numbers corresponding to 8 cluster centers, P the pixel values in 2 and C the pixel point numbers in 2 correspond to each other and belong to the same clustering result.

[0069] Step (5) specifically includes:

[0070] Combine the color palette color list of the significant region of the color matching style map obtained in step (3-2) P 1 and the color palette color list of the non-significant region of the color matching style map obtained in step (4-3) P 2 to obtain the final color palette color list of the color matching map P 3, and the specific method is as follows:

[0071] (5-1) Traverse the color palette color list of the significant region P 1, find the similar colors in the color palette color list of the non-significant region P 2, and use the Euclidean distance of two RGB vectors to measure the closeness of colors; set the distance threshold DIS equal to 15, and identify two colors with a vector Euclidean distance less than DIS as similar;

[0072] (5-2) Delete the similar colors found in step (5-1) from P 1, and merge the number of pixels of the colors belonging to this color category in C 1 into the number of pixels of the corresponding colors in C 2. Since this color exists in the non-significant area, reclassify it as a non-significant color;

[0073] (5-3) Multiply the list of the number of pixels of the significant area color palette C 1 by the weight W c ( W c Take 5), and then merge it with the list of the number of pixels of the non-significant area color palette C 2 to obtain the list of the number of pixels of the color palette for the overall color matching style diagram C 3; Merge the significant area color palette color list P 1 and the non-significant area color palette color list P 2 to obtain the color palette color list P 3 for the overall color matching style diagram. After merging P The pixel values in 3 correspond to the number of pixels in C 3 and belong to the same color category.

[0074] Step (6) specifically includes:

[0075] (6-1) Call the sort function to sort the list of the number of pixels of the color palette of the color matching style diagram C 3 from largest to smallest, and adjust the color order of the color palette color list P 3 of the color matching style diagram according to the sorting result;

[0076] (6-2) Fill the colors in P 3 into the color palette canvas with a size of 1024×512 in order. Specifically, arrange them in vertical strips from left to right. The height of each color strip is the same as the canvas, and the proportion of the width of the color strip in the total width of the canvas is set according to C The proportion of the number of pixels of each color category in 3;

[0077] (6-3) Output the color palette canvas after color filling as the color matching color palette diagram, as shown in Figure 4 shown.

Claims

1. A method for extracting a color palette of a color matching style combined with visual saliency, characterized in that: The method includes the following steps: Step 1: Input the initial color matching style diagram P orin , and obtain the color matching style diagram after preprocessing P scale ; Step 2: includes the following steps: Step 2-1: Use the pre-trained saliency detection network model to perform saliency detection on the color matching style map P scale to obtain the attention map M 0; Step 2-2: Process the attention map with Otsu's method M to obtain a saliency region mask M 1; Step 2-3: Convert the color matching style diagram P scale to the HSV space to obtain the color matching style diagram P hsv ; Step 2-4: Mask with the significant region M 1 pair of color matching style diagrams P hsv Perform region segmentation to obtain the significant region in the color matching style diagram P roi ; Step 3: Extract the color of the detected significant region P roi ; Step 4: Based on the significant regions P roi Obtain the non-significant regions P nroi , and for the non-significant regions P nroi perform color extraction; Step 5: Combine and adjust the color swatches of the significant regions and the colors extracted from the non-significant regions; Step 6: Generate a color-matching color swatch.

2. The method for extracting a color palette of a color matching style combining visual saliency according to claim 1, wherein: In the said step 1, the color matching style map is scaled to P orin by means of bilinear interpolation, and the resolution is scaled to W × H , obtaining the scaled color matching style map P scale .

3. A method for extracting a color palette of a color matching style combined with visual saliency according to claim 1, characterized in that: The said Step 3 includes the following steps: Step 3-1: Perform color clustering on all pixels in the significant region P roi using the K-means algorithm, and set K 1 clustering center, K where 1 is a positive integer; Step 3-2: Convert the pixel values of the K 1 clustering center from the HSV space to the RGB space to obtain the significant region P roi color swatch color list P 1 and the corresponding color swatch pixel point quantity list C 1, where P 1 contains K the pixel values of 1 clustering center, C 1 contains K the number of pixel points corresponding to 1 clustering center, P the pixel values in 1 and C the number of pixel points in 1 correspond.

4. A method for extracting a color palette of a color matching style combined with visual saliency according to claim 3, characterized in that: The said Step 4 includes the following steps: Step 4-1: Invert the significant region mask obtained in Step 2-2 M 1 to obtain a non-significant region mask M 2. Use the non-significant region mask M 2 to perform region segmentation on the color style map P hsv to obtain non-significant regions P nroi ; Step 4-2: For all pixels in the non-significant region P nroi perform clustering using the K-means algorithm, and set K 2 clustering centers, K where 2 is a positive integer; Step 4-3: Convert the pixel values of the two clustering centers obtained by clustering from the HSV space to the RGB space to obtain the color palette color list of the non-significant region K and the corresponding color palette pixel point quantity list P nroi 2, where P 2 contains C the pixel values of the two clustering centers, P and 2 contains K the pixel point quantities corresponding to the two clustering centers, C and the pixel values in 2 K correspond to the pixel point quantities in 2 P C C ​ 5. A method for extracting a color palette of a color matching style combined with visual saliency according to claim 4, characterized in that: In step 5, extract the significant region P roi colors to obtain a color palette color list P 1; Extract the non-significant region P nroi colors to obtain a color palette color list P 2; Based on the color palette color list P 1 and the color palette color list P 2, combine and adjust the colors extracted from the significant region color palette and the non-significant region 6. A method for extracting a color palette of a color matching style combining visual saliency according to claim 5, characterized in that: The said Step 5 includes the following steps: Step 5-1: Measure the proximity of colors using the Euclidean distance of the RGB vectors of two pixel values, and set a distance threshold DIS , DIS greater than 0; Step 5-2: Traverse the color list of the color palette P 1. Search in the color list of the color palette P 2 for pixels whose Euclidean distance from any color in the color list P 1 is less than DIS . If any exist, delete this color from P 1, and merge the number of pixels of this color class in C 1 into the number of pixels of the corresponding color in C 2; Step 5-3: Multiply the list of the number of pixels in the significant region color palette C 1 by the weight W c , W c if it is positive, and then merge it with the list of the number of pixels in the non-significant region color palette C 2 to obtain the list of the number of pixels in the color palette of the overall color matching style map C 3; Merge the significant region color palette color list P 1 and the non-significant region color palette color list P 2 to obtain the color palette color list of the overall color matching style map P 3. After merging, P the pixel values in 3 correspond to C the number of pixels in 3.

7. A method for extracting a color palette of a color matching style combined with visual saliency according to claim 6, characterized in that: In the said step 5-2, the color deleted from P 1 exists in the non-significant area P nroi and is classified as a non-significant color.

8. A method for extracting a color palette of a color matching style combined with visual saliency according to claim 6, characterized in that: The said Step 6 includes the following steps: Step 6-1: Sort the number of pixels in the color palette pixel number list C in 3 from largest to smallest, and correspondingly adjust the color order in the color palette color list P in 3; Step 6-2: Fill the colors in P 3 into the color palette canvas of W out × H out dimensions in strips in sequence. The height of each color strip is the same as that of the canvas, and the proportion of the width of the color strip in the total width of the canvas is set according to C the proportion of the number of pixels of each type of color in 3; Step 6-3: Output the color-filled color swatch canvas as the color-matching color swatch.

Images