A palette-based image recoloring method and apparatus

Abstract

The application discloses a kind of based on palette's image recoloring method and device, belong to image processing field.Step 1: obtain original RGB color image, set palette color quantity, iteration control variable range;Step 2: initialization palette, coefficient auxiliary matrix, iteration control variable is minimum value, iteration number k=1;Step 3: solve coefficient matrix ω k ;Step 4: update coefficient auxiliary matrix φ k+1 ;Step 5: update palette C k+1 ;Step 6: update iteration control variable β k+1 , if β k+1 It is less than maximum value, then k=k+1, return step 3, otherwise jump to step 7;Step 7: return palette C k+1 And coefficient matrix ω k , based on the palette and coefficient matrix of return original image I is recolored.Based on the smoothing term, sparse term, normalizing term constraint designed in the application, iteration formula is solved coefficient matrix, updates palette color, with better coloring effect.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to an image recoloring method and apparatus based on a color palette. Background Technology

[0002] Image recoloring is a general term for computer technology that refers to recoloring the original image to achieve color restoration or visual enhancement. Image recoloring has many applications in production and daily life, such as colorizing black and white photos to generate color photos; and visualizing depth images and infrared images.

[0003] The most advanced image recoloring method currently available is the palette-based image recoloring method proposed by Chang et al., and its variations. The main idea is to decompose the pixel colors of the original image into a superposition of several different colors, each color having a different weight on each pixel. All the decomposed colors form a palette, and the weights of all pixels form a coefficient matrix. By calculating the palette and coefficient matrix, and modifying the palette colors as needed, a new image can be re-superimposed, achieving the purpose of recoloring. However, regardless of the method, there are always some issues with color bleeding and color difference.

[0004] How to achieve better coloring results and reduce bleeding and color difference is a problem that researchers in related fields are dedicated to studying. Summary of the Invention

[0005] To address the aforementioned issues of color bleeding and color difference, this invention provides an image recoloring method and apparatus based on a color palette. Based on the smoothing term, sparse term, and normalization term constraints designed in this invention, the coefficient matrix is ​​iteratively solved and the color palette colors are updated, resulting in better coloring effects.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] In a first aspect, the present invention proposes an improved image recoloring method based on a color palette, comprising the following steps:

[0008] Step 1. Obtain the original image ,in For RGB color images, each pixel is described using a vector representing the three colors: red, green, and blue, and a color palette is set. Number of colors Iterative control variable minimum value Maximum value .

[0009] Step 2. Process the original image The set of pixels K-means clustering uses the cluster centers of the clustering results as the initial colors of the color palette. Initialize the auxiliary coefficient matrix The auxiliary coefficient matrix has the same size as the coefficient matrix of the palette, which is equivalent to the image size; initialize the iteration control variables. Number of iterations .

[0010] Step 3. Solve for the coefficient matrix by minimizing the following function. , For the number of iterations:

[0011]

[0012] in, Represents the original image pixels, Represents the original image pixels RGB values; Indicates the first In the coefficient matrix at the next iteration, pixels The first color in the corresponding color palette The weight of each color; Indicates the first The palette in the next iteration a variety of colors; This represents the weight of the smoothing term, and its value is the input parameter. Represents pixels closest A collection of pixels, For input parameters, the "distance" between any two pixels is defined as: the distance between pixels. RGB values with pixel coordinates Merge into a 5-dimensional vector As the feature vector of a pixel The Euclidean distance between feature vectors is used as the "distance" between pixels; This represents the weight of a term, and its value is the input parameter. Indicates the first Control variables for each iteration; Indicates the first In the coefficient auxiliary matrix at the next iteration, the pixel The first color in the corresponding color palette Auxiliary variables for each color; The weight of adjacent pixels is represented by the following formula:

[0013]

[0014]

[0015] Step 4. Solve for the coefficient auxiliary matrix by minimizing the following function to obtain... :

[0016]

[0017] in Indicates the first In the nth iteration, pixel p corresponds to the nth color in the palette. Auxiliary variables for each color, Indicates the first In the coefficient matrix of the next iteration, pixels The first color in the corresponding color palette The weight of each color, This represents the weight of the sparse term, and its value is an input parameter. Indicates the first The next iteration controls the variable and the function. Represents the L0 norm, a vector Indicates the first In the next iteration, pixels The vector is composed of the weights of all colors. The analytical expression for the parameters corresponding to the minimum value of the above function is:

[0018]

[0019] Step 5. Update the color palette according to the following formula to obtain... :

[0020]

[0021] in, This indicates the updated color palette. The A variety of colors.

[0022] Step 6. Update the iterative control variables ,like ,but If the above steps are not taken, return to step 3; otherwise, skip to step 7.

[0023] Step 7. Return to the color palette sum coefficient matrix Based on the returned palette and coefficient matrix, the original image is processed. Perform recoloring.

[0024] This method objectively evaluates PSNR and SSIM to be higher than existing advanced methods, and can effectively reduce the problems of color bleeding and color difference, which is of great significance to the field of image recoloring. Attached Figure Description

[0025] Figure 1This is a schematic flowchart of the image recoloring method of the present invention;

[0026] The implementation and functional features of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0027] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0028] Example 1

[0029] This embodiment is performed on a computer-generated dataset, which includes 50 image pairs, totaling 100 images. Each image pair consists of the original image and a recolored image, which are generated by different recoloring algorithms.

[0030] The steps in this embodiment are as follows:

[0031] Step 1. Obtain the original image ,in For RGB color images, each pixel is described using a vector representing the three colors: red, green, and blue, and a color palette is set. Number of colors Iterative control variable minimum value Maximum value Among them, the color palette Number of colors The calculation is as follows:

[0032] Step 1.1: Quantize each color channel of the original image into 16 values, obtaining... The histogram.

[0033] Step 1.2: Select the colors with the highest frequency in sequence until the sum of the selected frequencies reaches a certain percentage of the total. There may be 8 colors to choose from.

[0034] Step 1.3: Use the number of selected colors as a color palette Number of colors .

[0035] Step 2. Process the original image The set of pixels K-means clustering uses the cluster centers of the clustering results as the initial colors of the color palette. Initialize the auxiliary coefficient matrix The auxiliary coefficient matrix has the same size as the coefficient matrix of the palette, which is equivalent to the image size; initialize the iteration control variables. Number of iterations .

[0036] Step 3. Solve for the coefficient matrix by minimizing the following function. , For the number of iterations:

[0037]

[0038] in Represents the original image pixels, Represents the original image pixels RGB values; Indicates the first In the coefficient matrix at the next iteration, pixels The corresponding color palette of the first The weight of each color; Indicates the first The palette in the next iteration a variety of colors; This represents the weight of the smoothing term, and its value is the input parameter. Represents pixels closest A collection of pixels, For input parameters, the "distance" between any two pixels is defined as: the distance between pixels. RGB values with pixel coordinates Merge into a 5-dimensional vector As the feature vector of a pixel The Euclidean distance between feature vectors is used as the "distance" between pixels; This represents the weight of a term, and its value is the input parameter. Indicates the first Control variables for each iteration; Indicates the first In the coefficient auxiliary matrix at the next iteration, the pixel The first color in the corresponding color palette Auxiliary variables for each color; The weight of adjacent pixels is represented by the following formula:

[0039]

[0040]

[0041] in, This represents the weight of a term, and its value is... ; The above formula belongs to the quadratic programming problem.

[0042] Step 4. Solve for the coefficient auxiliary matrix by minimizing the following function to obtain... :

[0043]

[0044] in Indicates the first In the nth iteration, pixel p corresponds to the nth color in the palette. Auxiliary variables for each color, Indicates the first In the coefficient matrix of the next iteration, pixels The first color in the corresponding color palette The weight of each color, This represents the weight of the sparse term, with a value of 1. Indicates the first The next iteration controls the variable and the function. Represents the L0 norm, a vector Indicates the first In the next iteration, pixels The vector is composed of the weights of all colors. The analytical expression for the parameters corresponding to the minimum value of the above function is:

[0045]

[0046] Step 5. Update the color palette according to the following formula to obtain... :

[0047]

[0048] in, This indicates the updated color palette. The The formula above for each color belongs to a quadratic programming problem.

[0049] Step 6. Update the iterative control variables ,like ,but If the above steps are not taken, return to step 3; otherwise, skip to step 7.

[0050] Step 7. Return to the color palette sum coefficient matrix .

[0051] After completing the above steps, a color palette is obtained. The original image, color palette, and recolored image are randomly assigned to one of the 25 volunteers. By modifying the colors in the color palette, the original colors in the color palette are replaced with the target colors at the corresponding pixel positions in the recolored image, so that the modified image is subjectively similar to the recolored image. The modified image is displayed according to the coefficient matrix, and the PSNR and SSIM between the modified image and the recolored image are calculated.

[0052] The formula for calculating PSNR is as follows:

[0053]

[0054] The mean squared error (MSE) represents the maximum possible pixel value of the recolored image.

[0055]

[0056] Where a and b represent the length and width of the image, and X and Y represent the recolored image and the modified image, respectively.

[0057] PSNR measures the degree of distortion of a modified image relative to a recolored image; the higher the value, the lower the degree of distortion.

[0058] The formula for calculating SSIM is as follows:

[0059]

[0060] in, For the modified image and the recolored image, The average pixel value of the modified image. Similarly, for Pixel value standard deviation Similarly, for Pixel value covariance, , , To prevent constants with a denominator of 0.

[0061] SSIM measures the similarity between images; the higher the value, the greater the similarity.

[0062] In comparison, on the same dataset as in Example 1, four state-of-the-art recoloring methods—Chang, Tan, Aksoy, and Zhang—were used for image recoloring, and PSNR and SSIM were recorded. The comparative results are shown in Table 1.

[0063] Table 1. Comparison results of different coloring methods based on the dataset in Example 1.

[0064]

[0065] Referring to Table 1, it can be seen that compared with other advanced methods, the present invention has the highest PSNR and SSIM indices and the best recoloring effect.

[0066] Example 2

[0067] This embodiment is performed on a manually generated dataset, which includes 50 image pairs, totaling 100 images. Each image pair consists of the original image and a recolored image, which were generated by professionals using software such as Photoshop.

[0068] Except for the dataset, the rest of the steps in this embodiment are the same as in embodiment 1, recording PSNR and SSIM.

[0069] Similarly, for comparison, on the same dataset as in Example 2, four current state-of-the-art recoloring methods, namely Chang, Tan, Aksoy, and Zhang, were used for image recoloring, and PSNR and SSIM were recorded. The comparative results with the results of the present invention in Example 2 are shown in Table 2.

[0070] Table 2. Comparison results of different coloring methods based on the dataset in Example 2.

[0071]

[0072] Referring to Table 2, it can be seen that compared with other advanced methods, the present invention has the highest PSNR and SSIM indices and the best recoloring effect.

[0073] Example 3

[0074] Accordingly, the present invention provides an image recoloring apparatus based on a color palette. In this embodiment, the image recoloring apparatus includes:

[0075] The image acquisition module is used to acquire the original RGB color image and set the number of colors in the color palette. Iterative control variable minimum value Maximum value ;

[0076] The initialization module is used to initialize the color palette. Auxiliary matrix of coefficients Iterative control variables Number of iterations ;

[0077] The coefficient matrix solver module is used to solve the coefficient matrix by minimizing the following quadratic programming problem. :

[0078]

[0079] in, Represents the pixels of the original RGB color image. Pixels representing the original RGB color image RGB values; Indicates the first In the coefficient matrix at the next iteration, pixels The first color in the corresponding color palette The weight of each color; Indicates the first The palette in the next iteration a variety of colors; Indicates the weight of the smoothing term; Represents pixels closest A set of pixels; This indicates that the weight is assigned to a single item; Indicates the first Control variables for each iteration; Indicates the first In the coefficient auxiliary matrix at the next iteration, the pixel The first color in the corresponding color palette Auxiliary variables for each color, Indicates the weight of adjacent pixels;

[0080] The coefficient auxiliary moment update module is used to update the coefficient auxiliary matrix. ;

[0081] The palette update module is used to update the palette. ;

[0082] The decision module is used to update the iterative control variables. and judge Is it true? If so, then If the input returns to the coefficient matrix solution module, otherwise it jumps to the result output module.

[0083] The results output module is used to return the color palette. sum coefficient matrix Based on the returned palette and coefficient matrix, the original image is processed. Perform recoloring.

[0084] In one specific implementation, the update formula in the coefficient auxiliary moment update module is:

[0085]

[0086] in, This represents the weight of sparse terms.

[0087] In one specific implementation, the update formula in the palette update module is as follows:

[0088]

[0089] in, This indicates the updated color palette. The A variety of colors.

[0090] In one specific implementation, the formula for calculating the weight of adjacent pixels is as follows:

[0091]

[0092]

[0093] in, Represents adjacent pixels of the original RGB color image. Representing pixels eigenvectors.

[0094] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the description of the method embodiments, and the implementation methods of the remaining modules will not be repeated here. The device embodiments described above are merely illustrative, and some or all of the modules / units can be selected according to actual needs to achieve the purpose of the present invention. Those skilled in the art can understand and implement them without creative effort. The embodiments of the device of the present invention can be applied to any device with data processing capabilities, such as a computer. The device embodiments can be implemented by software, hardware, or a combination of software and hardware. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device that it is located in, reading the corresponding computer program instructions from the non-volatile memory into memory and running them.

[0095] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0096] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0097] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A palette-based image recoloring method, characterized in that, Includes the following steps: Step 1: Obtain the original RGB color image and set the number of colors in the palette. Iterative control variable minimum value Maximum value ; Step 2: Initialize the color palette Auxiliary matrix of coefficients Iterative control variables Number of iterations ; Step 3: Solve for the coefficient matrix by minimizing the following quadratic programming problem. : ； in, Represents the pixels of the original RGB color image. Represents pixels of a raw RGB color image RGB values; Indicates the first In the coefficient matrix at the next iteration, pixels The first color in the corresponding color palette The weight of each color; Indicates the first The palette in the next iteration a variety of colors; Indicates the weight of the smoothing term; Represents pixels closest A set of pixels; This indicates that the weight is assigned to a single item; Indicates the first Control variables for each iteration; Indicates the first In the coefficient auxiliary matrix at the next iteration, the pixel The first color in the corresponding color palette Auxiliary variables for each color, Indicates the weight of adjacent pixels; Step 4: Update the coefficient auxiliary matrix The updated formula is: ； in, Indicates the weight of sparse terms; Step 5: Update the color palette The updated formula is: ； in, This indicates the updated color palette. The a variety of colors; Step 6: Update the iterative control variables ,like ,but Otherwise, return to step 3; Step 7: Return to the color palette sum coefficient matrix Based on the returned palette and coefficient matrix, the original image is processed. Perform recoloring.

2. The image recoloring method based on a palette according to claim 1, characterized in that, The distance between pixels is calculated as follows: the RGB value and pixel coordinates of a pixel are used as the feature vector of the pixel, and the Euclidean distance between the feature vectors of two pixels is used as the distance between the pixels.

3. The image recoloring method based on a palette according to claim 1, characterized in that, The formula for calculating the weight of adjacent pixels is as follows: ； ； in, Represents adjacent pixels of the original RGB color image. Representing pixels eigenvectors.

4. The image recoloring method based on a color palette according to claim 1, characterized in that, The method for initializing the number of colors in the palette is as follows: Step 1.1: Quantize each color channel of the original RGB color image into 16 values, obtaining... Histogram; Step 1.2: Select the colors with the highest frequency in sequence until the sum of the selected frequencies reaches a certain percentage of the total. There may be 8 colors to choose from; Step 1.3: Use the number of selected colors as the number of colors in the palette. .

5. The image recoloring method based on a palette according to claim 1, characterized in that, The method for initializing the color palette is as follows: The pixel set of the original RGB color image is initialized... The K-means clustering method uses the cluster centers of the clustering results as the initial colors of the color palette.

6. A palette-based image recoloring apparatus, characterized in that, include: The image acquisition module is used to acquire the original RGB color image and set the number of colors in the color palette. Iterative control variable minimum value Maximum value ; The initialization module is used to initialize the color palette. Auxiliary matrix of coefficients Iterative control variables Number of iterations ; The coefficient matrix solver module is used to solve the coefficient matrix by minimizing the following quadratic programming problem. : ； in, Represents the pixels of the original RGB color image. Represents pixels of a raw RGB color image RGB values; Indicates the first In the coefficient matrix at the next iteration, pixels The first color in the corresponding color palette The weight of each color; Indicates the first The palette in the next iteration a variety of colors; Indicates the weight of the smoothing term; Represents pixels closest A set of pixels; This indicates that the weight is assigned to a single item; Indicates the first Control variables for each iteration; Indicates the first In the coefficient auxiliary matrix at the next iteration, the pixel The first color in the corresponding color palette Auxiliary variables for each color, Indicates the weight of adjacent pixels; The coefficient auxiliary moment update module is used to update the coefficient auxiliary matrix. The updated formula is: ； in, Indicates the weight of sparse terms; The palette update module is used to update the palette. The updated formula is: ； in, This indicates the updated color palette. The a variety of colors; The decision module is used to update the iterative control variables. and judge Is it true? If so, then If the input returns to the coefficient matrix solution module, otherwise it jumps to the result output module. The results output module is used to return the color palette. sum coefficient matrix Based on the returned palette and coefficient matrix, the original image is processed. Perform recoloring.

Images