A color transfer method based on semantic segmentation and clustering blocks

Abstract

This invention discloses a color transfer method based on semantic segmentation and clustering blocks, belonging to the field of digital image processing technology. The method includes: identifying object blocks in the target and source images using semantic segmentation, proposing a similarity matching algorithm to match the similarity between the source and target images, and implementing color transfer using a clustering block-based method. This invention solves the problem of excessive color distortion in corresponding regions after the overall transfer from the source to the target image in existing color transfer methods, achieving the effect of using different color schemes for different classification regions, effectively addressing the defect of excessive color discrepancies between certain classification regions and reality. Furthermore, by introducing semantic similarity, this invention solves the distortion and lack of personalization issues in traditional color transfer, achieving the effect of customized processing based on the target image.

Description

Technical Field

[0001] This invention relates to a color transfer method based on semantic segmentation and clustering blocks, belonging to the field of digital image processing technology. Background Technology

[0002] Image color transfer technology can significantly alter the visual effect of an image, enabling both pseudo-coloring of grayscale images and effects similar to adding filters to a picture.

[0003] In the past, Li Yanhao (Li Yanhao, Li Zhijiang, Cao Liqin. Research on color transfer algorithm based on neighborhood similarity [C]. 2015:2.) and Qian Xiaoyan (Qian Xiaoyan, Li Min, Han Lei, Wang Bangfeng. A fast color transfer algorithm [J]. Journal of Image and Graphics, 2011, 16(04):613-617.) and others have conducted relevant research and achieved good results, but they all have a common shortcoming: during color transfer, the corresponding regions are only similar in color, and the correspondence between the classification regions in the source image and the classification regions in the target image is not realized.

[0004] Therefore, traditional color transfer methods are prone to distortion of objects. For example, color transferring the sky from an image with a dominant sky to an image with a dominant grassland will result in a green sky. Furthermore, traditional color transfer methods transfer between overall image styles, failing to transfer customized styles. Thus, traditional color transfer methods lack personalization capabilities. Summary of the Invention

[0005] To address the issues that current color transfer algorithms cannot accurately migrate colors from different classification regions in the target image, and that the transferred image suffers from significant color distortion in certain regions compared to the original image, as well as the inability to personalize the color transfer, this invention provides a color transfer method based on semantic segmentation and clustering blocks. The technical solution is as follows:

[0006] The first objective of this invention is to provide a color migration method, comprising:

[0007] Step 1: Perform semantic segmentation on the source image and the target image to obtain the inter-domain classification results of the source image and the target image;

[0008] Step 2: Calculate the correspondence between the domains of the source image and the target image using the inter-domain matching algorithm;

[0009] Step 3: Perform intra-domain clustering on a corresponding region in the source image and the target image, and select the combination with the smallest sum of variance as the intra-domain matching result;

[0010] Step 4: Perform color transfer according to the color transfer algorithm to obtain the processing result of a region within a domain;

[0011] Step 5: Repeat steps 3 and 4 until all inter-domain correspondences have been processed;

[0012] Step 6: Output the result image.

[0013] Optionally, step 2 includes:

[0014] Step 21: Record the final classification number of the target image and the final classification number of the source image;

[0015] Step 22: From the classification heatmap of the target image, construct an m*n similarity matrix R using the classification number of the source image as the row index and the classification number of the target image as the column index, and use a matching algorithm. Here, m is the number of classifications of the source image, n is the number of classifications of the target image, and each element of the matrix represents the similarity of each classification.

[0016] Optionally, step 3 uses the K-Means++ algorithm to cluster the two regions separately, with K being the number of cluster centers and c1, c2, ..., c3 being the set of cluster centers. K The maximum number of iterations is T, and the termination threshold is σ.

[0017] Step 31: Randomly select a sample from the dataset as the initial cluster center;

[0018] Step 32: Calculate the shortest distance between each sample and the existing cluster centers, denoted by D(x), where x represents the sample to be clustered. Calculate the probability that each sample will be selected as the next cluster center: The next cluster center is selected using the roulette wheel method;

[0019] Step 33: Repeat step 32 until a total of K cluster centers are selected;

[0020] Step 34: For each sample x in the dataset i Calculate its distance to the K cluster centers and assign it to the class corresponding to the cluster center with the smallest distance;

[0021] Step 35: For each category c i Recalculate its cluster centers:

[0022]

[0023] Step 36: Repeat steps 34 and 35, terminating when the number of iterations reaches the maximum limit or the cluster centers no longer change;

[0024] Step 37: Calculate the mean values ​​of the R, G, and B components in each cluster domain of the source and target images, using them as feature points for that region. Then, construct a selection matrix and denote it as a matrix of size K*K. For any element in the matrix... ij , (i,j≤K), represents the Euclidean distance d between the feature points of the i-th region in the source image and the feature points of the j-th region in the target image. ij Let source i For the i-th feature point in the source image, target j Let j be the j-th feature point of the source image, then:

[0025] d ij =[source i (R)-target j (R)] 2 +[source i (G)-target j (G)] 2 +[source i (B)-target j (B)] 2

[0026] Step 38: Find the minimum sum of elements in different rows and columns of the matrix to minimize the overall offset within the region, and return the corresponding matrix row and column indices to form the corresponding intra-domain matching results.

[0027] Optionally, step 4 includes:

[0028] Step 41: Transfer the specified regions of the source image and the specified regions of the target image from the RGB space to the Iαβ color space;

[0029] Step 42: Calculate the mean and standard deviation of each cluster for each channel of the source image and the target image;

[0030] Step 43: Remove the corresponding mean from each cluster region of the target image, and scale the remaining part according to the ratio of the standard deviation of the cluster region h in the source image to the standard deviation of the current processing region;

[0031] Step 44: Add the mean of the clustering regions h of the source image;

[0032] Step 45: Calculate a weighted average of the processing results for each clustering region in the target image to obtain a pixel relationship expression between the target image and the source image.

[0033] The pixel value in the αβ space of the target image = (source standard deviation / target standard deviation) * (target pixel value - target mean) +

[0034] Source Mean

[0035] Let μ ι s μ α s μ β s σ represents the mean values ​​of the ι, α, and β channels of the source image, respectively. ι s σ α s σ β s These represent the standard deviations of the ι, α, and β channels of the source image, respectively; μ ι t μ α t μ β t σ represents the mean values ​​of the ι, α, and β channels of the source image, respectively. ι t σ α t σ β t ι, α, and β represent the standard deviations of the ι, α, and β channels of the source image, respectively; ι, α, and β are the ιαβ channel values ​​of the target image, and ι', α', and β' are the ιαβ channel values ​​of the result image.

[0036]

[0037] Step 46: Convert the resulting image from αβ space to RGB space.

[0038] Optionally, the process of transferring an image from the RGB color space to the αβ color space includes:

[0039] RGB => CIEXYZ conversion:

[0040]

[0041] CIEXY => LMS conversion:

[0042]

[0043] LMS => αβ conversion:

[0044]

[0045]

[0046]

[0047] Optionally, step 1 uses a fully convolutional neural network (FCN) for semantic segmentation.

[0048] Optionally, the fully convolutional neural network (FCN) is trained using the Pascal VOC2012 dataset.

[0049] Optionally, the training process augments the image data, and image augmentation techniques include: image size changes, mirroring, inversion, color space changes, and random cropping of the image.

[0050] A second object of the present invention is to provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the color migration method described above.

[0051] A third objective of the present invention is to provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform any of the color migration methods described above.

[0052] The beneficial effects of this invention are:

[0053] This invention classifies the source and target images into regions, and then selects the source image region with the highest similarity for each region in the target image to perform color migration from the source image to the target image. This solves the problem of excessive color distortion in the corresponding regions after the overall migration from the source image to the target image in existing color migration algorithms. It achieves the effect of using different color schemes for different classification regions, effectively solving the defect that the colors of some classification regions differ too much from reality. At the same time, the color contrast between different regions in the migrated image is not too high, avoiding the drawbacks that may occur when performing color migration by region.

[0054] Furthermore, by introducing semantic similarity, this invention solves the problems of distortion and lack of personalization in traditional color transfer, achieving the effect of customized processing based on the target image. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a traditional color migration effect diagram.

[0057] Figure 2 This is a flowchart of the clustering color migration based on semantic segmentation in this invention.

[0058] Figure 3 This is a schematic diagram of the image augmentation after embodiment two of the present invention.

[0059] Figure 4 This is a graph showing the model training loss and ACC index of Embodiment 2 of the present invention.

[0060] Figure 5 This is a semantic segmentation process diagram of Embodiment 2 of the present invention.

[0061] Figure 6 This is a flowchart of the inter-domain matching algorithm of the present invention.

[0062] Figure 7 It is an αβ space diagram.

[0063] Figure 8 This is a color migration effect diagram of semantic segmentation and clustering blocks in Embodiment 2 of the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0065] Example 1:

[0066] This embodiment provides a color migration method; see [link to relevant documentation] Figure 2 The methods include:

[0067] Step 1: Perform semantic segmentation on the source image and the target image to obtain the inter-domain classification results of the source image and the target image;

[0068] Step 2: Calculate the correspondence between the domains of the source image and the target image using the inter-domain matching algorithm;

[0069] Step 3: Perform intra-domain clustering on a corresponding region in the source image and the target image, and select the combination with the smallest sum of variance as the intra-domain matching result;

[0070] Step 4: Perform color transfer according to the color transfer algorithm to obtain the processing result of a region within a domain;

[0071] Step 5: Repeat steps 3 and 4 until all inter-domain correspondences have been processed;

[0072] Step 6: Output the result image.

[0073] Example 2:

[0074] This embodiment provides a color migration method; see [link to relevant documentation] Figure 2 The methods include:

[0075] Step 1: Perform semantic segmentation on the source image and the target image to obtain the inter-domain classification results of the source image and the target image;

[0076] This embodiment employs a fully convolutional neural network (FCN) for semantic segmentation to transform image pixels into pixel categories. Since semantic region annotation and prediction are pixel-level, this embodiment uses color transfer to perform similar semantic operations on two images, ensuring region similarity and preventing image distortion.

[0077] This embodiment uses the publicly available Pascal VOC2012 dataset to train the neural network. This dataset has 21 labeled classes, whose labels are as follows:

[0078] ['background','aeroplane','bicycle','bird','boat','bottle','bus','car','cat','chair','cow','diningtable','dog','horse','motorbike','person','potted plant','sheep','sofa','train','tv / monitor']

[0079] Considering that the amount of data can significantly affect the model's prediction performance, this embodiment augments the image data and reduces the model's dependence on certain attributes to improve the model's generalization ability. The image augmentation techniques used in this embodiment include: image size changes, mirroring, inversion changes, color space changes (brightness, contrast, hue, etc.), and random cropping of the image (center cropping, random cropping).

[0080] This embodiment uses a convolutional neural network to extract image features, then uses a 1×1 convolutional layer to transform the number of channels into the number of categories, and then uses a transposed convolutional layer to transform the height and width of the feature map into the size of the input image.

[0081] After multiple convolutions and pooling operations, the original image (with dimensions H*W) will gradually shrink in size and decrease in resolution. When the image size reaches... When creating a heatmap (the most important high-dimensional feature map), the original image is upsampled to its original size after obtaining the high-dimensional feature map. The final output is 21 heatmaps. Upsampling is used to transform them into the original image size. That is, the maximum numerical description (probability) of each pixel at that pixel position in the 21 images is used as the classification of that pixel.

[0082] To maximize GPU resource efficiency, this embodiment employs transfer learning. Transfer learning is a method that transfers learned model parameters to a new model to aid in its training. By sharing learned model parameters (or knowledge acquired by the model) with the new model, it accelerates and optimizes the learning efficiency, unlike most networks which learn from scratch. This embodiment uses a pre-trained ResNet18 network for transfer learning. Key code snippets of the training process are as follows:

[0083]

[0084] The trained ResNet18 network structure is as follows:

[0085]

[0086]

[0087] The loss function used during training is cross-entropy L:

[0088]

[0089] The above formula is implemented in code as follows:

[0090]

[0091] Cross-entropy is used to calculate the similarity between the processed image and the real image, with the goal of minimizing the cross-entropy.

[0092] Since the CNN is already trained, this embodiment only requires a small amount of training to achieve good results. The ACC of the training and test sets and the loss of the training set are as follows: Figure 4 As shown.

[0093] Step 2: Calculate the correspondence between the domains of the source image and the target image using the inter-domain matching algorithm;

[0094] In step 1, similar semantic segmentation was performed on the source image and the target image, resulting in the classification of the two images and heatmaps of their 21 categories.

[0095] Let m be the number of existing categories in the source image and n be the number of existing categories in the target image. In this embodiment, we need to achieve category-to-category matching based on the semantic similarity between the two images, that is, matching semantically similar regions in the target image with those in the source image to perform color transfer.

[0096] The classification is obtained by matching the recognition results of the source image and the recognition results of the target image. The corresponding value is selected from the heatmap of the target image according to the classification number of the source image to obtain the matching matrix.

[0097] The inter-domain matching algorithm process is as follows:

[0098] Record the final classification number of the target image (using argmax to obtain the category number sequence) as A, and record the final classification number of the source image as B. From the classification heatmap of the target image, use the classification number of the source image as the row index and the classification number of the target image as the column index to obtain an m*n similarity matrix.

[0099] An m*n matrix is ​​constructed according to the matching algorithm, where each matrix element represents the similarity between different categories. Here, softmax processing is performed on each region of the image beforehand (i.e., the aforementioned m*n similarity matrix), and then the average similarity of that region is taken.

[0100] Therefore, the matching algorithm can be simplified to: selecting one element from each row. Requirements: All columns must select elements, and multiple elements are allowed in each column. Let f be the decision function for the selected element matching scheme. Therefore, this embodiment can construct the following objective function:

[0101]

[0102] Here, the p-value is 2, which is the Euclidean distance. The matching domain obtained by performing a global search is the matching scheme.

[0103] The code for region matching is as follows:

[0104]

[0105] Step 3: Perform intra-domain clustering on a corresponding region in the source image and the target image, and select the combination with the smallest sum of variance as the intra-domain matching result;

[0106] Since color migration may result in indistinct texture information and unreasonable color migration within the domain after the source and target images are classified and matched, this embodiment requires intra-domain clustering of all domains that have been classified in order to avoid these problems as much as possible.

[0107] Let A be any region A after the source image is classified, with sample space s1, s2, ..., and B be the corresponding region B of the target image, with sample space t1, t2, ..., which we will temporarily refer to as x1, x2, ... . In this embodiment, the K-Means++ algorithm is used to cluster the two regions respectively. Let K be the number of cluster centers and c1, c2, ..., c KThe maximum number of iterations is T, and the termination threshold is σ.

[0108] Step 1: Randomly select a sample from the dataset as the initial cluster center c1;

[0109] Step 2: Calculate the shortest distance between each sample and the current existing cluster center (i.e., the distance to the nearest cluster center), denoted by D(x); then calculate the probability that each sample will be selected as the next cluster center:

[0110]

[0111] Finally, the next cluster center is selected using the roulette wheel method.

[0112] Step 3: Repeat step 2 until a total of K cluster centers are selected.

[0113] Step 4: For each sample x in the dataset i Calculate its distance to the K cluster centers and assign it to the class corresponding to the cluster center with the smallest distance.

[0114] Step 5: For each category c i Recalculate its cluster centers:

[0115]

[0116] Step 6: Repeat steps 4 and 5. The termination condition is that the number of iterations reaches the maximum limit or the cluster centers no longer change. Here, no longer changing generally means reaching the termination threshold.

[0117] Step 7: Calculate the mean values ​​of the R, G, and B components in each cluster domain of the source and target images. These values ​​are used as feature points for that region. Then, construct a selection matrix and denote it as a matrix of size K*K. For any element in the matrix... ij , (i,j≤K), represents the Euclidean distance d between the feature points of the i-th region in the source image and the feature points of the j-th region in the target image. ij Let source i For the i-th feature point in the source image, target j Let j be the j-th feature point of the source image, then:

[0118] d ij =[source i (R)-target j (R)] 2 +[source i (G)-target j(G)] 2 +[source i (B)-target j (B)] 2

[0119] Step 8: Since for any region within the target image, there must always be a corresponding region within the source image, the matrix needs to find the minimum value of the sum of elements in different rows and columns according to a certain strategy. At this time, the overall offset within the region is the smallest, indicating that the matching effect is good. The corresponding matrix row and column indices are returned to form the corresponding intra-domain matching results. Finally, color migration is performed between the intra-domain matches.

[0120] Since there will be multiple inter-domain correspondences between the source and target images, and each corresponding domain requires an intra-domain matching calculation, it can be predicted that steps 1 to 8 will be executed multiple times. The number of times is the number of correspondences, which is the smaller value of the number of source or target image classifications, and is used as the total number of times the intra-domain matching algorithm is executed.

[0121] Step 4: Perform color transfer according to the color transfer algorithm to obtain the processing result of a region within a domain;

[0122] The color transfer algorithm references the research of Reinhard et al., who proposed a set of color transfer formulas applicable to each color component based on the characteristic that the channels in the αβ color space are not correlated with each other, which effectively achieved color transfer between color images.

[0123] Step 1: Before performing color transfer, the specified regions of the source and target images need to be transferred from the RGB color space to the Iαβ color space. Colors in the RGB space are highly correlated, but after the space transfer, they become orthogonal to each other.

[0124] Converting from RGB to αβ space requires three transformations:

[0125] RGB=>CIEXYZ=>LMS=>ιαβ

[0126] RGB => CIEXYZ conversion:

[0127]

[0128] CIEXY => LMS conversion:

[0129]

[0130] LMS => αβ conversion:

[0131]

[0132]

[0133]

[0134] Step 2: Calculate the mean and standard deviation of each cluster for each channel.

[0135] Step 3: Remove the corresponding mean from each cluster region of the target image, and scale the remaining part according to the ratio of the standard deviation of the cluster region h in the source image to the standard deviation of the current processing region.

[0136] Step 4: Add the mean of the clustering regions h in the source image.

[0137] Step 5: Perform a weighted average of the processing results for each cluster region in the target graph.

[0138] After processing, the pixel relationship between the target image and the source image is expressed as follows: Pixels in the αβ space of the target image = (Source standard deviation / Target standard deviation) * (Target pixel - Target mean) + Source mean

[0139] Let μ ι s μ α s μ β s σ represents the mean values ​​of the ι, α, and β channels of the source image, respectively. ι s σ α s σ β s These represent the standard deviations of the ι, α, and β channels of the source image, respectively; μ ι t μ α t μ β t σ represents the mean values ​​of the ι, α, and β channels of the source image, respectively. ι t σ α t σ β t , , and β represent the standard deviations of the ι, α, and β channels of the source image, respectively; ι, α, and β are the channel values ​​of the target image, and ι', α', and β' are the ιαβ channel values ​​of the result image.

[0140]

[0141] Step 6: Convert the αβ space to the RGB space according to the conversion formula in Step 1.

[0142] ιαβ=>LMS=>RGB

[0143] ιαβ=>LMS conversion:

[0144]

[0145] LMS => RGB conversion:

[0146]

[0147]

[0148] Step 5: Repeat steps 3 and 4 until all inter-domain correspondences have been processed;

[0149] Step 6: Output the result image.

[0150] In this embodiment, two photos are selected, one as the source image and the other as the target image, and the color space of the source image is transferred to the color space of the target image.

[0151] First, using the pre-trained PKL format model, image segmentation is performed, assuming the source image is divided into {s1, s2, ..., s...}. m}, m≤21; the target image is divided into {t1,t2,…,t n}, n≤21, and then according to the inter-domain matching algorithm, the source region and the target region are matched. Each target region has at least one corresponding source region. If necessary, dimensional expansion processing is performed. After completion, in addition to the matching relationship, the coordinates of each point in each region should also be stored for subsequent retrieval. The code is as follows:

[0152]

[0153] Next, intra-domain clustering is performed on the inter-domain matching results of the source and target one by one. After intra-domain clustering, the best matching result is selected according to the above strategy. The code is as follows:

[0154]

[0155] Next, the Reinhard algorithm is used for color transfer between regions within each domain. This preserves some texture information in the target image, but the processing results need to be temporarily stored because only a portion of the image is returned after each processing step. Finally, after each inter-domain region is processed, the complete RGB space of the image is obtained.

[0156] The code for the entire color space conversion process is as follows:

[0157] (lMeanSrc,lStdSrc,aMeanSrc,aStdSrc,bMeanSrc,bStdSrc)=image_stats(sourc e_dealing)

[0158] (lMeanTar,lStdTar,aMeanTar,aStdTar,bMeanTar,bStdTar)=image_stats(targe t_dealing)

[0159] (l,a,b)=cv2.split(target_dealing)

[0160] l-=lMeanTar

[0161] a-=aMeanTar

[0162] b-=bMeanTar

[0163] l=(lStdTar / lStdSrc)*l

[0164] a=(aStdTar / aStdSrc)*a

[0165] b=(bStdTar / bStdSrc)*b

[0166] l+=lMeanSrc

[0167] a+=aMeanSrc

[0168] b+=bMeanSrc

[0169] l=np.clip(l,0,255)

[0170] a=np.clip(a,0,255)

[0171] b=np.clip(b,0,255)

[0172] transfer=cv2.merge([l,a,b])

[0173] transfer=cv2.cvtColor(transfer.astype("uint8"),cv2.COLOR_LAB2BGR)#Color space conversion

[0174] This embodiment classifies the source and target images into regions, and then selects the source image region with the highest similarity for each region in the target image to perform color migration from the source image to the target image. This solves the problem of excessive color distortion in the corresponding regions after the existing color migration algorithm migrates the entire source image to the target image. It achieves the effect of using different color schemes for different classification regions, effectively solving the defect that the colors of some classification regions differ too much from reality. At the same time, the color contrast between the regions in the migrated image is not too high, avoiding the drawbacks that may occur when performing color migration by region.

[0175] Some steps in the embodiments of the present invention can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0176] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A color migration method, characterized in that, The color migration method includes: Step 1: Perform semantic segmentation on the source image and the target image to obtain the inter-domain classification results of the source image and the target image; Step 2: Calculate the correspondence between the domains of the source image and the target image using the inter-domain matching algorithm; Step 3: Perform intra-domain clustering on a corresponding region in the source image and the target image, and select the combination with the smallest sum of variance as the intra-domain matching result; Step 4: Perform color transfer according to the color transfer algorithm to obtain the processing result of a region within a domain; Step 5: Repeat steps 3 and 4 until all inter-domain correspondences have been processed; Step 6: Output the result image; Step 2 includes: Step 21: Record the final classification number of the target image and the final classification number of the source image; Step 22: From the classification heatmap of the target image, construct an m*n similarity matrix R using the classification number of the source image as the row index and the classification number of the target image as the column index, and use a matching algorithm. Here, m is the number of categories of the source image, n is the number of categories of the target image, and each element of the matrix represents the similarity of each category. Step 3 uses the K-Means++ algorithm to cluster the two regions respectively. Let the number of cluster centers be K, and the set of cluster centers be c1, c2, ..., c K The maximum number of iterations is T, and the termination threshold is σ. Step 31: Randomly select a sample from the dataset as the initial cluster center; Step 32: Calculate the shortest distance between each sample and the existing cluster centers, denoted by D(x), where x represents the sample to be clustered. Calculate the probability that each sample will be selected as the next cluster center: The next cluster center is selected using the roulette wheel method; Step 33: Repeat step 32 until a total of K cluster centers are selected; Step 34: For each sample x in the dataset i Calculate its distance to the K cluster centers and assign it to the class corresponding to the cluster center with the smallest distance; Step 35: For each category c i Recalculate its cluster centers: Step 36: Repeat steps 34 and 35, terminating when the number of iterations reaches the maximum limit or the cluster centers no longer change; Step 37: Calculate the mean values ​​of the R, G, and B components in each cluster domain of the source and target images, using them as feature points for that region. Then, construct a selection matrix and denote it as a matrix of size K*K. For any element in the matrix... ij , (i,j≤K), representing the Euclidean distance d between the feature points of the i-th region in the source image and the feature points of the j-th region in the target image. ij Let source i For the i-th feature point in the source image, target j Let j be the j-th feature point of the target image, then: Step 38: Find the minimum sum of elements in different rows and columns of the matrix to minimize the overall offset within the region, and return the corresponding matrix row and column indices to form the corresponding intra-domain matching results; Step 4 includes: Step 41: Transfer the specified regions of the source image and the specified regions of the target image from the RGB space to the Iαβ color space; Step 42: Calculate the mean and standard deviation of each cluster for each channel of the source image and the target image; Step 43: Remove the corresponding mean from each cluster region of the target image, and scale the remaining part according to the ratio of the standard deviation of the cluster region h in the source image to the standard deviation of the current processing region; Step 44: Add the mean of the clustering regions h of the source image; Step 45: Calculate a weighted average of the processing results for each clustering region in the target image to obtain a pixel relationship expression between the target image and the source image. The pixel value in the αβ space of the target image is calculated as follows: (Source standard deviation / Target standard deviation) * (Target pixel value - Target mean) + Source mean. Let μ ι s μ α s μ β s σ represents the mean values ​​of the ι, α, and β channels of the source image, respectively. ι s σ α s σ β s These represent the standard deviations of the ι, α, and β channels of the source image, respectively; μ ι t μ α t μ β t σ represents the mean values ​​of the ι, α, and β channels of the target image, respectively. ι t σ α t σ β t , respectively, represent the standard deviations of the ι, α, and β channels of the target image; ι, α, and β are the ιαβ channel values ​​of the target image, and ι', α', and β' are the ιαβ channel values ​​of the resulting image. Step 46: Convert the resulting image from αβ space to RGB space.

2. The color migration method according to claim 1, characterized in that, The process of transferring an image from the RGB color space to the Iαβ color space includes: RGB => CIEXYZ conversion: CIEXY => LMS conversion: LMS => αβ conversion:

3. The color migration method according to claim 1, characterized in that, Step 1 uses a fully convolutional neural network (FCN) for semantic segmentation.

4. The color migration method according to claim 3, characterized in that, The fully convolutional neural network (FCN) was trained using the Pascal VOC2012 dataset.

5. The color migration method according to claim 4, characterized in that, The training process augments the image data, and the image augmentation techniques include: image size changes, mirroring, inversion, color space changes, and random cropping of the image.

6. An electronic device, comprising: At least one processor; The at least one processor is also connected in communication with a memory, wherein the memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the color migration method according to any one of claims 1-5.

7. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the color migration method according to any one of claims 1-5.

Images