Image hue migration method based on identification photo

Abstract

The invention discloses an image hue migration method based on an identification photo, and relates to the technical field of identification photo hue migration, and the method comprises the following steps: S1, data preparation: preparing model training data, including an input picture of a model and an output picture of the model; s2, model training: a generative model is trained according to the prepared data, model input is an original image to be subjected to color and hue migration, and model output is a result image corresponding to the original image after color and hue migration; obtaining a migrated result graph output by the model corresponding to the migrated result graph; according to the method, target hue migration can be carried out on identification photo pictures of different photographing scenes, the texture features of the to-be-migrated picture can be well reserved through hue migration, a good hue migration effect can be achieved, and the problem of artifacts or original picture structure change can be avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of ID photo tone migration, and in particular to an image tone migration method based on ID photos. Background Art

[0002] Nowadays, ID photos are required in many scenarios, but each user's own photography conditions are different (lighting, camera parameters, etc.). However, in some special scenarios, there are certain requirements for these ID photos: such as tone, lighting, color, etc., and the photos taken by users themselves cannot meet these requirements. In order to meet these requirements, manual photo editing is required, which will consume a lot of manpower and time.

[0003] Therefore, there are a few methods that can transfer the color tone of ID photos. There are two main solutions for the existing image color and tone transfer technology: 1. Color transfer based on the calculation of color values ​​of traditional image algorithms. This method calculates the color value (saturation / brightness / contrast) of the target color image.

[0004] / LAB / HSV / YUV, etc. mean, variance, maximum, minimum, histogram), and then calculate the relevant color values ​​of the original image, and adjust the relevant color values ​​of the original image to be consistent with the target color values ​​to achieve the effect of color and hue migration; 2. It is based on deep learning to train a generative color migration model. This method can avoid the drawbacks of traditional image color migration and achieve the effect of being consistent with the target color image. Therefore, the present invention is aimed at the problems existing in the above-mentioned technologies.

[0005] Although the above two existing methods can play a certain role in image tone migration, they have the following disadvantages: 1. The color values ​​calculated by this traditional method are limited, and the target color image cannot represent all the color values ​​that need color tone migration. Therefore, the adjusted image effect is different from the target effect, and there will also be color distortion. It cannot be applied to all original images under different lighting and different shooting scenes, resulting in weak generalization ability; 2. The generated image will also have artifacts or modify the structural content of the original image.

[0006] To this end, we designed an image tone migration method based on ID photos to solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to provide an image tone migration method based on ID photos, which can not only uniformly convert original images in different shooting scenes into target colors and tones while ensuring the effect, but also avoid the occurrence of artifacts or changes in the original image structure, so as to solve the problems raised in the above background technology.

[0008] In order to solve the above technical problems, the present invention provides a method for image tone migration based on ID photos, comprising the following steps:

[0009] S1. Data preparation: Prepare the data for model training, including the input image and output image of the model;

[0010] S2. Model training: A generative model is trained based on the prepared data. The model input is the original image to be subjected to color and tone migration, and the model output is the result image after the color and tone migration corresponding to the original image.

[0011] S3, CMM matrix preparation: first input the image to be migrated into the generative model to obtain the corresponding model output result image after migration; then synchronously divide the image to be migrated and the model result image into blocks to obtain a number of paired blocks that correspond to each other; then calculate the CMM matrix for each pair of blocks; according to the above calculated CMM matrix, recalculate the result image block after migration corresponding to the image block to be migrated. Note that the model's tone migration result image is not used directly here because the generative model will have the problem of changing the texture structure of the original image;

[0012] S4, block stitching: stitch the result image blocks of the calculated CMM matrix.

[0013] Furthermore, a smoothing processing module is also included: since there will be obvious gap marks at the splicing points, this step is used to smooth these marks.

[0014] Furthermore, the data preparation also includes: annotated data, which includes obtaining a large number of RGB images of the to-be-migrated tones in different photographing scenes, and then adjusting them through professional annotation software so that the tones reach the target effect, recording the adjusted images as target images, and matching the images in different photographing scenes with the annotated target images one by one as training data for the model.

[0015] Furthermore, the model training also includes: constructing a generative GAN network, taking the image to be migrated as the model input and the target image as the model output, performing model training and optimization, and finally obtaining a generative model that can perform tone migration. However, the model may generate artifacts or change the content structure of the original image. The following steps are required to avoid the occurrence of the above problems.

[0016] Furthermore, the CMM matrix preparation specifically includes obtaining a picture of the to-be-migrated tone, which is recorded as image, and then inputting the image into the above-mentioned trained generative model to obtain a tone migration result image of the model, which is recorded as model_image, and then performing the following operations on image and model_image:

[0017] Chunking: This step uses chunking operations to prepare for the CMM matrix in the following text. The reason for not directly using the image to be migrated and the model image for CMM matrix calculation is that the CMM matrix blocks are too small and the images are too large to map all color values. Therefore, reducing the mapped image range can have an obvious effect. Specific steps: Obtain the width and height of the image, denoted as w and h respectively (note: the width and height of both image and model_image are w and h), set the chunk size block_size, and the general rule for setting block_size is to set it to 2 n , where n is an integer and 3 < n < 7. The size of block_size can be controlled by adjusting n according to the actual usage effect. According to the size of block_size, divide image and model_image into blocks of size col * row respectively, where col is the number of columns divided and row is the number of rows divided. The calculation method is as follows:

[0018] col = int(w / block_size) + 1;

[0019] row = int(h / block_size) + 1;

[0020] Note: The int(*) operation directly takes the integer part of *, not rounding to the nearest integer operation;

[0021] CMM matrix calculation: Initialize an empty set Mats to store the matrix. Through the chunking operation, the block set of the image after chunking is denoted as image_blockes, and the block set of the model_image after chunking is denoted as model_image_blockes. The lengths of both sets are col * row. The specific information of the set elements is as follows (where i ∈ [0, col * row - 1]), and then calculate the CMM block matrix through the following steps (i, ii, iii...):

[0022] image_blockes = [image_blocke0, image_blocke1,....image_blocke i

[0023] model_image_blockes = [model_image_blocke0,

[0024] model_image_blocke1,....,model_image_blocke i

[0025] ​​Step i. Synchronously traverse the image_blockes and model_image_blockes collections, and record the elements in the traversed image_blocks as image_block i (where i∈[0,col*row-1]), the element in model_image_blockes is denoted as model_image_block i , image_block i with model_image_block i The shape and size of i , b_h i , 3).

[0026] Step ii. Image_block i 、model_image_blocke i Convert its shape to (b_w i *b_h i ,3), image_block i 、model_image_blocke i The results after the transformation are recorded as A i , b i ;

[0027] A i , b i Construct a linear equation system in the form of Ax=b, and use the least squares method to solve the vector x, where x is image_block i Hue conversion to model_image_block i The CMM correction matrix of shape (3, 3) is placed in the i-th position of the set Mats;

[0028] CMM matrix application: Through the above operations, the CMM matrix set Mats of the image block to be migrated to the target tone image block has been obtained. Now initialize an empty set result_blocks to store the tone migration result image blocks after the set image_blockes applies the matrix set Mats. The matrix application operation is as follows:

[0029] Step i. Synchronously traverse the image_blockes and Mats collections, and record the traversed elements as image_block i 、M i , image_block i The shape and size of (b_w i ,b_hi ,3), image_block i The shape is converted to (b_w i *b_h i , 3) The result after that is recorded as A i ;

[0030] Step ii. i With M i Perform matrix multiplication to obtain a matrix with shape (b_w i *b_h i , 3) is denoted as R i , and then the vector matrix R i The shape is converted to (b_w i ,b_h i ,3), after the shape conversion, R i The values ​​in are truncated to [0,255]. The truncation method is: values ​​less than 0 are directly set to 0, and values ​​greater than 255 are set to 255. After matrix multiplication, shape conversion, and value truncation, R i That is image_block i The corresponding result image block after tone migration is placed in the i-th position of the set result_blocks.

[0031] Furthermore, the splicing block also includes: splicing the model obtained by the GAN network in the order of elements from left to right and from top to bottom into a result image corrected by the CMM matrix, recorded as result_image, where each row of result_image consists of col elements and each column of row elements, and the width and height are consistent with the image to be migrated, which is (w, h, 3).

[0032] Furthermore, the smoothing processing module removes traces at the splicing of the spliced ​​images, initializes three empty area sets result_area, and then performs the following steps: m pixel values ​​are expanded to the left and right or up and down according to the splicing position, m is set to 3-5, and an area is obtained. This area in the result image result_image is first Gaussian blurred and then image sharpened. The final result_image effect is the tone migration result image, which retains the texture features of the image to be migrated and can achieve a good tone migration effect.

[0033] Compared with the prior art, the beneficial effect of the present invention is that the image tone migration method based on ID photos can migrate the target tone of ID photos in different shooting scenes. The tone migration can not only well preserve the texture characteristics of the image to be migrated, but also achieve a good tone migration effect, and avoid the problem of artifacts or changes in the original image structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0035] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0036] In the description of the present invention, it should be understood that if the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like are involved, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0037] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", and "set" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection. It can be a mechanical connection or an electrical connection. It can be directly connected or indirectly connected through an intermediate medium. It can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0038] See also Figure 1The present invention provides a technical solution: a method for image tone migration based on a certificate photo, comprising the following steps:

[0039] S1. Data preparation: Prepare the data for model training, including the input image and output image of the model;

[0040] S2. Model training: A generative model is trained based on the prepared data. The model input is the original image to be subjected to color and tone migration, and the model output is the result image after the color and tone migration corresponding to the original image.

[0041] S3, CMM matrix preparation: first input the image to be migrated into the generative model to obtain the corresponding model output result image after migration; then synchronously divide the image to be migrated and the model result image into blocks to obtain a number of paired blocks that correspond to each other; then calculate the CMM matrix for each pair of blocks; according to the above calculated CMM matrix, recalculate the result image block after migration corresponding to the image block to be migrated. Note that the model's tone migration result image is not used directly here because the generative model will have the problem of changing the texture structure of the original image;

[0042] S4, block stitching: stitch the result image blocks of the calculated CMM matrix.

[0043] In specific implementation, the image tone migration method based on ID photos can migrate the target tone of ID photos in different photo scenes. The tone migration can not only well preserve the texture features of the image to be migrated, but also achieve a good tone migration effect, and avoid the problem of artifacts or changes in the original image structure.

[0044] See also Figure 1 As shown, a smoothing processing module is also included: since there will be obvious gap marks at the joints, this step is used to smooth these marks.

[0045] See also Figure 1 ,Data preparation also includes: ,labeled data, which includes obtaining a large number of RGB images of the to-be-migrated tones in ,different shooting scenes, and then adjusting them through professional labeling ,software to achieve the target tones, recording the adjusted images as ,target images, and matching the images in different shooting scenes with the labeled ,target images one by one as training data for the model.

[0046] See also Figure 1, Model training also includes: constructing a generative GAN network, using the image to be transferred as the model input and the target image as the model output, training and optimizing the model, and finally obtaining a generative model that can perform tone transfer. However, this model may have problems such as generating artifacts or changing the content structure of the original image. The following steps are needed to avoid the occurrence of the above problems.

[0047] Refer to Figure 1 , The production of the CMM matrix specifically also includes obtaining the image of the tone to be transferred, denoted as image, then inputting image into the above-trained generative model to obtain the tone transfer result image of the model, denoted as model_image, and then performing the following operations on image and model_image:

[0048] Chunking: This step uses chunking operations to prepare for the CMM matrix below. The reason for not directly using the image to be transferred and the model image to calculate the CMM matrix is that the CMM matrix blocks are too small and the image is too large to map all color values. Therefore, reducing the mapped image range can significantly improve the effect. Specific steps: Obtain the width and height of the image image, denoted as w and h respectively (note: the width and height of image and model_image are both w and h), set the chunk size block_size, and the general rule for setting block_size is to set it to 2 n , where n is an integer and 3 < n < 7. The size of block_size can be controlled by adjusting n according to the actual usage effect. According to the size of block_size, image and model_image are respectively divided into col*row blocks, col is the number of divided columns, and row is the number of divided rows. The calculation method is as follows:

[0049] col = int(w / block_size) + 1;

[0050] row = int(h / block_size) + 1;

[0051] Note: The int(*) operation directly takes the integer of *, not rounding to the nearest integer operation;

[0052] CMM matrix calculation: Initialize an empty set Mats to store the matrix. Through the chunking operation, the set of image blocks after chunking image is denoted as image_blockes, and the set of model_image blocks after chunking model_image is denoted as model_image_blockes. The lengths of both sets are col*row. The specific information of the set elements is as follows (where i ∈ [0, col*row - 1]), and then the CMM block matrix is calculated through the following steps (i, ii, iii...):

[0053] image_blockes=[image_blocke0,image_blocke1,....image_blocke i ]

[0054] model_image_blockes=[model_image_blocke0,

[0055] model_image_blocke1,....,model_image_blocke i ]

[0056] Step i. Synchronously traverse the image_blockes and model_image_blockes collections, and record the elements in the traversed image_blocks as image_block i (where i∈[0,col*row-1]), the element in model_image_blockes is denoted as model_image_block i , image_block i with model_image_block i The shape and size of i , b_h i , 3).

[0057] Step ii. Image_block i 、model_image_blocke i Convert its shape to (b_w i *b_h i ,3), image_block i 、model_image_blocke i The results after the transformation are recorded as A i , b i ;

[0058] A i , b i Construct a linear equation system in the form of Ax=b, and use the least squares method to solve the vector x, where x is image_block i Hue conversion to model_image_block i The CMM correction matrix of shape (3, 3) is placed in the i-th position of the set Mats;

[0059] CMM matrix application: Through the above operations, the CMM matrix set Mats of the image block to be migrated to the target tone image block has been obtained. Now initialize an empty set result_blocks to store the tone migration result image blocks after the set image_blockes applies the matrix set Mats. The matrix application operation is as follows:

[0060] Step i. Synchronously traverse the image_blockes and Mats collections, and record the traversed elements as image_block i 、M i , image_block i The shape and size of (b_w i ,b_h i ,3), image_block i The shape is converted to (b_w i *b_h i , 3) The result after that is recorded as A i ;

[0061] Step ii. i With M i Perform matrix multiplication to obtain a matrix with shape (b_w i *b_h i , 3) is denoted as R i , and then convert the shape of the vector matrix Ri to (b_w i ,b_h i ,3), after the shape conversion, R i The values ​​in are truncated to [0,255]. The truncation method is: values ​​less than 0 are directly set to 0, and values ​​greater than 255 are set to 255. After matrix multiplication, shape conversion, and value truncation, R i That is image_block i The corresponding result image block after tone migration is placed in the i-th position of the set result_blocks.

[0062] See also Figure 1, The splicing block also includes: splicing the model obtained by the GAN network from left to right and from top to bottom in the order of elements into a CMM matrix correction result image, recorded as result_image, each row of result_image consists of col elements and each column of row elements, and the width and height are consistent with the image to be migrated image (w,h,3), the smoothing processing module removes the traces of the splicing at the splicing point after the splicing image, initializes three empty area sets result_area, and then performs the following steps: expand m pixel values ​​to the left or right or up and down according to the splicing position, (m is generally set to 3 to 5), and obtain an area, and Gaussian blur this area in the result image result_image and then perform image sharpening processing. The final result_image effect is the tone migration result image, which not only retains the texture characteristics of the image to be migrated, but also can achieve a good tone migration effect. In addition, (b_w i ,b_h i , 3), (w,h,3) where 3 represents the RGB channels of the image.

[0063] In addition, photo smoothing can be applied to image processing, computer vision, photography and other fields. For example, in photography, smoothing can reduce noise and spots in photos, improve the clarity and beauty of photos; in image processing, smoothing can be used as part of image preprocessing to provide better input data for subsequent image analysis and recognition;

[0064] The main methods are: Mean filtering: Principle: replace the value of a pixel by calculating the average value of a pixel in the image and its surrounding pixels; Features: simple and easy, but may cause loss of image details;

[0065] Median filter: Principle: Sort the values ​​of a certain pixel and its surrounding pixels in the image, and then take the middle value as the new value of the pixel; Features: It has a good suppression effect on salt and pepper noise (i.e. randomly appearing white or black pixels), and can better preserve the edge information of the image;

[0066] Gaussian filtering: Principle: Perform weighted smoothing on the image based on the shape and standard deviation of the Gaussian function; the closer the pixel is to the center pixel, the greater its weight; Features: It can better preserve the overall characteristics of the image and make the image smoother;

[0067] Bilateral filtering: Principle: Combine information in the spatial domain and the intensity domain to smooth the image; While smoothing the image, it can better preserve the edge and detail information of the image; Features: Suitable for scenes that need to preserve both image smoothness and edge information; Choose the appropriate filtering method: Choose the appropriate filtering method according to the specific situation and processing requirements of the photo. For example, for photos containing a lot of noise, you can choose median filtering or Gaussian filtering; for photos that need to preserve edge information, you can choose bilateral filtering;

[0068] Control the filter strength: Too strong a filter strength may cause loss of image details and blur. Therefore, the filter strength needs to be properly controlled during the processing to achieve the best visual effect;

[0069] Pay attention to color fidelity: When smoothing, you need to maintain the color fidelity of the photo as much as possible to avoid color distortion or deviation.

[0070] In addition, the present invention can also be applied to some real-life situations where a mobile phone is used to take photos of identity cards, residence permits, bank cards, etc. and upload them to a system to generate scanned file formats, which can save the step of using a printer and is more convenient and concise. It can also be used for distortion correction of other images containing quadrilateral objects.

[0071] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for image tone migration based on ID photos, characterized in that: It includes the following steps: S1. Data preparation: Prepare the data for model training, including: the input pictures of the model and the output pictures of the model; S2. Model training: Train a generative model according to the prepared data. The input of the model is the original picture to be color and tone transferred, and the output of the model is the result picture after color and tone transfer corresponding to the original picture; S3. CMM matrix production: First, input the picture to be transferred into the generative model to obtain the transferred result picture of the corresponding model output; then synchronously divide the picture to be transferred and the model result picture into blocks to obtain several pairs of corresponding blocks; then calculate the CMM matrix for each pair of blocks; S4. Stitching blocks: Stitch the result picture blocks of the calculated CMM matrix.

2. The method for image tone migration based on ID photo as claimed in claim 1, characterized in that: It also includes a smoothing processing module: Since there will be obvious gap traces at the stitching places, the function of this step is to smooth these traces.

3. The method for image tone migration based on ID photo as claimed in claim 1, characterized in that: The data preparation also includes: annotating data, which includes obtaining a large number of RGB pictures of the tone to be transferred under different photographing scenarios, and then adjusting them through professional annotation software to make their tones reach the target effect. The adjusted pictures are recorded as target pictures, and the pictures under different photographing scenarios are corresponded to the annotated target pictures one by one as the training data of the model.

4. The method for image tone migration based on ID photo as claimed in claim 1, characterized in that: The model training also includes: constructing a generative GAN network, using the picture to be transferred as the model input and the target picture as the model output, performing model training and optimization, and finally obtaining a generative model that can perform tone transfer.

5. The method for image tone migration based on ID photo as claimed in claim 4, characterized in that: The CMM matrix production specifically further includes obtaining the picture of the tone to be transferred, recording it as image, then inputting image into the GAN network to obtain the tone transfer result picture of the model, recorded as model_image, and then operating on image and model_image.

6. The method for image tone migration based on ID photo as claimed in claim 5, characterized in that: The operation on image and model_image includes: obtaining the width w and height h of the image picture, and setting the block size in blocks; the setting rule of block_size is 2n, n is an integer, and 3 < n < 7. According to the size of block_size, image and model_image are respectively divided into col * row blocks, col is the number of divided columns, and row is the number of divided rows; where, col = int(w / block_size) + 1; col = int(w / block_size) + 1.

7. The method for image tone migration based on ID photo as claimed in claim 6, characterized in that: The CMM matrix calculation includes initializing an empty set Mats for storing the matrix. Through the block operation, the block set of the divided image is recorded as image_blockes, and the block set of the divided model_image is recorded as model_image_blockes. The lengths of the sets are both col * row.

8. The method for image tone migration based on ID photo as claimed in claim 5, characterized in that: The CMM block matrix is obtained through the CMM matrix calculation. The CMM block matrix includes: image_blockes=[image_blocke0,image_blocke1,....image_blocke i ] model_image_blockes = [model_image_blocke0, model_image_blocke1,....,model_image_blocke i ]。 9. The method for image tone migration based on ID photo as claimed in claim 5, characterized in that: The splicing block also includes: splicing the model obtained by the GAN network in the order of elements from left to right and from top to bottom into a CMM matrix corrected result image, recorded as result_image.

10. The method for image tone migration based on ID photo as claimed in claim 2, characterized in that: The smoothing processing module removes traces at the splicing locations after the spliced ​​images are joined, initializes three empty area sets result_area, and then executes the steps.

Images