Method and device for intelligently changing ID photos

Abstract

The present invention relates to the technical field of image data processing, and more specifically to a method and device for intelligently changing clothing for ID photos. The method comprises: performing a cleaning operation on clothing and non-facial skin areas of a user image, and synchronously updating a corresponding mask image; calculating neck key points based on the user image after cleaning the clothing and non-facial skin areas and the mask image; using the neck key points, mapping a neck template to the corresponding area of ​​the user image; performing replacement and detail processing to obtain a regenerated neck user image; determining the position coordinates of the clothing to be replaced on the regenerated neck user image; and fusing the position coordinates with the regenerated neck user image to generate a final clothing change result image. The present invention can optimize the final clothing change effect of user-uploaded photos in which the neck portion is obscured by turtlenecks or scarves.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a method and device for intelligently changing ID photos. Background Art

[0002] ID photos are official photographs used for various documents, such as ID cards, passports, driver's licenses, and student ID cards. They must not only clearly depict the individual's features but also meet specific clothing requirements for their intended use. For example, standard ID photos typically require a dark top with a noticeable collar; uniforms or suspenders are to be avoided. For standard passport photos, white clothing with a light blue background is generally recommended; for other colored clothing, a white background is recommended.

[0003] However, temporarily preparing ID photos that meet these dress requirements often means that multiple offline photo shoots are required. This process is not only time-consuming and cumbersome, but also expensive. Traditional methods rely on professional image processing software to perform post-processing of photos. This processing method requires personnel with professional skills and operational capabilities to complete. Existing intelligent dressing technologies often fail to fully consider authenticity factors, such as skin color, gender, etc. when performing image processing, resulting in the final dressing effect not appearing realistic enough. In addition, the current mainstream ID photo intelligent dressing technology often does not produce satisfactory dressing results when processing photos uploaded by users where the neck is obscured by high-necked clothing or scarves, and there is still much room for improvement. Summary of the Invention

[0004] The mainstream smart dressing technology for ID photos in the existing technology does not produce good dressing effects when processing photos uploaded by users in which the neck part is obscured by high-necked clothing or scarves. The present invention provides a smart dressing processing method and device for ID photos, which optimize the final dressing effects of photos uploaded by users in which the neck part is obscured by high-necked clothing or scarves.

[0005] To achieve the above objectives, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a method for intelligently changing ID photos, comprising:

[0007] Acquire a user image, and perform a cleaning operation on clothing and non-face skin areas of the user image to obtain a user image with the clothing and non-face skin areas cleaned and a corresponding portrait parsing mask image;

[0008] Calculating a set of neck key points based on the user image with clothing and non-face skin areas removed and the corresponding portrait parsing mask image;

[0009] Mapping the neck template to a corresponding area of ​​the user image after removing clothing and non-face skin areas based on the neck key point set, to obtain a user image after regenerating the neck;

[0010] The position coordinates of the clothing to be replaced are determined on the user image after the neck is regenerated, and the clothing to be replaced is fused with the user image after the neck is regenerated to obtain a final clothing change result image.

[0011] In a second aspect, the present invention provides an ID photo intelligent dressing processing device, comprising:

[0012] An image cleaning unit is configured to obtain a user image, perform a cleaning operation on clothing and non-face skin areas of the user image, and obtain a user image with the clothing and non-face skin areas cleaned and a corresponding portrait parsing mask image;

[0013] A neck key point calculation module, configured to calculate a set of neck key points based on the user image with clothing and non-face skin areas removed and a corresponding portrait parsing mask image;

[0014] A neck generation module is configured to map a neck template to a corresponding area of ​​the user image after removing clothing and non-face skin areas based on the set of neck key points, thereby obtaining a user image after regenerating the neck;

[0015] The clothing replacement module is used to determine the position coordinates of the clothing to be replaced on the user image after the neck is regenerated, and to fuse the clothing to be replaced with the user image after the neck is regenerated to obtain a final clothing replacement result image.

[0016] In a third aspect, an embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0017] The memory is used to store programs;

[0018] The processor executes the program to implement the method described above.

[0019] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0020] In a fifth aspect, embodiments of the present invention further provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0021] Compared with the existing technology, the present invention has the following advantages: the method for intelligently changing ID photos provided by the present invention does not require users to have professional image processing skills or rely on complex image processing software. Users only need to upload the ID photo to be changed and select the image of the favorite clothing, and the system can automatically complete the changing process, realizing intelligent operation.

[0022] While catering to the personalized needs of ID photos for different occasions, the present invention ensures that the photos after changing clothes are natural and realistic, and fully considers key factors such as skin color and gender to optimize the overall quality of the photos after changing clothes. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 This is an overall flow chart of the ID photo intelligent replacement processing method of this embodiment;

[0025] Figure 2 It is a diagram that uses a mask image to visualize the specific operation of step 100;

[0026] Figure 3 It is a diagram that uses a mask image to visualize the specific operation of step 300;

[0027] Figure 4 It is a diagram of the neck template and the positions of the key points on the left and right sides of the neck;

[0028] Figure 5 This is a diagram showing the overall dressing effect of this embodiment;

[0029] Figure 6 This is another overall flow chart of the ID photo intelligent replacement processing method of this embodiment. DETAILED DESCRIPTION

[0030] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0031] Example:

[0032] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] See also Figures 1 to 5 The embodiment of the present invention provides a method for intelligently changing ID photos, which may include the following steps:

[0034] Step 100: Obtain a user image, perform a cleaning operation on the clothing and non-face skin areas of the user image, and obtain a user image with the clothing and non-face skin areas cleaned and a corresponding portrait parsing mask image. Figure 2 , Figure 2 This embodiment uses a mask image to visually display the specific operations of step 100.

[0035] Wherein, before the cleaning operation of the clothing and non-face skin area of ​​the user image is performed, the method further includes: performing face detection on the user image Image0 to obtain a face detection result; performing face key point detection on the face detection result to obtain a face key point set P0; performing a screening operation on the face key point set P0 to obtain a face outer contour point set PL0. For example, a detection model containing 106 key points is adopted, wherein the face key point set numbered 0 to 32 is identified as a face outer contour point set; Gaussian smoothing and cubic curve fitting are performed on the face outer contour point set PL0 in sequence to obtain a smoothed and dense face contour point set PL1; performing a face segmentation operation on the user image Image0 according to the face key point set P0 to obtain a face mask M0 marking the face area, performing morphological dilation and corrosion on the face mask M0, and performing a face mask M0 according to the smoothed and dense face contour point set. The face contour point set PL1 is collected to perform detailed optimization processing on the contour edge of the face mask image M0; the face posture angle of the face key point set P0 is calculated and the face correction operation is performed to obtain the corrected face key point set P1, the corrected user image Image1 and the corrected face mask image M1, wherein, after the correction matrix is ​​obtained by calculating the face posture angle, it can be directly applied to the face key point set P0, the user image Image0, and the face mask image M0 to obtain the corrected face key point set P1, the corrected user image Image1 and the corrected face mask image M1. Exemplarily, this operation can also be directly implemented using the existing functions in the opencv library; portrait analysis is performed on the corrected user image Image1 to obtain the portrait analysis mask image M2. Exemplarily, the adopted portrait analysis model can perform in-depth analysis on the input image, accurately distinguish and mark the four main semantic categories such as background, clothing, hair and skin.

[0036] The above steps are used to complete face detection and key point detection, facial contour processing, face segmentation and mask creation, face correction and portrait analysis operations.

[0037] The cleaning operation of the clothing and non-face skin area of ​​the user image specifically includes:

[0038] Based on the clothes semantic area identified in the portrait parsing mask image M2, a cleaning operation is performed on the clothes area corresponding to the user image Image1. At the same time, the pixel values ​​of the clothes semantic area of ​​the portrait parsing mask image M2 are updated to obtain the user image Image2 after the clothes are cleaned and the portrait parsing mask image M3; exemplarily, the cleaning operation involves adjusting the pixel values ​​of the corresponding area in the corrected user image Image1 to be consistent with the background, and using morphological dilation and erosion processing technology to accurately optimize the contour edges to obtain the user image Image2 after the clothes are cleaned; then, the pixel values ​​of the clothes semantic area of ​​the portrait parsing mask image M2 are updated to pixel values ​​consistent with the background semantic area to obtain the portrait parsing mask image M3 after the clothes are cleaned, for example, the pixel values ​​corresponding to the background semantics are 0.

[0039] Based on the skin semantic areas identified in the corrected face mask M1 and the portrait parsing mask M3, a cleaning operation is performed on the non-face skin areas of the user image Image2. At the same time, the pixel values ​​corresponding to the non-face skin areas in the portrait parsing mask M3 are updated to obtain the user image Image3 and the portrait parsing mask M4 from which the non-face skin areas have been cleaned. Exemplarily, after the user image Image2 after cleaning the clothes is cleaned from the skin portion of the non-face area, the contour edges are precisely optimized using morphological dilation and erosion processing techniques to obtain the user image Image3 from which the skin portion of the non-face area has been cleaned; then, the pixel values ​​corresponding to the skin portion of the non-face area in the portrait parsing mask M3 after cleaning the clothes are updated to pixel values ​​consistent with the background semantic areas to obtain the portrait parsing mask M4 from which the skin portion of the non-face area has been cleaned.

[0040] The above steps are implemented to complete the cleaning of clothes and non-human face skin.

[0041] Step 200: Based on the user image with clothes and non-face skin areas removed and the corresponding portrait parsing mask image, a set of neck key points is calculated.

[0042] Among them, the neck key point set is calculated based on the user image with clothes and non-face skin areas removed and the corresponding portrait parsing mask image, specifically including: according to the corrected face key point set P1, the horizontal coordinate, maximum horizontal coordinate and minimum horizontal coordinate of the neck center point are calculated; the neck width is determined by multiplying the difference between the maximum horizontal coordinate and the minimum horizontal coordinate by a preset threshold. For example, different thresholds K can be set for gender according to user needs. For example, the threshold required for calculating the neck width of men can be set to 0.8, and the threshold required for calculating the neck width of women can be set to 0.7; according to the horizontal coordinate of the neck center point and the neck width, the horizontal coordinates of the key points on the left and right sides of the neck are calculated; according to the horizontal coordinates of the key points on the left and right sides of the neck, a traversal search is performed on the corrected face mask image M1 to calculate the vertical coordinates of the left and right key points of the neck.

[0043] The above steps are used to calculate the center point and width of the neck and the key points of the left and right neck.

[0044] The above embodiment further includes: a detailed adjustment operation for the vertical coordinates of the left and right key points of the neck: if the difference between the vertical coordinates of the left and right key points of the neck exceeds a preset pixel value, a traversal search is performed on the corrected face mask image M1 based on the vertical coordinate of the key point on the lower side, and the horizontal and vertical coordinates of the key point on the other side are updated. Exemplarily, after obtaining the vertical coordinates of the left and right neck key points, a comparative analysis is performed. If the difference between the two vertical coordinates exceeds 2 pixel values, a traversal search is performed on the corrected face mask image M1 based on the vertical coordinate of the lower side to update the calculated horizontal and vertical coordinate information of the key point on the other side, thereby ensuring the consistency and accuracy of the coordinates of the neck key points.

[0045] Through the above steps, the detailed adjustment of the vertical coordinates of the left and right key points of the neck is completed.

[0046] Step 300: Based on the set of neck key points, the neck template is mapped to the corresponding area of ​​the user image after cleaning the clothes and non-face skin area, to obtain the user image after the neck is regenerated. Figure 3 , Figure 3 This embodiment uses a mask image to visually display the specific operations of step 300.

[0047] According to the set of neck key points, the neck template is mapped to the corresponding area of ​​the user image after cleaning the clothes and non-face skin area to obtain the user image after the neck is regenerated, which specifically includes: a preset neck template image and a corresponding template mask image, such as Figure 4As shown, the neck template image includes the coordinate information of the known left and right key points; based on the coordinate information of the known left and right key points, the area enclosed by the left and right key points of the template neck is determined; the scaling ratio is determined based on the area enclosed by the left and right key points of the template neck and the area enclosed by the neck key point set; the neck template image and the corresponding template mask image are scaled according to the scaling ratio. For example, the left and right key points of the neck in step 200 include coordinate information, and the neck template image also includes the coordinate information of the known left and right key points. By calculating the neck widths respectively, the ratio of the two neck widths is used as the scaling ratio, and the neck template image is scaled according to the scaling ratio. The height of the neck template image is also automatically scaled according to the scaling ratio; based on the neck key point set, the corresponding neck area is found on the user image Image3, and the scaled neck template image is replaced on the user image Image3 to obtain the user image Image4 with the regenerated neck; based on the neck key point set, the corresponding neck area is found on the portrait parsing mask image M4, and the scaled template mask image is replaced on the portrait parsing mask image M4 to obtain the portrait parsing mask image M5 with the regenerated neck.

[0048] The above steps are implemented to complete the neck template preparation and template scaling and replacement.

[0049] In the above embodiment, after obtaining the user image Image4 and the portrait parsing mask image M5, the method further includes: utilizing the distribution characteristics of the skin color component in the lɑβ space to obtain color distribution information of the user image Image4, wherein the color distribution information includes facial skin color information and neck skin color information; utilizing the portrait parsing mask image M5 to extract the facial skin color information and neck skin color information of the user image Image4; and adaptively converting the neck skin color information into a color that matches the facial skin color based on the facial skin color information and neck skin color information using a preset conversion rule. For example, the original face image and the target neck image can be extracted based on the portrait parsing mask image M5, and the LAB color information of the original face image, including the color range and average, is calculated. Similarly, the LAB color information of the target neck image is calculated. For each pixel in the target neck image, color correction is performed based on the color information of the original face image and the target neck image. Specifically, a linear transformation is performed based on the range and average of the A and B channels of the LAB color space. According to the brightness difference between the original face image and the target neck image, the brightness of the clothing image is appropriately adjusted to make the target neck image more closely match the skin color in the original face image.

[0050] The color adaptive transfer is completed through the above steps.

[0051] In the above embodiment, after obtaining the user image Image4 and the portrait parsing mask image M5, the process further includes: using edge detection technology to generate an edge contour E0 of the chin and the neck. For example, during the replacement process, the edge contour E0 of the chin and the neck is recorded using edge detection technology; performing an edge repair operation and an image blurring operation on the user image Image4 according to the edge contour E0; performing a detail adding operation on the user image Image4 after the edge repairing operation and the image blurring operation. For example, the detail adding operation includes: adding texture information operation, adding Adam's apple operation and adding shadow operation; wherein the adding texture information operation is: setting a fine line area in the middle area of ​​the neck, And deepen the color information in the fine line area; the operation of adding Adam's apple is: by setting a light spot in the middle area of ​​the neck, and ensuring that the pixel value of the light spot does not exceed the boundary (the pixel value does not exceed the boundary means that the pixel value of the small light spot superimposed on the image will exceed 255), the operation of adding shadows is: after performing morphological dilation on the edge contour E0, locate the corresponding area in the user image Image4, and reduce the brightness of the area to achieve the shadow effect; then, based on the color information of the chin area in the user image Image3, generate a horizontal color palette; render the horizontal color palette to the neck area of ​​the user image Image4 that has undergone the detail addition operation, and obtain the user image Image5 of the regenerated neck. The effect achieved through the above steps can be seen in Figure 5 The natural transition between skin color and texture can be seen in the overall dressing effect.

[0052] The above steps are used to complete edge detection and edge optimization, detail addition and color secondary correction.

[0053] Step 400: determining the position coordinates of the clothing to be replaced on the user image after the neck is regenerated, and fusing the clothing to be replaced with the user image after the neck is regenerated to obtain a final clothing change result image.

[0054] Among them, the position coordinates of the clothing to be replaced are determined on the user image after the neck is regenerated, and the clothing to be replaced is merged with the user image after the neck is regenerated to obtain the final dressing result image, which specifically includes: moving the vertical coordinates of the left and right key points of the neck of the user image Image5 downward to obtain two fitting key points; presetting the clothing image to be replaced and the corresponding clothing mask image, the clothing image to be replaced and the corresponding clothing mask image contain two neckline coordinate points; calculating the horizontal coordinate difference of the two fitting key points and the horizontal coordinate difference of the two neckline coordinate points; using the ratio between the two differences as the scaling ratio of the clothing image to be replaced, and using the scaling ratio to scale the clothing image to be replaced and the clothing mask image The code image is scaled proportionally to obtain a target clothing image and a target clothing mask image. At the same time, the two neckline coordinate points are updated using the scaling ratio. The position information required for the updated left neckline coordinate point to move to the left fitting key point is calculated, and the target clothing image is placed on the user image Image5 according to the position information. A black background image is generated, and the black background image has the same size as the user image Image5. The target clothing mask image is placed on the black background image according to the position information to obtain a final clothing template image. The user image Image5 on which the target clothing image has been placed is fused according to the final clothing template image to obtain a final dressing result image.

[0055] The above steps are used to complete the calculation of fitting key points, preparation of clothing to be changed, scaling of clothing, replacement of clothing and fusion operations.

[0056] Example 2

[0057] See also Figures 1 to 6 The embodiment of the present invention provides a method for intelligently changing ID photos, which may include the following steps:

[0058] Step 10: Clean up the clothing and non-face skin areas on the user image and update the corresponding mask map simultaneously.

[0059] Step 20: Calculate the neck key points based on the user image after cleaning the clothing and non-face skin areas and the mask image.

[0060] Step 30: Map the neck template to the corresponding area of ​​the user image using the neck key points, and perform replacement and detail processing to obtain the user image after the neck is regenerated.

[0061] Step 40: Determine the position coordinates of the clothing to be replaced on the user image after the neck is regenerated, and fuse them with the user image after the neck is regenerated to generate a final clothing change result image.

[0062] In certain embodiments, Figure 2 This embodiment uses a mask image to visualize the specific operations of step 10. In step 10, the specific operations of the cleaning area in this embodiment further include:

[0063] Step 11: Perform face detection on the user image Image0, perform facial key point detection based on the face detection result, and obtain the facial key point set P0.

[0064] Step 12: Filter the facial contour point set PL0 from the facial key point set P0. For example, in this embodiment, a detection model containing 106 key points is adopted, wherein the facial key point sets numbered 0 to 32 are identified as facial contour point sets.

[0065] Furthermore, Gaussian smoothing and cubic curve fitting are sequentially performed on the facial contour point set PL0 to obtain a smoothed and dense facial contour point set PL1.

[0066] Step 13: Perform facial segmentation on the user image according to the result of the facial key point set P0 to obtain a facial mask image M0 that marks the facial area.

[0067] The step 13 further comprises:

[0068] Step 131: Perform morphological dilation and erosion processing on the face mask image M0, and perform detailed optimization processing on the contour edge of the face mask image M0 based on the smoothed dense face contour point set PL1.

[0069] Step 14: Calculate the facial posture angle of the facial key point set P0 and perform face correction operation to obtain the corrected facial key point set P1, the corrected user image Image1 and the corrected face mask M1 ， Among them, after calculating the facial posture angle to obtain the correction matrix, it can be directly applied to the facial key point set P0, the user image Image0, and the facial mask map M0 to obtain the corrected facial key point set P1, the corrected user image Image1, and the corrected facial mask map M1. For example, this operation can also be directly implemented using existing functions in the opencv library.

[0070] Step 15: Perform portrait parsing on the corrected user image Image1 to generate a portrait parsing mask image M2. For example, the portrait parsing model used in this embodiment can perform in-depth analysis on the input image and accurately distinguish and mark the four main semantic categories of background, clothing, hair and skin.

[0071] Step 16: Based on the clothing semantic area identified in the portrait parsing mask image M2, the clothing area corresponding to the corrected user image Image1 is cleaned, and the pixel values ​​of the clothing semantic area of ​​the portrait parsing mask image M2 are updated to obtain the user image Image2 after cleaning the clothes and the portrait parsing mask image M3.

[0072] Wherein, the step 16 further comprises:

[0073] Step 161: The cleaning operation involves adjusting the pixel values ​​of the corresponding area in the corrected user image Image1 to be consistent with the background, and using morphological dilation and erosion processing techniques to accurately optimize the contour edges to obtain the user image Image2 after cleaning the clothes;

[0074] Step 162: Update the pixel values ​​of the clothes semantic area of ​​the portrait parsing mask image M2 to pixel values ​​consistent with the background semantic area, and obtain the portrait parsing mask image M3 after cleaning the clothes. For example, in this embodiment, the pixel value corresponding to the background semantics is 0.

[0075] Step 17: Based on the skin semantic areas identified in the corrected face mask image M1 and the portrait parsing mask image M3 after cleaning the clothes, the user image Image2 after cleaning the clothes is further processed to clean up the skin parts in the non-face areas. At the same time, the pixel values ​​corresponding to the skin areas in the non-face areas in the portrait parsing mask image M3 after cleaning the clothes are updated to obtain the user image Image3 with the skin parts in the non-face areas cleaned up and the portrait parsing mask image M4.

[0076] Wherein, the step 17 further comprises:

[0077] Step 171: After cleaning the skin portion of the non-face area of ​​the user image Image2 after cleaning the clothes, the contour edge is accurately optimized using morphological dilation and erosion processing technology to obtain the user image Image3 after cleaning the skin portion of the non-face area;

[0078] Step 172: Update the pixel values ​​corresponding to the skin area of ​​the non-face area in the portrait parsing mask image M3 after cleaning the clothes to pixel values ​​consistent with the background semantic area, and obtain the portrait parsing mask image M4 with the skin part of the non-face area cleaned.

[0079] In some embodiments, in step 20, the specific operation of calculating the neck key points in this embodiment further includes:

[0080] Step 21: Calculate the central horizontal coordinate of the corrected facial key point set P1 and the difference between the maximum and minimum horizontal coordinates.

[0081] The central horizontal coordinate is defined as the horizontal coordinate of the center point of the neck, and a threshold K is set to be multiplied by the difference between the maximum and minimum horizontal coordinates to determine the neck width.

[0082] Wherein, the step 21 further includes:

[0083] Step 211: Different thresholds K may be set for different genders according to user needs. For example, in this embodiment, the threshold required for calculating the neck width of men is set to 0.8, and the threshold required for calculating the neck width of women is set to 0.7.

[0084] Step 22: Calculate the horizontal coordinates of the key points on the left and right sides of the neck based on the horizontal coordinates of the neck center point and the neck width, and perform a traversal search on the corrected face mask M1 based on the horizontal coordinates of the key points on the left and right sides of the neck to determine the calculated vertical coordinates of the left and right neck key points, and compare them for detailed adjustments.

[0085] Wherein, the step 22 further includes:

[0086] Step 221: Detail adjustment refers to performing a comparative analysis after obtaining the vertical coordinates of the left and right neck key points. If the difference between the two vertical coordinate values ​​exceeds 2 pixel values, the vertical coordinate of the lower side is used as a reference to perform a traversal search on the corrected face mask M1 to update the calculated horizontal and vertical coordinate information of the key point on the other side to ensure the consistency and accuracy of the coordinates of the neck key points.

[0087] In certain embodiments, Figure 3 This embodiment uses a mask image to visualize the specific operations of step 30. In step 30, the specific operations of generating the neck in this embodiment further include:

[0088] Step 31: Figure 4 As shown, a neck template image and a corresponding template mask image are prepared in advance, and the coordinate information of the left and right key points in the neck template image is ensured to be known. According to the area size of the left and right neck key points calculated above, the neck template image and the corresponding template mask image are scaled to adapt to the size of the target area. In the specific implementation: the left and right key points of the neck calculated in step 22 are two-dimensional coordinates that can calculate the corresponding width. The coordinate information of the left and right key points in the template image can also calculate the width. The ratio of the two widths is used as the scaling ratio. The template image is scaled according to this scaling ratio. The height of the template image will also be automatically scaled according to this ratio, and then directly pasted to the area enclosed by the left and right neck key points on the user image.

[0089] Step 32: Based on the calculated left and right neck key points, the corresponding neck area is found in the user image Image3 after cleaning the skin part of the non-face area, and replaced with the scaled neck template image to obtain the regenerated neck user image Image4.

[0090] Step 33: Based on the calculated left and right neck key points, the corresponding neck area is found in the portrait parsing mask image M4 after removing the skin portion of the non-face area, and replaced with the scaled neck template mask image to obtain a regenerated neck portrait parsing mask image M5.

[0091] Wherein, the step 33 further includes:

[0092] Step 331: During the replacement process, edge detection technology is used to record the edge contour E0 of the chin and neck.

[0093] Step 34: Since the facial skin color and neck skin color of the user image after neck replacement are inconsistent, the color distribution information of the regenerated neck user image Image4 is calculated using the distribution characteristics of the skin color component in lɑβ space. The facial skin color information and neck skin color information are then extracted based on the regenerated neck portrait parsing mask M5. Adaptive conversion of facial skin color to neck skin color is achieved through corresponding conversion rules. In a specific implementation, the original face image and target neck image are extracted based on the portrait parsing mask M5. The LAB color information of the original face image, including the color range and average, is calculated. Similarly, the LAB color information of the target neck image is calculated. For each pixel in the target neck image, color correction is performed based on the color information of the original face image and the target neck image. Specifically, a linear transformation is performed based on the range and average of the A and B channels in the LAB color space. Based on the brightness difference between the original face image and the target neck image, the brightness of the clothing image is appropriately adjusted to ensure that the skin color of the target neck image more closely matches that of the original face image.

[0094] Step 35: performing edge restoration and image blurring operations on the user image Image4 after the color transfer according to the edge contour E0 to optimize the edge.

[0095] Step 36: Perform a detail adding operation on the edge-optimized user image Image4 by adding texture information, automatically adding Adam's apple information based on gender information, and adding shadow information to the chin area.

[0096] Wherein, the step 36 further comprises:

[0097] Step 361: Adding texture information means setting a fine line area in the middle area of ​​the neck and deepening the color information in the fine line area;

[0098] Step 362: Adding the Adam's apple is done by setting a small diverging light spot in the middle of the neck and ensuring that the pixel value of the light spot area does not exceed the bounds. The pixel value does not exceed the bounds when the small light spot is superimposed on the image.

[0099] Step 363: The shadow adding operation mainly involves performing a moderate morphological dilation on the edge contour E0, then locating the corresponding areas in the user image Image4, and performing a brightness reduction process on these areas to achieve a shadow effect.

[0100] Step 37: Calculate the color information of the chin area in the user image Image3 after cleaning the skin part of the non-face area, create a horizontal color palette based on the calculated color information, and then render it to the neck area of ​​the user image Image4 after the detail addition operation to achieve a natural transition and matching of colors, and obtain the regenerated neck user image Image5. The effect achieved by steps 34 to 37 can be seen in Figure 5 The natural transition between skin color and texture can be seen in the overall dressing effect.

[0101] In some embodiments, in step 40, the specific operation of replacing clothing in this embodiment further includes:

[0102] Step 41: The vertical coordinates of the left and right neck key points calculated above are appropriately shifted downward to obtain two fitting key points of the regenerated neck user image Image5. The two fitting key points are located on the edge contour lines on both sides and are used to connect the two connection points of the face area in the ID photo image to be replaced.

[0103] Step 42: Prepare in advance an image of the clothing to be changed and a corresponding clothing mask image, and ensure that the two collar coordinate points of the clothing to be changed image are known, wherein the clothing to be changed image is an image containing a transparent channel, wherein when synthesizing the images, the transparent channel allows the image to be seamlessly superimposed on another image to create a natural transition effect.

[0104] Step 43: Calculate the horizontal coordinate difference of the two fitting key points and the horizontal coordinate difference of the two neckline coordinate points. The ratio between the two differences is used as the scaling ratio of the clothing image to be changed. The clothing image to be changed and the clothing mask image are scaled proportionally using the scaling ratio to obtain the target clothing image and the target clothing mask image. At the same time, the two neckline coordinate points are updated using the scaling ratio.

[0105] Step 44: Calculate the position information required for the updated left neckline coordinate point to move to the left fitting key point, and place the target clothing image on the regenerated neck user image Image5 using the moved position information.

[0106] Step 45: Generate a black background image of the same size as the regenerated neck user image Image5, and place the target clothing mask image into the black background image using the moved position information to generate the final clothing template image.

[0107] Step 46: The final clothing template image is fused with the user image Image5 on which the target clothing image has been placed, to obtain an ID photo image after the clothing change.

[0108] Example 3

[0109] Based on the same inventive concept, an embodiment of the present invention further provides an ID photo intelligent dressing processing device, comprising:

[0110] An image cleaning unit is configured to obtain a user image, perform a cleaning operation on clothing and non-face skin areas of the user image, and obtain a user image with the clothing and non-face skin areas cleaned and a corresponding portrait parsing mask image;

[0111] A neck key point calculation module, configured to calculate a set of neck key points based on the user image with clothing and non-face skin areas removed and a corresponding portrait parsing mask image;

[0112] A neck generation module is configured to map a neck template to a corresponding area of ​​the user image after removing clothing and non-face skin areas based on the set of neck key points, thereby obtaining a user image after regenerating the neck;

[0113] The clothing replacement module is used to determine the position coordinates of the clothing to be replaced on the user image after the neck is regenerated, and to fuse the clothing to be replaced with the user image after the neck is regenerated to obtain a final clothing replacement result image.

[0114] Since the device is a device corresponding to the method for intelligent ID photo replacement processing in an embodiment of the present invention, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0115] Example 4

[0116] Based on the same inventive concept, an embodiment of the present invention also provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the above-mentioned method for intelligent ID photo replacement.

[0117] It is understood that the memory may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the data storage area may store data created based on the use of the server, etc.

[0118] The processor may include one or more processing cores. The processor utilizes various interfaces and circuits to connect various components within the server. It executes various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, as well as accessing data stored in memory. Optionally, the processor may be implemented using at least one of the following hardware forms: digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor may integrate one or a combination of a central processing unit (CPU) and a modem. The CPU primarily processes the operating system and application programs, while the modem handles wireless communications. It is understood that the modem may not be integrated into the processor and may be implemented separately via a single chip.

[0119] Since the electronic device is the electronic device corresponding to the method for intelligently changing ID photos in an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0120] Example 5

[0121] Based on the same inventive concept, an embodiment of the present invention also provides a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the above-mentioned method for intelligent ID photo replacement.

[0122] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0123] Since the storage medium is the storage medium corresponding to the ID photo intelligent replacement processing method of the embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0124] Example 6

[0125] In some possible implementations, various aspects of the methods of the embodiments of the present invention may also be implemented in the form of a program product, which includes program code. When the program product is executed on a computer device, the program code is used to cause the computer device to perform the steps of the ID photo intelligent re-dressing processing method according to various exemplary embodiments of the present application as described above in this specification. The executable computer program code or "code" used to perform the various embodiments may be written in a high-level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0126] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0127] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0128] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made based on the essence of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A method for intelligently changing ID photos, characterized in that: include: Acquire a user image, and perform a cleaning operation on clothing and non-face skin areas of the user image to obtain a user image with the clothing and non-face skin areas cleaned and a corresponding portrait parsing mask image; Based on the user image with the clothes and non-face skin areas removed and the corresponding portrait parsing mask map, a set of neck key points is calculated; wherein, based on the corrected face key point set P1, the horizontal coordinate, maximum horizontal coordinate and minimum horizontal coordinate of the neck center point are calculated; the neck width is determined by multiplying the difference between the maximum horizontal coordinate and the minimum horizontal coordinate by a preset threshold; based on the horizontal coordinate of the neck center point and the neck width, the horizontal coordinates of the key points on the left and right sides of the neck are calculated; based on the horizontal coordinates of the key points on the left and right sides of the neck, a traversal search is performed on the corrected face mask map M1 to calculate the vertical coordinates of the key points on the left and right sides of the neck; based on the horizontal coordinates of the key points on the left and right sides of the neck, a traversal search is performed on the corrected face mask map M1 to calculate the vertical coordinates of the key points on the left and right sides of the neck. The corrected face key point set P1 and the corrected face mask map M1 are obtained by calculating the facial posture angle of the facial key point set and performing a face correction operation; Mapping the neck template to a corresponding area of ​​the user image after removing clothing and non-face skin areas based on the neck key point set, to obtain a user image after regenerating the neck; The position coordinates of the clothing to be replaced are determined on the user image after the neck is regenerated, and the clothing to be replaced is fused with the user image after the neck is regenerated to obtain a final clothing change result image.

2. The method for intelligently changing ID photos according to claim 1, characterized in that: Before performing the cleaning operation on the clothing and non-face skin areas of the user image, the method further includes: Performing face detection on the user image Image0 to obtain a face detection result; Performing facial key point detection on the face detection result to obtain a facial key point set P0; Performing a screening operation on the facial key point set P0 to obtain a facial contour point set PL0; Performing Gaussian smoothing and cubic curve fitting on the facial contour point set PL0 in sequence to obtain a smoothed and dense facial contour point set PL1; Performing a face segmentation operation on the user image Image0 based on the face key point set P0 to obtain a face mask M0 that marks the face area, performing morphological dilation and erosion processing on the face mask M0, and performing detailed optimization processing on the contour edge of the face mask M0 based on the smoothed and dense face contour point set PL1; Calculating the facial posture angles and performing a facial correction operation on the facial key point set P0 to obtain a corrected facial key point set P1, a corrected user image Image1, and a corrected facial mask image M1; Perform portrait analysis on the corrected user image Image1 to obtain a portrait analysis mask image M2.

3. The method for intelligently changing ID photos according to claim 2, characterized in that: The operation of cleaning clothing and non-face skin areas of the user image specifically includes: Based on the clothing semantic region identified in the portrait parsing mask M2, a cleaning operation is performed on the clothing region corresponding to the user image Image1. At the same time, a pixel value of the clothing semantic region of the portrait parsing mask M2 is updated to obtain the user image Image2 after cleaning the clothing and the portrait parsing mask M3; Based on the skin semantic areas identified in the corrected face mask M1 and the portrait parsing mask M3, the non-face skin areas of the user image Image2 are cleaned. At the same time, the pixel values ​​corresponding to the non-face skin areas in the portrait parsing mask M3 are updated to obtain the user image Image3 and the portrait parsing mask M4 with the non-face skin areas cleaned off.

4. The method for intelligently changing ID photos according to claim 3, characterized in that: Also includes: Detailed adjustment operation on the vertical coordinates of the left and right key points of the neck: if the difference in the values ​​of the vertical coordinates of the left and right key points of the neck exceeds the preset pixel value, the vertical coordinate of the key point on the lower side is used as the reference, and a traversal search is performed on the corrected face mask M1 to update the horizontal and vertical coordinates of the key point on the other side.

5. The method for intelligently changing ID photos according to claim 4, characterized in that: Mapping the neck template to a corresponding area of ​​the user image after removing clothing and non-face skin areas based on the neck key point set to obtain a regenerated user image of the neck specifically includes: Preset a neck template image and a corresponding template mask image, wherein the neck template image contains coordinate information of known left and right key points; Determine the area enclosed by the left and right key points of the template neck based on the known coordinate information of the left and right key points; Determining a scaling ratio based on an area enclosed by the left and right key points of the template neck and an area enclosed by the set of neck key points, and scaling the neck template image and the corresponding template mask image based on the scaling ratio; According to the set of neck key points, a corresponding neck region is found on the user image Image3, and the scaled neck template image is replaced on the user image Image3 to obtain a regenerated neck user image Image4; According to the set of neck key points, the corresponding neck area is found on the portrait parsing mask image M4, and the scaled template mask image is replaced on the portrait parsing mask image M4 to obtain a regenerated neck portrait parsing mask image M5.

6. The method for intelligently changing ID photos according to claim 5, characterized in that: After obtaining the user image Image4 and the portrait parsing mask image M5, the method further includes: Using the distribution characteristics of the skin color component in the lɑβ space, color distribution information of the user image Image4 is obtained, wherein the color distribution information includes facial skin color information and neck skin color information; Extracting facial skin color information and neck skin color information of the user image Image4 using the portrait parsing mask image M5; Based on the facial skin color information and the neck skin color information, the neck skin color is adaptively converted into a color that matches the facial skin color through a preset conversion rule.

7. The method for intelligently changing ID photos according to claim 6, characterized in that: After obtaining the user image Image4 and the portrait parsing mask image M5, the method further includes: Using edge detection technology, generate the edge contour E0 of the chin and neck; Performing an edge repair operation and an image blur operation on the user image Image4 according to the edge contour E0; Performing a detail adding operation on the user image Image4 after the edge repair operation and the image blur operation, wherein the detail adding operation includes: adding texture information operation, adding Adam's apple operation and adding shadow operation; wherein, The operation of adding texture information is as follows: setting a fine line area in the middle area of ​​the neck and deepening the color information of the fine line area; The operation of adding the Adam's apple is as follows: setting a light spot in the middle area of ​​the neck and ensuring that the pixel value of the light spot does not exceed the boundary; The shadow adding operation is as follows: after performing morphological dilation on the edge contour E0, locating the corresponding area in the user image Image4, and performing a brightness reduction process on the area; Generate a horizontal color palette based on the color information of the chin area in the user image Image3; The horizontal color palette is rendered to the neck area of ​​the user image Image4 that has been subjected to the detail adding operation, to obtain a user image Image5 with a regenerated neck.

8. The method for intelligently changing ID photos according to claim 7, characterized in that: The step of determining the position coordinates of the clothing to be replaced on the user image after the neck is regenerated, and fusing the clothing to be replaced with the user image after the neck is regenerated to obtain a final clothing-changing result image, specifically includes: Shift the vertical coordinates of the left and right key points of the neck of the user image Image5 downward to obtain two fitting key points; Preset the clothing image to be changed and the corresponding clothing mask image, wherein the clothing image to be changed and the corresponding clothing mask image include two collar coordinate points; Calculating the difference between the horizontal coordinates of the two fitting key points and the difference between the horizontal coordinates of the two collar coordinate points; The ratio between the two differences is used as the scaling ratio of the clothing image to be changed, and the clothing image to be changed and the clothing mask image are scaled in equal proportion using the scaling ratio to obtain the target clothing image and the target clothing mask image. At the same time, the two collar coordinate points are updated using the scaling ratio; Calculate the position information required for the updated left neckline coordinate point to move to the left fitting key point, and place the target clothing image on the user image Image5 according to the position information; Generate a black background image, the black background image has the same size as the user image Image5, and place the target clothing mask image on the black background image according to the position information to obtain a final clothing template image; The user image Image5 on which the target clothing image has been placed is fused according to the final clothing template image to obtain a final clothing change result image.

9. An intelligent ID photo re-dressing device, characterized in that: include: An image cleaning unit is configured to obtain a user image, perform a cleaning operation on clothing and non-face skin areas of the user image, and obtain a user image with the clothing and non-face skin areas cleaned and a corresponding portrait parsing mask image; A neck key point calculation module is configured to calculate a set of neck key points based on the user image with clothing and non-face skin areas removed and the corresponding portrait parsing mask map; wherein, based on the corrected face key point set P1, the horizontal coordinate, maximum horizontal coordinate and minimum horizontal coordinate of the neck center point are calculated; the neck width is determined by multiplying the difference between the maximum horizontal coordinate and the minimum horizontal coordinate by a preset threshold; the horizontal coordinates of the key points on the left and right sides of the neck are calculated based on the horizontal coordinates of the key points on the left and right sides of the neck; a traversal search is performed on the corrected face mask map M1 based on the horizontal coordinates of the key points on the left and right sides of the neck to calculate the vertical coordinates of the key points on the left and right sides of the neck; a traversal search is performed on the corrected face mask map M1 based on the horizontal coordinates of the key points on the left and right sides of the neck to calculate the vertical coordinates of the key points on the left and right sides of the neck. The corrected face key point set P1 and the corrected face mask map M1 are obtained by calculating the facial posture angle of the facial key point set and performing a facial correction operation; A neck generation module is configured to map a neck template to a corresponding area of ​​the user image after removing clothing and non-face skin areas based on the set of neck key points, thereby obtaining a user image after regenerating the neck; The clothing replacement module is used to determine the position coordinates of the clothing to be replaced on the user image after the neck is regenerated, and to fuse the clothing to be replaced with the user image after the neck is regenerated to obtain a final clothing replacement result image.

Images