Identification photo intelligent reloading processing method and device

Abstract

The invention relates to the technical field of image data processing, in particular to an identification photo intelligent reloading processing method and device, and the method comprises the steps: carrying out the cleaning operation of clothes and non-face skin regions of a user image, and synchronously updating a corresponding mask image, and calculating neck key points according to the user image after cleaning the clothes and the non-human face skin area and the mask image, mapping a neck template to a corresponding area of the user image by using the neck key points, and performing replacement and detail processing to obtain the user image after the neck is regenerated. And determining position coordinates of clothes to be replaced on the user image after the neck is regenerated, and fusing the position coordinates with the user image after the neck is regenerated so as to generate a final clothes replacement result graph. According to the invention, the photo that the neck part is shielded by the tall-necked clothes or scarf and uploaded by the user can be optimized, and the final clothes changing effect is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a method and device for intelligently changing ID photos. Background Art

[0002] ID photos are official photos used on various documents, such as ID cards, passports, driver's licenses, student ID cards and other official documents. They not only need to clearly show the individual's facial features, but also must meet the specific clothing requirements for specific purposes. For example, standard ID photos usually require wearing a dark top with a clear collar, and avoid wearing uniforms or suspenders. For ordinary passport photos, it is generally recommended to wear white clothes and match with a light blue background; if wearing other colors, the background should be white.

[0003] However, temporarily preparing ID photos that meet these dress requirements often means that multiple offline photo shoots are required. This process is not only time-consuming and cumbersome, but also expensive. Traditional methods rely on professional image processing software to perform post-processing of photos, which requires personnel with professional skills and operational capabilities to complete. However, existing intelligent dressing technologies often fail to fully consider authenticity factors, such as skin color, gender, etc. when processing images, resulting in the final dressing effect not being realistic enough. In addition, the current mainstream ID photo intelligent dressing technology often has unsatisfactory dressing effects when processing photos uploaded by users where the neck is blocked by high-necked clothing or scarves, and there is still much room for improvement. Summary of the invention

[0004] In view of the fact that the mainstream intelligent dressing technology for ID photos in the prior art has poor dressing effect when processing photos uploaded by users in which the neck part is blocked by high-necked clothing or scarves, the present invention provides an intelligent dressing processing method and device for ID photos, which optimize the final dressing effect of photos uploaded by users in which the neck part is blocked by high-necked clothing or scarves.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a method for intelligently changing ID photos, comprising:

[0007] Acquire a user image, and perform a cleaning operation on clothing and non-face skin areas of the user image to obtain a user image with clothing and non-face skin areas cleaned and a corresponding portrait parsing mask image;

[0008] Based on the user image with clothes and non-face skin areas removed and the corresponding portrait parsing mask image, a set of neck key points is calculated;

[0009] According to the set of neck key points, mapping the neck template to the corresponding area of ​​the user image after the clothes and non-face skin area are removed, so as to obtain the user image after the neck is regenerated;

[0010] The position coordinates of the clothing to be replaced are determined on the user image after the neck is regenerated, and the clothing to be replaced is merged with the user image after the neck is regenerated to obtain a final clothing change result image.

[0011] In a second aspect, the present invention provides a device for intelligently changing ID photos, comprising:

[0012] An image cleaning unit, which is used to obtain a user image, perform a clothing and non-face skin area cleaning operation on the user image, and obtain a user image with the clothing and non-face skin area cleaned and a corresponding portrait analysis mask image;

[0013] A neck key point calculation module, which is used to calculate a set of neck key points based on the user image with clothes and non-face skin areas removed and the corresponding portrait parsing mask map;

[0014] A neck generation module, which is used to map the neck template to the corresponding area of ​​the user image after cleaning the clothes and non-face skin area according to the neck key point set, so as to obtain the user image after the neck is regenerated;

[0015] The clothing replacement module is used to determine the position coordinates of the clothing to be replaced on the user image after the neck is regenerated, and merge the clothing to be replaced with the user image after the neck is regenerated to obtain a final clothing change result image.

[0016] In a third aspect, an embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0017] The memory is used to store programs;

[0018] The processor executes the program to implement the method described above.

[0019] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0020] In a fifth aspect, an embodiment of the present invention further provides a computer program product or a computer program, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above method.

[0021] Compared with the prior art, the beneficial effect of the present invention is that the method for intelligently changing ID photos provided by the present invention does not require users to have professional image processing skills or rely on complex image processing software. Users only need to upload the ID photos to be changed and select the images of their favorite clothes, and the system can automatically complete the changing process, realizing intelligent operation.

[0022] The present invention caters to the personalized needs of ID photos for different occasions, ensures that the photos after changing clothes are natural and realistic, and fully considers key factors such as skin color and gender to optimize the overall quality of the photos after changing clothes. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is an overall flow chart of the ID photo intelligent replacement processing method of this embodiment;

[0025] Figure 2 It is a diagram that uses a mask image to visualize the specific operation of step 100;

[0026] Figure 3 It is a diagram that uses a mask image to visualize the specific operation of step 300;

[0027] Figure 4 It is a neck template diagram and a schematic diagram of the key points on the left and right sides of the neck;

[0028] Figure 5 This is a diagram showing the overall dressing effect of this embodiment;

[0029] Figure 6 This is another overall flow chart of the ID photo intelligent replacement processing method of this embodiment. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] Example:

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations of the embodiments of the present invention are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] See also Figures 1 to 5 The embodiment of the present invention provides a method for intelligently changing ID photos, which may include the following steps:

[0034] Step 100: Obtain a user image, and perform a cleaning operation on the clothing and non-face skin areas of the user image to obtain a user image with the clothing and non-face skin areas cleaned and a corresponding portrait analysis mask image. Figure 2 , Figure 2 This embodiment uses a mask image to visualize the specific operation of step 100.

[0035] Wherein, before the cleaning operation of the clothing and non-face skin area of ​​the user image is performed, it also includes: performing face detection on the user image Image0 to obtain a face detection result; performing face key point detection on the face detection result to obtain a face key point set P0; performing a screening operation on the face key point set P0 to obtain a face outer contour point set PL0, exemplarily, a detection model containing 106 key points is adopted, wherein the face key point set numbered 0 to 32 is identified as a face outer contour point set; performing Gaussian smoothing and cubic curve fitting on the face outer contour point set PL0 in sequence to obtain a smoothed and dense face contour point set PL1; performing a face segmentation operation on the user image Image0 according to the face key point set P0 to obtain a face mask M0 marking the face area, performing morphological dilation and corrosion on the face mask M0, and A face contour point set PL1 is collected to perform detailed optimization processing on the contour edge of the face mask image M0; the face posture angle is calculated for the face key point set P0 and a face correction operation is performed to obtain a corrected face key point set P1, a corrected user image Image1 and a corrected face mask image M1, wherein the correction matrix obtained by calculating the face posture angle can be directly applied to the face key point set P0, the user image Image0, and the face mask image M0 to obtain a corrected face key point set P1, a corrected user image Image1 and a corrected face mask image M1. Exemplarily, this operation can also be directly implemented using existing functions in the opencv library; portrait analysis is performed on the corrected user image Image1 to obtain a portrait analysis mask image M2. Exemplarily, the adopted portrait analysis model can perform in-depth analysis on the input image, accurately distinguish and mark four main semantic categories such as background, clothing, hair and skin.

[0036] The above steps are used to complete face detection and key point detection, facial contour processing, face segmentation and mask creation, face correction and portrait analysis operations.

[0037] The cleaning operation of the clothing and non-face skin area of ​​the user image specifically includes:

[0038] According to the clothes semantic area identified in the portrait parsing mask image M2, a cleaning operation is performed on the clothes area corresponding to the user image Image1, and at the same time, the pixel value of the clothes semantic area of ​​the portrait parsing mask image M2 is updated to obtain the user image Image2 and the portrait parsing mask image M3 after the clothes are cleaned up; exemplarily, the cleaning operation involves adjusting the pixel value of the corresponding area in the corrected user image Image1 to be consistent with the background, and using morphological dilation and erosion processing technology to accurately optimize the contour edge to obtain the user image Image2 after the clothes are cleaned up; then, the pixel value of the clothes semantic area of ​​the portrait parsing mask image M2 is updated to be consistent with the pixel value of the background semantic area, to obtain the portrait parsing mask image M3 after the clothes are cleaned up, for example, the pixel value corresponding to the background semantics is 0.

[0039] According to the skin semantic areas identified in the corrected face mask M1 and the portrait parsing mask M3, the non-face skin areas of the user image Image2 are cleaned, and at the same time, the pixel values ​​corresponding to the non-face skin areas in the portrait parsing mask M3 are updated to obtain the user image Image3 and the portrait parsing mask M4 with the non-face skin areas cleaned. Exemplarily, after the skin part of the non-face area is cleaned from the user image Image2 after the clothes are cleaned, the contour edge is accurately optimized using morphological dilation and corrosion processing technology to obtain the user image Image3 with the skin part of the non-face area cleaned; then, the pixel values ​​corresponding to the skin area of ​​the non-face area in the portrait parsing mask M3 after the clothes are cleaned are updated to pixel values ​​consistent with the background semantic area to obtain the portrait parsing mask M4 with the skin part of the non-face area cleaned.

[0040] The above steps are implemented to complete the cleaning of clothes and non-human face skin.

[0041] Step 200: Based on the user image with clothes and non-face skin areas removed and the corresponding portrait parsing mask image, a set of neck key points is calculated.

[0042] Among them, the neck key point set is calculated based on the user image with clothes and non-face skin areas removed and the corresponding portrait parsing mask image, specifically including: according to the corrected face key point set P1, the horizontal coordinate, the maximum horizontal coordinate and the minimum horizontal coordinate of the neck center point are calculated; the neck width is determined by multiplying the difference between the maximum horizontal coordinate and the minimum horizontal coordinate by a preset threshold. Exemplarily, different thresholds K can be set for gender according to user needs. For example, the threshold required for calculating the neck width of men can be set to 0.8, and the threshold required for calculating the neck width of women can be set to 0.7; according to the horizontal coordinate of the neck center point and the neck width, the horizontal coordinates of the key points on the left and right sides of the neck are calculated; according to the horizontal coordinates of the key points on the left and right sides of the neck, a traversal search is performed on the corrected face mask image M1 to calculate the vertical coordinates of the left and right key points of the neck.

[0043] The above steps are used to complete the calculation of the center point and width of the neck and the calculation of the left and right neck key points.

[0044] In the above embodiment, it also includes: a detailed adjustment operation of the vertical coordinates of the left and right key points of the neck: if the difference between the vertical coordinates of the left and right key points of the neck exceeds the preset pixel value, the vertical coordinate of the key point on the lower side is used as a reference, and a traversal search is performed on the corrected face mask map M1 to update the horizontal and vertical coordinates of the key points on the other side. Exemplarily, after obtaining the vertical coordinates of the left and right neck key points, a comparative analysis is performed. If the difference between the two vertical coordinate values ​​exceeds 2 pixel values, the vertical coordinate on the lower side is used as a reference, and a traversal search is performed on the corrected face mask map M1 to update the calculated horizontal and vertical coordinate information of the key points on the other side, so as to ensure the consistency and accuracy of the coordinates of the neck key points.

[0045] Through the above steps, the detailed adjustment of the vertical coordinates of the left and right key points of the neck is completed.

[0046] Step 300: Based on the set of neck key points, the neck template is mapped to the corresponding area of ​​the user image after the clothes and non-face skin area are removed, so as to obtain the user image after the neck is regenerated. Figure 3 , Figure 3 This embodiment uses a mask image to visually display the specific operation of step 300.

[0047] The method maps the neck template to the corresponding area of ​​the user image after removing the clothes and non-face skin area according to the neck key point set, so as to obtain the user image after the neck is regenerated, specifically including: a preset neck template image and a corresponding template mask image, such as Figure 4As shown, the neck template image includes the coordinate information of the known left and right key points; according to the known coordinate information of the left and right key points, the area surrounded by the left and right key points of the template neck is determined; the scaling ratio is determined according to the area surrounded by the left and right key points of the template neck and the area surrounded by the neck key point set; the neck template image and the corresponding template mask image are scaled according to the scaling ratio. Exemplarily, the left and right key points of the neck in step 200 include coordinate information, and the neck template image also includes the coordinate information of the known left and right key points. By calculating the neck widths respectively, the ratio of the two neck widths is used as the scaling ratio, and the neck template image is scaled according to the scaling ratio, and the height of the neck template image is automatically scaled according to the scaling ratio; according to the neck key point set, the corresponding neck area is found on the user image Image3, and the scaled neck template image is replaced on the user image Image3 to obtain the user image Image4 of the regenerated neck; according to the neck key point set, the corresponding neck area is found on the portrait parsing mask image M4, and the scaled template mask image is replaced on the portrait parsing mask image M4 to obtain the portrait parsing mask image M5 of the regenerated neck.

[0048] The above steps are implemented to complete the neck template preparation and template scaling and replacement.

[0049] In the above embodiment, after obtaining the user image Image4 and the portrait parsing mask image M5, it also includes: using the distribution characteristics of the skin color component in the lɑβ space to obtain the color distribution information of the user image Image4, the color distribution information includes the face skin color information and the neck skin color information; using the portrait parsing mask image M5 to extract the face skin color information and the neck skin color information of the user image Image4; based on the face skin color information and the neck skin color information, the neck skin color is adaptively converted into a color that matches the face skin color through a preset conversion rule. Exemplarily, the original face image and the target neck image can be extracted respectively according to the portrait parsing mask image M5, and the LAB color information of the original face image can be calculated, including the range and average value of the color. Similarly, the LAB color information of the target neck image is calculated. For each pixel of the target neck image, color correction is performed according to the color information of the original face image and the target neck image. Specifically, it performs a linear transformation according to the range and average value of the A and B channels of the LAB color space. According to the brightness difference between the original face image and the target neck image, the brightness of the clothing image is appropriately adjusted to make the target neck image more closely match the skin color in the original face image.

[0050] The color adaptive transfer is completed through the above steps.

[0051] In the above embodiment, after obtaining the user image Image4 and the portrait parsing mask image M5, it also includes: using edge detection technology to generate an edge contour E0 of the chin and the neck. Exemplarily, in the replacement process, the edge contour E0 of the chin and the neck is recorded using edge detection technology; according to the edge contour E0, an edge repair operation and an image blurring operation are performed on the user image Image4; after the edge repair operation and the image blurring operation, a detail adding operation is performed on the user image Image4. Exemplarily, the detail adding operation includes: adding texture information operation, adding Adam's apple operation and adding shadow operation; wherein, the adding texture information operation is: setting a fine line area in the middle area of ​​the neck, And deepen the color information in the fine line area; the operation of adding Adam's apple is: by setting a light spot in the middle area of ​​the neck, and ensuring that the pixel value of the light spot does not cross the boundary (the pixel value does not cross the boundary means that the pixel value of the small light spot superimposed on the image will exceed 255), the operation of adding shadows is: after performing morphological expansion on the edge contour E0, locate the corresponding area in the user image Image4, and reduce the brightness of the area to achieve the shadow effect; then, based on the color information of the chin area in the user image Image3, generate a horizontal color palette; render the horizontal color palette to the neck area of ​​the user image Image4 that has undergone the detail addition operation, and obtain the user image Image5 of the regenerated neck. Through the above steps, the effect achieved can be seen in Figure 5 The natural transition between skin color and texture can be seen in the overall dressing effect.

[0052] The above steps are used to implement edge detection and edge optimization, detail addition and secondary color correction.

[0053] Step 400: determining the position coordinates of the clothing to be replaced on the user image after the neck is regenerated, and fusing the clothing to be replaced with the user image after the neck is regenerated to obtain a final clothing change result image.

[0054] The method of determining the position coordinates of the clothing to be replaced on the user image after the neck is regenerated, and fusing the clothing to be replaced with the user image after the neck is regenerated to obtain the final dressing result image specifically includes: moving the vertical coordinates of the left and right key points of the neck of the user image Image5 downward to obtain two fitting key points; presetting the clothing image to be replaced and the corresponding clothing mask image, wherein the clothing image to be replaced and the corresponding clothing mask image include two neckline coordinate points; calculating the horizontal coordinate difference of the two fitting key points and the horizontal coordinate difference of the two neckline coordinate points; using the ratio between the two differences as the scaling ratio of the clothing image to be replaced, and using the scaling ratio to scale the clothing image to be replaced and the clothing mask image The code image is scaled in proportion to obtain a target clothing image and a target clothing mask image, and at the same time, the two neckline coordinate points are updated using the scaling ratio; the position information required for the updated left neckline coordinate point to move to the left fitting key point is calculated, and the target clothing image is placed on the user image Image5 according to the position information; a black background image is generated, and the black background image is consistent in size with the user image Image5, and the target clothing mask image is placed on the black background image according to the position information to obtain a final clothing template image; a fusion operation is performed on the user image Image5 where the target clothing image has been placed according to the final clothing template image to obtain a final dressing result image.

[0055] The above steps are used to complete the calculation of fitting key points, preparation of clothing to be changed, scaling of clothing, replacement of clothing and fusion operations.

[0056] Example 2

[0057] See also Figures 1 to 6 The embodiment of the present invention provides a method for intelligently changing ID photos, which may include the following steps:

[0058] Step 10: Perform clothing and non-face skin cleaning operations on the user image, and simultaneously update the corresponding mask map.

[0059] Step 20: Calculate the key points of the neck based on the user image and the mask image after cleaning the clothing and non-face skin areas.

[0060] Step 30: Map the neck template to the corresponding area of ​​the user image using the neck key points, and perform replacement and detail processing to obtain the user image after the neck is regenerated.

[0061] Step 40: Determine the position coordinates of the clothing to be replaced on the user image after the neck is regenerated, and merge them with the user image after the neck is regenerated to generate a final clothing change result image.

[0062] In certain embodiments, Figure 2 This embodiment uses a mask image to visualize the specific operation of step 10. In step 10, the specific operation of the cleaning area in this embodiment further includes:

[0063] Step 11: Perform face detection on the user image Image0, perform face key point detection based on the face detection result, and obtain the face key point set P0.

[0064] Step 12: Filter the facial contour point set PL0 from the facial key point set P0. For example, in this embodiment, a detection model including 106 key points is adopted, in which the facial key point sets numbered 0 to 32 are identified as facial contour point sets.

[0065] Furthermore, the facial contour point set PL0 is subjected to Gaussian smoothing processing and cubic curve fitting processing in sequence to obtain a smoothed and dense facial contour point set PL1.

[0066] Step 13: Perform face segmentation on the user image according to the result of the face key point set P0 to obtain a face mask image M0 marking the face area.

[0067] The step 13 further comprises:

[0068] Step 131: perform morphological dilation and erosion processing on the face mask image M0, and perform detailed optimization processing on the contour edge of the face mask image M0 according to the smoothed dense face contour point set PL1.

[0069] Step 14: Calculate the facial posture angle of the facial key point set P0 and perform a facial correction operation to obtain a corrected facial key point set P1, a corrected user image Image1 and a corrected facial mask image M1 ， Among them, after calculating the face posture angle to obtain the correction matrix, it can be directly applied to the face key point set P0, the user image Image0, and the face mask map M0 to obtain the corrected face key point set P1, the corrected user image Image1 and the corrected face mask map M1. Exemplarily, this operation can also be directly implemented using existing functions in the opencv library.

[0070] Step 15: Perform portrait analysis on the modified user image Image1 to generate a portrait analysis mask image M2. For example, the portrait analysis model used in this embodiment can perform in-depth analysis on the input image, accurately distinguish and mark four main semantic categories such as background, clothing, hair and skin.

[0071] Step 16: According to the clothes semantic area identified in the portrait parsing mask image M2, the clothes area corresponding to the corrected user image Image1 is cleaned, and the pixel value of the clothes semantic area of ​​the portrait parsing mask image M2 is updated to obtain the user image Image2 after cleaning the clothes and the portrait parsing mask image M3.

[0072] Wherein, the step 16 further comprises:

[0073] Step 161: The cleaning operation involves adjusting the pixel values ​​of the corresponding area in the modified user image Image1 to be consistent with the background, and using morphological dilation and erosion processing techniques to accurately optimize the contour edge to obtain the user image Image2 after cleaning the clothes;

[0074] Step 162: Update the pixel values ​​of the clothes semantic area of ​​the portrait parsing mask image M2 to pixel values ​​consistent with the background semantic area, and obtain the portrait parsing mask image M3 after cleaning the clothes. For example, in this embodiment, the pixel value corresponding to the background semantics is 0.

[0075] Step 17: Based on the skin semantic areas identified in the corrected face mask image M1 and the portrait parsing mask image M3 after cleaning the clothes, the user image Image2 after cleaning the clothes is further processed to clean up the skin parts in the non-face areas, and at the same time, the pixel values ​​corresponding to the skin areas in the non-face areas in the portrait parsing mask image M3 after cleaning the clothes are updated to obtain the user image Image3 and the portrait parsing mask image M4 with the skin parts in the non-face areas cleaned up.

[0076] Wherein, the step 17 further comprises:

[0077] Step 171: After cleaning the skin portion of the non-face area of ​​the user image Image2 after cleaning the clothes, the contour edge is accurately optimized using morphological dilation and erosion processing technology to obtain the user image Image3 after cleaning the skin portion of the non-face area;

[0078] Step 172: Update the pixel values ​​corresponding to the skin area of ​​the non-face area in the portrait analysis mask image M3 after cleaning the clothes to pixel values ​​consistent with the background semantic area, and obtain the portrait analysis mask image M4 with the skin part of the non-face area cleaned.

[0079] In some embodiments, in step 20, the specific operation of calculating the key points of the neck in this embodiment further includes:

[0080] Step 21: Calculate the central horizontal coordinate of the corrected face key point set P1 and the difference between the maximum and minimum horizontal coordinates.

[0081] The central horizontal coordinate is defined as the horizontal coordinate of the center point of the neck, and a threshold K is set to be multiplied by the difference between the maximum and minimum horizontal coordinates to determine the neck width.

[0082] Wherein, the step 21 further comprises:

[0083] Step 211: different thresholds K may be set for different genders according to user needs. For example, in this embodiment, the threshold required for calculating the neck width of men is set to 0.8, and the threshold required for calculating the neck width of women is set to 0.7.

[0084] Step 22: Calculate the horizontal coordinates of the key points on the left and right sides of the neck based on the horizontal coordinates of the neck center point and the neck width, and perform a traversal search on the corrected face mask image M1 based on the horizontal coordinates of the key points on the left and right sides of the neck to determine the calculated vertical coordinates of the left and right neck key points, and compare them for detailed adjustments.

[0085] Wherein, the step 22 further comprises:

[0086] Step 221: Detail adjustment refers to a comparative analysis after obtaining the vertical coordinates of the left and right neck key points. If the difference between the two vertical coordinate values ​​exceeds 2 pixel values, the vertical coordinate of the lower side is used as a reference to perform a traversal search on the corrected face mask image M1 to update the calculated horizontal and vertical coordinate information of the key points on the other side to ensure the consistency and accuracy of the coordinates of the neck key points.

[0087] In certain embodiments, Figure 3 This embodiment uses a mask image to visualize the specific operation of step 30. In step 30, the specific operation of generating a neck in this embodiment further includes:

[0088] Step 31: Figure 4 As shown, a neck template image and a corresponding template mask image are prepared in advance, and the coordinate information of the left and right key points in the neck template image is ensured to be known. According to the area size of the left and right neck key points calculated above, the neck template image and the corresponding template mask image are scaled to adapt to the size of the target area. In the specific implementation: the left and right key points of the neck calculated in step 22 are two-dimensional coordinates that can calculate the corresponding width, and the coordinate information of the left and right key points in the template image can also calculate the width. The ratio of the two widths is used as the scaling ratio. The template image is scaled according to this scaling ratio, and the height of the template image will also be automatically scaled according to this ratio, and then directly pasted to the area surrounded by the left and right neck key points on the user image.

[0089] Step 32: Find the corresponding neck area in the user image Image3 after cleaning the skin part of the non-face area according to the left and right neck key points calculated above, and replace it with the scaled neck template image to obtain the regenerated neck user image Image4.

[0090] Step 33: Find the corresponding neck area in the portrait parsing mask image M4 after removing the skin part of the non-face area according to the left and right neck key points calculated above, and replace it with the scaled neck template mask image to obtain the regenerated neck portrait parsing mask image M5.

[0091] Wherein, the step 33 further comprises:

[0092] Step 331: During the replacement process, edge detection technology is used to record the edge contour E0 of the chin and neck.

[0093] Step 34: Since the facial skin color and neck skin color of the user image after replacing the neck are inconsistent, the color distribution information of the user image Image4 of the regenerated neck is calculated by using the distribution characteristics of the skin color component in the lɑβ space, and the facial skin color information and the neck skin color information are extracted according to the portrait parsing mask image M5 of the regenerated neck, and the adaptive conversion of the facial skin color to the neck skin color is realized through the corresponding conversion rules. In the specific implementation: the original face image and the target neck image can be extracted respectively according to the portrait parsing mask image M5, and the LAB color information of the original face image is calculated, including the range and average value of the color. The LAB color information of the target neck image is calculated in the same way. For each pixel of the target neck image, color correction is performed according to the color information of the original face image and the target neck image. Specifically, it performs a linear transformation according to the range and average value of the A and B channels of the LAB color space. According to the brightness difference between the original face image and the target neck image, the brightness of the clothing image is appropriately adjusted to make the target neck image more matched with the skin color in the original face image.

[0094] Step 35: performing edge restoration and image blurring operations on the user image Image4 after the color transfer according to the edge contour E0 to optimize the edge.

[0095] Step 36: Perform a detail adding operation on the edge-optimized user image Image4, by adding texture information, automatically adding Adam's apple information according to gender information, and adding shadow information in the chin area.

[0096] Wherein, the step 36 further comprises:

[0097] Step 361: The operation of adding texture information is to set a fine line area in the middle area of ​​the neck and deepen the color information in the fine line area;

[0098] Step 362: The operation of adding the Adam's apple is mainly to set a small divergent light spot in the middle area of ​​the neck and ensure that the pixel value of the light spot area does not cross the boundary, wherein the pixel value does not cross the boundary means that the pixel value of the small light spot superimposed on the image will exceed 255;

[0099] Step 363: The shadow adding operation mainly involves performing a moderate morphological dilation on the edge contour E0, then locating the corresponding areas in the user image Image4, and performing a brightness reduction process on these areas to achieve a shadow effect.

[0100] Step 37: Calculate the color information of the chin area in the user image Image3 after cleaning the skin part of the non-face area, make a horizontal color palette based on the calculated color information, and then render it to the neck area of ​​the user image Image4 after the detail adding operation, so as to achieve a natural transition and matching of colors, and obtain the user image Image5 that regenerates the neck. The effects achieved by steps 34 to 37 can be seen in Figure 5 The natural transition between skin color and texture can be seen in the overall dressing effect.

[0101] In some embodiments, in step 40, the specific operation of replacing clothing in this embodiment further includes:

[0102] Step 41: The vertical coordinates of the left and right neck key points calculated in the above are appropriately moved downward to obtain two fitting key points of the user image Image5 of the regenerated neck. The two fitting key points are located on the edge contour lines on both sides and are used to connect the two connection points of the face area in the ID photo image to be replaced.

[0103] Step 42: Prepare in advance an image of clothing to be changed and a corresponding clothing mask image, and ensure that two collar coordinate points of the clothing image to be changed are known, wherein the clothing image to be changed is an image containing a transparent channel, wherein when synthesizing the images, the transparent channel allows the image to be seamlessly superimposed on another image to create a natural transition effect.

[0104] Step 43: Calculate the horizontal coordinate difference of the two fitting key points and the horizontal coordinate difference of the two neckline coordinate points, and use the ratio between the two differences as the scaling ratio of the clothing image to be replaced. Use the scaling ratio to scale the clothing image to be replaced and the clothing mask image in equal proportion to obtain the target clothing image and the target clothing mask image, and use the scaling ratio to update the two neckline coordinate points.

[0105] Step 44: Calculate the position information required for the updated left collar coordinate point to move to the left fitting key point, and place the target clothing image into the regenerated neck user image Image5 using the position information to be moved.

[0106] Step 45: Generate a black background image of the same size as the regenerated neck user image Image5, and place the target clothing mask image into the black background image according to the moved position information to generate the final clothing template image.

[0107] Step 46: The final clothing template image is fused with the user image Image5 on which the target clothing image has been placed, to obtain a ID photo image after the change of clothing.

[0108] Example 3

[0109] Based on the same inventive concept, an embodiment of the present invention further provides an ID photo intelligent dressing processing device, comprising:

[0110] An image cleaning unit, which is used to obtain a user image, perform a clothing and non-face skin area cleaning operation on the user image, and obtain a user image with the clothing and non-face skin area cleaned and a corresponding portrait analysis mask image;

[0111] A neck key point calculation module, which is used to calculate a set of neck key points based on the user image with clothes and non-face skin areas removed and the corresponding portrait parsing mask map;

[0112] A neck generation module, which is used to map the neck template to the corresponding area of ​​the user image after cleaning the clothes and non-face skin area according to the neck key point set, so as to obtain the user image after the neck is regenerated;

[0113] The clothing replacement module is used to determine the position coordinates of the clothing to be replaced on the user image after the neck is regenerated, and merge the clothing to be replaced with the user image after the neck is regenerated to obtain a final clothing change result image.

[0114] Since the device is a device corresponding to the method for intelligent replacement of ID photos in an embodiment of the present invention, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0115] Example 4

[0116] Based on the same inventive concept, an embodiment of the present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the intelligent ID photo replacement processing method as described above.

[0117] It is understood that the memory may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data created according to the use of the server, etc.

[0118] The processor may include one or more processing cores. The processor uses various interfaces and lines to connect the various parts of the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor can integrate one or a combination of a central processing unit (CPU) and a modem. Among them, the CPU mainly processes the operating system and application programs; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor, but implemented separately through a chip.

[0119] Since the electronic device is the electronic device corresponding to the method for intelligent replacement of ID photos in the embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0120] Example 5

[0121] Based on the same inventive concept, an embodiment of the present invention also provides a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the intelligent ID photo replacement processing method as described above.

[0122] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0123] Since the storage medium is the storage medium corresponding to the method for intelligent replacement of ID photos in an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0124] Example 6

[0125] In some possible implementations, various aspects of the method of the embodiments of the present invention may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the ID photo intelligent replacement processing method according to various exemplary embodiments of the present application described above in this specification. Among them, the executable computer program code or "code" used to execute each embodiment can be written in a high-level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, structured query language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0126] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0127] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0128] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for intelligently changing ID photos, characterized in that: include: Acquire a user image, and perform a cleaning operation on clothing and non-face skin areas of the user image to obtain a user image with clothing and non-face skin areas cleaned and a corresponding portrait parsing mask image; Based on the user image with clothes and non-face skin areas removed and the corresponding portrait parsing mask image, a set of neck key points is calculated; According to the set of neck key points, mapping the neck template to the corresponding area of ​​the user image after the clothes and non-face skin area are removed, so as to obtain the user image after the neck is regenerated; The position coordinates of the clothing to be replaced are determined on the user image after the neck is regenerated, and the clothing to be replaced is merged with the user image after the neck is regenerated to obtain a final clothing change result image.

2. The method for intelligently changing ID photos according to claim 1, characterized in that: Before performing the cleaning operation on the clothing and non-face skin areas of the user image, the method further includes: Performing face detection on the user image Image0 to obtain a face detection result; Performing facial key point detection on the face detection result to obtain a facial key point set P0; Performing a screening operation on the facial key point set P0 to obtain a facial contour point set PL0; The facial contour point set PL0 is subjected to Gaussian smoothing processing and cubic curve fitting processing in sequence to obtain a smoothed and dense facial contour point set PL1; Performing a face segmentation operation on the user image Image0 according to the face key point set P0 to obtain a face mask image M0 marking the face area, performing morphological dilation and erosion processing on the face mask image M0, and performing detailed optimization processing on the contour edge of the face mask image M0 according to the smoothed dense face contour point set PL1; Calculating the facial posture angle and performing a facial correction operation on the facial key point set P0 to obtain a corrected facial key point set P1, a corrected user image Image1 and a corrected facial mask image M1; Portrait analysis is performed on the modified user image Image1 to obtain a portrait analysis mask image M2.

3. The method for intelligently changing ID photos according to claim 2, characterized in that: The cleaning operation of the clothing and non-face skin area of ​​the user image specifically includes: According to the clothes semantic region identified in the portrait parsing mask image M2, a cleaning operation is performed on the clothes region corresponding to the user image Image1, and at the same time, an update operation is performed on the pixel value of the clothes semantic region of the portrait parsing mask image M2, so as to obtain the user image Image2 after the clothes are cleaned and the portrait parsing mask image M3; Based on the skin semantic areas identified in the corrected face mask image M1 and the portrait parsing mask image M3, the non-face skin areas of the user image Image2 are cleaned up. At the same time, the pixel values ​​corresponding to the non-face skin areas in the portrait parsing mask image M3 are updated to obtain the user image Image3 and the portrait parsing mask image M4 with the non-face skin areas cleaned up.

4. The method for intelligently changing ID photos according to claim 1, characterized in that: The step of calculating a set of neck key points based on the user image with clothes and non-face skin areas removed and the corresponding portrait analysis mask map specifically includes: According to the modified face key point set P1, the horizontal coordinate, the maximum horizontal coordinate and the minimum horizontal coordinate of the neck center point are calculated; Determine the neck width according to the multiplication of the difference between the maximum horizontal coordinate and the minimum horizontal coordinate and a preset threshold; Calculate the horizontal coordinates of the key points on the left and right sides of the neck according to the horizontal coordinate of the center point of the neck and the width of the neck; A traversal search is performed on the modified face mask image M1 according to the horizontal coordinates of the key points on the left and right sides of the neck, and the vertical coordinates of the key points on the left and right sides of the neck are calculated.

5. The method for intelligently changing ID photos according to claim 4, characterized in that: Also includes: Detailed adjustment operation on the vertical coordinates of the left and right key points of the neck: if the difference in the values ​​of the vertical coordinates of the left and right key points of the neck exceeds the preset pixel value, the vertical coordinates of the key points on the lower side are used as the reference to perform a traversal search on the corrected face mask image M1 to update the horizontal and vertical coordinates of the key points on the other side.

6. The method for intelligently changing ID photos according to claim 5, characterized in that: The step of mapping the neck template to the corresponding area of ​​the user image after the clothes and non-face skin area are removed according to the neck key point set to obtain the user image after the neck is regenerated specifically includes: Preset a neck template image and a corresponding template mask image, wherein the neck template image contains coordinate information of known left and right key points; Determine the area enclosed by the left and right key points of the template neck according to the known coordinate information of the left and right key points; Determine a scaling ratio according to an area enclosed by left and right key points of the template neck and an area enclosed by the set of neck key points, and scale the neck template image and the corresponding template mask image according to the scaling ratio; According to the set of neck key points, a corresponding neck region is found on the user image Image3, and the scaled neck template image is replaced on the user image Image3 to obtain a regenerated neck user image Image4; According to the set of neck key points, the corresponding neck area is found on the portrait parsing mask image M4, and the scaled template mask image is replaced on the portrait parsing mask image M4 to obtain a portrait parsing mask image M5 for regenerating the neck.

7. The method for intelligently changing ID photos according to claim 6, characterized in that: After obtaining the user image Image4 and the portrait analysis mask image M5, the method further includes: Using the distribution characteristics of the skin color component in the lɑβ space, the color distribution information of the user image Image4 is obtained, wherein the color distribution information includes the face skin color information and the neck skin color information; Extracting the face skin color information and neck skin color information of the user image Image4 using the portrait parsing mask image M5; Based on the face skin color information and the neck skin color information, the neck skin color is adaptively converted into a color matching the face skin color through a preset conversion rule.

8. The method for intelligently changing ID photos according to claim 7, characterized in that: After obtaining the user image Image4 and the portrait analysis mask image M5, the method further includes: Using edge detection technology, generate the edge contour E0 of the chin and neck; Performing edge repair and image blur operations on the user image Image4 according to the edge contour E0; A detail adding operation is performed on the user image Image4 after the edge repair operation and the image blur operation, wherein the detail adding operation includes: adding texture information operation, adding Adam's apple operation and adding shadow operation; wherein, The operation of adding texture information is as follows: setting a fine line area in the middle area of ​​the neck, and deepening the color information in the fine line area; The operation of adding the Adam's apple is as follows: a light spot is set in the middle area of ​​the neck and the pixel value of the light spot is ensured not to exceed the boundary; The shadow adding operation is: after performing morphological dilation on the edge contour E0, locating the corresponding area in the user image Image4, and performing a brightness reduction process on the area; Generate a horizontal color palette according to the color information of the chin area in the user image Image3; The horizontal color palette is rendered to the neck region of the user image Image4 after the detail adding operation, to obtain the user image Image5 with the neck regenerated.

9. The method for intelligently changing ID photos according to claim 8, characterized in that: The step of determining the position coordinates of the clothing to be replaced on the user image after the neck is regenerated, and fusing the clothing to be replaced with the user image after the neck is regenerated to obtain a final clothing change result image, specifically includes: The vertical coordinates of the left and right key points of the neck of the user image Image5 are moved downward to obtain two fitting key points; Preset the clothing image to be changed and the corresponding clothing mask image, wherein the clothing image to be changed and the corresponding clothing mask image include two collar coordinate points; Calculating the horizontal coordinate difference of the two fitting key points and the horizontal coordinate difference of the two collar coordinate points; According to the ratio between the two differences as the scaling ratio of the clothing image to be changed, the clothing image to be changed and the clothing mask image are scaled in equal proportion by using the scaling ratio to obtain the target clothing image and the target clothing mask image, and at the same time, the two collar coordinate points are updated by using the scaling ratio; Calculate and obtain the position information required for the updated left neckline coordinate point to move to the left fitting key point, and place the target clothing image on the user image Image5 according to the position information; Generate a black background image, the black background image has the same size as the user image Image5, and place the target clothing mask image into the black background image according to the position information to obtain a final clothing template image; According to the final clothing template image, a fusion operation is performed on the user image Image5 on which the target clothing image has been placed to obtain a final dressing result image.

10. An intelligent ID photo replacement processing device, characterized in that: include: An image cleaning unit, which is used to obtain a user image, perform a clothing and non-face skin area cleaning operation on the user image, and obtain a user image with the clothing and non-face skin area cleaned and a corresponding portrait analysis mask image; A neck key point calculation module, which is used to calculate a set of neck key points based on the user image with clothes and non-face skin areas removed and the corresponding portrait parsing mask map; A neck generation module, which is used to map the neck template to the corresponding area of ​​the user image after cleaning the clothes and non-face skin area according to the neck key point set, so as to obtain the user image after the neck is regenerated; The clothing replacement module is used to determine the position coordinates of the clothing to be replaced on the user image after the neck is regenerated, and merge the clothing to be replaced with the user image after the neck is regenerated to obtain a final clothing change result image.

Images