Image processing method, device, equipment and storage medium

Abstract

This application discloses an image processing method, apparatus, device, and storage medium, belonging to the field of computer vision technology. This application preprocesses an image to be processed, determines the type of hair of a person in the image to be processed, matches a corresponding adjustment strategy based on the hair type, and adjusts the hair in the image to eliminate gaps between the hair and the sides of the neck / shoulders based on the adjustment strategy, thereby obtaining a target image. Specifically, the image processing method of this application adjusts the hair based on the hair type, fills the gaps between the hair and the neck, and repairs the gaps between the hair and the shoulders, thereby improving the authenticity of the ID photo.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to an image processing method, apparatus, device and storage medium. Background Art

[0002] With the rapid development of technology and the rapid adoption of mobile devices, more and more offline needs have shifted online. For example, ID photo modifications, which previously required visiting a store, can now be accomplished simply by downloading a mobile app. ID photo modifications include changing clothing. Existing technology for these changes focuses solely on matching the collar with the neck, ignoring details like hair, which reduces the photo's authenticity.

[0003] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide an image processing method, device, equipment and storage medium, aiming to solve the problem of low authenticity of ID photos obtained after ID photos are replaced and modified in the existing technology.

[0005] To achieve the above objectives, the present application provides an image processing method, which includes the following steps:

[0006] Preprocessing the image to be processed to determine the hair type of the portrait in the image to be processed;

[0007] According to the hair type, matching the corresponding adjustment strategy;

[0008] According to the adjustment strategy, the hair in the image to be processed is adjusted to eliminate the gap between the hair and the sides of the neck / shoulder neck, thereby obtaining a target image.

[0009] Optionally, the step of pre-processing the image to be processed and determining the hair type of the portrait in the image to be processed includes:

[0010] Acquire a first mask image of the hair area and a second mask image of the face area of ​​the image to be processed;

[0011] extracting a left shoulder-neck curve and a right shoulder-neck curve from the image to be processed;

[0012] Calculating the number of pixels of the first mask image below the left shoulder-neck curve and the right shoulder-neck curve, and comparing the number of pixels with the second mask image to obtain a comparison result;

[0013] If the comparison result is greater than or equal to a preset threshold, determining that the hair type of the portrait in the image to be processed is front-length hair;

[0014] If the comparison result is less than a preset threshold, it is determined that the hair-falling type of the portrait in the image to be processed is the back-falling hair.

[0015] Optionally, when the hair type is hair hanging down, the step of adjusting the hair in the image to be processed according to the adjustment strategy to eliminate gaps between the hair and both sides of the neck / shoulder neck to obtain the target image includes:

[0016] Generate a well-shaped positioning frame;

[0017] Based on the well-shaped positioning frame, determining a neck region adjustment rectangle and a shoulder and neck region adjustment rectangle;

[0018] Adjusting the rectangle based on the neck area to adjust the distance between the hair and both sides of the neck;

[0019] The rectangle is adjusted based on the neck and shoulder area, and the distance between the hair and the neck and shoulder is adjusted to obtain a target image.

[0020] Optionally, the neck area adjustment rectangle includes a left adjustment rectangle and a right adjustment rectangle, and the step of adjusting the distance between the hair and both sides of the neck based on the neck area adjustment rectangle includes:

[0021] horizontally stretching the right edge of the left adjustment rectangle to a preset position, and mapping the texture of the hair area within the left adjustment rectangle to the stretched area;

[0022] horizontally stretching the left edge of the right adjustment rectangle to a preset position, and mapping the texture of the hair area within the right adjustment rectangle to the stretched area;

[0023] Wherein, the preset position is the midline of the neck area.

[0024] Optionally, the step of adjusting the distance between the hair and the neck and shoulder based on the adjustment rectangle of the neck and shoulder region includes:

[0025] Adjust the coordinates of each vertex of the rectangle according to the shoulder and neck area to determine a first preset coordinate point and a second preset coordinate point;

[0026] Horizontally stretch the lower left point of the shoulder and neck area adjustment rectangle to the first preset coordinate point, horizontally stretch the lower right point of the shoulder and neck area adjustment rectangle to the second preset coordinate point, and map the hair area texture within the shoulder and neck area adjustment rectangle to the stretched area.

[0027] Optionally, when the hair type is front-down hair, the step of adjusting the hair in the image to be processed according to the adjustment strategy to eliminate gaps between the hair and both sides of the neck / shoulder neck to obtain the target image includes:

[0028] Generate a well-shaped positioning frame;

[0029] Acquire a third mask image of the head region, a fourth mask image of the neck region, and a fifth mask image of the clothing region of the image to be processed;

[0030] Determining, based on the third mask image, the fourth mask image, and the fifth mask image, an area in the tic-tac-toe locating frame where the mask is empty;

[0031] Collecting the hair area texture to fill the empty area of ​​the mask;

[0032] The edge of the filled area is fused with the original hair area to obtain the target image.

[0033] Optionally, the step of generating a well-shaped positioning frame includes:

[0034] Obtaining facial key points and preset feature positioning points of the portrait in the image to be processed;

[0035] A tic-tac-toe positioning frame is generated based on the facial key points and the preset feature positioning points.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides an image processing device, the device comprising:

[0037] A preprocessing module, configured to preprocess the image to be processed and determine the hair type of the portrait in the image to be processed;

[0038] A strategy determination module, configured to match a corresponding adjustment strategy according to the hair type;

[0039] The adjustment module is used to adjust the hair in the image to be processed according to the adjustment strategy to eliminate the gap between the hair and the sides of the neck / shoulder neck, so as to obtain a target image.

[0040] In addition, to achieve the above-mentioned purpose, the present application also provides an image processing device, which includes: a memory, a processor, and an image processing program stored on the memory and executable on the processor, wherein the image processing program is configured to implement the steps of the image processing method described above.

[0041] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, on which an image processing program is stored. When the image processing program is executed by a processor, the steps of the image processing method described above are implemented.

[0042] This application discloses an image processing method, apparatus, device, and storage medium. Compared to the existing technique of modifying ID photos to obtain ID photos with low authenticity, this application pre-processes the image to be processed, determines the hair type of the person in the image to be processed; matches a corresponding adjustment strategy based on the hair type; and adjusts the hair in the image to eliminate gaps between the hair and the sides of the neck / shoulders based on the adjustment strategy to obtain a target image. Specifically, the image processing method of this application adjusts the hair based on the hair type, fills the gaps between the hair and the neck, and repairs the gaps between the hair and the shoulders, thereby improving the authenticity of the ID photo. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0045] Figure 1 It is a structural diagram of an image processing device in a hardware operating environment involved in an embodiment of the present application;

[0046] Figure 2 This is a flowchart of the first embodiment of the image processing method of the present application;

[0047] Figure 3 This is a schematic diagram of the shoulder and neck points for this application;

[0048] Figure 4 A schematic diagram of the locations of the applicant's facial key points and preset feature positioning points;

[0049] Figure 5 This is a scene application diagram of adjusting the distance between the hair and both sides of the neck based on the rectangle of the neck area;

[0050] Figure 6 This is a scene application diagram of adjusting the distance between the hair and the neck and shoulders based on the rectangle of the neck and shoulders area;

[0051] Figure 7 This is a scene application diagram for adjusting the hair in the image to be processed when the hair type of this application is front-hanging hair;

[0052] Figure 8This is a schematic diagram of the functional modules of the first embodiment of the image processing device of the present application.

[0053] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0055] Reference Figure 1 , Figure 1 This is a schematic diagram of the image processing device structure of the hardware operating environment involved in the embodiment of the present application.

[0056] like Figure 1 As shown, the image processing device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0057] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the image processing device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0058] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and an image processing program. The operating system is a program that manages and controls the hardware and software resources of the image processing device and supports the operation of the image processing program and other software or programs.

[0059] exist Figure 1In the image processing device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the image processing device calls the image processing program stored in the memory 1005 through the processor 1001, and executes the image processing method provided in the embodiment of the present application.

[0060] The present application provides an image processing method, referring to Figure 2 , Figure 2 This is a flowchart of a first embodiment of an image processing method of the present application.

[0061] In this embodiment, the image processing method includes:

[0062] Step S10: pre-process the image to be processed to determine the hair type of the portrait in the image to be processed.

[0063] Specifically, preprocessing the image to be processed to determine the hair type of the portrait in the image to be processed includes:

[0064] Step S11: Acquire a first mask image of the hair area and a second mask image of the face area of ​​the image to be processed.

[0065] In this embodiment, obtaining a first mask image of a hair region and a second mask image of a face region of an image to be processed may include: obtaining a mean and a standard deviation of pixels in the image to be processed; performing pixel normalization processing on the image according to the mean and the standard deviation to obtain a pixel-normalized image; and performing segmentation processing on the pixel-normalized image using an image segmentation network to obtain the first mask image of the hair region and the second mask image of the face region of the image to be processed.

[0066] In this embodiment, obtaining a first mask image of the hair area and a second mask image of the face area of ​​the image to be processed may further include: hair-level portrait segmentation to obtain a third mask image of the head area of ​​the image to be processed that includes the hair and the face; through portrait semantic segmentation, obtaining a second mask image of the face area, a fourth mask image of the neck area, and a fifth mask image of the clothing area of ​​the image to be processed; and subtracting the third mask image from the second mask image to obtain the first mask image.

[0067] Reference Figure 3 , Figure 3 This is a schematic diagram of the shoulder and neck points for this application.

[0068] Step S12: extracting a left shoulder-neck curve and a right shoulder-neck curve from the image to be processed.

[0069] a1. Obtain a large number of portrait images marked with 18 shoulder and neck points as a training sample set;

[0070] a2. Input the training sample set into the neural network model to be trained, and obtain the predicted coordinates of each shoulder and neck point as the output result;

[0071] a3. Based on the output results and the real coordinates, make judgments through R-square calculation;

[0072] a4. If the R-squared value is greater than or equal to the preset R-squared threshold, the current neural network model to be trained is determined to be a shoulder and neck point recognition model;

[0073] a5. If the R-squared value is less than the preset R-squared threshold, adjust the parameters of the current neural network model to be trained and return to step a1;

[0074] a6. Use the trained shoulder and neck point recognition model to identify the image to be processed and obtain the shoulder and neck points of the portrait in the image to be processed;

[0075] a7. Fit the 9 shoulder and neck points of the left shoulder and neck using the least squares method to obtain the left shoulder and neck curve, and fit the 9 shoulder and neck points of the right shoulder and neck using the least squares method to obtain the right shoulder and neck curve;

[0076] The value range of the R-squared threshold is between 0 and 1. The specific value can be determined according to actual application conditions and is not specifically limited in this embodiment.

[0077] Step S13: Calculate the number of pixels of the first mask image below the left shoulder-neck curve and the right shoulder-neck curve, and compare them with the number of pixels of the second mask image to obtain a comparison result.

[0078] In this embodiment, calculating the number of pixels of the first mask image below the left shoulder and neck curve and the right shoulder and neck curve and comparing them with the number of pixels of the second mask image can include: respectively calculating the area of ​​the first mask image below the left shoulder and neck curve and the right shoulder and neck curve, calculating the area of ​​the second mask image, and comparing the area below the left shoulder and neck curve and the right shoulder and neck curve of the first mask image with the area of ​​the second mask image.

[0079] Step S14: If the comparison result is greater than or equal to a preset threshold, it is determined that the hair type of the portrait in the image to be processed is front-length hair.

[0080] Step S15: If the comparison result is less than a preset threshold, it is determined that the hair type of the portrait in the image to be processed is the back-length hair.

[0081] Step S20: Match the corresponding adjustment strategy according to the hair type.

[0082] Step S30: According to the adjustment strategy, the hair in the image to be processed is adjusted to eliminate the gap between the hair and the sides of the neck / shoulder neck, thereby obtaining a target image.

[0083] It should be noted that there are two types of hair-hanging: front hair-hanging and back hair-hanging, and each type corresponds to a different adjustment strategy.

[0084] When the hair type is back-down hair, the hair in the image to be processed is adjusted according to the adjustment strategy to eliminate the gap between the hair and the sides of the neck / shoulder neck to obtain a target image, including:

[0085] Step S31: Generate a well-shaped positioning frame.

[0086] Specifically, generating a well-shaped positioning frame includes:

[0087] Step S311: Acquire facial key points and preset feature positioning points of the portrait in the image to be processed.

[0088] Reference Figure 4 , Figure 4 This is a schematic diagram of the applicant's facial key points and preset feature positioning points.

[0089] In this embodiment, the facial key points are points 0, 6, 26 and 32 among the 106 facial key points. Figure 4 The points 0, 6, 26, and 32 marked in the figure are the intersection points of the neck and chin on both sides. Figure 4 The two points A and B marked in the figure, as well as the outermost points of the left and right shoulders, such as Figure 4 The two points marked C and D.

[0090] Among them, the method for obtaining facial key points is:

[0091] A1. Obtain a large number of facial images marked with 0, 6, 26, and 32 of the 106 facial key points as training sample sets.

[0092] A2. Input the training sample set into the neural network model to be trained and obtain the predicted coordinates of each facial key point as the output result;

[0093] A3. Based on the output results and the real coordinates, make judgments through R-square calculations;

[0094] A4. If R-squared is greater than or equal to a preset R-squared threshold, determine that the current neural network model to be trained is a facial key point recognition model;

[0095] A5. If the R-squared value is less than the preset R-squared threshold, adjust the parameters of the current neural network model to be trained and return to step A1;

[0096] A6. Use the trained facial key point recognition model to identify the image to be processed and obtain the facial key points of the person in the image to be processed;

[0097] The value range of the R-squared threshold is between 0 and 1. The specific value can be determined according to actual application conditions and is not specifically limited in this embodiment.

[0098] Among them, the method for obtaining the intersection of neck and chin is:

[0099] A1. Obtain a large number of portrait images with the neck-chin intersection marked as a training sample set;

[0100] A2. Input the training sample set into the neural network model to be trained and obtain the predicted coordinates of the neck-chin intersection as the output result;

[0101] A3. Based on the output results and the real coordinates, make judgments through R-square calculations;

[0102] A4. If the R-squared value is greater than or equal to the preset R-squared threshold, the current neural network model to be trained is determined to be a neck-chin intersection recognition model;

[0103] A5. If the R-squared value is less than the preset R-squared threshold, adjust the parameters of the current neural network model to be trained and return to step A1;

[0104] A6. Use the trained neck-chin intersection recognition model to identify the image to be processed and obtain the neck-chin intersection of the person in the image to be processed;

[0105] The value range of the R-squared threshold is between 0 and 1. The specific value can be determined according to actual application conditions and is not specifically limited in this embodiment.

[0106] Among them, the method for obtaining the outermost point of the shoulder is:

[0107] A1. Obtain a large number of portrait images with the outermost points of the shoulders marked as a training sample set;

[0108] A2. Input the training sample set into the neural network model to be trained and obtain the predicted coordinates of the outermost point of the shoulder as the output result;

[0109] A3. Based on the output results and the real coordinates, make judgments through R-square calculations;

[0110] A4. If the R-squared value is greater than or equal to the preset R-squared threshold, the current neural network model to be trained is determined to be a shoulder outermost point recognition model;

[0111] A5. If the R-squared value is less than the preset R-squared threshold, adjust the parameters of the current neural network model to be trained and return to step A1;

[0112] A6. Use the trained shoulder outermost point recognition model to identify the image to be processed and obtain the shoulder outermost point of the portrait in the image to be processed;

[0113] The value range of the R-squared threshold is between 0 and 1. The specific value can be determined according to actual application conditions and is not specifically limited in this embodiment.

[0114] Step S312: Generate a tic-tac-toe positioning frame based on the facial key points and the preset feature positioning points.

[0115] In this embodiment, when the facial key points are points 0, 6, 26, and 32 among the 106 facial key points, and the preset feature positioning points are the neck-chin intersection points A and B on the left and right sides and the outermost points C and D of the left and right shoulders, this application generates a tic-tac-toe positioning frame based on the facial key points and the preset feature positioning points, specifically including:

[0116] Take the horizontal coordinates of the facial key points 0 and 32, and in the image coordinate system, use the vertical line where the horizontal coordinate of point 0 is located as the left vertical positioning line of the well-shaped positioning frame, and use the vertical line where the horizontal coordinate of point 32 is located as the right vertical positioning line of the well-shaped positioning frame;

[0117] Take the maximum value of the vertical coordinates of the 6 and 26 facial key points, and in the image coordinate system, use the horizontal line where the maximum value of the vertical coordinates of the 6 and 26 facial key points is located as the upper positioning line of the tic-tac-toe positioning frame;

[0118] Take the maximum value of the vertical coordinates of the outermost points C and D of the shoulder. In the image coordinate system, use the horizontal straight line where the maximum value of the vertical coordinates of the outermost points C and D of the shoulder is located as the upper positioning line of the tic-tac-toe positioning frame.

[0119] Reference Figure 5 , Figure 5 This is a scene application diagram of adjusting the rectangle based on the neck area and adjusting the distance between the hair and both sides of the neck.

[0120] Reference Figure 6 , Figure 6 This is a scene application diagram of adjusting the rectangle based on the shoulder and neck area and adjusting the distance between the hair and the shoulder and neck.

[0121] Step S32: Based on the well-shaped positioning frame, determine the neck area adjustment rectangle and the shoulder and neck area adjustment rectangle.

[0122] In this embodiment, the neck area adjustment rectangle includes a left adjustment rectangle and a right adjustment rectangle. The left adjustment rectangle is determined as follows: the left vertical positioning line of the well-shaped positioning frame is used as the straight line where the left edge of the left adjustment rectangle is located, the upper positioning line of the well-shaped positioning frame is used as the straight line where the upper edge of the left adjustment rectangle is located, the lower positioning line of the well-shaped positioning frame is used as the straight line where the lower edge of the left adjustment rectangle is located, and the vertical line where the horizontal coordinate of the neck-chin intersection point A is located is used as the straight line where the right edge of the left adjustment rectangle is located. The four straight lines intersect to obtain the left adjustment rectangle, as shown in FIG. Figure 5 The left rectangle in the middle tic-tac-toe positioning box.

[0123] The right adjustment rectangle is determined as follows: the right vertical positioning line of the well-shaped positioning frame is used as the straight line for the right edge of the right adjustment rectangle, the upper positioning line of the well-shaped positioning frame is used as the straight line for the upper edge of the right adjustment rectangle, the lower positioning line of the well-shaped positioning frame is used as the straight line for the lower edge of the right adjustment rectangle, and the vertical line where the horizontal coordinate of point B, the intersection of the neck and chin, is located is used as the straight line for the left edge of the right adjustment rectangle. The four straight lines intersect to obtain the right adjustment rectangle, as shown in the figure below. Figure 5 The right rectangle in the well-shaped positioning box.

[0124] The shoulder and neck area adjustment rectangle is determined as follows: the lower positioning line of the tic-tac-toe positioning frame is used as the straight line for the lower edge of the shoulder and neck area adjustment rectangle, the vertical line where the horizontal coordinate of the outermost point C of the shoulder is located is used as the straight line for the left edge of the shoulder and neck area adjustment rectangle, the vertical line where the horizontal coordinate of the outermost point D of the shoulder is located is used as the straight line for the right edge of the shoulder and neck area adjustment rectangle, and the horizontal line where the minimum value of the vertical coordinates of the intersection of the neck and chin points A and B is located is used as the straight line for the upper edge of the shoulder and neck area adjustment rectangle, as shown in the figure below. Figure 6 The lower rectangle of the middle well-shaped positioning frame.

[0125] Step S33: adjusting the rectangle based on the neck area to adjust the distance between the hair and both sides of the neck.

[0126] In this embodiment, the neck region adjustment rectangle includes a left adjustment rectangle and a right adjustment rectangle.

[0127] Based on the left adjustment rectangle, the distance between the hair and the left side of the neck is adjusted, including: keeping the position of the left edge of the left adjustment rectangle unchanged, horizontally stretching the right edge of the left adjustment rectangle to a preset position, and mapping the texture of the hair area within the left adjustment rectangle to the stretched area, where the stretched area refers to the enlarged portion of the left adjustment rectangle after stretching compared to the left adjustment rectangle before stretching.

[0128] Based on the right adjustment rectangle, the distance between the hair and the right side of the neck is adjusted, including: keeping the position of the right edge of the right adjustment rectangle unchanged, horizontally stretching the left edge of the right adjustment rectangle to a preset position, and mapping the texture of the hair area within the right adjustment rectangle to the stretched area, where the stretched area refers to the enlarged portion of the right adjustment rectangle after stretching compared to the right adjustment rectangle before stretching.

[0129] It should be noted that the texture of the hair area within the left / right adjustment rectangle is mapped to the stretched area, and the mapping method adopted is texture mapping in graphics rendering.

[0130] Among them, the method for determining the preset position is as follows: calculate the midpoints of the facial key points 6 and 26, as well as the midpoints of the outermost points C and D of the shoulders, connect the two midpoints to obtain the midpoint line, and extend the midpoint line upward and downward, intersecting with the upper and lower positioning lines of the tic-tac-toe positioning frame to obtain the midpoint of the neck area. The midpoint of the neck area is the preset position.

[0131] It should be noted that, in this embodiment, by adjusting the rectangle based on the neck area and adjusting the distance between the hair and both sides of the neck, image distortion caused by excessive stretching can be avoided.

[0132] Step S34: adjusting the rectangle based on the neck and shoulder area, adjusting the distance between the hair and the neck and shoulder to obtain a target image.

[0133] Based on the shoulder and neck area adjustment rectangle, the distance between the hair and the shoulder and neck is adjusted, including: determining a first preset coordinate point and a second preset coordinate point according to the coordinates of each vertex of the shoulder and neck area adjustment rectangle, keeping the position of the upper edge of the shoulder and neck area adjustment rectangle unchanged, horizontally stretching the lower left point of the shoulder and neck area adjustment rectangle to the first preset coordinate point, horizontally stretching the lower right point of the shoulder and neck area adjustment rectangle to the second preset coordinate point, and mapping the hair area texture within the shoulder and neck area adjustment rectangle to the stretched area, where the stretched area refers to the enlarged portion of the shoulder and neck area adjustment rectangle compared to the shoulder and neck area adjustment rectangle before stretching.

[0134] It should be noted that the texture of the hair area within the shoulder and neck area adjustment rectangle is mapped to the stretched area, and the mapping method adopted is the texture mapping in graphics rendering.

[0135] The method for determining the first preset coordinate point and the second preset coordinate point is as follows:

[0136] Calculate the straight-line distance d between the 6 and 26 facial key points;

[0137] Get the coordinates of each vertex of the shoulder and neck area adjustment rectangle, the upper left vertex (X lu , Y lu ), the lower left vertex (X ld, Y ld )、the upper right vertex (X ru , Y ru ) and the lower right vertex (X rd , Y rd );

[0138] Calculate the first preset coordinate point and the second preset coordinate point using the formula:

[0139] Y ld '=Y lu +max((Y ld -Y lu )×1.7, d), that is, the first preset coordinate point is (X ld , Y ld ');

[0140] Y rd '=Y ru +max((Y rd -Y ru )×1.7, d), that is, the second preset coordinate point is (X ld , Y rd ').

[0141] It should be noted that, in this embodiment, by adjusting the rectangle based on the neck and shoulder area and adjusting the distance between the hair and the neck and shoulder, image distortion caused by excessive stretching can be avoided.

[0142] Reference Figure 7 , Figure 7 This is a scene application diagram for adjusting the hair in the image to be processed when the hair type of this application is front-facing hair.

[0143] When the hair type is front-down hair, adjusting the hair in the image to be processed according to the adjustment strategy to eliminate gaps between the hair and both sides of the neck / shoulder neck to obtain a target image includes:

[0144] Step S301: Generate a well-shaped positioning frame.

[0145] It should be noted that the specific process steps of generating the well-shaped positioning frame in step S301 are the same as those in step S31 and will not be repeated here.

[0146] Step S302: Acquire a third mask image of the head region, a fourth mask image of the neck region, and a fifth mask image of the clothing region of the image to be processed.

[0147] In this embodiment, obtaining a third mask image of the head area, a fourth mask image of the neck area, and a fifth mask image of the clothing area of ​​the image to be processed may include: hair-level portrait segmentation to obtain a third mask image of the head area of ​​the image to be processed containing hair and face; and obtaining a fourth mask image of the neck area and a fifth mask image of the clothing area of ​​the image to be processed through portrait semantic segmentation.

[0148] Step S303: Determine an area in the tic-tac-toe positioning frame where the mask is empty based on the third mask image, the fourth mask image, and the fifth mask image.

[0149] In this embodiment, based on the third mask image, the fourth mask image and the fifth mask image, determining the area in the tic-tac-toe positioning frame where the mask is empty may include: superimposing the third mask image, the fourth mask image and the fifth mask image in the tic-tac-toe positioning frame, and calculating the area in the tic-tac-toe positioning frame where the pixel value is 0 according to the pixel values ​​corresponding to the third mask image, the fourth mask image and the fifth mask image, and this area is the area in the tic-tac-toe positioning frame where the mask is empty.

[0150] Step S304: Collect the hair area texture to fill the empty area of ​​the mask.

[0151] In this embodiment, the hair area texture is collected to fill the empty area of ​​the mask, and the filling technology used is the inpainting filling technology.

[0152] It should be noted that there are two types of hair types: hair hanging in front and hair hanging in back. The area of ​​hair that needs to be adjusted when the hair is hanging in back is much larger than the area of ​​hair that needs to be adjusted when the hair is hanging in front. For adjusting a small area of ​​hair, the inpainting filling technology can be used. The inpainting filling technology has a better filling effect in a small area than in a large area.

[0153] Step S305: Merge the edge of the filling area with the original hair area to obtain a target image.

[0154] In this embodiment, the edge of the filling area is fused with the original hair area, and the fusion technology adopted is an edge fusion method.

[0155] Compared to existing techniques that modify ID photos, resulting in gaps between the portrait's hair and neck, and a discontinuity between the hair and shoulders after changing their appearance, which reduces the authenticity of the ID photo, the present application pre-processes the image to be processed, determines the hair type of the portrait in the image to be processed; matches a corresponding adjustment strategy based on the hair type; and adjusts the hair in the image to eliminate the gaps between the hair and the sides of the neck / shoulders based on the adjustment strategy, thereby obtaining a target image. Specifically, the image processing method of the present application adjusts the hair based on the hair type, fills the gaps between the portrait's hair and neck, and repairs the discontinuity between the hair and shoulders, thereby improving the authenticity of the ID photo.

[0156] The embodiment of the present application further provides a second embodiment based on the first embodiment of the image processing method.

[0157] In this embodiment, before the step of pre-processing the image to be processed and determining the type of hair of the portrait in the image to be processed, the method further includes:

[0158] Determine hair type based on user-uploaded images;

[0159] In the case where the hair type is long hair, the portrait in the image is scaled according to a preset ratio, and the scaled portrait is placed at a position in the image that meets a preset condition to obtain an image to be processed.

[0160] It's important to note that when modifying ID photos, if the user uploads an image with short hair, the hair area doesn't need to be stretched or deformed to produce a more authentic ID photo. Therefore, determining hair type in user-uploaded images allows for early screening, reducing the workload of subsequent image processing.

[0161] In this embodiment, determining the hair type of an image uploaded by a user may include: after segmenting the hair region and the clothing region using a preset semantic segmentation model, first determining whether the hair region and the clothing region are adjacent. If the hair region and the clothing region are determined not to be adjacent, determining the hair type corresponding to the hair region as short hair. If the hair region and the clothing region are determined to be adjacent, determining the hair type corresponding to the hair region as long hair.

[0162] It should be noted that the size of the portrait in different ID photos is different. Therefore, the portrait in the image needs to be scaled according to a preset ratio based on the type of ID photo. In this embodiment, when scaling the portrait in the image according to a preset ratio, the head width and head length need to be scaled according to the preset ratio at the same time.

[0163] In this embodiment, the preset conditions for placing the scaled portrait at a position in the image that meets the preset conditions may include: the straight-line distance between the highest point of the portrait's head and the upper edge of the image is L1, and the straight-line distances between the outer points of the left and right shoulders of the portrait and the left and right edges of the image are both L2.

[0164] The present application also provides an image processing device for use on a video platform. Figure 8 , Figure 8 This is a schematic diagram of the functional modules of the first embodiment of the image processing device of the present application.

[0165] In this embodiment, the image processing device includes:

[0166] A preprocessing module 10 is used to preprocess the image to be processed and determine the hair type of the portrait in the image to be processed;

[0167] A strategy determination module 20 is used to match a corresponding adjustment strategy according to the hair type;

[0168] The adjustment module 30 is configured to adjust the hair in the image to be processed according to the adjustment strategy to eliminate the gap between the hair and the sides of the neck / shoulder neck, thereby obtaining a target image.

[0169] Optionally, the preprocessing module includes:

[0170] A first acquisition subunit is configured to acquire a first mask image of a hair region and a second mask image of a face region of an image to be processed;

[0171] an extraction subunit, configured to extract a left shoulder-neck curve and a right shoulder-neck curve from the image to be processed;

[0172] a comparison subunit, configured to calculate the number of pixels of the first mask image below the left shoulder-neck curve and the right shoulder-neck curve, and compare the number of pixels with the second mask image to obtain a comparison result;

[0173] The type determination subunit is used to determine that the hair type of the portrait in the image to be processed is front-hanging hair if the comparison result is greater than or equal to a preset threshold, and to determine that the hair type of the portrait in the image to be processed is back-hanging hair if the comparison result is less than the preset threshold.

[0174] Optionally, when the hair-down type is hair-down at the back, the adjustment module includes:

[0175] The first determining subunit is used to generate a well-shaped positioning frame;

[0176] A second determining subunit is configured to determine a neck region adjustment rectangle and a shoulder and neck region adjustment rectangle based on the well-shaped positioning frame;

[0177] A first adjusting subunit, configured to adjust the rectangle based on the neck area to adjust the distance between the hair and both sides of the neck;

[0178] The second adjustment subunit is used to adjust the rectangle based on the neck and shoulder area, adjust the distance between the hair and the neck and shoulder, and obtain a target image.

[0179] Optionally, the neck area adjustment rectangle includes a left adjustment rectangle and a right adjustment rectangle, which are used to implement:

[0180] horizontally stretching the right edge of the left adjustment rectangle to a preset position, and mapping the texture of the hair area within the left adjustment rectangle to the stretched area;

[0181] horizontally stretching the left edge of the right adjustment rectangle to a preset position, and mapping the texture of the hair area within the right adjustment rectangle to the stretched area;

[0182] Wherein, the preset position is the midline of the neck area.

[0183] Optionally, the second adjusting subunit is configured to implement:

[0184] Adjust the coordinates of each vertex of the rectangle according to the shoulder and neck area to determine a first preset coordinate point and a second preset coordinate point;

[0185] Horizontally stretch the lower left point of the shoulder and neck area adjustment rectangle to the first preset coordinate point, horizontally stretch the lower right point of the shoulder and neck area adjustment rectangle to the second preset coordinate point, and map the hair area texture within the shoulder and neck area adjustment rectangle to the stretched area.

[0186] Optionally, when the hair-down type is front-down hair, the adjustment module includes:

[0187] The third determining subunit is used to generate a well-shaped positioning frame;

[0188] a second acquisition subunit, configured to acquire a third mask image of the head region, a fourth mask image of the neck region, and a fifth mask image of the clothing region of the image to be processed;

[0189] A fourth determining subunit is configured to determine an area in the tic-tac-toe positioning frame where the mask is empty based on the third mask image, the fourth mask image, and the fifth mask image;

[0190] A filling subunit, configured to collect the texture of the hair region to fill the empty region of the mask;

[0191] The fusion subunit is used to fuse the edge of the filling area with the original hair area to obtain the target image.

[0192] Optionally, the first determining subunit and the third determining subunit are configured to implement:

[0193] Obtaining facial key points and preset feature positioning points of the portrait in the image to be processed;

[0194] A tic-tac-toe positioning frame is generated based on the facial key points and the preset feature positioning points.

[0195] Optionally, the image processing device further includes:

[0196] A judgment module is used to judge the hair type of the image uploaded by the user;

[0197] The processing module is used to scale the portrait in the image according to a preset ratio when the hair type is long hair, and place the scaled portrait at a position in the image that meets preset conditions to obtain an image to be processed.

[0198] The specific implementation of the image processing device of the present application is basically the same as the embodiments of the above-mentioned image processing method, and will not be repeated here.

[0199] An embodiment of the present application further provides a storage medium, on which an image processing program is stored. When the image processing program is executed by a processor, the steps of the image processing method described above are implemented.

[0200] The specific implementation of the storage medium of the present application is basically the same as the embodiments of the above-mentioned image processing method, and will not be repeated here.

[0201] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0202] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0203] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0204] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image processing method, characterized in that: The image processing method comprises the following steps: Preprocessing the image to be processed to determine the hair type of the portrait in the image to be processed; According to the hair type, matching the corresponding adjustment strategy; According to the adjustment strategy, the hair in the image to be processed is adjusted to eliminate the gap between the hair and the sides of the neck / shoulder neck, thereby obtaining a target image; When the hair type is the back-down hair, the step of adjusting the hair in the image to be processed according to the adjustment strategy to eliminate the gap between the hair and the sides of the neck / shoulder neck to obtain the target image includes: Generate a well-shaped positioning frame; Based on the well-shaped positioning frame, determining a neck region adjustment rectangle and a shoulder and neck region adjustment rectangle; Adjusting the rectangle based on the neck area to adjust the distance between the hair and both sides of the neck; Adjusting the rectangle based on the neck and shoulder area, adjusting the distance between the hair and the neck and shoulder to obtain a target image; When the hair type is front-down hair, the step of adjusting the hair in the image to be processed according to the adjustment strategy to eliminate the gap between the hair and the sides of the neck / shoulder neck to obtain the target image includes: Generate a well-shaped positioning frame; Acquire a third mask image of the head region, a fourth mask image of the neck region, and a fifth mask image of the clothing region of the image to be processed; Determining, based on the third mask image, the fourth mask image, and the fifth mask image, an area in the tic-tac-toe locating frame where the mask is empty; Collecting the hair area texture to fill the empty area of ​​the mask; The edge of the filled area is fused with the original hair area to obtain the target image.

2. The image processing method according to claim 1, wherein: The step of pre-processing the image to be processed and determining the hair type of the portrait in the image to be processed includes: Acquire a first mask image of the hair area and a second mask image of the face area of ​​the image to be processed; extracting a left shoulder-neck curve and a right shoulder-neck curve from the image to be processed; Calculating the number of pixels of the first mask image below the left shoulder-neck curve and the right shoulder-neck curve, and comparing the number of pixels with the second mask image to obtain a comparison result; If the comparison result is greater than or equal to a preset threshold, determining that the hair type of the portrait in the image to be processed is front-length hair; If the comparison result is less than a preset threshold, it is determined that the hair-falling type of the portrait in the image to be processed is the back-falling hair.

3. The image processing method according to claim 1, wherein: The neck area adjustment rectangle includes a left adjustment rectangle and a right adjustment rectangle. The step of adjusting the distance between the hair and both sides of the neck based on the neck area adjustment rectangle includes: horizontally stretching the right edge of the left adjustment rectangle to a preset position, and mapping the texture of the hair area within the left adjustment rectangle to the stretched area; horizontally stretching the left edge of the right adjustment rectangle to a preset position, and mapping the texture of the hair area within the right adjustment rectangle to the stretched area; Wherein, the preset position is the midline of the neck area.

4. The image processing method according to claim 1, wherein: The step of adjusting the distance between the hair and the neck and shoulder based on the adjustment rectangle of the neck and shoulder area includes: Adjust the coordinates of each vertex of the rectangle according to the shoulder and neck area to determine a first preset coordinate point and a second preset coordinate point; Horizontally stretch the lower left point of the shoulder and neck area adjustment rectangle to the first preset coordinate point, horizontally stretch the lower right point of the shoulder and neck area adjustment rectangle to the second preset coordinate point, and map the hair area texture within the shoulder and neck area adjustment rectangle to the stretched area.

5. The image processing method according to claim 1, wherein: The step of generating a well-shaped positioning frame includes: Obtaining facial key points and preset feature positioning points of the portrait in the image to be processed; A tic-tac-toe positioning frame is generated based on the facial key points and the preset feature positioning points.

6. An image processing device, characterized in that The device comprises: A preprocessing module, configured to preprocess the image to be processed and determine the hair type of the portrait in the image to be processed; A strategy determination module, configured to match a corresponding adjustment strategy according to the hair type; an adjustment module, configured to adjust the hair in the image to be processed according to the adjustment strategy to eliminate the gap between the hair and the sides of the neck / shoulder neck, thereby obtaining a target image; When the hair-hanging type is back-hanging, the adjustment module is used to implement: Generate a well-shaped positioning frame; Based on the well-shaped positioning frame, determining a neck region adjustment rectangle and a shoulder and neck region adjustment rectangle; Adjusting the rectangle based on the neck area to adjust the distance between the hair and both sides of the neck; Adjusting the rectangle based on the neck and shoulder area, adjusting the distance between the hair and the neck and shoulder to obtain a target image; When the hair-hanging type is front-hanging hair, the adjustment module is used to implement: Generate a well-shaped positioning frame; Acquire a third mask image of the head region, a fourth mask image of the neck region, and a fifth mask image of the clothing region of the image to be processed; Determining, based on the third mask image, the fourth mask image, and the fifth mask image, an area in the tic-tac-toe locating frame where the mask is empty; Collecting the hair area texture to fill the empty area of ​​the mask; The edge of the filled area is fused with the original hair area to obtain the target image.

7. An image processing device, characterized in that The device includes: a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program is configured to implement the steps of the image processing method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores an image processing program, which implements the steps of the image processing method according to any one of claims 1 to 5 when executed by the processor.

Images