Face shape transformation method, device, equipment and storage medium

Abstract

The application provides a face shape transformation method, device, equipment and storage medium. The method comprises the following steps: obtaining a predetermined part length and a first key point located on a face contour from a user face shape image; obtaining a second key point corresponding to the first key point from a target face shape image; determining a transformed coordinate of each pixel point in a first key point corresponding to a to-be-adjusted region according to the predetermined part length, the first key point and the second key point; and stretching and transforming each pixel point according to a preset stretching coefficient and the transformed coordinate of each pixel point. The improved inverse transformation algorithm is used to control the stretching amplitude of the pixel points in the to-be-adjusted region, and the mutual transformation of various face shapes is adapted. The attenuation coefficient is further used to limit the stretching amplitude, so that the unnatural effect caused by the excessively large adjustment amplitude is avoided. Before the face shape transformation, the left and right sides and the lower side of the face boundary in the user face shape image are mirror filled, so that the problem of pseudo texture is avoided.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to a face shape transformation method, apparatus, device, and storage medium. Background Technology

[0002] Currently, many image processing scenarios require facial transformation processing of human face images. For example, in virtual beauty and plastic surgery scenarios, it is often necessary to transform the user's face image to adjust the user's face shape to a satisfactory one.

[0003] A face transformation method is provided in related technologies. This method selects multiple key points from a user's face image and performs a fixed-amplitude scaling on each key point to obtain the transformed face image. However, this related technology performs a fixed-amplitude scaling on all user face images, which has poor adaptability and easily leads to unnatural face shapes after transformation. Summary of the Invention

[0004] This application proposes a face shape transformation method, apparatus, device, and storage medium. A preset number of key points are selected in a user face shape image and a target face shape image. Based on the selected key points, the area to be adjusted is determined. The stretching amplitude of each pixel in the area to be adjusted is appropriately controlled, resulting in a more natural face shape transformation that can adapt to the mutual transformation between various face shapes.

[0005] The first aspect of this application proposes a face shape transformation method, including:

[0006] Obtain the length of a predetermined part and a first key point located on the facial contour from the user's face image, and obtain the second key point corresponding to the first key point from the target face image;

[0007] Based on the predetermined part length, the first key point, and the second key point, the transformed coordinates of each pixel in the area to be adjusted corresponding to the first key point are determined.

[0008] Based on the preset stretching coefficient and the transformed coordinates of each pixel, the pixel in the user's face image is stretched and transformed.

[0009] In some embodiments of this application, determining the transformed coordinates of each pixel in the region to be adjusted corresponding to the first key point based on the predetermined part length, the first key point, and the second key point includes:

[0010] A first circular region centered on the first key point is determined from the user's face image, and the first circular region is taken as the region to be adjusted corresponding to the first key point.

[0011] Based on the predetermined part length, the coordinates of the first key point, the coordinates of the second key point, and the coordinates of each pixel in the area to be adjusted, the transformed coordinates of each pixel are calculated respectively.

[0012] In some embodiments of this application, the step of stretching and transforming each pixel in the user's face image according to a preset stretching coefficient and the transformed coordinates of each pixel includes:

[0013] Based on the preset stretching coefficient, the predetermined part length, the first key point, the second key point, and the transformed coordinates of the first pixel, the original coordinates of the first pixel are calculated, where the first pixel is any pixel in the area to be adjusted.

[0014] Based on the user's face image and the coordinates before transformation corresponding to the first pixel, determine the pixel value corresponding to the coordinates before transformation;

[0015] Modify the pixel value of the pixel at the transformed coordinates in the result image to the pixel value at the original coordinates in the user image.

[0016] In some embodiments of this application, calculating the original coordinates of the first pixel based on a preset stretching coefficient, the predetermined part length, the first key point, the second key point, and the transformed coordinates of the first pixel includes:

[0017] Based on the preset stretching coefficient, the predetermined part length, the transformed coordinates of the first key point and the first pixel point, calculate the amplitude control coefficient corresponding to the first key point;

[0018] Calculate the stretch distance between the first key point and the second key point;

[0019] The product of the stretching distance and the amplitude control coefficient is determined as the stretching amplitude corresponding to the first pixel.

[0020] Calculate the difference between the transformed coordinates of the first pixel and the stretching amplitude, and determine the difference as the original coordinates of the first pixel.

[0021] In some embodiments of this application, determining the pixel value corresponding to the pre-transform coordinates based on the user's face image and the pre-transform coordinates of the first pixel includes:

[0022] Obtain the pixel value of each pixel adjacent to the original coordinates of the first pixel in the user's face image;

[0023] Interpolation is performed on the pixel values ​​of each adjacent pixel to obtain the pixel value corresponding to the coordinates before the transformation.

[0024] In some embodiments of this application, obtaining the first key point located on the facial contour from the user's face image includes:

[0025] Perform face detection on user face images and identify all facial key points located on the facial contour;

[0026] Select the chin key point and multiple facial key points mirrored on both sides of the facial contour from all identified facial key points;

[0027] The selected chin key point and the multiple mirror-distributed facial key points are determined as the first key point corresponding to the user's face image.

[0028] In some embodiments of this application, before performing stretching transformation on each pixel in the user face image according to a preset stretching coefficient and the transformed coordinates corresponding to each pixel, the method further includes:

[0029] Calculate the stretching distance from the first keypoint to the second keypoint;

[0030] Based on the stretching distance, determine the attenuation coefficient corresponding to the first key point;

[0031] Calculate the attenuated second key point based on the attenuation coefficient, the first key point, and the second key point.

[0032] In some embodiments of this application, before performing stretching transformation on each pixel in the user face image according to a preset stretching coefficient and the transformed coordinates corresponding to each pixel, the method further includes:

[0033] In the user's face image, the left and right sides of the face boundary are filled with pixels of a preset width, and the lower side of the face boundary is filled with pixels of a preset height.

[0034] In some embodiments of this application, after stretching and transforming each pixel in the user's face image according to a preset stretching coefficient and the transformed coordinates corresponding to each pixel, the method further includes:

[0035] In the user's face image, the portion of the image filled on the left, right, and bottom sides of the face boundary is deleted.

[0036] In some embodiments of this application, after calculating the original coordinates of the first pixel based on a preset stretching coefficient and the transformed coordinates of the first pixel, the method further includes:

[0037] If the calculated pre-transformation coordinates are located outside the facial boundary in the user's face image, then the orientation relationship between the pre-transformation coordinates and the facial boundary is determined, and the orientation relationship is used to indicate whether the pre-transformation coordinates are located to the left, right, or bottom of the facial boundary.

[0038] In the user's face image, pixel mirroring is performed on one side of the face boundary indicated by the orientation relationship;

[0039] Based on the user's face image after filling, the operation of determining the pixel value corresponding to the coordinates before transformation is performed.

[0040] An embodiment of the second aspect of this application provides a face shape changing device, comprising:

[0041] The acquisition module is used to acquire the length of a predetermined part and a first key point located on the facial contour from the user's face image, and to acquire a second key point corresponding to the first key point from the target face image.

[0042] The determining module is used to determine the transformed coordinates of each pixel in the region to be adjusted corresponding to the first key point based on the predetermined part length, the first key point, and the second key point.

[0043] The transformation module is used to stretch and transform each pixel in the user's face image according to a preset stretching coefficient and the transformed coordinates of each pixel.

[0044] An embodiment of the third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0045] An embodiment of the fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method described in the first aspect above.

[0046] The technical solutions provided in this application embodiment have at least the following technical effects or advantages:

[0047] In this embodiment, a predetermined part length and a first key point are obtained from a user's face image, and a second key point is obtained from a target face image. Based on the first and second key points, the transformed coordinates of each pixel within the area to be adjusted in the user's face image are determined. Then, each pixel within the area to be adjusted is stretched according to a preset stretching coefficient and the transformed coordinates. Given the first key point that needs to be stretched for face shape transformation, face shape transformation can be achieved by locally translating and deforming the area to be adjusted corresponding to each first key point. Furthermore, during the transformation process, the stretching amplitude of each pixel within the area to be adjusted corresponding to the first key point is controlled by a preset stretching coefficient, avoiding excessive stretching that leads to unnatural face shape transformation effects. This face shape transformation process is applicable to transformation scenarios between any two face shapes, even for asymmetrical face shape transformations. As long as a user's face image and a target face image are given, the face shape in the user's face image can be adjusted to the face shape in the target face image. The face shape transformation is highly accurate, the effect is more natural, and the applicability is stronger.

[0048] Furthermore, a preset number of key points were selected in both the user's face image and the target face image. Based on these key points, the area to be adjusted was determined. An improved inverse transform algorithm was used to appropriately control the stretching amplitude of each pixel in the area to be adjusted, enabling it to adapt to various face shape transformations. An attenuation coefficient was also added to further limit the stretching amplitude of each pixel, preventing excessive adjustment that could result in an unnatural stretching effect. Before performing the face shape transformation, pixel mirroring was applied to the left, right, and bottom edges of the face in the user's face image to avoid pseudo-texture issues after the transformation.

[0049] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0050] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0051] Figure 1 A flowchart of a face shape transformation method provided in an embodiment of this application is shown;

[0052] Figure 2 A schematic diagram illustrating the selection of key points provided in an embodiment of this application is shown;

[0053] Figure 3This illustration shows the effect of adding an attenuation coefficient to the front face shape transformation according to an embodiment of this application;

[0054] Figure 4 This illustration shows the effect of facial shape transformation after adding an attenuation coefficient according to an embodiment of this application;

[0055] Figure 5 This illustration shows a schematic diagram of pixel mirroring filling on the left and right sides and the bottom side of the face boundary in a user face image provided in an embodiment of this application;

[0056] Figure 6 This illustration shows a schematic diagram of the face shape transformation effect provided in an embodiment of this application;

[0057] Figure 7 This invention provides a schematic diagram of the structure of a face shape changing device according to an embodiment of the present application.

[0058] Figure 8 This illustration shows a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0059] Figure 9 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0060] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0061] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0062] The following description, in conjunction with the accompanying drawings, describes a face shape transformation method, apparatus, device, and storage medium according to embodiments of this application.

[0063] Traditional face reshaping methods currently employ a process of selecting multiple key points from a user's facial image. If the user desires a slimmer face, each key point is contracted inwards by a fixed amount. Conversely, if a fuller face is desired, each key point is stretched outwards by a fixed amount. Because this fixed-amount contraction or stretching is applied to all user facial images, traditional face reshaping methods have poor adaptability, often resulting in unnatural-looking faces. Furthermore, traditional methods only offer simple slimming or fuller face adjustments, failing to address the transformation of more diverse face shapes, such as square faces, round faces, or even asymmetrical faces.

[0064] Addressing the problems of traditional face shape transformation methods, this application proposes a face shape transformation method. This method, based on local translation deformation, determines the pixels that need adjustment when transforming from one face shape to another, and appropriately controls the stretching amplitude of each pixel, enabling it to adapt to transformations between various face shapes. Furthermore, this application incorporates an attenuation coefficient to limit the stretching amplitude of each pixel, preventing excessive adjustment that could lead to unnatural stretching effects. Before performing face shape transformation, this application also performs pixel mirroring fill on the left, right, and bottom edges of the user's face image to avoid pseudo-texture issues after transformation.

[0065] See Figure 1 The method specifically includes the following steps:

[0066] Step 101: Obtain the length of a predetermined part and the first key point located on the facial contour from the user's face image, and obtain the second key point corresponding to the first key point from the target face image.

[0067] The execution subject of this application embodiment is a device such as a terminal or server used for face shape transformation. This device stores a target face shape library, or can obtain the target face shape library from the cloud. The target face shape library stores a large number of target face shape images, covering various face shapes such as square, round, oval, and heart-shaped faces. All target face shape images are aligned with a standard face. The standard face, also known as the average face, is obtained by averaging the positions of the eyes, nose tip, corners of the mouth, eyebrows, etc., from a massive number of face images. Affine transformation aligns the original target face shape image with the standard face, effectively making the positions of the eyes, nose tip, corners of the mouth, eyebrows, etc., in the target face shape image identical to those in the standard face.

[0068] When a user needs to change their face shape, they submit a face image to the device. This image can be taken with a camera, selected from the device's locally stored face images, or downloaded from the internet. The user also needs to select a target face image from the device's target face image library. The device then adjusts the face shape in the submitted image to match the selected target face image.

[0069] After receiving the user's submitted face image, the device also aligns the user's face image with a standard face through affine transformation, so that the orientation of the face in the user's face image and the position of each part in the image are consistent, thus improving the accuracy of subsequent face transformation based on the user's face image and the target face image.

[0070] After obtaining the target face image and the user face image, both aligned with a standard face, the device first performs face detection on the user face image, identifying various facial key points and numbering each key point. The identified facial key points include those located on the facial contour, eyebrows, eyes, nose, and mouth, and the number of identified key points can be 67 or 96, etc. Based on the identified key points, a predetermined length of a specific area is obtained from the user face image. This predetermined length can be the distance between the center points of the two eyes, the distance between the center points of the two eyebrows, the distance between the key points of the two corners of the mouth, etc. Furthermore, all facial key points located on the facial contour are obtained from the user face image, and a predetermined number of first key points are selected from all the facial key points on the facial contour. The predetermined number can be 2N+1 facial key points, where N is an integer greater than or equal to 1. Specifically, from all facial key points on the facial contour, the chin key point located at the lowest position of the chin is selected, and 2N personal facial key points that are mirrored on both sides of the facial contour are selected. The selected chin key point and 2N personal facial key points are determined as the first key point corresponding to the user's face image.

[0071] As an example, such as Figure 2 As shown, 67 facial key points were identified from the user's face image, and each facial key point was numbered. Two facial key points numbered 3 and 13 were selected from the mirror distribution of the facial contour, two facial key points numbered 5 and 11 were selected from the mirror distribution, and a facial key point numbered 8 was selected from the chin. Thus, five facial key points numbered 3, 5, 8, 11 and 13 were selected as the first key points.

[0072] Similarly, the device performs face detection on the target face image submitted by the user, identifies each facial key point in the target face image, and assigns a number to each facial key point. It then obtains the second key point corresponding to each first key point from the target face image. Specifically, for each first key point, a second key point with the same number as the first key point is obtained from the target face image. That is, the corresponding first and second key points are located in the same position on the face.

[0073] Step 102: Based on the obtained predetermined part length, first key point and second key point, determine the transformed coordinates of each pixel in the area to be adjusted corresponding to the first key point.

[0074] For each of the preset number of first key points obtained in step 101, the corresponding local deformation adjustment operation is the same. Therefore, in this embodiment, only one first key point is used as an example for detailed explanation.

[0075] Specifically, a first circular region centered on a first key point is determined from the user's face image. This first circular region is designated as the region to be adjusted corresponding to the first key point, which is the region corresponding to the first key point before face transformation. The radius of this adjustment region can be the predetermined part length mentioned above. Based on the predetermined part length, the coordinates of the first key point, the coordinates of the second key point, and the coordinates of each pixel in the region to be adjusted, the transformed coordinates of each pixel are calculated. Specifically, the transformed coordinates of each pixel can be calculated using the following formula (1).

[0076]

[0077] In formula (1), u is the coordinate of the first pixel before transformation, and the first pixel is any pixel in the area to be adjusted. x is the coordinate of the first pixel after transformation, and r max Let c be the predetermined length of the key point, c be the coordinates of the first key point, and m be the coordinates of the second key point. In this embodiment, the user's face image and the target face image are in the same coordinate system. In the figure, → represents the vector pointing from the origin to the corresponding point. The vector pointing from the origin of the coordinate system to the first pixel before the face shape transformation. This is the vector pointing from the origin of the coordinate system to the first pixel after the face shape transformation. The vector pointing from the origin to the first key point. The vector pointing from the origin to the second key point.

[0078] Step 103: Based on the preset stretching coefficient and the transformed coordinates of each pixel in the area to be adjusted, stretch and transform each of the above-mentioned pixels in the user's face image.

[0079] This application embodiment performs local translation deformation on the area to be adjusted. The principle is to stretch the center of the area to be adjusted (i.e., the first key point) to the second key point in the target face image, and transform the circumference of the area to be adjusted and any point inside the circle to the corresponding point in the target face image. The coordinates of each point are adjusted to the transformed coordinates through local translation. In the actual calculation process, this application embodiment uses the inverse transformation algorithm. Based on the target face image, the transformed area corresponding to the area to be adjusted can be known. For the coordinates of any point in the transformed area, they are the known transformed coordinates. According to the known transformed coordinates, the corresponding original coordinates can be calculated by the inverse transformation algorithm shown in formula (1). Based on the user face image, the pixel value at the original coordinates can be obtained. The pixel value at the transformed coordinates in the user face image is replaced with the pixel value at the original coordinates, thus achieving the effect of stretching the pixel point at the original coordinates to the transformed coordinates. Each pixel point in the area to be adjusted corresponding to each first key point is stretched and transformed, thus achieving the adjustment of the face in the user face image to the face in the target face image.

[0080] To reduce the unnatural appearance of the face after excessive stretching, this application embodiment improves upon the inverse transformation algorithm shown in formula (1) above, using the improved inverse transformation algorithm to stretch each pixel. First, the inverse transformation formula shown in formula (1) is modified to obtain formula (2):

[0081]

[0082] When x is located at position m Based on this, formula (3) is derived from formula (2):

[0083]

[0084] The position of x at m mentioned here is for the purpose of simplifying the formula. In this case, x and m do not represent any actual meaning; they are only used to identify any coordinate point after the face shape transformation.

[0085] In this embodiment of the application, the formula (3) is... Transformed into And expressed using the amplitude control coefficient "infect". Thus, we obtain formula (4):

[0086]

[0087] In formula (4), `infect` is the amplitude control coefficient, and `curve` is the preset stretching coefficient. The preset stretching coefficient `curve` is used to control the stretching amplitude of each pixel to avoid excessive stretching that would result in unnatural facial transformation.

[0088] Applying formula (4) to the face shape transformation in the embodiments of this application, it can be seen from formula (4) that... The stretching distance is used to stretch the first keypoint in the user's face image to the second keypoint in the target face image. This is the curve relating the coordinates before transformation to the stretching distance, where the value of curve affects the curvature of the curve.

[0089] Where, influence ∈ [0, 1]. When hour, That is, points located on the circumference do not move. When hour, The coordinates u before transformation are moved to the second keypoint m in the target face image. At that time, the distance moved was

[0090] Since the area to be adjusted is a circular area, the improved inverse transformation algorithm shown in formula (4) above performs image translation deformation processing within the circular area. The image is stretched with the first key point c as the center. The closer to the center, the greater the translation distance; that is, the closer to the center, the greater the deformation. The closer to the boundary of the area to be adjusted, the smaller the deformation. There is no deformation at the boundary, and the deformation is smooth. The preset stretching coefficient curve affects the curvature of the correlation curve between the coordinates before transformation and the stretching distance. The larger the curve, the smoother and more rounded the stretching deformation; the smaller the curve, the more abrupt and sharp the stretching deformation. If the adjustment is too large, an unnatural effect will occur. In this embodiment, the preset stretching coefficient curve can be 0.6, 0.7, or 0.8, etc. This embodiment does not limit the value of the preset stretching coefficient curve; in practical applications, it can be set according to the requirements for a natural stretching effect.

[0091] This application embodiment uses the improved inverse transformation algorithm shown in formula (4) above, specifically performing stretching transformation on each pixel through the following steps A1-A3, including:

[0092] A1: Calculate the coordinates of the first pixel before transformation based on the preset stretching coefficient, the predetermined part length, the first key point, the second key point, and the transformed coordinates of the first pixel. The first pixel is any pixel in the area to be adjusted.

[0093] First, based on the preset stretching coefficient, the predetermined part length, the first key point, and the transformed coordinates of the first pixel, the amplitude control coefficient corresponding to the first key point is calculated. The preset stretching coefficient is used to control the stretching amplitude corresponding to the first pixel. That is, the amplitude control coefficient corresponding to the first key point is calculated using the following formula (5).

[0094]

[0095] In formula (5), `infect` is the amplitude control coefficient, `curve` is the preset stretching coefficient, `x` is the transformed coordinate of the first pixel, and `r` is the amplitude control coefficient. max denoted as 'c', where 'c' represents the length of the predetermined section and 'c' represents the coordinates of the first key point.

[0096] Then calculate the stretch distance between the first keypoint and the second keypoint. The product of the stretching distance and the amplitude control coefficient is used to determine the stretching amplitude corresponding to the first pixel. stretching range Substitute into formula (4) to calculate the transformed coordinates of the first pixel. With stretching range The difference between the two values ​​is used to determine the coordinates of the first pixel before the transformation.

[0097] A2: Determine the pixel value corresponding to the original coordinates based on the user's face image and the original coordinates of the first pixel.

[0098] The position of the first pixel before transformation is determined from the user's face image, and the pixel value of the pixel at that position is used as the pixel value corresponding to the original coordinates. Then, the first pixel is stretched using this pixel value in step A3.

[0099] Because the pixel stretching operation in step A3 directly uses the pixel value of the pixel at the coordinates before transformation, there may be a significant difference between the stretched pixel value and its neighboring pixel values, resulting in a jagged visual effect and distortion of the face shape transformation. To avoid this, this embodiment of the application can also obtain the pixel value of each pixel adjacent to the first pixel at the coordinates before transformation from the user's face image, instead of directly using the pixel value of the pixel at the coordinates before transformation; and perform interpolation calculation on the pixel value of each adjacent pixel to obtain the pixel value corresponding to the coordinates before transformation. The interpolation calculation can employ conventional interpolation algorithms such as nearest neighbor interpolation, bilinear interpolation, or bicubic interpolation.

[0100] A3: Modify the pixel value of the pixel at the transformed coordinates in the result image to the pixel value at the original coordinates in the user's face image.

[0101] After obtaining the pixel value corresponding to the original coordinates of the first pixel point through any of the methods in step A2, modify the pixel value of the pixel point at the transformed coordinates of the first pixel point in the result image to the pixel value corresponding to the original coordinates, thereby stretching the first pixel point from the original coordinates to the transformed coordinates.

[0102] For each pixel in the region to be adjusted corresponding to the first key point, the stretching transformation is performed on each pixel according to steps A1-A3 above, thus achieving local translation deformation of the region to be adjusted corresponding to the first key point. For each other first key point in the preset number of first key points in the user face image obtained in step 101, the local translation deformation of the region to be adjusted corresponding to each first key point is performed according to steps 102 and 103 above, thus completing the face transformation of the user face image, transforming the face in the user face image into the face in the target face image.

[0103] In the aforementioned face transformation process, given the number of first key points, the face transformation is achieved by locally translating and deforming the area to be adjusted corresponding to each first key point. Furthermore, during the transformation process, a magnitude control coefficient corresponding to each first key point is calculated using a preset stretching coefficient. This magnitude control coefficient controls the stretching amplitude of each pixel within the area to be adjusted corresponding to the first key point, preventing excessive stretching that would result in an unnatural face transformation effect. This face transformation process is applicable to any transformation scenario between two face shapes, even asymmetrical face transformations. Given a user's face image and a target face image, the face shape in the user's face image can be adjusted to match the face shape in the target face image. The face transformation is highly accurate, produces a more natural effect, and has wider applicability.

[0104] In other embodiments of this application, to further avoid unnatural deformation effects caused by excessive stretching, an attenuation coefficient can be introduced to further limit the stretching amplitude. Specifically, before performing step 103, the stretching distance from the first key point to the second key point is calculated. The stretching distance can be calculated using the following formula (6):

[0105]

[0106] In formula (6), d i The stretch distance corresponding to the i-th first keypoint. Let i be the coordinate vector of the first key point. Let x be the coordinate vector of the second key point corresponding to the i-th first key point, (x) i1 y i1 (x) represents the coordinates of the i-th first key point.i2 y i2 ) represents the coordinates of the second key point corresponding to the i-th first key point.

[0107] Based on the stretching distance, the attenuation coefficient corresponding to the first key point is determined. In this embodiment, lamba_arr can be used to represent the attenuation coefficient. In this embodiment, different distance intervals are preset, and the attenuation coefficients are different when the stretching distance belongs to different distance intervals. Specifically, the calculation method of the attenuation coefficient when the stretching distance belongs to different distance intervals is as shown in formula (7):

[0108]

[0109] After calculating the stretching distance corresponding to the first key point using the above formula (6), determine the distance interval to which the stretching distance belongs, and then calculate the attenuation coefficient corresponding to the first key point using the calculation method corresponding to the distance interval to which it belongs according to the above formula (7).

[0110] After determining the attenuation coefficient corresponding to the first key point using the above method, the attenuation coefficient, the first key point, and the second key point are used to calculate the attenuated second key point. Specifically, the second key point after introducing the attenuation coefficient can be calculated using the following formula (8):

[0111] m' i (x' i2 ,y' i2 ) = c i (x i1 ,y i1 )+lamda_arr*(m i (x i2 ,y i2 )-c i (x i1 ,y i1 ))…(8)

[0112] In formula (8), m' i (x' i2 ,y' i2 ) represents the second keypoint corresponding to the i-th first keypoint after the introduction of the attenuation coefficient, c i (x i1 ,y i1 Let ) be the i-th first key point, m i (x i2 ,y i2 ) represents the second key point corresponding to the i-th first key point before the introduction of the attenuation coefficient, and lamba_arr is the attenuation coefficient.

[0113] The second key point after introducing the attenuation coefficient is calculated using the above formula (8). Then, step 103 is executed, and the attenuated second key point is substituted into formula (4) to calculate the coordinates of the first pixel before transformation after introducing the attenuation coefficient. Then, the first pixel is stretched and deformed using the coordinates before transformation.

[0114] Figure 3 The image shown is a schematic diagram illustrating the effect of facial reshaping without applying an attenuation coefficient. Figure 4 The image shown illustrates the effect of applying an attenuation coefficient to transform facial features. Figure 3 and Figure 4 The face images on the left are user face images, the face images in the middle are target face images, and the face images on the right are the results of face transformation. From Figure 3 and Figure 4 By comparing the before and after images, it can be seen that the face in the face transformation image is more natural and the effect is better after adding the attenuation coefficient.

[0115] In this embodiment, the attenuation coefficient is automatically calculated based on the stretching distance corresponding to the first key point. By using this attenuation coefficient to limit the stretching amplitude of the pixel, unnatural stretching effects such as excessive stretching amplitude will not occur. This makes the face shape transformation method of this embodiment more adaptable and flexible, thus enabling it to be widely applied to various scenarios that require face shape transformation.

[0116] In some other embodiments of this application, the coordinates before transformation of the first pixel are calculated using the above formula (4). These coordinates may fall outside the boundary of the user's face in the user's face image. In this case, if the pixel value corresponding to the coordinates before transformation is directly obtained and the first pixel is stretched using the obtained pixel value, a false texture that should not exist will appear, causing the user's face image after face transformation to be distorted. To avoid this false texture situation, this embodiment of the application can also perform pixel mirror filling processing of a preset width on the left and right sides of the face boundary and pixel mirror filling processing of a preset height on the lower side of the face boundary in the user's face image before performing step 103. The preset width can be a multiple of the predetermined part length, and the preset height can be a multiple of the predetermined part length, such as two or three times.

[0117] Specifically, for the left side of the facial boundary, such as Figure 5 As shown, pixel mirroring is performed from left to right along the width of the face. Pixel mirroring means that for pixels in the same row, the pixel value is taken from the pixel value at the right edge of that row where it meets the user's avatar boundary; pixels in the same row are filled with the same pixel value. Similarly, for the right side of the face boundary, as shown... Figure 5As shown, pixel mirroring is performed from right to left along the width of the face. For pixels in the same row, the pixel value is taken from the pixel at the left edge of that row where it meets the user's avatar's border. For the lower edge of the face, as shown... Figure 5 As shown, pixel mirroring is performed from top to bottom along the height direction of the face. For pixels in the same column, the pixel value is taken from the pixel value at the position where the upper side of the column meets the boundary of the user's avatar.

[0118] After the above pixel mirroring filling process, for any pixel point whose pre-transformation coordinates are calculated by formula (4), even if the calculated pre-transformation coordinates fall outside the user's face boundary in the user's face image, the pixel value corresponding to the pre-transformation coordinates is still the pixel value within the user's face boundary. Using the pixel value corresponding to the pre-transformation coordinates to stretch and transform the pixel point will not produce pseudo textures that should not actually exist, thus avoiding face shape distortion caused by the calculated pre-transformation coordinates falling outside the user's face boundary.

[0119] After stretching and transforming the pixels in each region to be adjusted in the user's face image using the above method, the portion of the image filled by the mirrored pixels is then deleted, specifically the filled portions on the left, right, and bottom sides of the user's face image. By cropping the filled portions after the face transformation is complete, the final face transformation result maintains the same resolution as the original user's face image.

[0120] In some other embodiments of this application, the mirror pixel filling process described above may not be applied to all user face images before face transformation. Instead, step 103 is executed first, and the pre-transformation coordinates of the pixel are calculated using formula (4). If the pre-transformation coordinates fall within the user's face boundary, the pixel is stretched and deformed according to step 103. If the calculated pre-transformation coordinates fall outside the user's face boundary, the mirror pixel filling method described above is used to avoid the appearance of false textures.

[0121] Specifically, when the calculated pre-transformation coordinates fall outside the user's face boundary, the orientational relationship between these coordinates and the face boundary is first determined. This orientational relationship indicates whether the pre-transformation coordinates are located to the left, right, or bottom of the face boundary. Then, in the user's face image, pixel mirroring is performed on one side of the face boundary indicated by this orientational relationship. That is, if the calculated pre-transformation coordinates fall to the left of the face boundary, a preset width of pixel mirroring is performed on the left side of the face boundary in the user's face image. If the pre-transformation coordinates fall to the right of the face boundary, a preset width of pixel mirroring is performed on the right side of the face boundary. If the pre-transformation coordinates fall to the bottom of the face boundary, a preset height of pixel mirroring is performed on the bottom of the face boundary.

[0122] After the pixel mirroring fill operation described above, based on the filled user face image, the operation of determining the pixel value corresponding to the coordinates before transformation is performed. That is, steps A2 and A3 above are executed to stretch the pixels. This ensures that the pixel value corresponding to the coordinates before transformation is still within the user's face boundary. Using this pixel value to stretch the pixels avoids the appearance of unwanted textures and prevents face distortion caused by the calculated coordinates falling outside the user's face boundary.

[0123] After stretching the pixels, the portion of the user's face image filled with the mirrored pixels is deleted to ensure that the final face transformation image has the same resolution as the original user's face image.

[0124] Figure 6 The diagram shown is a schematic diagram of face shape transformation using the method provided in the embodiments of this application. Figure 6 The image on the left is the user's face image, the image in the middle is the target face image, and the image on the right is the result of the face transformation. From Figure 6 As can be seen, the effect is very natural when changing from a thinner face shape to a fuller face shape. The embodiments of this application can not only slim the face, but also make the face fuller, or change from a square face to a round face, or from a round face to an oval face, etc., and can achieve a very natural deformation effect.

[0125] In this embodiment, a preset number of key points are selected in the user's face image and the target face image. Based on the selected key points, the area to be adjusted is determined. An improved inverse transform algorithm is used to appropriately control the stretching amplitude of each pixel in the area to be adjusted, which can adapt to the mutual transformation between various face shapes. An attenuation coefficient is also added to further limit the stretching amplitude of each pixel, avoiding the situation where the stretching effect is unnatural due to excessive adjustment amplitude. Before performing face transformation, pixel mirroring is performed on the left and right sides and the bottom of the face boundary in the user's face image to avoid the problem of pseudo texture after face transformation.

[0126] This application also provides a face shape transformation device, which is used to perform the face shape transformation method provided in any of the above embodiments. See also Figure 7 The device includes:

[0127] The acquisition module 701 is used to acquire the length of a predetermined part and a first key point located on the facial contour from the user's face image, and to acquire the second key point corresponding to the first key point from the target face image.

[0128] The determining module 702 is used to determine the transformed coordinates of each pixel in the area to be adjusted corresponding to the first key point based on the predetermined part length, the first key point, and the second key point.

[0129] The transformation module 703 is used to stretch and transform each pixel in the user's face image according to the preset stretching coefficient and the transformed coordinates of each pixel.

[0130] The determining module 702 is used to determine a first circular region centered on a first key point from the user's face image, and to take the first circular region as the region to be adjusted corresponding to the first key point; and to calculate the transformed coordinates of each pixel point according to the predetermined part length, the coordinates of the first key point, the coordinates of the second key point and the coordinates of each pixel point in the region to be adjusted.

[0131] The transformation module 703 is used to calculate the original coordinates of the first pixel point based on the preset stretching coefficient, the predetermined part length, the first key point, the second key point, and the transformed coordinates of the first pixel point. The first pixel point is any pixel point in the area to be adjusted. Based on the user's face image and the original coordinates of the first pixel point, the pixel value corresponding to the original coordinates is determined. The pixel value of the pixel point at the transformed coordinates in the result image is modified to the pixel value corresponding to the original coordinates in the user's face image.

[0132] The transformation module 703 is used to calculate the amplitude control coefficient corresponding to the first key point based on the preset stretching coefficient, the predetermined part length, the first key point and the transformed coordinates corresponding to the first pixel; calculate the stretching distance between the first key point and the second key point; determine the stretching amplitude corresponding to the first pixel by multiplying the stretching distance and the amplitude control coefficient; and calculate the difference between the transformed coordinates of the first pixel and the stretching amplitude, and determine the difference as the original coordinates of the first pixel.

[0133] The transformation module 703 is used to obtain the pixel value of each pixel adjacent to the original coordinates of the first pixel in the user's face image; and to perform interpolation calculation on the pixel value of each adjacent pixel to obtain the pixel value corresponding to the original coordinates.

[0134] The acquisition module 701 is used to perform face detection on the user's face image and identify all facial key points located on the face contour; select the chin key point and multiple facial key points mirrored on both sides of the face contour from all the identified facial key points; and determine the selected chin key point and multiple mirrored facial key points as the first key point corresponding to the user's face image.

[0135] The device also includes: an attenuation module for calculating the stretching distance from the first key point to the second key point; determining the attenuation coefficient corresponding to the first key point based on the stretching distance; and calculating the attenuated second key point based on the attenuation coefficient, the first key point, and the second key point.

[0136] The device also includes a mirror fill module, which performs pixel mirror fill processing of a preset width on the left and right sides of the face boundary and pixel mirror fill processing of a preset height on the lower side of the face boundary in the user's face image.

[0137] The device also includes a deletion module, which deletes the portion of the image filled on the left and right sides and the bottom of the face boundary in the user's face image after the transformation module 703 has been executed.

[0138] The mirror fill module is also used to determine the orientation relationship between the pre-transformation coordinates and the face boundary if the calculated pre-transformation coordinates are located outside the face boundary in the user's face image. The orientation relationship is used to indicate whether the pre-transformation coordinates are located on the left, right, or bottom side of the face boundary. In the user's face image, pixel mirror fill processing is performed on one side of the face boundary indicated by the orientation relationship. Based on the filled user's face image, the operation of determining the pixel value corresponding to the pre-transformation coordinates is performed.

[0139] The face shape transformation device and the face shape transformation method provided in the above embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0140] This application also provides an electronic device for performing the above-described face shape transformation method. Please refer to... Figure 8 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 8 As shown, the electronic device 8 includes: a processor 800, a memory 801, a bus 802, and a communication interface 803. The processor 800, the communication interface 803, and the memory 801 are connected via the bus 802. The memory 801 stores a computer program that can run on the processor 800. When the processor 800 runs the computer program, it executes the face transformation method provided in any of the foregoing embodiments of this application.

[0141] The memory 801 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 803 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0142] Bus 802 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 801 is used to store programs. After receiving an execution instruction, the processor 800 executes the program. The face transformation method disclosed in any of the foregoing embodiments of this application can be applied to the processor 800, or implemented by the processor 800.

[0143] The processor 800 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 800 or by instructions in software form. The processor 800 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 801. Processor 800 reads the information in memory 801 and, in conjunction with its hardware, completes the steps of the above method.

[0144] The electronic device provided in this application embodiment and the face shape transformation method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0145] This application also provides a computer-readable storage medium corresponding to the face shape transformation method provided in the foregoing embodiments. Please refer to... Figure 9 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the face transformation method provided in any of the foregoing embodiments.

[0146] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0147] The computer-readable storage medium provided in the above embodiments of this application and the face shape transformation method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0148] It should be noted that:

[0149] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0150] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting a schematic diagram in which the claimed application requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0151] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0152] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A face shape transformation method, characterized in that, include: Obtain the length of a predetermined part and a first key point located on the facial contour from the user's face image, and obtain the second key point corresponding to the first key point from the target face image; Based on the predetermined part length, the first key point, and the second key point, the transformed coordinates of each pixel in the area to be adjusted corresponding to the first key point are determined. Based on the preset stretching coefficient and the transformed coordinates of each pixel, the pixel in the user's face image is stretched and transformed respectively. Before performing the stretching transformation on each pixel in the user's face image according to the preset stretching coefficient and the transformed coordinates of each pixel, the method further includes: Calculate the stretching distance from the first keypoint to the second keypoint; Based on the stretching distance, determine the attenuation coefficient corresponding to the first key point; Calculate the attenuated second key point based on the attenuation coefficient, the first key point, and the second key point.

2. The method according to claim 1, characterized in that, The step of determining the transformed coordinates of each pixel in the region to be adjusted corresponding to the first key point based on the predetermined part length, the first key point, and the second key point includes: A first circular region centered on the first key point is determined from the user's face image, and the first circular region is taken as the region to be adjusted corresponding to the first key point. Based on the predetermined part length, the coordinates of the first key point, the coordinates of the second key point, and the coordinates of each pixel in the area to be adjusted, the transformed coordinates of each pixel are calculated respectively.

3. The method according to claim 1, characterized in that, The step of stretching and transforming each pixel in the user's face image according to a preset stretching coefficient and the transformed coordinates of each pixel includes: Based on the preset stretching coefficient, the predetermined part length, the first key point, the second key point, and the transformed coordinates of the first pixel, the original coordinates of the first pixel are calculated, where the first pixel is any pixel in the area to be adjusted. Based on the user's face image and the coordinates before transformation corresponding to the first pixel, determine the pixel value corresponding to the coordinates before transformation; Modify the pixel value of the pixel at the transformed coordinates in the result image to the pixel value at the original coordinates in the user image.

4. The method according to claim 3, characterized in that, The step of calculating the original coordinates of the first pixel based on the preset stretching coefficient, the predetermined part length, the first key point, the second key point, and the transformed coordinates of the first pixel includes: Based on the preset stretching coefficient, the predetermined part length, the transformed coordinates of the first key point and the first pixel point, calculate the amplitude control coefficient corresponding to the first key point; Calculate the stretch distance between the first key point and the second key point; The product of the stretching distance and the amplitude control coefficient is determined as the stretching amplitude corresponding to the first pixel. Calculate the difference between the transformed coordinates of the first pixel and the stretching amplitude, and determine the difference as the original coordinates of the first pixel.

5. The method according to claim 3, characterized in that, The step of determining the pixel value corresponding to the pre-transformation coordinates based on the user's face image and the pre-transformation coordinates of the first pixel includes: Obtain the pixel value of each pixel adjacent to the original coordinates of the first pixel in the user's face image; Interpolation is performed on the pixel values ​​of each adjacent pixel to obtain the pixel value corresponding to the coordinates before the transformation.

6. The method according to any one of claims 1-5, characterized in that, The step of obtaining the first key point located on the facial contour from the user's face image includes: Perform face detection on user face images and identify all facial key points located on the facial contour; Select the chin key point and multiple facial key points mirrored on both sides of the facial contour from all identified facial key points; The selected chin key point and the multiple mirror-distributed facial key points are determined as the first key point corresponding to the user's face image.

7. The method according to any one of claims 1-5, characterized in that, Before performing the stretching transformation on each pixel in the user's face image according to the preset stretching coefficient and the transformed coordinates of each pixel, the method further includes: In the user's face image, the left and right sides of the face boundary are filled with pixels of a preset width, and the lower side of the face boundary is filled with pixels of a preset height.

8. The method according to claim 7, characterized in that, After stretching and transforming each pixel in the user's face image according to a preset stretching coefficient and the transformed coordinates of each pixel, the process further includes: In the user's face image, the portion of the image filled on the left, right, and bottom sides of the face boundary is deleted.

9. The method according to any one of claims 3-5, characterized in that, After calculating the original coordinates of the first pixel based on the preset stretching coefficient and the transformed coordinates of the first pixel, the method further includes: If the calculated pre-transformation coordinates are located outside the facial boundary in the user's face image, then the orientation relationship between the pre-transformation coordinates and the facial boundary is determined, and the orientation relationship is used to indicate whether the pre-transformation coordinates are located to the left, right, or bottom of the facial boundary. In the user's face image, pixel mirroring is performed on one side of the face boundary indicated by the orientation relationship; Based on the user's face image after filling, the operation of determining the pixel value corresponding to the coordinates before transformation is performed.

10. A face shape changing device, characterized in that, include: The acquisition module is used to acquire the length of a predetermined part and a first key point located on the facial contour from the user's face image, and to acquire a second key point corresponding to the first key point from the target face image. The determining module is used to determine the transformed coordinates of each pixel in the region to be adjusted corresponding to the first key point based on the predetermined part length, the first key point, and the second key point. The transformation module is used to stretch and transform each pixel in the user's face image according to a preset stretching coefficient and the transformed coordinates of each pixel. Before performing the stretching transformation on each pixel in the user's face image according to the preset stretching coefficient and the transformed coordinates of each pixel, the method further includes: Calculate the stretching distance from the first keypoint to the second keypoint; Based on the stretching distance, determine the attenuation coefficient corresponding to the first key point; Calculate the attenuated second key point based on the attenuation coefficient, the first key point, and the second key point.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-9.

Images