Methods, apparatuses, electronic devices and storage media for correcting facial images

Abstract

This application relates to a method, apparatus, computer device, and storage medium for correcting facial images. The method includes: performing face detection on an original image; if the original image meets the conditions for facial distortion correction, determining the face center region, face edge region, and background region in the original image; acquiring first grids containing the face center region, second grids containing the background region, and third grids containing the face edge region; generating an irregular grid of the original image based on the first grids, second grids, and third grids; wherein the first and second grids are both larger than the third grid; and performing facial distortion correction on the original image using the irregular grid to obtain a target image. This method can more accurately correct facial images.

Description

Technical Field

[0001] This application relates to computer technology, and in particular to a method, apparatus, electronic device, and computer-readable storage medium for correcting facial images. Background Technology

[0002] With the development of computer technology, more and more electronic devices are equipped with cameras, which can capture videos or images. Cameras include lenses, usually convex or concave lenses. As a result, the captured images can easily differ from the actual scene, and the distortion is more severe closer to the edges, requiring image correction.

[0003] When an image contains faces, the faces become distorted, resulting in a non-realistic representation of the person. To correct this distortion and present the most accurate information about the person, a global approach is typically used to correct facial distortion across the entire image.

[0004] However, this method of correcting facial images can distort other areas of the image while correcting the face, resulting in image distortion and inaccurate correction. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and computer-readable storage medium for correcting facial images, which can improve the accuracy of facial image correction.

[0006] A method for correcting facial images, comprising:

[0007] Perform face detection on the original image;

[0008] If the original image is found to meet the face distortion correction conditions, the face center region, face edge region, and background region in the original image are determined.

[0009] Obtain each first grid containing the center region of the face, each second grid containing the background region, and each third grid containing the edge region of the face. Generate an irregular grid of the original image based on each first grid, each second grid, and each third grid. Both the first grid and the second grid are larger than the third grid.

[0010] The irregular grid is used to correct facial distortion in the original image to obtain the target image.

[0011] A facial image correction device, comprising:

[0012] The detection module is used to perform face detection on the original image;

[0013] The region determination module is used to determine the face center region, face edge region, and background region in the original image when the original image is detected to meet the face distortion correction conditions;

[0014] An irregular grid generation module is used to obtain each first grid where the center region of the face is located, each second grid where the background region is located, and each third grid where the edge region of the face is located, and to generate an irregular grid of the original image based on each first grid, each second grid, and each third grid; the first grid and the second grid are both larger than the third grid;

[0015] The correction module is used to correct facial distortion in the original image using the irregular grid to obtain the target image.

[0016] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the face image correction method described above.

[0017] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0018] The aforementioned face image correction method, apparatus, electronic device, and computer-readable storage medium perform face detection on the original image; when the original image meets the face distortion correction conditions, it determines the face center region, face edge region, and background region in the original image; it obtains each first grid containing the face center region, each second grid containing the background region, and each third grid containing the face edge region, and generates an irregular grid of the original image based on each first grid, each second grid, and each third grid; the first grid and the second grid are both larger than the third grid. During the distortion correction of the face in the face image, the face edge region is located at the junction of the face and the background, requiring more refined correction. Therefore, each third grid containing the face edge region is smaller, thus more accurately correcting the face contour to a faithful state. Conversely, the face center region is located at the center of the face, and the background region is a region with weak correlation to the face to be distorted. Therefore, each first grid containing the face center region and each second grid containing the background region are larger, meaning both the first grid and the second grid are larger than the third grid. Therefore, using this irregular grid to correct facial distortion in the original image can not only correct the overall facial contour in the face image to a more realistic state that matches the actual scene, but also ensure that the lines in the central area of ​​the face and the background area do not change abruptly, thus obtaining a more accurate target image and improving the accuracy of face image correction. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a face image correction method in one embodiment;

[0021] Figure 2 This is a flowchart of the steps in one embodiment to obtain the first grids containing the center region of the face, the second grids containing the background region, and the third grids containing the edge region of the face;

[0022] Figure 3 This is a flowchart illustrating the steps of determining the face center region, face edge region, and background region in the original image in one embodiment.

[0023] Figure 4 A flowchart of a face image correction method in another embodiment;

[0024] Figure 5 A flowchart of a face image correction method in another embodiment;

[0025] Figure 6 This is a structural block diagram of a face image correction device in one embodiment;

[0026] Figure 7 This is a schematic diagram of the internal structure of an electronic device in one embodiment. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0028] It is understood that the terms "first," "second," etc., used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of this application, a first grid may be referred to as a second grid, and similarly, a second grid may be referred to as a first grid. Both the first grid and the second grid are grids, but they are not the same grid.

[0029] In one embodiment, such as Figure 1As shown, a method for correcting facial images is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, or to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0030] Step 102: Perform face detection on the original image.

[0031] The original image refers to the image to be corrected. The format of the original image is not limited; it can be an RGB (Red, Green, Blue) image, a YUV image, etc. Here, "Y" represents luminance (or Luma), which is the grayscale value, while "U" and "V" represent chrominance (or Chroma), which describe the image's color and saturation and are used to specify the color of a pixel.

[0032] In one embodiment, the electronic device is equipped with a camera to capture an original image, and then performs face detection on the original image. In another embodiment, the electronic device retrieves the original image from local memory and then performs face detection on the original image. In yet another embodiment, the electronic device receives an original image sent by another device and then performs face detection on the original image.

[0033] The original image may or may not contain faces. If the original image includes faces, the number of faces is not limited; there can be one or more faces. The position of the faces is also not limited; they can be located in the center or at the edge of the original image. If the original image does not contain faces, and it is determined that the original image does not meet the conditions for face distortion correction, then no correction processing is performed on the original image.

[0034] Specifically, the electronic device uses a face detection model to detect faces in the original image. This face detection model can be pre-trained. By performing face detection on the original image, the electronic device can obtain information including at least whether the original image contains faces, the number of faces, the location of the faces, and the size of the faces.

[0035] Step 104: If the original image is found to meet the face distortion correction conditions, determine the face center region, face edge region and background region in the original image.

[0036] The face distortion correction condition is used to determine whether to perform face distortion correction on the original image. The face distortion correction condition can be set as needed. In one embodiment, the face distortion correction condition is that the original image contains a face. Further, the face distortion correction condition is that the original image contains a face, and the face is located at a specified position in the original image. The specified position can also be set as needed. For example, the specified position can be the central region of the original image, the edge region of the original image, or any specified position in the original image. Further, the face distortion correction condition is that the original image contains a face, and the size of the face conforms to a face scale condition. Specifically, the face scale condition can be that the face size is greater than a preset area, or that the proportion of the face size to the original image size is greater than a preset proportion. Both the preset area and the preset proportion can be set as needed.

[0037] The face center region is the area located at the center of the face. The face edge region is the area located at the edge of the face. The background region is the area in the original image that belongs to the background.

[0038] If the original image meets the conditions for face distortion correction, then face distortion correction is required. In this case, the face center region, face edge region, and background region in the original image are determined. If the original image does not meet the conditions for face distortion correction, then face distortion correction is not required, and no correction processing is performed on the original image.

[0039] Specifically, when the original image is detected to meet the facial distortion correction conditions, the electronic device identifies the facial region from the original image and performs morphological operations on the facial region to determine the facial center region, facial edge region, and background region in the original image. These morphological operations include dilation and erosion. Erosion removes the edges of the image, essentially eliminating the "burrs" at the edges of the target. Dilation enlarges the edges of the image, filling in the edges or internal pits of the target.

[0040] Electronic devices use face segmentation models to determine the mask to which a face belongs from the original image, and then determine the face region where the mask is located. The face segmentation model can be a semantic segmentation model.

[0041] Step 106: Obtain the first grids containing the center region of the face, the second grids containing the background region, and the third grids containing the edge region of the face. Generate an irregular grid of the original image based on the first grids, the second grids, and the third grids. The first grids and the second grids are both larger than the third grids.

[0042] The first grid is the grid mapped to the center region of the face. The second grid is the grid mapped to the background region. The third grid is the grid mapped to the edge region of the face. An irregular grid refers to a grid in which the included grid sub-units do not have a uniform pattern. For example, in an irregular grid, some grid sub-units are larger, while others are smaller. Or, in an irregular grid, some grid sub-units are rectangular, while others are square.

[0043] In one implementation, the electronic device directly acquires each first grid set for the center region of the face, each second grid set for the background region, and each third grid set for the edge region of the face, and stitches the first grid, the second grid, and the third grid together to generate an irregular grid of the original image.

[0044] In another implementation, the electronic device maps a preset regular grid onto the original image. Within this grid mapped to the original image, the grid sub-units containing the central region of the face are merged to obtain first grids; each grid sub-unit is the smallest unit of the preset regular grid. The grid sub-units containing the background region are merged to obtain second grids; and third grids containing the edge regions of the face are obtained. The preset regular grid is a grid in which all included grid sub-units exhibit a uniform pattern. The grid sub-units included in the preset regular grid have a uniform size, shape, and arrangement. The preset regular grid can be a low-resolution grid, meaning that the number of grid sub-units in each row is less than the preset number of rows, and the number of grid sub-units in each column is less than the preset number of columns. Using a low-resolution grid can save computer resources and reduce the power consumption of the electronic device. For example, the preset regular grid can be a 100x80 resolution grid, meaning each row has 100 grid sub-units and each column has 80 grid sub-units.

[0045] Step 108: Use an irregular grid to correct facial distortion in the original image to obtain the target image.

[0046] The target image is the image obtained after correcting facial distortion in the original image. When an electronic device uses an irregular grid to correct facial distortion in the original image, the face in the resulting target image is closer to the face in the actual scene, meaning the target image is more faithful to reality.

[0047] Understandably, when electronic devices use irregular grids to correct facial distortion in the original image, they control the grid sub-units within the irregular grid—namely, each first grid, each second grid, and each third grid—to perform overall correction on the pixels within the corresponding grid in the original image. Therefore, the larger the grid sub-units in a certain region of the original image, the more intact the pixels in that region can be when performing overall correction. Conversely, if the grid sub-units in the facial edge region of the original image are smaller, then the overall correction of pixels in each third grid in the facial edge region can be more accurately controlled, resulting in more refined correction of the pixels in that facial edge region.

[0048] Therefore, the first and second grids are both larger than the third grid, which can ensure the integrity of pixels in the central area of ​​the face and the background area as much as possible, avoid distorting the lines in the background area and the central area of ​​the face during the distortion correction process, and at the same time, it can more finely correct the edge area of ​​the face, correcting the overall outline of the face to a faithful state.

[0049] Specifically, the electronic device uses an irregular grid to interpolate the original image to obtain the target image. This interpolation can be bilinear interpolation. Bilinear interpolation is a linear extension of an interpolation function with two variables; its core idea is to perform linear interpolation in two directions separately.

[0050] The aforementioned face image correction method involves performing face detection on the original image. If the original image meets the face distortion correction conditions, the face center region, face edge region, and background region are determined. First grids containing the face center region, second grids containing the background region, and third grids containing the face edge region are obtained. An irregular grid of the original image is generated based on these first, second, and third grids. The first and second grids are both larger than the third grid. During the distortion correction process, the face edge region, located at the junction of the face and background, requires more refined correction. Therefore, the third grids containing the face edge region are smaller, resulting in more accurate correction of the face contour to a faithful state. Conversely, the face center region is located at the center of the face, while the background region is less correlated with the face to be distorted. Therefore, the first grids containing the face center region and the second grids containing the background region are both larger, meaning the first and second grids are both larger than the third grid. Therefore, using this irregular grid to correct facial distortion in the original image can not only correct the overall facial contour in the face image to a more realistic state that matches the actual scene, but also ensure that the lines in the central area of ​​the face and the background area do not change abruptly, thus obtaining a more accurate target image and improving the accuracy of face image correction.

[0051] In another embodiment, the electronic device can also adjust the irregular grid in real time, dynamically generate the irregular grid, and use the dynamically generated irregular grid to correct facial distortion in the original image to obtain the target image.

[0052] In one embodiment, such as Figure 2 As shown, the process involves obtaining the first grid cells containing the center region of the face, the second grid cells containing the background region, and the third grid cells containing the edge regions of the face, including:

[0053] Step 202: In the preset regular grid mapped to the original image, merge the grid sub-units where the center region of the face is located to obtain each first grid; the grid sub-unit is the smallest unit of the preset regular grid.

[0054] A preset regular grid is a grid in which all grid sub-units follow a uniform pattern. The grid sub-units within a preset regular grid have a uniform size, shape, and arrangement.

[0055] The electronic device maps a preset regular grid onto the original image. Within the preset grid mapped to the original image, it merges the grid sub-units containing the central region of the face according to a first merging method to obtain each first grid. Specifically, the first merging method includes: within the preset grid mapped to the original image, the electronic device determines first grid groups, each containing at least two adjacent grid sub-units, from the grid sub-units containing the central region of the face; and merges the grid sub-units in each first grid group to obtain each first grid.

[0056] Step 204: Merge the grid sub-units containing the background area to obtain the second grids.

[0057] The electronic device merges the grid sub-units containing the background region within a preset grid mapped to the original image according to a second merging method to obtain each second grid. Specifically, the second merging method includes: the electronic device determining, from each grid sub-unit containing the background region, a second grid group comprising at least two adjacent grid sub-units, and merging the grid sub-units in each second grid group to obtain each second grid.

[0058] The first merging method can be the same as or different from the second merging method.

[0059] Step 206: Obtain the third grids containing the edge regions of the face.

[0060] In this embodiment, in the preset regular grid mapped to the original image, the grid sub-units where the center region of the face is located and the grid sub-units where the background region is located are merged to obtain the first grid and the second grid respectively; and the third grids where the edge region of the face is located are obtained, so that the irregular grid of the original image can be generated, thereby using the irregular grid to correct the original image more accurately.

[0061] In one embodiment, merging the grid sub-units containing the central region of the face to obtain each first grid includes: determining, from each grid sub-unit containing the central region of the face, a first grid group comprising at least two adjacent grid sub-units, and merging the grid sub-units in each first grid group to obtain each first grid; merging the grid sub-units containing the background region to obtain each second grid includes: determining, from each grid sub-unit containing the background region, a second grid group comprising at least two adjacent grid sub-units, and merging the grid sub-units in each second grid group to obtain each second grid.

[0062] The first grid group is a set consisting of at least two grid sub-units defined in each grid sub-unit containing the central region of the face. For example, from each grid sub-unit containing the central region of the face, a first grid group can be defined that includes 16 (4×4) adjacent grid sub-units, or a first grid group can be defined that includes 9 (3×3) adjacent grid sub-units, and there is no limitation.

[0063] The electronic device merges each grid sub-unit in each first grid group to obtain each first grid, that is, each first grid group obtains a first grid.

[0064] The second grid group is a set consisting of at least two grid sub-units defined in each grid sub-unit containing the background region. For example, a second grid group consisting of 16 (4×4) adjacent grid sub-units can be defined from each grid sub-unit containing the background region, or a second grid group consisting of 9 (3×3) adjacent grid sub-units can be defined, and there is no limitation.

[0065] The electronic device merges each grid sub-unit in each second grid group to obtain each second grid, that is, each second grid group obtains a second grid.

[0066] It should be noted that the number and arrangement of the grid sub-units in the first and second grid groups can be the same or different. For example, the first grid group includes 16 (4×4) adjacent grid sub-units, and the second grid group also includes 16 (4×4) adjacent grid sub-units. Alternatively, the first grid group may include 16 (4×4) adjacent grid sub-units, and the second grid group may include 36 (6×6) adjacent grid sub-units.

[0067] In one embodiment, the electronic device merges the grid sub-units containing the center region of the face in the preset regular grid to obtain the first grids, and merges the grid sub-units containing the background region in the preset regular grid to obtain the second grids, thereby generating an irregular grid, which can be used to more accurately correct the original image.

[0068] In one embodiment, obtaining each third grid where the face edge region is located includes: taking each grid sub-unit where the face edge region is located as a third grid; or determining a third grid group from each grid sub-unit where the face edge region is located, which includes at least two adjacent grid sub-units, and merging the grid sub-units in each third grid group to obtain each third grid.

[0069] In one implementation, the electronic device treats each grid sub-unit in the area where the face edge region is located as a third grid, which can ensure that the first grid and the second grid are both larger than the third grid.

[0070] In another implementation, the electronic device merges the grid sub-units containing the face edge region in a preset grid mapped to the original image according to a third merging method to obtain third grids. The number of grid sub-units included in the first grid group and the second grid group is greater than the number of grid sub-units included in the third grid group.

[0071] The third merging method specifically includes: the electronic device determines a third grid group from each grid sub-unit where the face edge region is located, which includes at least two adjacent grid sub-units, and merges the grid sub-units in each third grid group to obtain each third grid.

[0072] In this embodiment, the electronic device treats each grid sub-unit where the face edge region is located as a third grid; or from each grid sub-unit where the face edge region is located, it determines a third grid group that includes at least two adjacent grid sub-units, merges the grid sub-units in each third grid group to obtain each third grid, thereby obtaining each third grid where the face edge region is located, and thus generating an irregular grid with each first grid and each second grid.

[0073] In one embodiment, such as Figure 3 As shown, the face center region, face edge region, and background region in the original image are determined, including:

[0074] Step 302: Erosion processing is performed on the face region in the original image to obtain the center region of the face.

[0075] Erosion removes the edges of an image, essentially eliminating the "burrs" at the edges of the target.

[0076] Specifically, the electronic device determines the face region from the original image and performs erosion processing on the face region in the original image to obtain the center region of the face. The parameter for the erosion size is 7, meaning the electronic device performs an erosion process of 7 on the face region in the original image to obtain the center region of the face. In other embodiments, other parameters, such as 6 or 8, can be used for the erosion processing, and this is not limited.

[0077] Step 304: Dilate the face region in the original image to obtain the background region.

[0078] Dilation expands the edges of an image, its function being to fill in the edges or pits of a target.

[0079] The electronic device dilates the face region in the original image to obtain the background region. The dilation parameter is 7, meaning the electronic device dilates the face region in the original image by 7 to obtain the background region. In other embodiments, other parameters such as 6 or 8 can be used for the dilation process, and this is not limited to any particular parameter.

[0080] Step 306: Obtain the face edge region based on the original image, the face center region, and the background region.

[0081] Specifically, the electronic device performs differential processing on the original image, the central region of the face, and the background region, that is, removes the central region of the face and the background region from the original image to obtain the edge region of the face.

[0082] In this embodiment, the electronic device performs erosion processing on the face region in the original image to obtain the face center region; performs dilation processing on the face region in the original image to obtain the background region; and obtains the face edge region based on the original image, the face center region, and the background region, thereby accurately obtaining the face center region, the background region, and the face edge region in the original image.

[0083] In one embodiment, such as Figure 4 As shown, the above method also includes:

[0084] Step 402: Determine the target weights for the first grid, the second grid, and the third grid, respectively.

[0085] The target weight is used to represent the importance of the corresponding grid. The higher the target weight, the more important the grid corresponding to that target weight.

[0086] In one implementation, the electronic device acquires the target weights of the first grid, the second grid, and the third grid set by the user.

[0087] In another implementation, the electronic device can be configured to use the irregular grid and the target weights of the first, second, and third grids of the roof respectively.

[0088] Step 404: Construct an optimization function based on the target weights of the first grid, the second grid, and the third grid; solve the optimization function to obtain the optimized irregular grid.

[0089] The optimization function is a function that optimizes an irregular mesh. The optimization function includes a first mesh term for the first mesh, a second mesh term for the second mesh, and a third mesh term for the third mesh. The first mesh term is multiplied by the target weight of the first mesh, the second mesh term is multiplied by the target weight of the second mesh, and the third mesh term is multiplied by the target weight of the third mesh. The first and second mesh terms are the mesh curvature terms, and the third mesh term is the mesh regularization term.

[0090] Therefore, after solving the optimization function, the electronic device can obtain a more optimized irregular grid based on the target weights of the first, second, and third grids. That is, the grid sub-units with higher importance in the irregular grid can more accurately correct the corresponding pixels in the original image.

[0091] Face distortion correction is performed on the original image using an irregular grid to obtain the target image, including:

[0092] Step 406: The original image is corrected for facial distortion using the optimized irregular grid to obtain the target image.

[0093] In this embodiment, the electronic device optimizes the irregular grid, allowing for more accurate facial distortion correction of the original image and resulting in a more accurate target image. Furthermore, by constructing a new optimization function, the optimized irregular grid achieves a correction effect on the facial region that is essentially the same as that of a regular grid.

[0094] In one embodiment, the irregular grid is obtained based on a preset regular grid; determining the target weights of the first grid, the second grid, and the third grid respectively includes: obtaining the scales of the first grid, the second grid, and the third grid respectively; determining the target weight of the first grid based on the scale of the first grid and the preset original weights of the preset regular grid; determining the target weight of the second grid based on the scale of the second grid and the preset original weights of the preset regular grid; and determining the target weight of the third grid based on the scale of the third grid and the preset original weights of the preset regular grid.

[0095] In the preset rule grid, all grid sub-units have the same size and arrangement, and all grid sub-units have the same preset original weight.

[0096] It is understandable that the irregular grid is obtained based on a preset regular grid. The generation method of the irregular grid specifically includes: in the preset regular grid mapped to the original image, merging the grid sub-units where the center region of the face is located to obtain each first grid; the grid sub-unit is the smallest unit of the preset regular grid; merging the grid sub-units where the background region is located to obtain each second grid; and obtaining each third grid where the edge region of the face is located; and generating the irregular grid of the original image based on each first grid, each second grid and each third grid.

[0097] The electronic device acquires the scales of the first grid, the second grid, and the third grid respectively. The scale can be the perimeter of the first grid, the second grid, and the third grid, or the side length of the longest side of the first grid, the second grid, and the third grid, and is not limited to this.

[0098] In one implementation, the electronic device divides the preset original weight of the preset regular grid by the scale of the first grid to obtain the target weight of the first grid. The formula for calculating the target weight is as follows: Target weight of the first grid = Original weight / Scale Len of the first grid.

[0099] In another implementation, the electronic device subtracts the scale of the first grid from the preset original weight of the preset rule grid to obtain the target weight of the first grid. That is, the scale of the first grid is inversely proportional to the target weight of the first grid.

[0100] Similarly, the target weights of the second and third grids can be calculated using the same method as the target weights of the first grid, which will not be elaborated here. The scale of the second grid is inversely proportional to its target weight, and the scale of the third grid is inversely proportional to its target weight.

[0101] It is understandable that the smaller the scale of the grid, the more refined the region of the original image containing that grid needs to be corrected. Therefore, the scale of the first grid is inversely proportional to the target weight of the first grid. The larger the target weight, the more important the first grid and the face edge region are.

[0102] The first and second grids are both larger than the third grid. The face edge region is located between the background region and the face center region, requiring more refined correction processing to more accurately correct the face contour. Therefore, the third grids where the face edge region is located are smaller, meaning the scale of the third grid is smaller, the target weight of the third grid is larger, and the importance of the third grid is higher.

[0103] Since the first grids containing the face center region and the second grids containing the background region are relatively large, the target weights of the first and second grids are determined to be smaller. This ensures the integrity of pixels in the face center region and the background region as much as possible, and also saves the amount of computation in the face distortion correction process, reducing the computational complexity.

[0104] In one embodiment, such as Figure 5 As shown, the electronic device executes step 502, inputs the original image, and performs face detection and portrait segmentation on the original image; executes step 504, determines the face region based on the face detection result and the portrait segmentation result; executes step 506, generates an irregular mesh based on the original image and the face region; executes step 508, constructs an optimization function based on the irregular mesh, and solves the optimization function; executes step 510, uses the optimized irregular mesh to correct face distortion in the original image.

[0105] It should be understood that, although Figures 1 to 5 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 1 to 5 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0106] Figure 6 This is a structural block diagram of a face image correction device according to one embodiment. Figure 6 As shown, a face image correction device is provided, comprising: a detection module 602, a region determination module 604, an irregular grid generation module 606, and a correction module 608, wherein:

[0107] The detection module 602 is used to perform face detection on the original image.

[0108] The region determination module 604 is used to determine the face center region, face edge region and background region in the original image when the original image is detected to meet the face distortion correction conditions.

[0109] The irregular grid generation module 606 is used to obtain each first grid where the center region of the face is located, each second grid where the background region is located, and each third grid where the edge region of the face is located, and to generate an irregular grid of the original image based on each first grid, each second grid and each third grid; the first grid and the second grid are both larger than the third grid.

[0110] The correction module 608 is used to correct facial distortion in the original image using an irregular grid to obtain the target image.

[0111] The aforementioned face image correction device performs face detection on the original image. If the original image meets the face distortion correction conditions, it determines the face center region, face edge region, and background region in the original image. It acquires the first grids containing the face center region, the second grids containing the background region, and the third grids containing the face edge region. Based on these first, second, and third grids, it generates an irregular grid for the original image. The first and second grids are both larger than the third grid. During the distortion correction process for the face in the face image, the face edge region is located at the junction of the face and the background, requiring more refined correction. Therefore, the third grids containing the face edge region are smaller, thus more accurately correcting the face contour to a faithful state. Conversely, the face center region is located at the center of the face, while the background region is less correlated with the face to be distorted. Therefore, the first grids containing the face center region and the second grids containing the background region are both larger, meaning the first and second grids are both larger than the third grid. Therefore, using this irregular grid to correct facial distortion in the original image can not only correct the overall facial contour in the face image to a more realistic state that matches the actual scene, but also ensure that the lines in the central area of ​​the face and the background area do not change abruptly, thus obtaining a more accurate target image and improving the accuracy of face image correction.

[0112] In one embodiment, the irregular grid generation module 606 is further configured to merge the grid sub-units where the center region of the face is located in the preset regular grid mapped to the original image to obtain each first grid; the grid sub-unit is the smallest unit of the preset regular grid; merge the grid sub-units where the background region is located to obtain each second grid; and obtain each third grid where the edge region of the face is located.

[0113] In one embodiment, the irregular grid generation module 606 is further configured to determine, from each grid sub-unit where the face center region is located, a first grid group comprising at least two adjacent grid sub-units, and merge the grid sub-units in each first grid group to obtain each first grid; and to determine, from each grid sub-unit where the background region is located, a second grid group comprising at least two adjacent grid sub-units, and merge the grid sub-units in each second grid group to obtain each second grid.

[0114] In one embodiment, the irregular grid generation module 606 is further configured to treat each grid sub-unit where the face edge region is located as a third grid; or to determine a third grid group consisting of at least two adjacent grid sub-units from each grid sub-unit where the face edge region is located, and to merge the grid sub-units in each third grid group to obtain each third grid.

[0115] In one embodiment, the region determination module 604 is further configured to perform erosion processing on the face region in the original image to obtain the face center region; perform dilation processing on the face region in the original image to obtain the background region; and obtain the face edge region based on the original image, the face center region, and the background region.

[0116] In one embodiment, the above-mentioned apparatus further includes an optimization module, which is used to determine the target weights of the first grid, the second grid, and the third grid respectively; construct an optimization function based on the target weights of the first grid, the second grid, and the third grid; solve the optimization function to obtain an optimized irregular grid; and the above-mentioned correction module 608 is also used to perform face distortion correction on the original image using the optimized irregular grid to obtain a target image.

[0117] In one embodiment, the irregular mesh is obtained based on a preset regular mesh; the optimization module is further configured to obtain the scales of the first mesh, the second mesh, and the third mesh respectively; determine the target weight of the first mesh based on the scale of the first mesh and the preset original weights of the preset regular mesh; determine the target weight of the second mesh based on the scale of the second mesh and the preset original weights of the preset regular mesh; and determine the target weight of the third mesh based on the scale of the third mesh and the preset original weights of the preset regular mesh.

[0118] The division of the various modules in the above-described face image correction device is only for illustrative purposes. In other embodiments, the face image correction device can be divided into different modules as needed to complete all or part of the functions of the face image correction device.

[0119] Specific limitations regarding the facial image correction device can be found in the limitations of the facial image correction method described above, and will not be repeated here. Each module in the aforementioned facial image correction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0120] Figure 7 This is a schematic diagram of the internal structure of an electronic device in one embodiment. For example... Figure 7 As shown, the electronic device includes a processor and a memory connected via a system bus. The processor provides computing and control capabilities to support the operation of the entire electronic device. The memory may include non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The computer programs can be executed by the processor to implement a face image correction method provided in the following embodiments. The internal memory provides a cached operating environment for the operating system computer programs in the non-volatile storage media. The electronic device can be any terminal device such as a mobile phone, tablet computer, PDA (Personal Digital Assistant), POS (Point of Sales), in-vehicle computer, wearable device, etc.

[0121] The various modules in the face image correction device provided in this application embodiment can be implemented in the form of a computer program. This computer program can run on a terminal or server. The program modules constituted by this computer program can be stored in the memory of an electronic device. When the computer program is executed by a processor, it implements the steps of the method described in the embodiments of this application.

[0122] This application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, cause the processors to perform the steps of a face image correction method.

[0123] A computer program product containing instructions that, when run on a computer, causes the computer to perform a method for correcting a human face image.

[0124] Any references to memory, storage, databases, or other media used in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which is used as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0125] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for correcting a face image, characterized by, The method comprises the following steps: performing face detection on an original image; in the case that it is detected that the original image meets a face distortion correction condition, determining a face center region, a face edge region and a background region in the original image; the face distortion correction condition is that the original image includes a face; the face edge region is located at a connection between the face center region and the background region; obtaining each first grid in which the face center region is located, each second grid in which the background region is located and each third grid in which the face edge region is located, and generating an irregular grid of the original image according to the first grids, the second grids and the third grids; the first grid and the second grid are both larger than the third grid; determining a target weight of each of the first grid, the second grid and the third grid; the higher the target weight is, the more important the grid corresponding to the target weight is; the scale of the first grid is in inverse proportion to the target weight of the first grid; the scale of the second grid is in inverse proportion to the target weight of the second grid; the scale of the third grid is in inverse proportion to the target weight of the third grid; constructing an optimization function according to the target weight of the first grid, the target weight of the second grid and the target weight of the third grid, solving the optimization function and obtaining an optimized irregular grid; performing face distortion correction on the original image by using the optimized irregular grid to obtain a target image.

2. The method of claim 1, wherein, The obtaining each first grid in which the face center region is located, each second grid in which the background region is located and each third grid in which the face edge region is located comprises: in a preset regular grid mapped to the original image, merging each grid subunit in which the face center region is located to obtain each first grid; the grid subunit is the smallest unit of the preset regular grid; merging each grid subunit in which the background region is located to obtain each second grid; and obtaining each third grid in which the face edge region is located.

3. The method of claim 2, wherein, The merging each grid subunit in which the face center region is located to obtain each first grid comprises: from each grid subunit in which the face center region is located, determining a first grid group including at least two adjacent grid subunits, and merging the grid subunits in each first grid group to obtain each first grid. The merging each grid subunit in which the background region is located to obtain each second grid comprises: from each grid subunit in which the background region is located, determining a second grid group including at least two adjacent grid subunits, and merging the grid subunits in each second grid group to obtain each second grid.

4. The method of claim 2, wherein, The obtaining each third grid in which the face edge region is located comprises: regarding each grid subunit in which the face edge region is located as a third grid; or from each grid subunit in which the face edge region is located, determining a third grid group including at least two adjacent grid subunits, and merging the grid subunits in each third grid group to obtain each third grid.

5. The method of claim 1, wherein, The determining the face center region, the face edge region and the background region in the original image comprises: eroding the face region in the original image to obtain the face center region; dilating the face region in the original image to obtain the background region; obtaining the face edge region according to the original image, the face center region and the background region.

6. The method of claim 1, wherein, The irregular grid is obtained based on a preset regular grid; The determining the target weight of the first grid, the second grid and the third grid respectively comprises: obtaining the scale of the first grid, the second grid and the third grid respectively; determining the target weight of the first grid based on the scale of the first grid and the preset original weight of the preset regular grid; determining the target weight of the second grid based on the scale of the second grid and the preset original weight of the preset regular grid; determining the target weight of the third grid based on the scale of the third grid and the preset original weight of the preset regular grid.

7. A facial image correction device, characterized in that, comprises: a detection module configured to detect a face in an original image; a region determination module configured to determine a face center region, a face edge region and a background region in the original image when the original image is detected to meet a face distortion correction condition; the face distortion correction condition is that the original image includes a face; the face edge region is located at a connection between the face center region and the background region; an irregular grid generation module configured to obtain each first grid in which the face center region is located, each second grid in which the background region is located and each third grid in which the face edge region is located, and generate an irregular grid of the original image according to the first grids, the second grids and the third grids; the first grid and the second grid are both larger than the third grid; an optimization module configured to determine a target weight of the first grid, the second grid and the third grid respectively; The higher the target weight is, the more important the grid corresponding to the target weight is; the scale of the first grid is in inverse proportion to the target weight of the first grid; the scale of the second grid is in inverse proportion to the target weight of the second grid; the scale of the third grid is in inverse proportion to the target weight of the third grid; an optimization function is constructed according to the target weight of the first grid, the target weight of the second grid and the target weight of the third grid, the optimization function is solved, and an optimized irregular grid is obtained; a correction module configured to correct a face distortion of the original image by using the optimized irregular grid to obtain a target image.

8. An electronic device comprising a memory and a processor, said memory having stored therein a computer program, characterized in that, The computer program, when executed by the processor, causes the processor to perform the steps of the face image correction method according to any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1 to 6.

Images