Face image correction method and device, computer readable medium and electronic device

Abstract

The present disclosure provides a facial image correction method and device, a computer-readable medium and an electronic device, and relates to the field of image processing technology. The method includes: in response to detecting a distorted face image, the distorted face image includes a face area, determining a non-distorted area in the distorted face image; if the non-distorted area is detected to include a partial area of ​​the face area, determining the adaptive correction strength corresponding to the face area; performing distortion correction on the face area according to the adaptive correction strength to obtain a face image after distortion correction. The present disclosure can effectively improve the distortion correction effect of larger faces in selfie images, improve the restoration of face content, ensure the accuracy of expression of face content after distortion correction, make the face content in the image more consistent with the real face, and improve the user's photo-taking experience.

Description

Background Art

[0002] With the continuous improvement of people's living standards, the way of taking portrait photos through mobile terminals is becoming more and more popular. The perspective projection in camera imaging will cause perspective deformation, which causes the shape of the object to change, and may cause the face area in the captured image to be distorted. The degree of face distortion caused by perspective projection is related to its position in the picture. The closer to the edge, the greater the degree of face distortion, but the straight line shape can be maintained.

[0003] At present, the related face distortion correction solutions generally use the polar spherical projection (Stereographic) and the equiangular cylindrical projection (Mercator) to reduce the face distortion caused by perspective distortion. However, the side effect of these overall projections is that the straight lines in the image are bent, resulting in a decrease in the restoration of the photo image. Summary of the invention

[0004] The purpose of the present disclosure is to provide a facial image correction method, a facial image correction device, a computer-readable medium and an electronic device, thereby improving the restoration degree of facial content after distortion correction at least to a certain extent, so that the facial content in the image is more consistent with the real face.

[0005] According to a first aspect of the present disclosure, a facial image correction method is provided, comprising:

[0006] In response to detecting a distorted face image, the distorted face image includes a face region, determining a non-distorted region in the distorted face image;

[0007] If it is detected that the distortion-free area includes a partial area of ​​the face area, determining an adaptive correction strength corresponding to the face area;

[0008] The face region is subjected to distortion correction according to the adaptive correction strength to obtain a face image after distortion correction.

[0009] According to a second aspect of the present disclosure, there is provided a facial image correction device, comprising:

[0010] A distortion-free region determination module, configured to respond to detection of a distorted face image, the distorted face image including a face region, and to determine a distortion-free region in the distorted face image;

[0011] A correction strength determination module, configured to determine an adaptive correction strength corresponding to the face area if it is detected that the distortion-free area includes a partial area of ​​the face area;

[0012] The image distortion correction module is used to perform distortion correction on the face area according to the adaptive correction strength to obtain a face image after distortion correction.

[0013] According to a third aspect of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented.

[0014] According to a fourth aspect of the present disclosure, there is provided an electronic device, characterized in that it includes:

[0015] Processor; and

[0016] The memory is used to store one or more programs. When the one or more programs are executed by one or more processors, the one or more processors implement the above method.

[0017] A facial image correction method provided by an embodiment of the present disclosure can determine a non-distorted area in a distorted facial image when a distorted facial image including a facial area is detected, and when it is detected that the non-distorted area includes a partial area of ​​the facial area, it can determine an adaptive correction strength corresponding to the facial area, and then perform distortion correction on the facial area according to the adaptive correction strength to obtain a distortion-corrected facial image. On the one hand, when it is determined that the face area in the face distortion image is partially in the non-distortion area and partially in the distortion area, the face area is subjected to distortion correction through adaptive correction strength, so as to achieve different degrees of correction according to the distortion amplitude of the image content in different areas, thereby avoiding the situation in the related technical scheme where the same correction degree is used for different areas and resulting in a low degree of restoration of the corrected face, thereby effectively improving the expression accuracy of the face image after distortion correction, and improving the degree of restoration and authenticity of the corrected face image; on the other hand, by performing distortion correction on the face area through adaptive correction strength, it can effectively ensure a smooth transition between the image correction at the edge of the non-distortion area and the image correction at the edge of the distorted area, and there will be no obvious "boundary" problem caused by different correction strengths, thereby effectively improving the smoothness of the face area correction, and further improving the degree of restoration and authenticity of the corrected face image.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0020] Figure 1 A schematic diagram showing an exemplary system architecture to which embodiments of the present disclosure may be applied;

[0021] Figure 2 A flowchart schematically illustrates a method for correcting a facial image in an exemplary embodiment of the present disclosure;

[0022] Figure 3 A schematic diagram of a process for determining adaptive correction strength in an exemplary embodiment of the present disclosure is schematically shown;

[0023] Figure 4 A schematic diagram of a process of performing distortion correction on a face area in an exemplary embodiment of the present disclosure is schematically shown;

[0024] Figure 5 A schematic diagram of a process for detecting a distorted face image in an exemplary embodiment of the present disclosure is schematically shown;

[0025] Figure 6 A schematic diagram of a process for determining a non-distorted area in a distorted face image in an exemplary embodiment of the present disclosure is schematically shown;

[0026] Figure 7 A schematic diagram of a process for determining a partial area of ​​a human face area included in a distortion-free area in an exemplary embodiment of the present disclosure is schematically shown;

[0027] Figure 8 A schematic diagram schematically shows a principle of determining that a part of a face region is in a distortion-free region based on area in an exemplary embodiment of the present disclosure;

[0028] Fig. 9 A schematic diagram schematically shows the composition of a facial image correction device in an exemplary embodiment of the present disclosure;

[0029] Fig.10 A schematic diagram of an electronic device to which the embodiments of the present disclosure can be applied is shown. DETAILED DESCRIPTION

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0031] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0032] Figure 1 A schematic diagram showing a system architecture of an exemplary application environment in which a method and apparatus for facial image correction according to an embodiment of the present disclosure can be applied.

[0033] like Figure 1 As shown, the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The terminal devices 101, 102, 103 may be various electronic devices with image processing functions, including but not limited to desktop computers, portable computers, smart phones, tablet computers, etc. It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.

[0034] The face image correction method provided in the embodiment of the present disclosure is generally executed by the terminal devices 101, 102, and 103, and accordingly, the face image correction device is generally set in the terminal devices 101, 102, and 103. However, it is easy for those skilled in the art to understand that the face image correction method provided in the embodiment of the present disclosure can also be executed by the server 105, and accordingly, the face image correction device can also be set in the server 105, which is not particularly limited in this exemplary embodiment.

[0035] For example, in an exemplary embodiment, a user may obtain a facial image through terminal devices 101, 102, and 103, and when it is determined that a distorted facial image is detected, the distorted facial image is uploaded to server 105. After the server generates a distortion-corrected facial image through the facial image correction method provided in the embodiment of the present disclosure, the server transmits the distortion-corrected facial image to terminal devices 101, 102, 103, etc.

[0036] At present, for perspective projection, the closer the face area is to the edge of the image, the more obvious the image distortion is, while the distortion of the face area within the 50° field of view is extremely small and can be ignored; polar spherical projection is a common conformal projection method, but for the face in the middle area of ​​the image (such as within the 50° field of view), if the face is close to the camera when taking a photo (such as taking a selfie with the front camera), the face area will appear to be obviously bulged and enlarged, and visually appear to be widened.

[0037] In response to this phenomenon, the relevant solutions directly treat the faces within the 50° field of view as protected faces and do not perform distortion correction processing, that is, the unprocessed area. However, there is a common way to take selfies with the front camera: the face is in the middle of the picture and occupies a large area in the picture. Since the position of the face area in this type of scene exceeds the unprocessed range, the face part near the edge of the image has obvious distortion and will be listed as a face to be corrected. Therefore, after the distortion correction is implemented, the face will appear convex and widened.

[0038] Based on one or more problems in the related art, the present disclosure first provides a facial image correction method. The facial image correction method of an exemplary embodiment of the present disclosure is specifically described below by taking a terminal device executing the method as an example.

[0039] Figure 2 A schematic flow chart of a facial image correction method in this exemplary embodiment is shown, which may include the following steps S210 to S230:

[0040] In step S210, in response to detecting a distorted face image, the distorted face image including a face region, a non-distorted region is determined in the distorted face image.

[0041] In an exemplary embodiment, a face-distorted image refers to an image in which a qualified face area appears in the face image. For example, the face-distorted image may be a face image in which the face area occupies a large proportion of the entire image, such as a front-facing selfie face. It may also be a face image in which the face area is at the edge of the image. Of course, it may also be a face image that includes a face area at the edge of the image and has a larger area. This exemplary embodiment does not specifically limit this.

[0042] The face area refers to the image area where the face image content is located in the face distorted image. For example, the face distorted image can be recognized by a face detection and recognition network, and the image area corresponding to the obtained face frame can be considered as the image area where the face image content is located. Of course, the face area can also be an image area composed of a face edge contour determined by an image edge detection algorithm, but this embodiment is not limited to this.

[0043] The distortion-free area refers to an image area in the face distortion image obtained by perspective projection where there is no image distortion or the image distortion is weak. For example, the distortion-free area can be an image area within a 50° field of view angle in the face distortion image, or it can be an image area within a 40° field of view angle in the face distortion image. Of course, the specific range of the distortion-free area can be customized according to actual conditions (such as camera parameters), and this example embodiment does not impose any special limitations on this.

[0044] A facial image can be obtained from an image sensor or a storage unit and detected. When it is determined that the facial image belongs to a facial distortion image, a non-distortion area is determined in the facial distortion image. For example, the non-distortion area can be determined in the facial distortion image by drawing a region shape in the facial distortion image using preset parameters, but the present example embodiment is not limited to this.

[0045] In step S220, if it is detected that the distortion-free area includes a partial area of ​​the face area, the adaptive correction strength corresponding to the face area is determined.

[0046] In an exemplary embodiment, adaptive correction strength refers to parameter data for achieving different degrees of correction for different areas in an image. For example, the adaptive correction strength may be a correction strength distribution table for achieving weak correction processing for the central area of ​​the image and strong correction processing for the edge area of ​​the image. For example, the correction strength distribution table may include the correction strength corresponding to each pixel in different areas. The adaptive correction strength may also be an adaptive projection grid using a gradient correction strength. For example, the gradient correction strength may be determined by the distance data of each pixel in the image from the center of the image, and an adaptive projection grid may be constructed based on the gradient correction strength to achieve adaptive distortion correction for different areas or different image contents in the image through the adaptive projection grid. Of course, other types of adaptive correction strengths may also be used, as long as the data is capable of achieving different degrees of distortion correction processing for different areas in the image. This exemplary embodiment does not impose any special limitation on this.

[0047] It can be detected whether the distortion-free area contains part of the face area. If it is detected that the distortion-free area does not contain the face area, it means that the face area is entirely located in the edge area or the distorted area, and the face area can be directly subjected to strong correction processing; if it is detected that the distortion-free area contains part of the face area, it means that part of the face area is in the distortion-free area and part of it is in the edge area or the distorted area. If the face area in the distortion-free area is directly protected and the face area in the edge area or the distorted area is subjected to strong correction processing, the face may appear raised and widened, and an obvious "border" may appear at the boundary between the distortion-free area and the edge area or the distorted area, resulting in unrealistic facial content after correction, low restoration degree, and poor distortion correction effect.

[0048] Therefore, when it is detected that the distortion-free area contains part of the face area, the adaptive correction intensity can be used to effectively improve the expression accuracy of the face image after distortion correction, improve the restoration and authenticity of the corrected face image, and effectively improve the smoothness of the image content display in the corrected face area, thereby improving the effect and quality of distortion correction.

[0049] In step S230, distortion correction is performed on the face region according to the adaptive correction strength to obtain a distortion-corrected face image.

[0050] In an exemplary embodiment, after determining the adaptive correction strength corresponding to the face area, the face area in the face distortion image is corrected according to the adaptive correction strength, so that a face image with a higher degree of restoration and better distortion correction effect can be obtained, ensuring that the face area in the face image after distortion correction is displayed smoothly, improving the authenticity of the face, and enhancing the user's selfie experience.

[0051] Steps S210 to S230 are described in detail below.

[0052] In an exemplary embodiment, step S220 may be performed by Figure 3 The steps in the above table are used to determine the adaptive correction strength corresponding to the face area. Figure 3 As shown, it may specifically include:

[0053] Step S310, determining the image center coordinates of the distorted face image;

[0054] Step S320, calculating first distance data from each pixel point in the face area to the center coordinates of the image;

[0055] Step S330: determining an adaptive correction strength corresponding to the face area according to the first distance data.

[0056] Among them, the image center coordinates refer to the coordinates corresponding to the point in the face distortion image that is at the exact center of the image. For example, the coordinates of the four vertices of the face distortion image can be determined, and the image center coordinates of the face distortion image can be calculated based on the vertex coordinates. Of course, the midpoints of the four sides of the face distortion image can also be determined, and the image center coordinates of the face distortion image can be determined by the midpoints of the four sides. This example embodiment does not impose any special limitation on the method of calculating the image center coordinates of the face distortion image.

[0057] The first distance data refers to a set of distances from each pixel point in the face distortion image to the center of the image. For example, the first distance data may be the Euclidean distance from each pixel point to the center of the image. Specifically, the Euclidean distance may be calculated based on the pixel coordinates corresponding to each pixel point and the coordinates of the center of the image. The first distance data may also be the number of pixels between the pixel point and the center of the image, that is, the distance from each pixel point to the center of the image is measured by the number of pixels. Of course, other data that can measure the distance from each pixel point to the center of the image may also be used, and the present example embodiment is not limited thereto.

[0058] Generally speaking, the degree of face distortion caused by perspective projection is related to the position of the face in the image. The closer the face is to the edge of the image, the greater the degree of face distortion. Therefore, the adaptive correction strength for correcting the face area is determined based on the first distance data from each pixel to the center of the image, which can achieve weak correction processing for some face areas in the distortion-free area and strong correction processing for some face areas outside the distortion-free area. While ensuring the smoothness of the correction result, it effectively improves the distortion correction effect and improves the restoration degree of the corrected face image.

[0059] In an exemplary embodiment, a pre-constructed target projection model may be acquired, and the distortion of the face region may be corrected by using an adaptive correction strength and the target projection model.

[0060] Among them, the target projection model refers to a correction model used to reduce the facial distortion caused by perspective projection distortion. For example, the target projection model can be a projection model constructed based on the polar spherical projection principle, or it can be a projection model constructed based on the equiangular cylindrical projection principle. Of course, the target projection model can also be other types of projection models used to reduce the facial distortion caused by perspective projection distortion, and this example embodiment is not limited to this.

[0061] Taking the target projection model constructed based on the polar spherical projection principle as an example, the target projection model used for image distortion correction can be expressed by equation (1):

[0062]

[0063] Among them, r uIt can represent the radial distance from the current pixel to the center of the image under polar spherical projection; r0 can represent the mapping scale factor, which can be customized according to the actual situation and is not specifically limited here; r p It can represent the radial distance from the current pixel to the center of the image under perspective projection, and f can represent the focal length of the camera corresponding to the current pixel.

[0064] From equation (1), we can see that the smaller the focal length f, the smaller the calculated r u The smaller the r u The smaller the value, the greater the correction strength for the current pixel. However, in general, since the camera that captures the image is fixed and the focal length is also fixed, the adaptive projection distortion correction with gradual correction strength can be achieved by introducing a parameter for adjusting the focal length f in equation (1).

[0065] In this embodiment, the adaptive correction strength can be expressed as a pre-set focal length coefficient, which is used to adjust the focal length f in the target projection model represented by equation (1) to achieve weak correction processing for the partial face area in the distortion-free area and strong correction processing for the partial face area not in the distortion-free area.

[0066] Furthermore, you can Figure 4 The steps in the figure are used to implement the process of distortion correction of the face area through adaptive correction strength and target projection model. Figure 4 As shown, it may specifically include:

[0067] Step S410, determining the focal length coefficient corresponding to each pixel point in the face area according to the first distance data;

[0068] Step S420: adjusting the focal length parameters in the target projection model by using the focal length coefficient, and performing distortion correction on the face region according to the adjusted target projection model.

[0069] The focal length coefficient refers to pre-set data for adjusting the focal length parameters of the projection model. The focal length coefficient can be determined by the distance data from each pixel in the face area to the center of the image. Specifically, the focal length coefficient can be inversely proportional to the distance data.

[0070] The focal length parameter in the target projection model can be adjusted by the focal length coefficient to achieve different correction intensities in different areas. The farther the pixel is from the center of the image, the greater the image distortion formed by the perspective projection. At this time, the smaller the focal length coefficient is, the smaller the focal length parameter in the projection model is, and the smaller the radial distance from the current pixel to the center of the image under the polar spherical projection is, the greater the correction intensity is. Conversely, the closer the pixel is to the center of the image, the smaller the image distortion formed by the perspective projection is, and at this time, the larger the focal length coefficient is, the larger the focal length parameter in the projection model is, and the larger the radial distance from the current pixel to the center of the image under the polar spherical projection is, the smaller the correction intensity is.

[0071] For example, the focal length parameter in the pre-constructed target projection model can be adjusted by the focal length coefficient, and the adjusted target projection model can be expressed by equation (2):

[0072]

[0073] Among them, r u It can represent the radial distance from the current pixel to the center of the image under polar spherical projection; r0 can represent the mapping scale factor, which can be customized according to the actual situation and is not specifically limited here; r p It can represent the radial distance from the current pixel to the center of the image under perspective projection, a can represent the focal length coefficient, and f can represent the focal length of the camera corresponding to the current pixel.

[0074] In another example embodiment, the distance data from each pixel point in the part of the face region located in the non-distortion region to the image center coordinates can be determined, and the gradient correction strength for the image content located in the non-distortion region can be determined based on the distance data; the fixed correction strength of the part of the face region in the distortion region can be determined, and the adaptive correction strength for the face region can be determined based on the gradient correction strength of the face region in the non-distortion region and the fixed correction strength of the face region in the distortion region. This is also within the protection scope of this example embodiment, and this example embodiment is not limited thereto.

[0075] In an exemplary embodiment, the Figure 5 The steps in the above code are used to detect the distorted face image. Figure 5 As shown, it may specifically include:

[0076] Step S510, obtaining a face image to be processed, and performing face recognition on the face image to determine a face area in the face image;

[0077] Step S520, determining attribute data corresponding to the face region, and determining the degree of distortion of the face region according to the attribute data;

[0078] Step S530: In response to the distortion degree being greater than a preset distortion degree threshold, determining the face image as a distorted face image.

[0079] Among them, the facial image to be processed refers to an image containing facial content. For example, the facial image to be processed can be a facial image collected in real time by an image sensor, or a facial image stored in a storage unit. This example embodiment does not make any special limitations on this.

[0080] A face region refers to an image region in a face image where face content exists. For example, a face image can be identified by a face recognition detection model, and a face annotation box can be present in the output face image. The image region corresponding to the face annotation box can be determined as the face region in the face image. Of course, the face image can also be identified by an image segmentation algorithm, and the edge contour corresponding to the face content in the face image can be output, and the image region corresponding to the edge contour can be determined as the face region in the face image.

[0081] Attribute data refers to relevant parameters corresponding to the facial area in the facial image. For example, the attribute data can be the area size corresponding to the facial area, or the position coordinates of the facial area in the facial image. Of course, it can also be other types of attribute data related to the facial area. This example implementation does not make any special limitations on this.

[0082] The degree of distortion refers to the ratio of the distorted area of ​​the face area in the face image to the entire face image. In perspective projection, the farther the pixel point is from the center of the image, the greater the image distortion formed by the perspective projection, and the greater the degree of distortion of the face area.

[0083] In an exemplary embodiment, the attribute data may include face position data, which refers to the position of the face area in the face image. For example, the face position data may be the coordinates of the position of the face area in the face image, or it may be the distance data from the center point of the face area to the center point of the face image. This exemplary embodiment does not specifically limit this.

[0084] Optionally, the following steps may be used to determine the degree of distortion of a face region based on the attribute data of the face region: first, the second distance data from the center point of the face image to the center point of the face region may be determined, and then the face position data of the face region in the face image may be determined based on the second distance data, and then the degree of distortion of the face region may be determined based on the face position data. Specifically, the mapping relationship between different distance data and the degree of distortion may be pre-calculated under face images of different sizes, and the degree of distortion of the face region may be quickly determined based on the mapping relationship and the second distance data from the center point of the face region to the center point of the face region.

[0085] It should be noted that the “first” and “second” in the “first distance data” and “second distance data” in the embodiment of the present disclosure are only used to distinguish different distances and have no special meaning and should not cause any special limitation to this exemplary embodiment.

[0086] In an exemplary embodiment, the attribute data may include facial area data. For example, assuming that the face region is a regular region formed by a face frame, the facial area data corresponding to the face region can be calculated based on the size of the face frame; if the face region is an irregular region formed by edge contours detected by an edge detection algorithm, the facial area data corresponding to the face region can be calculated based on the number of pixels in the face region and the pixel size.

[0087] Optionally, the degree of distortion of the face region can be determined based on the attribute data of the face region through the following steps: first, the face area data can be determined based on the face frame corresponding to the face region, and then when it is detected that the face area data is greater than a preset area threshold, the degree of distortion of the face region can be determined through the face area data.

[0088] As the size of the face region increases, the more parts are close to the edge of the face image, the larger the proportion of the distorted part is, the greater the distortion amplitude is, that is, the greater the degree of distortion of the face region is. Specifically, the mapping relationship between the size of the face region and the degree of distortion can be determined in advance under face images of different sizes, and the degree of distortion of the face region can be quickly determined through the mapping relationship and the size of the face region.

[0089] In an exemplary embodiment, step S210 may be performed by Figure 6 The steps in the above are used to determine the distortion-free area in the face distortion image. Figure 6 As shown, it may specifically include:

[0090] Step S610, determining a target radial circle based on the image center coordinates of the face distortion image and preset radius data;

[0091] Step S620: taking the area where the target radial circle is located as the non-distortion area in the distorted face image.

[0092] The radius data refers to the pre-set data used to determine the size of the distortion-free area in the face distortion image. For example, assuming that the size of the face distortion image is 100mm*100mm, and the radius of the circular area corresponding to the 50° field of view in the face distortion image is 25mm, then the radius data of the distortion-free area corresponding to the face distortion image at this size can be set to 25mm. Of course, the radius data of the distortion-free area can be set according to actual conditions, such as setting the radius data of the distortion-free area according to the parameters of the camera, and this exemplary embodiment does not specifically limit this.

[0093] The target radial circle refers to a circular area determined in the distorted face image using radius data and the image center coordinates of the distorted face image.

[0094] The target radial circle can be determined by the image center coordinates of the face distortion image and pre-stored radius data, and then the target radial circle can be used as the distortion-free area in the face distortion image.

[0095] In an exemplary embodiment, step S220 may be performed by Figure 7 The steps in the above figure are used to detect whether part of the face area is in the distortion-free area. Figure 7 As shown, it may specifically include:

[0096] Step S710, determining the area intersection of the face region in the face distortion image and the target radial circle;

[0097] Step S720, determining a first area ratio according to the area intersection and the face area data;

[0098] Step S730, determining a second area ratio according to the area intersection and the area of ​​the target radial circle;

[0099] Step S740: If the first area ratio is not 1, and the first area ratio or the second area ratio is greater than a ratio threshold, it is determined that the distortion-free area is detected to include a partial area of ​​the face area.

[0100] Among them, the first area ratio refers to data measuring the ratio of the overlapping area between the face area and the target radial circle to the face area, and the second area ratio refers to data measuring the ratio of the overlapping area between the face area and the target radial circle to the area of ​​the target radial circle.

[0101] It should be noted that the “first” and “second” in the “first area ratio” and “second area ratio” in this embodiment are only used to distinguish different area ratios and do not have any other special meanings and should not cause any special limitations to this embodiment.

[0102] When it is detected that the first area ratio is not 1, it can be considered that the distortion-free area may include a part of the face area, and the face area needs to be distorted. At this time, it can be further determined whether the first area ratio or the second area ratio is greater than the ratio threshold. If the first area ratio or the second area ratio is greater than the ratio threshold, it can be determined that the distortion-free area includes a part of the face area, and the face image of the face area can be used as a face distortion image.

[0103] In an exemplary embodiment, step S220 may further include: if it is detected that the distortion-free area includes a face area, performing distortion correction processing on the area other than the face area in the face distortion image.

[0104] Since the detected face area is completely in the non-distortion area, it means that the face area is not distorted, or the distortion is small and can be ignored. Therefore, after the face area is protected, the face image is globally corrected.

[0105] Specifically, when it is detected that the first area ratio is 1, it is determined that the distortion-free area includes the face area. Since the ratio of the overlapping area of ​​the face area and the target radial circle to the face area is 1, it means that the overlapping area of ​​the face area and the target radial circle is equal to the face area data of the face area, and it can be considered that the distortion-free area includes the face area.

[0106] Figure 8 The schematic diagram schematically shows a principle of determining that a part of a face region is in a distortion-free region based on area in an exemplary embodiment of the present disclosure.

[0107] refer to Figure 8 As shown, a non-distorted area 802 is determined in a distorted face image 801, and face recognition is performed on the face image 801 to obtain a face area 803. The area of ​​the target radial circle corresponding to the non-distorted area 802 is S1, and the face area data of the face area 803 is S2.

[0108] The area intersection S3 of the target radial circle corresponding to the face area 803 and the undistorted area 802 in the face distortion image 801 can be determined, and a first area ratio p1 is determined according to the area intersection and the face area data, where p1=S3 / S2, and a second area ratio p2 is determined according to the area intersection and the area of ​​the target radial circle, where p2=S3 / S1. When it is detected that the first area ratio is not 1, it can be considered that the undistorted area may contain a part of the face area, and the face area needs to be distorted. If the first area ratio or the second area ratio is greater than the ratio threshold, it can be considered that the undistorted area does contain a part of the face area, and the face area can be distorted by the determined adaptive correction strength. When the first area ratio is detected to be 1, it means that the overlapping area between the face region and the target radial circle is equal to the face area data of the face region. It can be considered that the distortion-free region contains the complete face region. At this time, since there is no face distortion in the face region or the face distortion is weak and can be ignored, the face region can be protected and then the face distortion image can be subjected to conventional distortion correction processing.

[0109] The face image correction method in the disclosed embodiment optimizes and improves the front selfie effect. Since the face area in the face distortion image occupies a large area in the entire image, the edge part is obviously distorted, while the middle part is in the unprocessed range. The disclosed embodiment introduces an adaptive projection grid with a gradient correction intensity. Then, through adaptive correction intensity, weak correction processing is used to target the bulge in the middle of the face distortion image, so that the final imaging effect approaches perspective projection; strong correction processing is used to target the face distortion at the edge of the face distortion image, so that the imaging effect approaches polar spherical projection, thereby achieving a smooth transition of correction intensity from the middle area to the edge area of ​​the face, avoiding unnatural stretching.

[0110] To summarize, in this exemplary embodiment, when a distorted face image including a face area is detected, a non-distorted area in the distorted face image can be determined, and when it is detected that the non-distorted area includes a partial area of ​​the face area, the adaptive correction strength corresponding to the face area can be determined, and then the face area can be distorted according to the adaptive correction strength to obtain a face image after distortion correction. On the one hand, when it is determined that the face area in the face distortion image is partially in the non-distortion area and partially in the distortion area, the face area is subjected to distortion correction through adaptive correction strength, so as to achieve different degrees of correction according to the distortion amplitude of the image content in different areas, thereby avoiding the situation in the related technical scheme where the same correction degree is used for different areas and resulting in a low degree of restoration of the corrected face, thereby effectively improving the expression accuracy of the face image after distortion correction, and improving the degree of restoration and authenticity of the corrected face image; on the other hand, by performing distortion correction on the face area through adaptive correction strength, it can effectively ensure a smooth transition between the image correction at the edge of the non-distortion area and the image correction at the edge of the distorted area, and there will be no obvious "boundary" problem caused by different correction strengths, thereby effectively improving the smoothness of the face area correction, and further improving the degree of restoration and authenticity of the corrected face image.

[0111] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0112] For further reference, Fig. 9 As shown, in the embodiment of this example, a facial image correction device 900 is also provided, including a distortion-free region determination module 910, a correction strength determination module 920 and an image distortion correction module 930. Among them:

[0113] The distortion-free region determination module 910 is used to respond to detecting a distorted face image, the distorted face image including a face region, and to determine a distortion-free region in the distorted face image;

[0114] The correction strength determination module 920 is used to determine the adaptive correction strength corresponding to the face area if it is detected that the distortion-free area includes a partial area of ​​the face area;

[0115] The image distortion correction module 930 is used to perform distortion correction on the face area according to the adaptive correction strength to obtain a distortion-corrected face image.

[0116] In an exemplary embodiment, the correction strength determination module 920 may be used to:

[0117] Determining the image center coordinates of the distorted face image;

[0118] Calculate first distance data from each pixel point in the face area to the center coordinates of the image;

[0119] Determining the adaptive correction strength corresponding to the face area according to the first distance data;

[0120] The adaptive correction strength is used to perform weak correction processing on a part of the face area in the distortion-free area and to perform strong correction processing on a part of the face area outside the distortion-free area.

[0121] In an exemplary embodiment, the correction strength determination module 920 may be used to:

[0122] The face region is subjected to distortion correction by using the adaptive correction strength and a pre-constructed target projection model.

[0123] In an exemplary embodiment, the adaptive correction strength may include a focal length factor, and the correction strength determination module 920 may be used to:

[0124] Determine the focal length coefficient corresponding to each pixel point in the face area according to the first distance data;

[0125] The focal length parameters in the target projection model are adjusted by the focal length coefficient, and the distortion of the face area is corrected according to the adjusted target projection model.

[0126] In an exemplary embodiment, the distortion-free region determining module 910 may include:

[0127] A face region determination unit, used to obtain a face image to be processed, perform face recognition on the face image, and determine a face region in the face image;

[0128] a distortion degree determination unit, configured to determine attribute data corresponding to the face region, and determine the distortion degree of the face region according to the attribute data;

[0129] The face distortion image screening unit is used to determine the face image as a face distortion image in response to the distortion degree being greater than a preset distortion degree threshold.

[0130] In an exemplary embodiment, the attribute data may include face position data, and the distortion degree determination unit may be used to:

[0131] Determine second distance data from the center point of the face image to the center point of the face area;

[0132] Determine face position data of the face area in the face image according to the second distance data;

[0133] The degree of distortion of the face area is determined by the face position data.

[0134] In an exemplary embodiment, the attribute data may include face area data, and the distortion degree determination unit may be used to:

[0135] Determine the face area data based on the face frame corresponding to the face area;

[0136] If it is detected that the face area data is greater than a preset area threshold, the degree of distortion of the face region is determined according to the face area data.

[0137] In an exemplary embodiment, the distortion-free region determining module 910 may also be used to:

[0138] Determining a target radial circle based on the image center coordinates of the face distortion image and preset radius data;

[0139] The area where the target radial circle is located is used as the non-distortion area in the face distorted image.

[0140] In an exemplary embodiment, the correction strength determination module 920 may be used to:

[0141] Determine the area intersection of the face region in the face distortion image and the target radial circle;

[0142] Determine a first area ratio according to the area intersection and the face area data;

[0143] Determine a second area ratio according to the area intersection and the area of ​​the target radial circle

[0144] If the first area ratio is not 1, and the first area ratio or the second area ratio is greater than a ratio threshold, it is determined that the distortion-free area is detected to include a partial area of ​​the face area.

[0145] In an exemplary embodiment, the facial image correction device 900 may further include a face protection unit, which may be used to:

[0146] If it is detected that the distortion-free area includes the complete face area, distortion correction processing is performed on the area other than the face area in the face distortion image.

[0147] In an exemplary embodiment, the face protection unit may also be used to:

[0148] If it is detected that the first area ratio is 1, it is determined that the distortion-free area includes the complete face area.

[0149] The specific details of each module in the above device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0150] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0151] The exemplary embodiment of the present disclosure also provides an electronic device. The electronic device may be the above-mentioned terminal device 101, 102, 103 or server 105. Generally, the electronic device may include a processor and a memory, the memory is used to store executable instructions of the processor, and the processor is configured to execute the above-mentioned face image correction method by executing the executable instructions.

[0152] Below Fig.10 The mobile terminal 1000 in FIG. 1 is taken as an example to exemplify the structure of the electronic device. It should be understood by those skilled in the art that, in addition to the components specifically used for mobile purposes, Fig.10 The construction in can also be applied to fixed type equipment.

[0153] like Fig.10 As shown, the mobile terminal 1000 may specifically include: a processor 1001, a memory 1002, a bus 1003, a mobile communication module 1004, an antenna 1, a wireless communication module 1005, an antenna 2, a display screen 1006, a camera module 1007, an audio module 1008, a power module 1009 and a sensor module 1010.

[0154] Processor 1001 may include one or more processing units, for example: processor 1001 may include AP (Application Processor), modem processor, GPU (Graphics Processing Unit), ISP (Image Signal Processor), controller, encoder, decoder, DSP (Digital Signal Processor), baseband processor and / or NPU (Neural-Network Processing Unit), etc.

[0155] The encoder can encode (i.e. compress) an image or video to reduce the data size for easy storage or transmission. The decoder can decode (i.e. decompress) the encoded data of an image or video to restore the image or video data. The mobile terminal 1000 can support one or more encoders and decoders, such as: image formats such as JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), BMP (Bitmap), and video formats such as MPEG (Moving Picture Experts Group) 1, MPEG10, H.1063, H.1064, and HEVC (High Efficiency Video Coding).

[0156] The processor 1001 may be connected to the memory 1002 or other components via a bus 1003 .

[0157] The memory 1002 may be used to store computer executable program codes, which may include instructions. The processor 1001 executes various functional applications and data processing of the mobile terminal 1000 by running the instructions stored in the memory 1002. The memory 1002 may also store application data, such as images, videos, and other files.

[0158] The communication function of the mobile terminal 1000 can be implemented by the mobile communication module 1004, antenna 1, wireless communication module 1005, antenna 2, modulation and demodulation processor and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 1004 can provide 3G, 4G, 5G and other mobile communication solutions applied to the mobile terminal 1000. The wireless communication module 1005 can provide wireless communication solutions such as wireless LAN, Bluetooth, near field communication, etc. applied to the mobile terminal 1000.

[0159] The display screen 1006 is used to implement display functions, such as displaying user interfaces, images, videos, etc. The camera module 1007 is used to implement shooting functions, such as shooting images, videos, etc. The audio module 1008 is used to implement audio functions, such as playing audio, collecting voice, etc. The power module 1009 is used to implement power management functions, such as charging the battery, powering the device, monitoring the battery status, etc.

[0160] The sensor module 1010 may include one or more sensors for implementing corresponding sensing detection functions. For example, the sensor module 1010 may include an inertial sensor for detecting the motion posture of the mobile terminal 1000 and outputting inertial sensing data.

[0161] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above-mentioned method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of the present specification.

[0162] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0163] In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0164] In addition, program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0165] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0166] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A facial image correction method, characterized in that: include: In response to detecting a distorted face image, the distorted face image includes a face region, determining a non-distorted region in the distorted face image; If it is detected that the distortion-free area includes a partial area of ​​the face area, determining an adaptive correction strength corresponding to the face area, wherein the adaptive correction strength is used to perform a weak correction process on the partial face area in the distortion-free area and to perform a strong correction process on the partial face area outside the distortion-free area; The face region is subjected to distortion correction according to the adaptive correction strength to obtain a face image after distortion correction.

2. The method according to claim 1, characterized in that The determining the adaptive correction strength corresponding to the face area includes: Determining the image center coordinates of the distorted face image; Calculate first distance data from each pixel point in the face area to the center coordinates of the image; The adaptive correction strength corresponding to the face area is determined according to the first distance data.

3. The method according to claim 2, characterized in that The performing distortion correction on the face region according to the adaptive correction strength includes: The face region is subjected to distortion correction by using the adaptive correction strength and a pre-constructed target projection model.

4. The method according to claim 3, characterized in that The adaptive correction strength includes a focal length factor, The step of performing distortion correction on the face region by using the adaptive correction strength and a pre-built target projection model includes: Determine the focal length coefficient corresponding to each pixel point in the face area according to the first distance data; The focal length parameters in the target projection model are adjusted by the focal length coefficient, and the distortion of the face area is corrected according to the adjusted target projection model.

5. The method according to claim 1, characterized in that The response detects a distorted face image, including: Acquire a face image to be processed, perform face recognition on the face image, and determine a face area in the face image; Determining attribute data corresponding to the face region, and determining the degree of distortion of the face region according to the attribute data; In response to the distortion degree being greater than a preset distortion degree threshold, the face image is determined as a distorted face image.

6. The method according to claim 5, characterized in that The attribute data includes face position data, and determining the degree of distortion of the face region according to the attribute data includes: Determine second distance data from the center point of the face image to the center point of the face area; Determine face position data of the face area in the face image according to the second distance data; The degree of distortion of the face area is determined by the face position data.

7. The method according to claim 6, characterized in that The attribute data includes face area data, and determining the degree of distortion of the face area according to the attribute data includes: Determine the face area data based on the face frame corresponding to the face area; If it is detected that the face area data is greater than a preset area threshold, the degree of distortion of the face region is determined according to the face area data.

8. The method according to claim 1, characterized in that The determining of the non-distortion area in the face distortion image includes: Determining a target radial circle based on the image center coordinates of the face distortion image and preset radius data; The area where the target radial circle is located is used as the non-distortion area in the face distorted image.

9. The method according to claim 8, characterized in that The detecting that the distortion-free area includes a partial area of ​​the face area includes: Determine the area intersection of the face region in the face distortion image and the target radial circle; Determine a first area ratio according to the area intersection and the face area data; Determining a second area ratio according to the area intersection and the area of ​​the target radial circle; If the first area ratio is not 1, and the first area ratio or the second area ratio is greater than a ratio threshold, it is determined that the distortion-free area is detected to include a partial area of ​​the face area.

10. The method according to claim 1, characterized in that The method further comprises: If it is detected that the distortion-free area includes the complete face area, distortion correction processing is performed on the area other than the face area in the face distortion image.

11. The method according to claim 9 or 10, characterized in that: If it is detected that the distortion-free area includes the complete face area, the method comprises: If it is detected that the first area ratio is 1, it is determined that the distortion-free area includes the complete face area.

12. A facial image correction device, characterized in that: include: A distortion-free region determination module, configured to respond to detection of a distorted face image, the distorted face image including a face region, and to determine a distortion-free region in the distorted face image; a correction strength determination module, configured to determine an adaptive correction strength corresponding to the face region if it is detected that the distortion-free region includes a partial region of the face region, wherein the adaptive correction strength is used to perform a weak correction process on the partial face region in the distortion-free region and to perform a strong correction process on the partial face region outside the distortion-free region; The image distortion correction module is used to perform distortion correction on the face area according to the adaptive correction strength to obtain a face image after distortion correction.

13. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

14. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 11 by executing the executable instructions.

Images