Image distortion correction method, device, electronic device, chip and storage medium

Abstract

This application is applicable to the field of image processing technology, and provides an image distortion correction method, device, electronic device, chip and storage medium. The method includes: if there is a first body area in at least one body area of the image to be processed, performing distortion correction on the first body area; if there is a second body area in at least one of the body areas of the image to be processed, calculating the area of the second body area and the area of the first overlapping area; calculating a first area ratio; the first area ratio refers to the ratio of the area of the first overlapping area to the area of the second body area; if the first area ratio is less than or equal to a first threshold, performing distortion correction on the second body area. Through this application, the distorted body areas in the image can be corrected for distortion.

Description

Technical Field

[0001] This application belongs to the technical field of image processing, and particularly relates to an image distortion correction method, device, electronic device, chip and storage medium. Background Art

[0002] In recent years, in order to obtain images with a larger field of view, the field of view angle of the camera in electronic devices has become larger and larger. Images captured by a camera with a larger field of view angle usually have a problem of portrait distortion. When the face in the image is distorted, it is usually necessary to correct the distorted face so that the face conforms to human eye vision. Summary of the Invention

[0003] Embodiments of this application provide an image distortion correction method, device, electronic device, chip and storage medium to correct the distorted body area in the image.

[0004] In a first aspect, embodiments of this application provide an image distortion correction method, including:

[0005] If there is a first body area in at least one body area of the image to be processed, correct the distortion of the first body area; the body area refers to the area in the portrait area of the image to be processed that does not contain face data; the first body area refers to the body area that does not overlap with the preset area; the preset area refers to the local area centered on the center point of the image to be processed in the image to be processed;

[0006] If there is a second body area in at least one of the body areas of the image to be processed, calculate the area of the second body area and the area of the first overlapping area; the second body area refers to the body area that partially overlaps with the preset area; the first overlapping area refers to the overlapping area between the second body area and the preset area;

[0007] Calculate a first area ratio; the first area ratio refers to the ratio of the area of the first overlapping area to the area of the second body area;

[0008] If the first area ratio is less than or equal to a first threshold, correct the distortion of the second body area.

[0009] In the embodiments of the present application, by determining whether the body region overlaps with the preset region, it can be determined whether there is a first body region with distortion and a second body region that may have distortion in the image to be processed. If there is a first body region with distortion, distortion correction is performed on the first body region. If there is a second body region that may have distortion, by calculating the area of the second body region and the area of the overlapping region between the second body region and the preset region (i.e., the area of the first overlapping region), and comparing the ratio of the area of the first overlapping region to the area of the second body region (i.e., the first area ratio) with the first threshold, it can be further determined whether the second body region has distortion. When the first area ratio is less than or equal to the first threshold, it is determined that the second body region has distortion, and at this time, distortion correction needs to be performed on the second body region. In the above process, by performing distortion correction on the first body region and the second body region with distortion, distortion correction of the body region in the image to be processed can be achieved.

[0010] In a second aspect, an image distortion correction device provided by an embodiment of the present application includes:

[0011] A first correction module, configured to perform distortion correction on the first body region if there is a first body region in at least one body region of the image to be processed; the body region refers to the region in the portrait region of the image to be processed that does not contain face data; the first body region refers to the body region that does not overlap with the preset region; the preset region refers to the local region centered on the center point of the image to be processed in the image to be processed;

[0012] An area calculation module, configured to calculate the area of the second body region and the area of the first overlapping region if there is a second body region in at least one of the body regions of the image to be processed; the second body region refers to the body region that partially overlaps with the preset region; the first overlapping region refers to the overlapping region between the second body region and the preset region;

[0013] A ratio calculation module, configured to calculate the first area ratio; the first area ratio refers to the ratio of the area of the first overlapping region to the area of the second body region;

[0014] A second correction module, configured to perform distortion correction on the second body region if the first area ratio is less than or equal to the first threshold.

[0015] In a third aspect, an electronic device provided by an embodiment of the present application includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the image distortion correction method described in the first aspect above are implemented.

[0016] Fourthly, an embodiment of the present application provides a chip, including a processor, which is configured to read and execute a computer program stored in a memory to perform the steps of the image distortion correction method as described in the first aspect above.

[0017] Optionally, the memory is connected to the processor through a circuit or a wire.

[0018] Fifthly, an embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the image distortion correction method as described in the first aspect above are implemented.

[0019] Sixthly, an embodiment of the present application provides a computer program product, and when the computer program product runs on an electronic device, the electronic device is enabled to perform the steps of the image distortion correction method as described in the first aspect above.

[0020] It can be understood that the second aspect, the third aspect, the fourth aspect, the fifth aspect, and the sixth aspect provided above are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be elaborated here. Description of the Drawings

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0022] Figure 1a It is an example diagram of a to-be-processed image with human portrait distortion;

[0023] Figure 1b It is Figure 1a an example diagram of the result of global correction of the to-be-processed image in;

[0024] Figure 1c It is Figure 1a an example diagram of the result of face local correction of the to-be-processed image in;

[0025] Figure 1d It is Figure 1c an example diagram of the result of distortion correction of the body area in;

[0026] Figure 2 It is a schematic diagram of the implementation process of the image distortion correction method provided by an embodiment of the present application;

[0027] Figure 3 It is an example diagram of a preset area;

[0028] Figure 4 It is a schematic diagram of the implementation process of the image distortion correction method provided by another embodiment of the present application;

[0029] Figure 5 It is a schematic diagram of the implementation process of the image distortion correction method provided by yet another embodiment of the present application;

[0030] Figure 6 It is an example diagram of a body frame;

[0031] Figure 7 It is a schematic diagram of the structure of the image distortion correction device provided by an embodiment of the present application;

[0032] Figure 8 It is a schematic diagram of the structure of the electronic device provided by an embodiment of the present application. Detailed implementation manners

[0033] In the following description, specific details such as specific system architectures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0034] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0035] In addition, in the description of the specification of the present application and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0036] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that specific features, structures, or characteristics described in connection with that embodiment are included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized.

[0037] The image distortion correction method provided by the embodiments of this application can be applied to electronic devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiments of this application do not impose any restrictions on the specific types of electronic devices.

[0038] It should be understood that the magnitudes of the sequence numbers of the steps in this embodiment do not mean the order of execution is prior or subsequent. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this application.

[0039] When a user takes a photo using a camera with a relatively large field of view angle (such as a wide-angle camera, an ultra-wide-angle camera, etc.), it usually causes stretching and distortion of the edge portrait, resulting in distortion. As Figure 1a shown is an example diagram of the image to be processed with portrait distortion, Figure 1a In which 101 represents a straight line in the background area of the image to be processed, 102, 103, and 104 represent the portrait areas in the image to be processed. The portrait areas 102 and 104 are distorted, and the portrait area 103 is not distorted. Among them, the portrait area in the image to be processed can be understood as the foreground of the image to be processed (i.e., the object of interest to the user), and then the background area can refer to the area other than the portrait area in the image to be processed.

[0040] In order to achieve distortion correction of the portrait area in the image to be processed, the existing two solutions are respectively to perform global correction on the image to be processed and to perform face local correction on the image to be processed.

[0041] Global correction of the image to be processed is to correct the distortion of the entire image. Although this solution can correct the distortion of the portrait area, it is easy to bend the straight lines in the background of the image to be processed and change the shape of the undistorted portrait area in the image to be processed, resulting in distortion of the image to be processed. Figure 1b The shown is Figure 1a The result of global correction of the image to be processed in is shown in the figure below. Figure 1b It can be seen that through Figure 1a The global result of the image to be processed in the image can achieve distortion correction of the portrait area 102 and the portrait area 104 in the image to be processed, but it causes the straight line 101 to bend and the portrait area 103 to be distorted.

[0042] The local face correction of the processed image is to correct the distortion of the face area in the processed image. Although this solution can correct the distortion of the face area, it is easy to cause the head-to-body ratio to be inconsistent (for example, the face area is larger and the body area is smaller). Figure 1c The shown is Figure 1a The example diagram of the result of local face correction on the image to be processed in . Figure 1c It can be seen that by performing local face correction on the processed image, the distortion correction of the face area in the portrait area 102 and the face area in the portrait area 104 in the processed image can be achieved, but the body area in the portrait area 102 and the portrait area 104 is still distorted, resulting in an uncoordinated head-to-body ratio in the portrait area 102 and the portrait area 104. The body area refers to the area in the portrait area that does not contain face data. For example, Figure 1a , Figure 1b and Figure 1c The areas other than the face area in the portrait area 102 and the portrait area 104, and the entire portrait area 103 are all body areas.

[0043] In order to achieve distortion correction of the face area in the image to be processed while reducing the influence of distortion correction on the background area and improving the problem of the uncoordinated head-to-body ratio. An embodiment of the present application provides an image distortion correction method, which, on the basis of performing local face correction on the image to be processed, determines whether the body area overlaps with a preset area, and when there is a partial overlap with the preset area, based on the area of the body area in the preset area and the ratio of the body area to its own area, can screen out the distorted body area from the image to be processed, and perform distortion correction on the distorted body area to achieve distortion correction of the body area in the image to be processed. This application, on the basis of performing local face correction on the image to be processed, performs distortion correction on the distorted body area, and does not perform distortion correction on the background area, can reduce the influence of distortion correction on the background area and improve the problem of the uncoordinated head-to-body ratio. Figure 1d The shown is Figure 1c An example of the results of distortion correction of the body region in Figure 2.

[0044] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0045] See also Figure 2 , is a schematic diagram of an implementation flow of an image distortion correction method provided by an embodiment of the present application, and the image distortion correction method is applied to electronic equipment. Figure 2 As shown, the image distortion correction method may include the following steps:

[0046] Step 201: If a first body region exists in at least one body region of the image to be processed, distortion correction is performed on the first body region.

[0047] The above-mentioned image to be processed can be an image stored in the electronic device, an image sent to the electronic device by other devices, or an image captured in real time by the camera of the electronic device (for example, the image displayed in the preview area in the camera interface of a mobile phone), and is not limited here.

[0048] The camera of the electronic device may be a front camera or a rear camera of the electronic device, or a camera controlled by the electronic device in a wired or wireless manner. The above-mentioned camera may be a camera with a large field of view such as a wide-angle camera or an ultra-wide-angle camera.

[0049] After obtaining the image to be processed, the electronic device can obtain the body area and the face area in the portrait area based on whether the portrait area of the image to be processed contains face data. The face data may refer to the location information, size and other data of facial features such as nose, eyes, mouth, eyebrows and ears.

[0050] In an optional embodiment, if there is a portrait area in the image to be processed that does not contain face data, the portrait area is determined as the body area; if there is a portrait area in the image to be processed that contains face data, based on the position information in the face data, the body area in the portrait area is determined.

[0051] Among them, the position information in the face data may refer to the position information of facial features such as the nose, eyes, mouth, eyebrows, and ears in the image to be processed.

[0052] To determine whether a portrait area contains face data, it can be determined whether the portrait area contains data of all facial features such as the nose, eyes, mouth, eyebrows, and ears. Of course, it can also be determined whether the portrait area contains data of some facial features among the five sense organs such as the nose, eyes, mouth, eyebrows, and ears, which is not limited here.

[0053] For a portrait area containing face data, based on the position information in the face data, the electronic device can determine the face area in the portrait area, and then the area other than the face area in the portrait area is the body area.

[0054] The electronic device can obtain all the body areas and face areas in the image to be processed by performing portrait segmentation and face detection on the image to be processed. For example, all portrait areas can be obtained from the image to be processed through portrait segmentation first, and then face detection is performed on each portrait area to detect whether each portrait area contains face data; for a portrait area containing face data, the portrait area can be divided into a face area and a body area based on the position information in the face data; for a portrait area that does not contain face data, the entire portrait area can be determined as the body area.

[0055] It should be noted that this application does not limit the algorithms used for portrait segmentation and face detection. For example, a portrait segmentation algorithm based on a convolutional neural network model can be used to perform portrait segmentation on the image to be processed; a face detection algorithm based on binary wavelet transform or based on histogram rough segmentation and singular value features can be used to perform face detection on the image to be processed.

[0056] Among them, the first body area may refer to the body area that does not overlap with the preset area.

[0057] The preset area may refer to a local area centered on the center point of the image to be processed in the image to be processed. For example, the preset area is a circular area with the center point of the image to be processed as the origin and the minimum value of the height and width of the image to be processed as the radius. The portrait area located within the preset area is not likely to be distorted or the distortion generated is small and can be ignored.

[0058] ​In the image to be processed, distortion usually occurs symmetrically from the center point of the image to be processed. The closer the area is to the center point, the smaller the degree of distortion; the farther the area is from the center point, the greater the degree of distortion. Therefore, by setting a local area centered on the center point of the image to be processed as the preset area, it is possible to more accurately determine whether distortion has occurred in the body area of the image to be processed.

[0059] Step 202: If there is a second body area in at least one body area of the image to be processed, calculate the area of the second body area and the area of the first overlapping area.

[0060] Among them, the second body area can refer to the body area that partially overlaps with the preset area; the first overlapping area can refer to the overlapping area between the second body area and the preset area. The partial overlap of the two areas can be understood as the intersection of the two areas.

[0061] By determining whether at least one body area in the image to be processed overlaps with the preset area, the electronic device can determine whether there is a first body area, a second body area, and a third body area in the image to be processed. Among them, the third body area can refer to the body area located within the preset area.

[0062] As Figure 3 shown is an example diagram of the preset area. Figure 3 The circular area formed by the dotted line in Figure 3 is the preset area. It can be seen from

[0063] that the body area in the portrait area 104 does not overlap with the preset area, so it can be determined that the body area in the portrait area 104 is the first body area; the body area in the portrait area 102 partially overlaps with the preset area, so it can be determined that the body area in the portrait area 102 is the second body area; the portrait area 103 is located within the preset area, so it can be determined that the portrait area 103 has not undergone distortion and no distortion correction is required.

[0064] It should be noted that there may be both a first body area and a second body area in the image to be processed, or there may be only a first body area or a second body area. If there is a first body area in the image to be processed, the number of first body areas may be one or at least two. If there is a second body area in the image to be processed, the number of second body areas may be one or at least two.

[0064] To simplify the calculation of the areas of the second body area and the first overlapping area, the number of pixels in the second body area and the number of pixels in the first overlapping area can be counted. The number of pixels in the second body area is used as the area of the second body area, and the number of pixels in the first overlapping area is used as the area of the first overlapping area. Of course, other methods can also be used to calculate the areas of the second body area and the first overlapping area, which are not limited here.

[0065] Step 203, calculate the first area ratio; the first area ratio refers to the ratio of the area of the first overlapping region to the area of the second body region.

[0066] Taking Figure 3 the human figure region 102 in

[0067] as an example, if the area of the body region of the human figure region 102 is A, and the area of the overlapping region between the body region of the human figure region 102 and the preset region is B, then the first area ratio corresponding to the human figure region 102 is

[0068] Step 204, if the first area ratio is less than or equal to the first threshold, perform distortion correction on the second body region.

[0069] Among them, the first threshold is used to determine whether distortion correction needs to be performed on the second body region. If the first area ratio is less than or equal to the first threshold, it can be determined that the second body region has distortion and distortion correction needs to be performed on the second body region; if the first area ratio is greater than the first threshold, it can be determined that the second body region has no distortion or the distortion generated can be ignored, and there is no need to perform distortion correction on the second body region.

[0069] For example, set the first threshold to 0.6. If Figure 3 the first area ratio corresponding to the human figure region 102 in then it can be determined that the body region in the human figure region 102 has distortion and needs to be corrected for distortion.

[0070] In the embodiment of the present application, by judging whether the body region overlaps with the preset region, and when there is partial overlap with the preset region, based on the area of the body region in the preset region and the ratio of its own area, the distorted body region can be screened out from the to-be-processed image. By performing distortion correction on the distorted body region, distortion correction of the body region in the to-be-processed image can be achieved.

[0071] See Figure 4 , which is a schematic implementation flowchart of an image distortion correction method provided by another embodiment of the present application. This image distortion correction method is applied to an electronic device. As Figure 4 shown, this image distortion correction method may include the following steps:

[0072] Step 401, if there is a first body region in at least one body region of the to-be-processed image, perform distortion correction on the first body region.

[0073] This step is the same as step 201. For specific details, please refer to the relevant description of step 201 and will not be elaborated here.

[0074] Step 402: If there is a second body area in at least one body area of the image to be processed, calculate the area of the second body area and the area of the first overlapping area.

[0075] This step is the same as step 202. For specific details, please refer to the relevant description of step 202 and will not be elaborated here.

[0076] Step 403: Calculate the first area ratio; the first area ratio refers to the ratio of the area of the first overlapping area to the area of the second body area.

[0077] This step is the same as step 203. For specific details, please refer to the relevant description of step 203 and will not be elaborated here.

[0078] Step 404: If the first area ratio is less than or equal to the first threshold, perform distortion correction on the second body area to obtain the corrected body area corresponding to the second body area.

[0079] This step is the same as step 204. For specific details, please refer to the relevant description of step 204 and will not be elaborated here.

[0080] Step 405: Based on the original grid of the image to be processed, determine the corrected grid of the area to be corrected and the original grid of the background area in the image to be processed.

[0081] In this embodiment, when there is no protected area in the image to be processed, step 405 can be executed. Among them, a grid can be established in the image to be processed (for example, using low-resolution grid points to represent the coordinate points of each position in the image to be processed), and this grid is the original grid of the image to be processed. From the original grid of the image to be processed, the original grid of the area to be corrected and the original grid of the background area in the image to be processed can be determined; perform distortion correction on the original grid of the area to be corrected to obtain the corrected grid of the area to be corrected. Among them, conformal projection can be used in this application to correct the area to be corrected, so the corrected grid of the area to be corrected can also be called the conformal projection grid of the area to be corrected.

[0082] Among them, the protected area includes the face area that does not require distortion correction and / or the body area that does not require distortion correction. The absence of a protected area in the image to be processed means that there is neither a face area that does not require distortion correction nor a body area that does not require distortion correction in the image to be processed.

[0083] The first body area in step 401 and the second body area in step 402 are both body areas to be corrected. The area to be corrected in step 405 includes the body area to be corrected; if the image to be processed also includes a face area to be corrected, then the area to be corrected also includes the face area to be corrected.

[0084] In an alternative embodiment, if a third body region exists in the image to be processed, the third body region can be determined as a protected region; if a fourth body region exists in the image to be processed, the fourth body region can also be determined as a protected region. Herein, the fourth body region refers to a body region that partially overlaps with a preset region and the ratio of the area of the overlapping region to the area of the body region itself is greater than a first threshold (i.e., the second body region that does not require distortion correction).

[0085] Since the area of the face region is smaller than that of the body region, when a face region exists in the image to be processed, it can be determined whether the face region needs distortion correction by judging whether the face region is within the preset region; if the face region is within the preset region, it can be determined that the face region does not require distortion correction and the face region is a protected region; if the face region is not within the preset region (for example, the face region does not overlap with the preset region or partially overlaps), it can be determined that the face region needs distortion correction and the face region is a face region to be corrected. By performing distortion correction on the face region, a corrected face region can be obtained.

[0086] Step 406: Based on the corrected grid of the region to be corrected and the optimized grid of the region to be corrected, construct a first optimization term.

[0087] Herein, the first optimization term represents the difference between the corrected grid and the optimized grid of the region to be corrected.

[0088] The number of regions to be corrected in the image to be processed can be one or at least two. When the number of regions to be corrected is at least two, the optimization term corresponding to each region to be corrected can be calculated first, and by adding up the optimization terms corresponding to all regions to be corrected, the first optimization term can be obtained.

[0089] In an alternative embodiment, the optimization term corresponding to each region to be corrected can be expressed as follows:

[0090]

[0091] Herein, E c,k represents the optimization term corresponding to the kth region to be corrected; i represents the index of the grid point; ω i represents the weight of the grid point with index i; v i represents the coordinate of the grid point with index i in the optimized grid of the kth region to be corrected; u i represents the coordinate of the grid point with index i in the corrected grid of the kth region to be corrected; t k represents the translation term of the kth region to be corrected, which is used to perform an overall translation transformation on the corrected grid of the kth region to be corrected; S kThe similarity transformation term for the k-th area to be corrected is used to perform an overall similarity transformation on the corrected grid of the k-th area to be corrected.

[0092] The first optimization term can be expressed as follows:

[0093]

[0094] Step 407: Based on the original grid of the background area and the optimized grid of the background area, construct a second optimization term.

[0095] Among them, the second optimization term represents the difference between the original grid of the background area and the optimized grid of the background area.

[0096] The second optimization term may include a straight-line angle preservation constraint condition and a grid interval constraint condition. The straight-line angle preservation constraint condition is used to constrain the position offset of the original grid and the optimized grid in the background area, so that the background area in the finally obtained first optimized image is similar to the background area in the image to be processed. The grid interval constraint condition is used to constrain the grid size offset of the original grid and the optimized grid in the background area to prevent grid mutations in the background area.

[0097] The expression of the straight-line angle preservation constraint condition is as follows:

[0098]

[0099] Among them, g represents the index of the grid point; j ∈ N(g) represents the surrounding grid points of the grid point with index g; e gj is the unit vector in the original grid of the background area, and e gj = p g - p j , p g and p j respectively represent the coordinates of the grid points with indices g and j in the original grid of the background area; v g and v j respectively represent the coordinates of the grid points with indices g and j in the optimized grid of the background area.

[0100] The expression of the grid interval constraint condition is as follows:

[0101]

[0102] Step 408: Based on the first optimization term and the second optimization term, construct a first optimization function.

[0103] Among them, the first optimization function is a function that aims to minimize the difference between the corrected grid of the area to be corrected and the optimized grid of the area to be corrected, as well as the difference between the original grid of the background area and the optimized grid of the background area.

[0104] The first optimization term and the second optimization term are weighted and summed to obtain the first optimization function.

[0105] The first optimization function can be expressed as follows:

[0106] E t1 = ω c E c + ω b E b + ω r E r

[0107] Where ω c represents the weight of the first optimization term; ω b represents the weight of the straight-line angle maintenance constraint condition; E b represents the straight-line angle maintenance constraint condition; ω r represents the weight of the grid interval constraint condition; E r represents the grid interval constraint condition.

[0108] Step 409: By adjusting the optimized grid of the area to be corrected and the optimized grid of the background area, the first optimization function is minimized to obtain the first optimized image.

[0109] By solving the first optimization function, the first optimized grid of the image to be processed can be obtained. Interpolation is performed on the first optimized grid to obtain the first optimized image. Among them, the first optimized grid includes the optimized grid of the area to be corrected and the optimized grid of the background area that minimize the first optimization function.

[0110] In the embodiment of the present application, by constructing the first optimization function based on the area to be corrected and the background area in the image to be processed, the corrected area can be optimized through the first optimization function, and the shape of the background area in the first optimized image is made consistent with its shape in the image to be processed, so that the first optimized image is more natural and realistic.

[0111] See Figure 5 , which is a schematic flowchart of the implementation of an image distortion correction method provided by another embodiment of the present application. This image distortion correction method is applied to an electronic device. As Figure 5 shown, this image distortion correction method may include the following steps:

[0112] Step 501: If there is a first body area in at least one body area of the image to be processed, perform distortion correction on the first body area.

[0113] This step is the same as step 201. For specific details, please refer to the relevant description of step 201 and will not be elaborated here.

[0114] Step 502, if there is a second body area in at least one body area of the image to be processed, calculate the area of the second body area and the area of the first overlapping area.

[0115] This step is the same as step 202. For specific details, please refer to the relevant description of step 202 and will not be elaborated here.

[0116] Step 503, calculate the first area ratio; the first area ratio is the ratio of the area of the first overlapping area to the area of the second body area.

[0117] This step is the same as step 203. For specific details, please refer to the relevant description of step 203 and will not be elaborated here.

[0118] Step 504, if the first area ratio is less than or equal to the first threshold, perform distortion correction on the second body area.

[0119] This step is the same as step 204. For specific details, please refer to the relevant description of step 204 and will not be elaborated here.

[0120] Step 505, based on the original grid of the image to be processed, determine the corrected grid of the area to be corrected in the image to be processed and the original grid of the background area in the image to be processed.

[0121] This step is the same as step 405. For specific details, please refer to the relevant description of step 405 and will not be elaborated here.

[0122] Step 506, based on the corrected grid of the area to be corrected and the optimized grid of the area to be corrected, construct the first optimization term.

[0123] This step is the same as step 406. For specific details, please refer to the relevant description of step 406 and will not be elaborated here.

[0124] Step 507, based on the original grid of the background area and the optimized grid of the background area, construct the second optimization term.

[0125] This step is the same as step 407. For specific details, please refer to the relevant description of step 407 and will not be elaborated here.

[0126] Step 508, based on the original grid of the protection area and the optimized grid of the protection area, construct the third optimization term.

[0127] Among them, the third optimization term represents the difference between the original grid of the protection area and the optimized grid of the protection area. The third optimization term can reduce the stretching deformation of the shape of the protection area, so that the shape of the protection area in the second optimized image is consistent with the shape in the image to be processed.

[0128] The number of protected regions in the image to be processed can be one, or at least two. When the number of protected regions is at least two, the optimization terms corresponding to each protected region can be calculated first, and all the optimization terms corresponding to the protected regions can be added up to obtain the third optimization term.

[0129] In an alternative embodiment, the optimization term corresponding to each protected region can be expressed as follows:

[0130]

[0131] where E p,h represents the optimization term corresponding to the h-th protected region; d represents the index of the grid point; ω d represents the weight of the grid point with index d; v d represents the coordinate of the grid point with index d in the optimized grid of the h-th protected region; p d represents the coordinate of the grid point with index d in the original grid of the h-th protected region; t h represents the translation term of the h-th protected region, which is used to perform an overall translation transformation on the original grid of the h-th protected region.

[0132] The third optimization term can be expressed as follows:

[0133]

[0134] Step 509, construct a second optimization function based on the first optimization term, the second optimization term, and the third optimization term.

[0135] Among them, the second optimization function is a function that minimizes the differences between the corrected grid of the region to be corrected and the optimized grid of the region to be corrected, the original grid of the background region and the optimized grid of the background region, and the original grid of the protected region and the optimized grid of the protected region.

[0136] By performing a weighted sum on the first optimization term, the second optimization term, and the third optimization term, the second optimization function can be obtained.

[0137] The second optimization function can be expressed as follows:

[0138] E t2 = ω P E p + ω c E c + ω b E b + ω r E r

[0139] where ω P represents the weight of the third optimization term.

[0140] Step 510: By adjusting the optimized meshes of the region to be corrected, the background region, and the protection region, minimize the second optimization function to obtain a second optimized image.

[0141] By solving the second optimization function, the second optimized mesh of the image to be processed can be obtained. By interpolating the second optimized mesh, a second optimized image can be obtained. Among them, the second optimized mesh includes the optimized meshes of the region to be corrected, the background region, and the protection region that minimize the second optimization function.

[0142] In an optional embodiment, when the number of protection regions is at least two and there are overlapping protection regions among the at least two protection regions, it further includes:

[0143] Calculate the area of the first protection region, the area of the second protection region, and the area of the second overlapping region; the first protection region and the second protection region refer to any two overlapping protection regions among the at least two protection regions; the second overlapping region refers to the overlapping region between the first protection region and the second protection region;

[0144] Based on the area of the first protection region and the area of the second protection region, determine the minimum value of the areas of the first protection region and the second protection region;

[0145] Calculate a second area ratio, where the second area ratio is the ratio of the area of the second overlapping region to the minimum value of the areas;

[0146] If the second area ratio is greater than the second threshold, merge the first protection region and the second protection region into one protection region.

[0147] By merging two protection regions with a larger overlapping region into one protection region, the electronic device can reduce the computation amount in the process of solving the second optimization function, accelerate the solution of the second optimization function, and improve the optimization speed of the image to be processed.

[0148] In the embodiments of the present application, by constructing a second optimization function based on the region to be corrected, the protection region, and the background region in the image to be processed, the corrected region can be optimized through the second optimization function. At the same time, the shape of the background region in the second optimized image is kept consistent with its shape in the image to be processed, and the shape of the protection region in the second optimized image is kept consistent with its shape in the image to be processed, so that the second optimized image is more natural and realistic.

[0149] For Figure 4 and Figure 5An embodiment of the image distortion correction method shown. In these two embodiments, the first optimization term may include the optimization term corresponding to the corrected body region; the first optimization term includes the optimization term corresponding to the body region to be corrected; before constructing the first optimization term based on the corrected grid of the region to be corrected and the optimized grid of the region to be corrected, it further includes:

[0150] Obtain the target correction degree of the body region to be corrected;

[0151] Based on the target correction degree, determine the weights of each grid point in the corrected grid of the body region to be corrected and the optimized grid of the body region to be corrected;

[0152] Construct the first optimization term based on the corrected grid of the region to be corrected and the optimized grid of the region to be corrected, including:

[0153] Construct the optimization term corresponding to the body region to be corrected based on the corrected grid of the body region to be corrected, the optimized grid of the body region to be corrected, and the weights of each grid point in the corrected grid of the body region to be corrected and the optimized grid of the body region to be corrected, so that the distortion correction degree of the body region to be corrected reaches the target correction degree through this optimization term.

[0154] By adjusting the weights of each grid point in the corrected grid of the body region to be corrected and the optimized grid of the body region to be corrected, different degrees of distortion correction can be performed on the body region to be corrected, thereby adaptively adjusting the distortion correction degree of the body region to be corrected.

[0155] In an optional embodiment, the body region to be corrected is framed by a body frame; obtaining the target distortion correction degree of the body region to be corrected includes:

[0156] Judge whether the horizontal coordinate of the left vertex of the body frame is less than the third threshold, and / or whether the horizontal coordinate of the right vertex of the body frame is greater than the fourth threshold, where the third threshold is greater than zero and less than the fourth threshold;

[0157] If the horizontal coordinate of the left vertex of the body frame is less than the third threshold, and / or the horizontal coordinate of the right vertex of the body frame is greater than the fourth threshold, then determine the target correction degree as the first target correction degree, otherwise the target correction degree is the second target correction degree, and the first target correction degree is greater than the second target correction degree.

[0158] Wherein, the above body frame may be the minimum circumscribed rectangle of the body region to be corrected.

[0159] In the image to be processed, distortion usually occurs symmetrically from the center point of the image to be processed. The closer the area is to the center point, the smaller the degree of distortion; the farther the area is from the center point, the greater the degree of distortion. Therefore, on the left side of the image to be processed, the horizontal coordinate of the left vertex of the body frame can be compared with the third threshold, and on the right side of the image to be processed, the horizontal coordinate of the right vertex of the body frame can be compared with the fourth threshold, so as to judge the degree of distortion of the body area to be corrected, and then obtain the target correction degree corresponding to the degree of distortion of the body area to be corrected. As Figure 6 shown in the example diagram of the body frame, Figure 6 the rectangular frame represented by the dotted line in it is the body frame.

[0160] By obtaining the target correction degree of the body area to be corrected, the target correction degree of the body area to be corrected can be adaptively adjusted based on the degree of distortion of the body area to be corrected in the image to be processed.

[0161] In an alternative embodiment, before judging whether the horizontal coordinate of the left vertex of the body frame is less than the third threshold, and / or whether the horizontal coordinate of the right vertex of the body frame is greater than the fourth threshold, it further includes:

[0162] Based on the maximum horizontal coordinate of the image to be processed, determine the third threshold and the fourth threshold, where the fourth threshold is greater than zero and less than the maximum horizontal coordinate.

[0163] The electronic device sets the third threshold and the fourth threshold based on the maximum horizontal coordinate of the image to be processed (for example, the third threshold is and the fourth threshold is ), and different thresholds can be set for images to be processed with different maximum horizontal coordinates, so as to realize the adaptive setting of the third threshold and the fourth threshold. Among them, the above maximum horizontal coordinate refers to the width of the image to be processed.

[0164] See Figure 7 , which is a schematic structural diagram of an image distortion correction device provided by an embodiment of the present application. For the sake of convenience of description, only the parts related to the embodiments of the present application are shown.

[0165] The above image distortion correction device includes:

[0166] The first correction module 71 is used to correct the distortion of the first body area if there is a first body area in at least one body area of the image to be processed; the body area refers to the area in the portrait area of the image to be processed that does not contain face data; the first body area refers to the body area that does not overlap with the preset area; the preset area refers to the local area centered on the center point of the image to be processed;

[0167] An area calculation module 72, configured to calculate the area of the second body area and the area of the first overlapping area if there is a second body area in at least one body area of the to-be-processed image; the second body area refers to a body area that overlaps with a preset area; the first overlapping area refers to the overlapping area between the second body area and the preset area;

[0168] A ratio calculation module 73, configured to calculate a first area ratio; the first area ratio refers to the ratio of the area of the first overlapping area to the area of the second body area;

[0169] A second correction module 74, configured to perform distortion correction on the second body area if the first area ratio is less than or equal to a first threshold.

[0170] Optionally, the first body area and the second body area subjected to distortion correction are areas to be corrected, and the above image distortion correction device further includes:

[0171] A first determination module, configured to, in the case that there is no protected area in the to-be-processed image, determine the corrected grid of the area to be corrected in the to-be-processed image and the original grid of the background area in the to-be-processed image based on the original grid of the to-be-processed image; the protected area includes a face area that does not require distortion correction and / or a body area that does not require distortion correction; the area to be corrected includes the area to be corrected body area; when there is an area to be corrected face area in the to-be-processed image, the area to be corrected further includes the area to be corrected face area;

[0172] A first construction module, configured to construct a first optimization term based on the corrected grid of the area to be corrected and the optimized grid of the area to be corrected; the first optimization term represents the difference between the corrected grid of the area to be corrected and the optimized grid of the area to be corrected;

[0173] A second construction module, configured to construct a second optimization term based on the original grid of the background area and the optimized grid of the background area; the second optimization term represents the difference between the original grid of the background area and the optimized grid of the background area;

[0174] A first optimization module, configured to construct a first optimization function based on the first optimization term and the second optimization term; the first optimization function is a function with the goal of minimizing the difference between the corrected grid of the area to be corrected and the optimized grid of the area to be corrected and the difference between the original grid of the background area and the optimized grid of the background area;

[0175] A first adjustment module, configured to minimize the first optimization function by adjusting the optimized grid of the area to be corrected and the optimized grid of the background area, and obtain a first optimized image.

[0176] Optionally, the first body area and the second body area subjected to distortion correction are areas to be corrected, and the above image distortion correction device further includes:

[0177] A second determination module, configured to, when there is a protected area in the image to be processed, determine a corrected grid of the area to be corrected, an original grid of the background area, and an original grid of the protected area in the image to be processed based on the original grid of the image to be processed; the protected area includes a face area that does not require distortion correction and / or a body area that does not require distortion correction; the area to be corrected includes a body area to be corrected; when there is a face area to be corrected in the image to be processed, the area to be corrected further includes the face area to be corrected;

[0178] A first construction module, configured to construct a first optimization term based on the corrected grid of the area to be corrected and the optimized grid of the area to be corrected; the first optimization term represents the difference between the corrected grid and the optimized grid of the area to be corrected;

[0179] A second construction module, configured to construct a second optimization term based on the original grid of the background area and the optimized grid of the background area; the second optimization term represents the difference between the original grid and the optimized grid of the background area;

[0180] A third construction module, configured to construct a third optimization term based on the original grid of the protected area and the optimized grid of the protected area; the third optimization term represents the difference between the original grid and the optimized grid of the protected area;

[0181] A second optimization module, configured to construct a second optimization function based on the first optimization term, the second optimization term, and the third optimization term; the second optimization function is a function that aims to minimize the differences between the corrected grid and the optimized grid of the area to be corrected, the original grid and the optimized grid of the background area, and the original grid and the optimized grid of the protected area;

[0182] A second adjustment module, configured to minimize the second optimization function by adjusting the optimized grids of the area to be corrected, the background area, and the protected area, so as to obtain a second optimized image.

[0183] Optionally, when the number of protected areas is at least two and there are overlapping protected areas among the at least two protected areas, the above image distortion correction device further includes:

[0184] A first calculation module, configured to calculate the area of a first protected area, the area of a second protected area, and the area of a second overlapping area; the first protected area and the second protected area refer to any two overlapping protected areas among the at least two protected areas; the second overlapping area refers to the overlapping area between the first protected area and the second protected area;

[0185] A minimum value determination module, configured to determine the minimum value of the areas of the first protection area and the second protection area based on the area of the first protection area and the area of the second protection area;

[0186] A second calculation module, configured to calculate a second area ratio, where the second area ratio is the ratio of the area of the second overlapping area to the minimum value of the areas;

[0187] An area synthesis module, configured to merge the first protection area and the second protection area into one protection area if the second area ratio is greater than a second threshold.

[0188] Optionally, the first optimization item includes an optimization item corresponding to the body area to be corrected; the above image distortion correction device further includes:

[0189] A target acquisition module, configured to acquire the target correction degree of the body area to be corrected;

[0190] A weight determination module, configured to determine the weight of each grid point in the corrected grid of the body area to be corrected and the optimized grid of the body area to be corrected based on the target correction degree;

[0191] The above first construction module is specifically configured to:

[0192] Based on the corrected grid of the body area to be corrected, the optimized grid of the body area to be corrected, and the weight of each grid point in the corrected grid of the body area to be corrected and the optimized grid of the body area to be corrected, construct an optimization item corresponding to the body area to be corrected, so that the distortion correction degree of the body area to be corrected reaches the target correction degree through this optimization item.

[0193] Optionally, the body area to be corrected is framed by a body frame; the above target acquisition module includes:

[0194] A threshold judgment unit, configured to judge whether the horizontal coordinate of the left vertex of the body frame is less than a third threshold, and / or whether the horizontal coordinate of the right vertex of the body frame is greater than a fourth threshold, where the third threshold is greater than zero and less than the fourth threshold;

[0195] A degree determination unit, configured to determine the target correction degree as the first target correction degree if the horizontal coordinate of the left vertex of the body frame is less than the third threshold, and / or the horizontal coordinate of the right vertex of the body frame is greater than the fourth threshold, otherwise determine the target correction degree as the second target correction degree, where the first target correction degree is greater than the second target correction degree.

[0196] Optionally, the above target acquisition module further includes:

[0197] A threshold calculation unit, configured to determine the third threshold and the fourth threshold based on the maximum horizontal coordinate of the image to be processed, where the fourth threshold is greater than zero and less than the maximum horizontal coordinate.

[0198] The image distortion correction device provided by the embodiment of the present application can be applied in the foregoing method embodiment. For details, refer to the description of the foregoing method embodiment, which will not be repeated here.

[0199] Figure 8 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 8 shown, the electronic device 8 of this embodiment includes: one or more processors 80 (only one is shown in the figure), a memory 81, and a computer program 82 stored in the memory 81 and executable on the at least one processor 80. When the processor 80 executes the computer program 82, the steps in the foregoing method embodiments of various image distortion correction methods are implemented.

[0200] The electronic device may include, but is not limited to, the processor 80 and the memory 81. Those skilled in the art can understand that Figure 8 it is only an example of the electronic device 8, and does not constitute a limitation on the electronic device 8. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0201] The so-called processor 80 may be a central processing unit (CPU), and this processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.

[0202] The memory 81 may be an internal storage unit of the electronic device 8, such as a hard disk or memory of the electronic device 8. The memory 81 may also be an external storage device of the electronic device 8, such as a plug-in hard disk equipped on the electronic device 8, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 81 may also include both the internal storage unit of the electronic device 8 and the external storage device. The memory 81 is used to store the computer program and other programs and data required by the electronic device. The memory 81 may also be used to temporarily store the data that has been output or will be output.

[0203] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working process of the units and modules in the above device can refer to the corresponding process in the foregoing method embodiments and will not be elaborated herein.

[0204] This application embodiment also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented.

[0205] This application embodiment also provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the foregoing method embodiments when executed.

[0206] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0207] Those of ordinary skill in the art will realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0208] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical or other forms.

[0209] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0210] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of this application, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0211] The above-described embodiments are only used to illustrate the technical solutions of this application, rather than to limit them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of this application, and should all be included in the protection scope of this application.

Claims

1. An image distortion correction method, characterized in that, Including: If there is a first body area in at least one body area of the image to be processed, perform distortion correction on the first body area; the body area refers to the area in the portrait area of the image to be processed that does not contain face data; if there is a portrait area that does not contain face data in the image to be processed, determine this portrait area as the body area; the first body area refers to the body area that does not overlap with the preset area; the preset area refers to the local area centered on the center point of the image to be processed in the image to be processed; If there is a second body area in at least one of the body areas of the image to be processed, calculate the area of the second body area and the area of the first overlapping area; the second body area refers to the body area that partially overlaps with the preset area; the first overlapping area refers to the overlapping area between the second body area and the preset area; Calculate the first area ratio; the first area ratio refers to the ratio of the area of the first overlapping area to the area of the second body area; If the first area ratio is less than or equal to the first threshold, perform distortion correction on the second body area.

2. The image distortion correction method according to claim 1, characterized in that, The first body area and the second body area that undergoes distortion correction are the body areas to be corrected, and it further includes: In the case where there is no protected area in the image to be processed, based on the original grid of the image to be processed, determine the corrected grid of the area to be corrected in the image to be processed and the original grid of the background area in the image to be processed; the protected area includes the face area that does not require distortion correction and / or the body area that does not require distortion correction; the area to be corrected includes the body areas to be corrected; when there is an area to be corrected for the face in the image to be processed, the area to be corrected further includes the area to be corrected for the face; Based on the corrected grid of the area to be corrected and the optimized grid of the area to be corrected, construct a first optimization term; the first optimization term represents the difference between the corrected grid of the area to be corrected and the optimized grid of the area to be corrected; Based on the original grid of the background area and the optimized grid of the background area, construct a second optimization term; the second optimization term represents the difference between the original grid of the background area and the optimized grid of the background area; Based on the first optimization term and the second optimization term, construct a first optimization function; the first optimization function is a function with the goal of minimizing the difference between the corrected grid of the area to be corrected and the optimized grid of the area to be corrected and the difference between the original grid of the background area and the optimized grid of the background area; By adjusting the optimized grid of the area to be corrected and the optimized grid of the background area, minimize the first optimization function to obtain a first optimized image.

3. The image distortion correction method according to claim 1, wherein The first body area and the second body area that undergoes distortion correction are the body areas to be corrected, and it further includes: When there is a protected area in the image to be processed, based on the original grid of the image to be processed, determine the corrected grid of the area to be corrected, the original grid of the background area, and the original grid of the protected area in the image to be processed; the protected area includes a face area that does not require distortion correction and / or a body area that does not require distortion correction; the area to be corrected includes the body area to be corrected; when there is a face area to be corrected in the image to be processed, the area to be corrected further includes the face area to be corrected; Based on the corrected grid of the area to be corrected and the optimized grid of the area to be corrected, construct a first optimization term; the first optimization term represents the difference between the corrected grid and the optimized grid of the area to be corrected; Based on the original grid of the background area and the optimized grid of the background area, construct a second optimization term; the second optimization term represents the difference between the original grid and the optimized grid of the background area; Based on the original grid of the protected area and the optimized grid of the protected area, construct a third optimization term; the third optimization term represents the difference between the original grid and the optimized grid of the protected area; Based on the first optimization term, the second optimization term, and the third optimization term, construct a second optimization function; the second optimization function is a function that aims to minimize the differences between the corrected grid and the optimized grid of the area to be corrected, the original grid and the optimized grid of the background area, and the original grid and the optimized grid of the protected area; By adjusting the optimized grids of the area to be corrected, the background area, and the protected area, minimize the second optimization function to obtain a second optimized image.

4. The image distortion correction method according to claim 3, wherein When the number of protected areas is at least two and there are overlapping protected areas among the at least two protected areas, it further includes: Calculate the area of the first protected area, the area of the second protected area, and the area of the second overlapping area; the first protected area and the second protected area refer to any two overlapping protected areas among the at least two protected areas; the second overlapping area refers to the overlapping area between the first protected area and the second protected area; Based on the area of the first protected area and the area of the second protected area, determine the minimum value of the areas of the first protected area and the second protected area; Calculate a second area ratio, where the second area ratio is the ratio of the area of the second overlapping area to the minimum value of the areas; If the second area ratio is greater than a second threshold, merge the first protected area and the second protected area into one protected area.

5. The image distortion correction method according to claim 2 or 3, characterized in that, The first optimization term includes the optimization term corresponding to the body area to be corrected; before constructing the first optimization term based on the corrected grid of the area to be corrected and the optimized grid of the area to be corrected, it further includes: Obtain the target correction degree of the body area to be corrected; Based on the target correction degree, determine the weights of each grid point in the corrected grid of the body area to be corrected and the optimized grid of the body area to be corrected; Based on the corrected grid of the area to be corrected and the optimized grid of the area to be corrected, construct a first optimization term, including: Based on the corrected grid of the body area to be corrected, the optimized grid of the body area to be corrected, and the weights of each grid point in the corrected grid of the body area to be corrected and the optimized grid of the body area to be corrected, construct the optimization term corresponding to the body area to be corrected, so that the distortion correction degree of the body area to be corrected reaches the target correction degree through this optimization term.

6. The image distortion correction method according to claim 5, wherein The body area to be corrected is framed by a body frame; obtaining the target distortion correction degree of the body area to be corrected includes: Judge whether the horizontal coordinate of the left vertex of the body frame is less than a third threshold, and / or whether the horizontal coordinate of the right vertex of the body frame is greater than a fourth threshold, where the third threshold is greater than zero and less than the fourth threshold; If the horizontal coordinate of the left vertex of the body frame is less than the third threshold, and / or the horizontal coordinate of the right vertex of the body frame is greater than the fourth threshold, then determine that the target correction degree is the first target correction degree, otherwise the target correction degree is the second target correction degree, and the first target correction degree is greater than the second target correction degree.

7. The image distortion correction method according to claim 6, wherein Before judging whether the horizontal coordinate of the left vertex of the body frame is less than the third threshold, and / or whether the horizontal coordinate of the right vertex of the body frame is greater than the fourth threshold, it also includes: Based on the maximum horizontal coordinate of the image to be processed, determine the third threshold and the fourth threshold, where the fourth threshold is greater than zero and less than the maximum horizontal coordinate.

8. The image distortion correction method according to any one of claims 1 to 3, characterized in that It also includes: If there is a portrait area containing face data in the image to be processed, determine the body area in the portrait area based on the position information in the face data.

9. An image distortion correction device, characterized in that, Includes: A first correction module, which is used to perform distortion correction on the first body area if there is a first body area in at least one body area of the image to be processed; the body area refers to the area in the portrait area of the image to be processed that does not contain face data; if there is a portrait area that does not contain face data in the image to be processed, then determine that portrait area as the body area; the first body area refers to the body area that does not overlap with the preset area; the preset area refers to the local area centered on the center point of the image to be processed in the image to be processed; An area calculation module, which is used to calculate the area of the second body area and the area of the first overlapping area if there is a second body area in at least one of the body areas of the image to be processed; the second body area refers to the body area that partially overlaps with the preset area; the first overlapping area refers to the overlapping area between the second body area and the preset area; A ratio calculation module for calculating a first area ratio; the first area ratio refers to the ratio of the area of the first overlapping region to the area of the second body region; A second correction module for performing distortion correction on the second body region if the first area ratio is less than or equal to a first threshold.

10. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the image distortion correction method according to any one of claims 1 to 8.

11. A chip, comprising a processor, characterized in that, The processor is used to read and execute the computer program stored in the memory to execute the steps of the image distortion correction method according to any one of claims 1 to 8.

12. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the image distortion correction method according to any one of claims 1 to 8.

Images