Distorted Image Correction Method, Device, Computer-Readable Medium, and Electronic Device

Abstract

The present disclosure provides a distortion image correction method and apparatus, a computer-readable medium, and an electronic device, relating to the technical field of image processing. The method includes: obtaining a distorted image to be corrected; dividing the distorted image into regions, and determining a first image region and a second image region in the distorted image; wherein, the first image region includes the image content that needs to be corrected; performing a protection process on the second image region and a correction process on the first image region, so as to avoid deforming the second image region when correcting the first image region, and completing the correction of the distorted image. The present disclosure can avoid abnormally stretching the adjacent image regions of the corrected part when correcting a distorted image, especially a distorted image containing a human face, improve the correction accuracy of the distorted image, and avoid abnormal stretching of the corrected image.

Description

Background Art

[0002] With the continuous improvement of people's living standards, the related technologies of taking pictures and videos have attracted more and more attention. However, in the images obtained by shooting at present, the distortion of the human face increases as the field of view angle of the lens becomes larger. There is no distortion at the center of the lens, and the distortion at the edge is obvious. Especially for ultra-wide-angle lenses, the deformation of the human face at the edge is significant. Therefore, it is necessary to correct the distorted part in the image.

[0003] At present, in the related technologies, when correcting distorted images, especially images containing portrait content, either the image area adjacent to the human face image part will be abnormally stretched when correcting the human face image part, or the distorted portrait content will be missed in the correction, resulting in a low accuracy of image content correction and a poor correction effect. Summary of the Invention

[0004] The purpose of the present disclosure is to provide a method for correcting distorted images, a device for correcting distorted images, a computer-readable medium, and an electronic device, so as to at least to a certain extent avoid the problem that when correcting the part to be corrected of a distorted image in the related technologies, the adjacent image area will be abnormally stretched, resulting in a low accuracy of image content correction and a poor correction effect.

[0005] According to the first aspect of the present disclosure, a method for correcting a distorted image is provided, including:

[0006] Obtaining a distorted image to be corrected;

[0007] Dividing the distorted image into regions, and determining a first image region and a second image region in the distorted image; wherein, the first image region includes the image content that needs to be corrected;

[0008] Performing a protection process on the second image region and a correction process on the first image region to avoid deforming the second image region when correcting the first image region, and completing the correction of the distorted image.

[0009] According to the second aspect of the present disclosure, a device for correcting a distorted image is provided, including:

[0010] A distorted image acquisition module, configured to acquire a distorted image to be corrected;

[0011] An image region division module, configured to divide the distorted image into regions, and determine a first image region and a second image region in the distorted image; wherein, the first image region includes the image content that needs to be corrected;

[0012] The distortion image correction module is used to perform protection processing on the second image area and correction processing on the first image area, so as to avoid distorting the second image area when correcting the first image area, and complete the correction of the distortion image.

[0013] According to the third aspect of the present disclosure, there is provided a computer-readable medium having a computer program stored thereon, and when the computer program is executed by a processor, the above method is implemented.

[0014] According to the fourth aspect of the present disclosure, there is provided an electronic device, characterized by including:

[0015] a processor; and

[0016] a memory for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the above method.

[0017] In the distortion image correction method provided by an embodiment of the present disclosure, the acquired distortion image is divided into regions, the first image area that needs to be corrected and the second image area that does not need to be corrected in the distortion image are determined. When correcting, after performing protection processing on the second image area, correction processing is performed on the first image area to avoid distorting the second image area when correcting the first image area, and finally the correction of the distortion image is completed. On the one hand, the acquired distortion image is divided into regions, correctly distinguishing the first image area that needs to be corrected and the second image area that does not need to be corrected, and all the first image areas that need to be corrected can be corrected, avoiding missed correction and improving the correction accuracy of the distortion image; on the other hand, protection processing is performed on the second image area that does not need to be corrected, and then correction is performed on the first image area that needs to be corrected, which can effectively avoid distorting the second image area when correcting the first image area, effectively improving the correction effect, and avoiding abnormal stretching deformation that does not conform to human visual perception in the corrected image.

[0018] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings here are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts. In the drawings:

[0020] Figure 1The figure shows a schematic diagram of an exemplary system architecture to which embodiments of the present disclosure can be applied;

[0021] Figure 2 The figure shows a schematic diagram of an electronic device to which embodiments of the present disclosure can be applied;

[0022] Figure 3 The figure schematically shows a flowchart of a distortion image correction method in an exemplary embodiment of the present disclosure;

[0023] Figure 4 The figure schematically shows a flowchart of dividing a region of a distorted image in an exemplary embodiment of the present disclosure;

[0024] Figure 5 The figure schematically shows a flowchart of determining a target face region in a target portrait region in an exemplary embodiment of the present disclosure;

[0025] Figure 6 The figure schematically shows a flowchart of another method of dividing a region of a distorted image in an exemplary embodiment of the present disclosure;

[0026] Figure 7 The figure schematically shows a flowchart of determining a first image region in an exemplary embodiment of the present disclosure;

[0027] Figure 8 The figure schematically shows a composition diagram of a distortion image correction device in an exemplary embodiment of the present disclosure. Detailed implementation manners

[0028] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art. The features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments.

[0029] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0030] Figure 1 The figure shows a schematic diagram of a system architecture of an exemplary application environment of a distortion image correction method and device to which embodiments of the present disclosure can be applied.

[0031] As shown Figure 1 in FIG. 1, the system architecture 100 may include one or more of the terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc. The terminal devices 101, 102, 103 may be various electronic devices with image processing functions, including but not limited to desktop computers, portable computers, smartphones, and tablet computers, etc. It should be understood that Figure 1 the numbers of terminal devices, networks, and servers in FIG. 1 are merely illustrative. According to the implementation requirements, there may be any number of terminal devices, networks, and servers. For example, the server 105 may be a server cluster composed of multiple servers, etc.

[0032] The distortion image correction method provided by the embodiments of the present disclosure is generally executed by the terminal devices 101, 102, 103. Correspondingly, the distortion image correction device is generally disposed in the terminal devices 101, 102, 103. However, those skilled in the art can easily understand that the distortion image correction method provided by the embodiments of the present disclosure can also be executed by the server 105. Correspondingly, the distortion image correction device can also be disposed in the server 105. No special limitation is made in this exemplary embodiment. For example, in an exemplary embodiment, it may be that the user uploads the collected distorted image to be corrected to the server 105 through the terminal devices 101, 102, 103. After the server completes the correction of the distorted image through the distortion image correction method provided by the embodiments of the present disclosure, the corrected image is transmitted to the terminal devices 101, 102, 103, etc.

[0033] An exemplary embodiment of the present disclosure provides an electronic device for implementing the distortion image correction method, which may be Figure 1 the terminal devices 101, 102, 103 or the server 105 in FIG. 1. The electronic device includes at least a processor and a memory. The memory is used to store executable instructions of the processor, and the processor is configured to execute the distortion image correction method by executing the executable instructions.

[0034] Next, taking Figure 2 the mobile terminal 200 in FIG. 1 as an example, the structure of the electronic device will be described exemplarily. Those skilled in the art should understand that, except for the components specifically for mobile purposes, Figure 2The structure in [the above description] can also be applied to fixed-type devices. In some other embodiments, the mobile terminal 200 may include more or fewer components than those illustrated, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. The interface connection relationships between the components are only schematically shown and do not constitute a structural limitation on the mobile terminal 200. In some other embodiments, the mobile terminal 200 may also adopt an interface connection method different from that of Figure 2 or a combination of multiple interface connection methods.

[0035] As Figure 2 shown, the mobile terminal 200 may specifically include: a processor 210, an internal memory 221, an external memory interface 222, a Universal Serial Bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 271, a receiver 272, a microphone 273, a headphone interface 274, a sensor module 280, a display screen 290, a camera module 291, an indicator 292, a motor 293, keys 294, and a subscriber identification module (SIM) card interface 295, etc. Among them, the sensor module 280 may include a depth sensor 2801, a pressure sensor 2802, a gyroscope sensor 2803, etc.

[0036] The processor 210 may include one or more processing units. For example, the processor 210 may include an Application Processor (AP), a modem processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a video codec, a Digital Signal Processor (DSP), a baseband processor, and / or a Neural-Network Processing Unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0037] The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission pattern between human brain neurons, it can quickly process input information and can also continuously learn on its own. Through the NPU, applications such as intelligent cognition of the mobile terminal 200 can be realized, such as image recognition, face recognition, speech recognition, text understanding, etc.

[0038] A memory is provided in the processor 210. The memory can store instructions for implementing six modular functions: detection instructions, connection instructions, information management instructions, analysis instructions, data transmission instructions, and notification instructions, and is controlled by the processor 210 for execution.

[0039] The charging management module 240 is used to receive charging input from a charger. The power management module 241 is used to connect the battery 242, the charging management module 240, and the processor 210. The power management module 241 receives inputs from the battery 242 and / or the charging management module 240 and supplies power to the processor 210, the internal memory 221, the display screen 290, the camera module 291, the wireless communication module 260, etc.

[0040] The wireless communication function of the mobile terminal 200 can be implemented through antenna 1, antenna 2, the mobile communication module 250, the wireless communication module 260, the modem processor, and the baseband processor, etc. Among them, antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals; the mobile communication module 250 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied to the mobile terminal 200; the modem processor can include a modulator and a demodulator; the wireless communication module 260 can provide solutions for wireless communications including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), etc. applied to the mobile terminal 200. In some embodiments, antenna 1 of the mobile terminal 200 is coupled to the mobile communication module 250, and antenna 2 is coupled to the wireless communication module 260, so that the mobile terminal 200 can communicate with the network and other devices through wireless communication technologies.

[0041] The mobile terminal 200 realizes the display function through the GPU, the display screen 290, and the application processor, etc. The GPU is a microprocessor for image processing, and is connected to the display screen 290 and the application processor. The GPU is used to execute mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or change display information.

[0042] The mobile terminal 200 can implement the shooting function through the ISP, the camera module 291, the video codec, the GPU, the display screen 290, the application processor, etc. Among them, the ISP is used to process the data fed back by the camera module 291; the camera module 291 is used to capture static images or videos; the digital signal processor is used to process digital signals, and in addition to processing digital image signals, it can also process other digital signals; the video codec is used to compress or decompress digital videos, and the mobile terminal 200 can also support one or more video codecs.

[0043] The external memory interface 222 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile terminal 200. The external memory card communicates with the processor 210 through the external memory interface 222 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0044] The internal memory 221 can be used to store computer-executable program codes, and the executable program codes include instructions. The internal memory 221 can include a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, the image playback function, etc.). The data storage area can store the data created during the use of the mobile terminal 200 (such as audio data, phone book, etc.). In addition, the internal memory 221 can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, a flash memory device, a Universal Flash Storage (UFS), etc. The processor 210 executes various functional applications and data processing of the mobile terminal 200 by running the instructions stored in the internal memory 221 and / or the instructions stored in the memory provided in the processor.

[0045] The mobile terminal 200 can implement the audio function through the audio module 270, the speaker 271, the receiver 272, the microphone 273, the headphone interface 274, the application processor, etc. For example, music playback, recording, etc.

[0046] The depth sensor 2801 is used to obtain the depth information of the scene. In some embodiments, the depth sensor can be disposed in the camera module 291.

[0047] The pressure sensor 2802 is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 2802 can be disposed in the display screen 290. There are many types of pressure sensors 2802, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc.

[0048] The gyroscope sensor 2803 can be used to determine the motion posture of the mobile terminal 200. In some embodiments, the angular velocity of the mobile terminal 200 around three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor 2803. The gyroscope sensor 2803 can be used for anti-shake shooting, navigation, motion-sensing game scenarios, etc.

[0049] In addition, sensors with other functions can be set in the sensor module 280 according to actual needs, such as a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0050] The mobile terminal 200 may further include other devices providing auxiliary functions. For example, the key 294 includes a power-on key, a volume key, etc., and the user can input through the key to generate a key signal input related to the user settings and function control of the mobile terminal 200. Another example is the indicator 292, the motor 293, the SIM card interface 295, etc.

[0051] The distortion of the human face increases as the field of view angle of the lens becomes larger. There is no distortion at the center of the lens, and the distortion within a field of view angle of 60° can be ignored. The distortion at the edge is obvious, especially for an ultra-wide-angle lens, and the distortion of the human face at the edge is significant. This kind of distortion belongs to perspective distortion, and the cause is the perspective projection in the lens imaging process.

[0052] In the related art, in order to solve the problem of human face distortion caused by perspective projection, stereographic projection is usually used to correct the perspective distortion. Stereographic projection is a conformal projection that can well restore the true situation of the human face, but it cannot maintain the shape of a straight line, while perspective projection can maintain the shape of a straight line. Therefore, the human face distortion correction method usually combines the advantages of perspective projection and conformal projection to correct the image. First, face detection and portrait segmentation are performed on the image, and then the conformal projection is used to correct the face area to be corrected; the rest of the face area is protected to avoid being deformed as the background; at the same time, the background information is maintained, and the transition between the portrait and the background is smooth. However, in this technical solution, during face correction, in addition to performing deformation correction on the face image part, it also considers correcting the body part of the portrait at the edge to keep the proportion of the face and the body consistent in the correction result, but ignores that the body part of the portrait adjacent to the edge portrait will be abnormally stretched; during the face correction process, for the area with only portrait segmentation information but no face detection information, if this area intersects with the image boundary and the size of this area is large, protection processing is performed, ignoring the fact that the portrait at the edge is originally significantly distorted, resulting in the problem of missed correction. After correction, there is deformation that does not conform to human eye vision, and the correction effect of the distorted image is poor.

[0053] Based on the above one or more problems, taking a terminal device as an example below, the distortion image correction method according to the exemplary embodiments of the present disclosure will be specifically described.

[0054] Figure 3 The flowchart of a distortion image correction method in this exemplary embodiment is shown, including the following steps S310 to step S330:

[0055] In step S310, a distorted image to be corrected is obtained.

[0056] In an exemplary embodiment, a distorted image refers to an image with stretching deformation at the edge obtained by lens shooting. For example, the distorted image can be a distorted image taken by a wide-angle lens, or a panoramic distorted image taken by a panoramic lens. Of course, the distorted image can also be a normal distorted image taken by a normal lens. This exemplary embodiment does not make special limitations on this.

[0057] In step S320, the distorted image is divided into regions, and a first image region and a second image region in the distorted image are determined; wherein, the first image region includes the image content that needs to be corrected.

[0058] In an exemplary embodiment, region division refers to a processing process of dividing the image region that needs to be corrected and the image region that does not need to be corrected in the distorted image. The first image region refers to the image region corresponding to the image content that needs to be corrected in the distorted image. For example, the first image region can be the distorted face part in the image, or the distorted foreground object part in the image. This exemplary embodiment does not make special limitations on this. The second image region refers to the image region corresponding to the image content that does not need to be corrected in the distorted image or the image region adjacent to the first image region. For example, the second image region can be the body part corresponding to the distorted face part in the image, or the image region that is not distorted in the image. This exemplary embodiment does not make special limitations on this.

[0059] In step S330, protection processing is performed on the second image region, and correction processing is performed on the first image region to avoid deforming the second image region when correcting the first image region, thereby completing the correction of the distorted image.

[0060] In an exemplary embodiment, the protection process refers to the process of modifying and locking the content in the image area. When the distortion correction algorithm corrects the distorted image, the protected image area will not be modified and corrected. After the second image area in the distorted image is protected, the first image area in the distorted image is corrected, and the overall correction of the distorted image is completed, effectively avoiding the deformation of the second image area when the first image area is corrected and ensuring the visual effect of the corrected distorted image.

[0061] The following further describes steps S310 to S330.

[0062] In an exemplary embodiment, image correction can be achieved after performing body protection processing on the portrait area with face detection information in the distorted image. First, the first image area and the second image area can be divided from the distorted image through the following steps. Refer to Figure 4 As shown, it can specifically include:

[0063] Step S410, perform portrait detection on the distorted image to determine the target portrait area;

[0064] Step S420, if there is a target face area in the target portrait area, then use the target face area as the first image area, and use the body area corresponding to the target face area in the target portrait area as the second image area.

[0065] Among them, the target portrait area refers to the portrait area in the distorted image that needs to be corrected for distortion. For example, the portrait detection model (such as a deep learning-based portrait detection model) or portrait segmentation algorithm (such as an edge detection-based segmentation algorithm) trained in advance can be used to perform portrait detection on the distorted image to determine multiple portrait areas in the distorted image, and then the target portrait area that needs to be corrected for distortion is selected from the multiple portrait areas. The target face area refers to the image area or face frame obtained by performing face area detection on the distorted image or the target portrait area. For example, the pre-trained face region of interest detection model or face key point detection model can be used to perform portrait detection on the distorted image or the target portrait area to determine the target face area corresponding to each target portrait area.

[0066] In an exemplary embodiment, the distorted image can be segmented into multiple portrait areas, and then the target portrait area can be selected from the obtained multiple portrait areas according to the following conditions:

[0067] The multiple portrait areas form a connected region; and

[0068] The width of the connected region is greater than or equal to the preset width threshold;

[0069] At least two sides of the boundary of the connected region intersect with the adjacent boundaries of the distorted image.

[0070] Among them, the fact that multiple portrait regions form a connected region means that all portrait segmentation regions are connected, and there are no independent portrait segmentation regions. Multiple portrait regions form a connected region. For example, a distorted image may include portrait region A, portrait region B, portrait region C, portrait region D, and portrait region E. Suppose portrait region A and portrait region B are independent, and portrait region C, portrait region D, and portrait region E are connected. That is, portrait region A and portrait region B correspond to a complete portrait contour, and the image content shows a single portrait, while portrait region C, portrait region D, and portrait region E correspond to an overlapping image region, and the image content shows multiple related portraits standing together. Therefore, the connected region formed by portrait region C, portrait region D, and portrait region E can be used as the target portrait region. Since the relevant correction algorithm can achieve good image correction for a single portrait region and will not cause abnormal stretching of the body part, when performing face correction on connected or overlapping multiple portrait regions, due to the difficulty in identifying the body part, it is very easy to cause abnormal stretching of the image in the adjacent region of the face during face correction. Therefore, it is necessary to screen out connected multiple portrait regions for body protection to improve the correction effect.

[0071] The width threshold refers to a preset value for screening connected regions. For example, suppose the width of the distorted image is 10 cm, then the width threshold can be 3 cm. Of course, the width threshold can also be 1 cm, etc. The specific width threshold can be custom-set according to the width of the distorted image or the actual situation. This example embodiment does not make special limitations on this. If the width of the connected region is greater than or equal to the preset width threshold, it means that the image content corresponding to this connected region occupies a relatively large proportion of the distorted image, and body protection processing is required to improve the correction effect; if the width of the connected region is less than the preset width threshold, it means that the image content corresponding to this connected region occupies a relatively small proportion of the distorted image. At this time, even if abnormal stretching occurs in the body part during correction, it will not have a great visual impact. Therefore, body protection processing may not be required, reducing the computational load of the system and improving the efficiency of image correction processing.

[0072] If at least two sides of the boundary of the connected region intersect with the adjacent boundaries of the distorted image, it means that this connected region is at the four corners of the distorted image. At this time, the distortion of the image content in the connected region is relatively large, and the stretching amplitude during face correction is also relatively large. Therefore, it is necessary to perform protection processing on the body part.

[0073] When screening the target portrait area, the target portrait area can meet any one of the above three conditions, or can meet any two of the above three conditions, or can meet all of the above three conditions. This exemplary embodiment does not make special limitations on this.

[0074] In an exemplary embodiment, it can be determined whether there is a target face area in the target portrait area through the steps Figure 5 in, referring to Figure 5 shown, specifically including:

[0075] Step S510, perform face detection on the target portrait area to determine the face area;

[0076] Step S520, take the face area in the face area whose distance from the boundary of the distorted image is less than a preset distance threshold and whose face area is greater than or equal to a preset first area threshold as the first face area;

[0077] Step S530, take the face area with the smallest distance from the boundary of the distorted image in the first face area as the second face area;

[0078] Step S540, take the face area with the smallest distance from the second face area in the first face area and whose corresponding body area is greater than or equal to a preset second area threshold as the third face area;

[0079] Step S550, take the second face area and the third face area as the target face area.

[0080] Among them, the distance threshold refers to the value used to screen the face area close to the boundary of the distorted image. For example, the distance threshold can be 5 mm or 1 cm. Specifically, it can be custom-set according to the size of the distorted image or the actual application scenario. This exemplary embodiment does not make special limitations on this. The boundary of the distorted image can be, for example, the upper boundary, lower boundary, left boundary or right boundary of the distorted image, etc. The first area threshold refers to the value used to screen the large-size face area close to the boundary of the distorted image. For example, the first area threshold can be 1 square centimeter or 25 square millimeters. Specifically, it can be custom-set according to the size of the distorted image or the actual application scenario. This exemplary embodiment does not make special limitations on this.

[0081] The first face area refers to the image area corresponding to the large-size face close to the edge of the distorted image. For the large-size face close to the edge of the distorted image, due to the large distortion amplitude, when correcting the large-size face, it will cause a large abnormal stretch to the area adjacent to the face. Therefore, for the portrait area containing the large-size face close to the edge of the distorted image, body protection and doctor correction effect are required.

[0082] Further, the face region in the first face region with the minimum distance from the boundary of the distorted image is used as the second face region. For example, assuming there are first portrait regions with distances of 1 cm, 2 cm, and 3 cm from the boundary of the distorted image respectively, then the portrait region with the minimum distance of 1 cm from the boundary of the distorted image in the first portrait regions is used as the second face region. And based on the second face region obtained by screening, the third face region is further screened from the remaining first face regions. Among them, the third face region is the one in the first face regions except the second face region, with the minimum distance from the second face region, and the area of the body region corresponding to this portrait region is greater than or equal to a preset second area threshold. The face region that meets the conditions is used as the third face region.

[0083] Finally, the second face region and the third face region that meet the conditions can be used as the target face regions, and the target portrait region containing the target face regions is divided into a first image region and a second image region. Among them, the first image region corresponds to the target face region in the target portrait region, and the second image region corresponds to the body region in the target portrait region except the target face region. When correcting the distorted image, protection processing is performed on the second image region, that is, the body region in the target portrait region, and correction processing is performed on the first image region, that is, the target face region in the target portrait region, so as to realize the correction of the distorted image, avoid unreasonable abnormal stretching, improve the accuracy of correction, and effectively improve the correction effect of the distorted image.

[0084] In an exemplary embodiment, regional division can be performed on the portrait region in the distorted image that does not have face detection information to avoid the problem of missed correction. First, it can be achieved through Figure 6 the steps in to divide the distorted image into a first image region and a second image region. Referring to Figure 6 shown, it can specifically include:

[0085] Step S610, perform portrait detection and face detection on the distorted image to determine the portrait region without a face;

[0086] Step S620, use the portrait region without a face in the portrait region without a face that intersects two adjacent boundaries of the distorted image or intersects one boundary of the distorted image as the first image region;

[0087] Step S630, use the portrait region without a face in the portrait region without a face that intersects at least three boundaries of the distorted image or intersects two non-adjacent boundaries of the distorted image as the second image region.

[0088] Among them, the area without a face portrait refers to a portrait area that does not contain face detection information. For example, the area without a face portrait can be a portrait area where the image content shows a person facing away from the field of view angle, or it can be a portrait area that only contains the body part. This exemplary embodiment does not make special limitations on this. Not containing face detection information can also be simply understood as detecting a portrait area, but no face bounding box or face key point information is output for this portrait area.

[0089] The area without a face portrait that intersects two adjacent boundaries of the distorted image or intersects one boundary of the distorted image in the area without a face portrait can be used as the area to be corrected, that is, the first image area. For example, the area without a face portrait that intersects the upper boundary (or lower boundary) and the left boundary (or right boundary) of the distorted image can be used as the first image area, or the area without a face portrait that intersects the upper boundary (or lower boundary, or left boundary, or right boundary) of the distorted image can be used as the first image area. This exemplary embodiment does not make special limitations on this.

[0090] The area without a face portrait that intersects at least three boundaries of the distorted image or intersects two non-adjacent boundaries of the distorted image in the area without a face portrait can be used as the protection area, that is, the second image area. For example, the area without a face portrait that intersects the upper boundary, the left boundary, and the lower boundary (or the left boundary, the lower boundary, and the right boundary, etc., at least three adjacent boundaries, or the four boundaries of the distorted image) of the distorted image can be used as the second image area, or the area without a face portrait that intersects the upper boundary and the lower boundary (or the left boundary and the right boundary) of the distorted image can be used as the second image area. This exemplary embodiment does not make special limitations on this.

[0091] Of course, the area without a face portrait that does not intersect the boundary of the distorted image and is within the target field of view angle in the area without a face portrait can also be used as the protection area, that is, the second image area. Among them, when judging whether it is within the target field of view angle, it can be determined by judging whether the radial distance between each pixel point in the area without a face portrait and the center point of the distorted image is less than a preset radial distance threshold. If the radial distance between each pixel point in the area without a face portrait and the center point of the distorted image is less than the preset radial distance threshold, it can be considered that the area without a face portrait is within the target field of view angle, otherwise it can be considered that the area without a face portrait is not within the target field of view angle.

[0092] In an exemplary embodiment, after using the area without a face portrait that intersects two adjacent boundaries of the distorted image or intersects one boundary of the distorted image as the first image area, Figure 7 the steps in Figure 7 are further used to process the first image area to improve the correction effect. As shown in

[0093] Step S710, calculate the ratio of the maximum width of the target area in the area without a face portrait to the height of the area without a face portrait;

[0094] Step S720, if the ratio is greater than or equal to a preset ratio threshold, then regard the entire area without a face portrait as the first image area;

[0095] Step S730, if the ratio is less than the preset ratio threshold, then regard the target area as the first image area.

[0096] Wherein, the target area refers to the image area in the area without a face portrait where a face may exist. For example, the target area can be the image area corresponding to the upper half of the area without a face portrait (such as a horizontal dividing line can be determined in the area without a face portrait, and the part above the dividing line is used as the upper half), or it can be the image area in the area without a face portrait where the average width is less than the average width of other areas. This exemplary embodiment does not make special limitations on this.

[0097] If there is an area without a face portrait to be corrected, then make the following judgment for each area without a face portrait to be corrected: the ratio of the maximum width of the target area in the area without a face portrait to the height of the area without a face portrait. If the obtained ratio is greater than or equal to the preset ratio threshold, then regard the entire area without a face portrait as the area to be corrected for the face, that is, the first image area; otherwise, intercept the area without a face portrait from top to bottom, and the intercepted length is the maximum width corresponding to the target area, and regard the intercepted target area as the area to be corrected for the face, that is, the first image area.

[0098] In summary, in this exemplary embodiment, the obtained distorted image is divided into regions, the first image region to be corrected and the second image region not to be corrected in the distorted image are determined. During correction, after performing protection processing on the second image region, correction processing is performed on the first image region to avoid deforming the second image region when correcting the first image region. Finally, the correction of the distorted image is completed. On the one hand, the obtained distorted image is divided into regions, and the first image region to be corrected and the second image region not to be corrected are correctly distinguished, so that all the first image regions to be corrected can be corrected, avoiding missed correction and improving the correction accuracy of the distorted image; on the other hand, protection processing is performed on the second image region not to be corrected, and then correction is performed on the first image region to be corrected, which can effectively avoid deforming the second image region when correcting the first image region, effectively improve the correction effect, and avoid abnormal stretching deformation that does not conform to human visual perception in the corrected image.

[0099] It should be noted that the above drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, rather than for limiting purposes. It is easy to understand that the processes shown in the above drawings do not indicate or limit the chronological order of these processes. Additionally, it is also easy to understand that these processes can be executed synchronously or asynchronously in, for example, multiple modules.

[0100] Further, referring to Figure 8 as shown, in the embodiment of this example, a distortion image correction device 800 is further provided, which may include a distortion image acquisition module 810, an image region division module 820, and a distortion image correction module 830. Among them:

[0101] The distortion image acquisition module 810 is configured to acquire a distortion image to be corrected;

[0102] The image region division module 820 is configured to divide the distortion image into regions, and determine a first image region and a second image region in the distortion image; wherein, the first image region includes the image content that needs to be corrected;

[0103] The distortion image correction module 830 is configured to perform a protection process on the second image region, and perform a correction process on the first image region, so as to avoid deforming the second image region when correcting the first image region, and complete the correction of the distortion image.

[0104] In an exemplary embodiment, the image region division module 820 may include:

[0105] A human figure segmentation unit, which can be used to perform human figure detection on the distortion image to determine a target human figure region;

[0106] An image region determination unit, which can be used to, if there is a target human face region in the target human figure region, use the target human face region as the first image region, and use the body region corresponding to the target human face region in the target human figure region as the second image region.

[0107] In an exemplary embodiment, the human figure segmentation unit may further be used for:

[0108] Performing human figure segmentation on the distortion image to obtain a human figure region, and using the human figure region that meets the following conditions as the target human figure region:

[0109] The human figure regions form a connected region; and

[0110] The width of the connected region is greater than or equal to a preset width threshold;

[0111] At least two sides of the boundary of the connected region intersect with the adjacent boundaries of the distortion image.

[0112] In an exemplary embodiment, the distortion image correction device 800 may further include a target face area screening unit, and the target face area screening unit may be configured to:

[0113] Perform face detection on the target portrait area to determine the face area;

[0114] Use the face area in which the distance from the boundary of the distortion image is less than a preset distance threshold and the face area is greater than or equal to a preset first area threshold as the first face area;

[0115] Use the face area with the smallest distance from the boundary of the distortion image in the first face area as the second face area;

[0116] Use the face area with the smallest distance from the second face area in the first face area and with the corresponding body area greater than or equal to a preset second area threshold as the third face area;

[0117] Use the second face area and the third face area as the target face area.

[0118] In an exemplary embodiment, the image area division module 820 may include:

[0119] A unit for determining a portrait area without a face, which may be configured to perform portrait detection and face detection on the distortion image to determine a portrait area without a face;

[0120] A first image area determination unit, which may be configured to use the portrait area without a face that intersects two adjacent boundaries of the distortion image or intersects one boundary of the distortion image in the portrait area without a face as the first image area;

[0121] A second image area determination unit, which may be configured to use the portrait area without a face that intersects at least three boundaries of the distortion image or intersects two non-adjacent boundaries of the distortion image in the portrait area without a face as the second image area.

[0122] In an exemplary embodiment, the second image area determination unit may further be configured to:

[0123] Use the portrait area without a face that does not intersect the boundary of the distortion image and is within the target field of view angle in the portrait area without a face as the second image area.

[0124] In an exemplary embodiment, the first image area determination unit may further be configured to:

[0125] Calculate the ratio of the maximum width of the target area in the area without a face portrait to the height of the area without a face portrait;

[0126] If the ratio is greater than or equal to a preset ratio threshold, then use the entire area without a face portrait as the first image area;

[0127] If the ratio is less than the preset ratio threshold, then use the target area as the first image area.

[0128] The specific details of each module in the above device have been described in detail in the implementation manner of the method part. For the details not disclosed, reference can be made to the implementation manner content of the method part, so they will not be elaborated here.

[0129] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method, or a program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation manner, a complete software implementation manner (including firmware, microcode, etc.), or an implementation manner combining hardware and software aspects, which can be collectively referred to as "circuit", "module", or "system" here.

[0130] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium, on which a program product capable of implementing the above method of this specification is stored. In some possible implementation manners, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" part of this specification. For example, it can execute Figures 3 to 7 any one or more of the steps.

[0131] It should be noted that the computer-readable medium shown in the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0132] In the present disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination of the above.

[0133] In addition, the program code for performing the operations of the present disclosure can be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).

[0134] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and embodiments are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

[0135] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A method for correcting distorted images, characterized in that, Including: Obtaining a distorted image to be corrected; Dividing the distorted image into regions to determine a first image region and a second image region in the distorted image; wherein, the first image region includes image content that needs to be corrected; Performing protection processing on the second image region and performing correction processing on the first image region to avoid deformation of the second image region when correcting the first image region, thereby completing the correction of the distorted image; Wherein, the dividing the distorted image into regions to determine the first image region and the second image region in the distorted image includes: Performing human figure detection and face detection on the distorted image to determine a human figure region without a face; Taking the human figure region without a face that intersects two adjacent boundaries of the distorted image or intersects one boundary of the distorted image as the first image region; Taking the human figure region without a face that intersects at least three boundaries of the distorted image or intersects two non-adjacent boundaries of the distorted image as the second image region.

2. The method according to claim 1, wherein The dividing the distorted image into regions to determine the first image region and the second image region in the distorted image further includes: Performing human figure detection on the distorted image to determine a target human figure region; If there is a target face region in the target human figure region, taking the target face region as the first image region and taking the body region corresponding to the target face region in the target human figure region as the second image region.

3. The method according to claim 2, wherein Performing human figure detection on the distorted image to determine a target human figure region includes: Performing human figure segmentation on the distorted image to obtain a human figure region, and taking the human figure region that meets the following conditions as the target human figure region: The human figure regions form a connected region; and The width of the connected region is greater than or equal to a preset width threshold; At least two sides of the boundary of the connected region intersect with the adjacent boundaries of the distorted image.

4. The method according to claim 2, characterized in that, The method further includes: Performing face detection on the target human figure region to determine a face region; Taking the face region whose distance from the boundary of the distorted image is less than a preset distance threshold and whose face region area is greater than or equal to a preset first area threshold as a first face region; Taking the face region with the smallest distance from the boundary of the distorted image in the first face region as a second face region; Taking the face region with the smallest distance from the second face region in the first face region and whose corresponding body region area is greater than or equal to a preset second area threshold as a third face region; Taking the second face region and the third face region as the target face region.

5. The method according to claim 1, characterized in that The method further includes: Taking the human figure region without a face that does not intersect the boundary of the distorted image and is within the target field of view angle as the second image region.

6. The method according to claim 1, wherein Taking the non - human - face portrait regions that intersect with two adjacent boundaries of the distorted image or intersect with one boundary of the distorted image in the non - human - face portrait region as the first image region further includes: Calculating the ratio of the maximum width of the target region in the non - human - face portrait region to the height of the non - human - face portrait region; If the ratio is greater than or equal to a preset ratio threshold, taking the entire non - human - face portrait region as the first image region; If the ratio is less than the preset ratio threshold, taking the target region as the first image region.

7. A distortion image correction device, characterized in that, Including: A distorted image acquisition module, configured to acquire a distorted image to be corrected; An image region division module, configured to divide the distorted image into regions, and determine a first image region and a second image region in the distorted image; wherein, the first image region includes the image content to be corrected; A distorted image correction module, configured to perform a protection process on the second image region and a correction process on the first image region to avoid deforming the second image region when correcting the first image region, and complete the correction of the distorted image; Wherein, the dividing the distorted image into regions and determining the first image region and the second image region in the distorted image includes: performing a portrait detection and a face detection on the distorted image to determine a non - human - face portrait region; taking the non - human - face portrait regions that intersect with two adjacent boundaries of the distorted image or intersect with one boundary of the distorted image in the non - human - face portrait region as the first image region; taking the non - human - face portrait regions that intersect with at least three boundaries of the distorted image or intersect with two non - adjacent boundaries of the distorted image in the non - human - face portrait region as the second image region.

8. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, Including: A processor; And A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the method according to any one of claims 1 to 6 by executing the executable instructions.

Images