An image distortion correction method, device and equipment

Abstract

The present invention discloses an image distortion correction method, device and equipment. The method includes: obtaining an image to be corrected, and determining the pixel coordinates of a foreground object in the image to be corrected; determining at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in a preset distortion correction network; wherein, the preset distortion correction network has the same size as the image to be corrected, and the distortion correction network includes a plurality of distortion correction grids, and the plurality of distortion correction grids are overlapped and combined; within the corresponding distortion correction grid, correcting the pixel coordinates of the foreground object according to a preset correction model to obtain the corrected pixel coordinates of the foreground object. This correction method has robustness and universality. Through the preset correction model, it is not necessary to segment the foreground object, nor to judge the distortion state of the foreground object, which improves the performance of image distortion correction and gives users a good visual experience.

Description

Technical Field

[0001] The present invention relates to the field of Internet technologies, and particularly to an image distortion correction method, apparatus, and device. Background Art

[0002] With the development of information technology, people's demand for camera functions of terminal devices has increased sharply, which has promoted the rapid development of camera functions. The camera functions of intelligent terminals including mobile phones and tablets are becoming more and more powerful and have become one of the important functions of terminal devices. Among them, the lens field of view of terminal devices has a great impact on the camera function. However, as the lens field of view becomes larger and larger, the problem of image distortion in captured images has become more and more prominent. Summary of the Invention

[0003] The present invention provides an image distortion correction method, apparatus, and device. The method can directly complete the correction process for foreground objects of different shapes by performing image distortion correction in a preset distortion correction grid. The preset correction model does not need to be reset according to the change in the shape of the foreground object, so it has strong robustness and universality.

[0004] In a first aspect, an embodiment of the present invention provides an image distortion correction method, which includes:

[0005] Obtain the pixel coordinates of the foreground object in the image to be corrected;

[0006] Determine at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in a preset distortion correction network; wherein, the preset distortion correction network has the same size as the image to be corrected, and the distortion correction network includes a plurality of distortion correction grids, and the plurality of distortion correction grids overlap and combine;

[0007] In the corresponding distortion correction grid, correct the pixel coordinates of the foreground object according to a preset correction model to obtain the corrected pixel coordinates of the foreground object.

[0008] In one or some alternative embodiments, the determining at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in a preset distortion correction network includes:

[0009] Obtain the set of pixel coordinates of each distortion correction grid in the preset distortion correction network;

[0010] For each distortion correction grid, based on the fact that the pixel coordinates of the foreground object belong to the set of pixel coordinates of the corresponding distortion correction grid, determine that the pixel coordinates of the foreground object correspond to the distortion correction grid.

[0011] In one or some alternative embodiments, obtaining the set of pixel coordinates of each distortion correction grid in the preset distortion correction network includes:

[0012] Obtaining the original pixel coordinates of each pixel point in the preset distortion correction network;

[0013] For each distortion correction grid in the preset distortion correction network:

[0014] Transform the original pixel coordinates of each pixel point in the distortion correction grid to determine the transformed pixel coordinates of each pixel point, and obtain the set of pixel coordinates of the distortion correction grid.

[0015] In one or some alternative embodiments, the method further includes:

[0016] Determining the original pixel coordinates of the foreground object in the image to be corrected;

[0017] Performing coordinate transformation on the original pixel coordinates of the foreground object in the image to obtain the pixel coordinates of the foreground object in the image.

[0018] In one or some alternative embodiments, the following formula 1 is used to perform coordinate transformation on the original pixel coordinates:

[0019]

[0020] where xi represents the row coordinate of the original pixel coordinate, and yi represents the column coordinate of the original pixel coordinate;

[0021] w represents the width of the distortion correction grid, and h represents the height of the distortion correction grid;

[0022] u0i represents the row coordinate of the pixel coordinate of the foreground object, and v0i represents the column coordinate of the pixel coordinate of the foreground object.

[0023] In one or some alternative embodiments, the preset correction model is a spherical projection model; according to the preset correction model, correcting the pixel coordinates of the foreground object to obtain the pixel coordinates of the corrected foreground object includes:

[0024] Substituting the pixel coordinates of the foreground object into the spherical mapping relationship formula of the spherical projection model to obtain the pixel coordinates of the corrected foreground object;

[0025] The spherical mapping relationship formula of the spherical projection model is:

[0026]

[0027] where, θ0 = atan(v0 i / u0i ), u0i represents the row coordinate of the pixel coordinates of the foreground object, v0i represents the column coordinate of the pixel coordinates of the foreground object, u1i represents the row coordinate of the pixel coordinates of the foreground object after correction, v1i represents the column coordinate of the pixel coordinates of the foreground object after correction, γ i is the scaling factor of the distortion correction grid.

[0028] In one or some alternative embodiments, the method further includes: obtaining the scaling factor γ of each distortion correction grid in the preset distortion correction network in the following manner i :

[0029] Obtain the distortion parameters of the camera corresponding to the preset distortion correction network;

[0030] Map the distortion parameters of the camera to the preset distortion correction network to obtain the distortion parameter values of the center positions of each distortion correction grid in the preset distortion correction network;

[0031] Obtain the scaling factor γ of each distortion correction grid according to the distortion parameter value of the center position of each distortion correction grid and the preset correction coefficient value i .

[0032] In one or some alternative embodiments, the preset correction model is a cylindrical projection model or a perspective projection model.

[0033] In one or some alternative embodiments, before obtaining the pixel coordinates of the foreground object in the image to be corrected, the method further includes:

[0034] Determine whether the image to be corrected includes a foreground object;

[0035] If so, determine the pixel coordinates of the foreground object in the image to be corrected.

[0036] In one or some alternative embodiments, the shape of each distortion correction grid in the preset distortion correction grid is an ellipse, a circle, a square or a rectangle.

[0037] In a second aspect, an embodiment of the present invention provides an image distortion correction device, including:

[0038] A coordinate determination module, configured to obtain the pixel coordinates of the foreground object in the image to be corrected;

[0039] A matching module, configured to determine at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in the preset distortion correction network; wherein, the preset distortion correction network has the same size as the image to be corrected, and the distortion correction network includes a plurality of distortion correction grids, and the plurality of distortion correction grids overlap and combine;

[0040] A correction module, configured to correct the pixel coordinates of the foreground object according to a preset correction model within a corresponding correction grid, so as to obtain the pixel coordinates of the corrected foreground object.

[0041] In a third aspect, an embodiment of the present invention provides an image distortion correction device, and the image distortion correction device includes the above-mentioned image distortion correction apparatus.

[0042] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the image distortion correction method as described above.

[0043] In a fifth aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, it implements the image distortion correction method as described above.

[0044] The beneficial effects of the above technical solutions provided by the embodiments of the present invention at least include:

[0045] The image distortion correction method provided by the present invention identifies the pixel coordinates of the foreground object in the image to be corrected, determines the distortion correction grid corresponding to the pixel coordinates of the foreground object, and completes the correction of the pixel coordinates of the foreground object according to the preset correction model within the corresponding distortion correction grid, and finally obtains a complete corrected image. By performing image distortion correction in a preset distortion correction grid, the correction process can be directly completed for foreground objects of different shapes, and the preset correction model does not need to be reset according to the change of the shape of the foreground object, so it has strong robustness and universality. Moreover, compared with the image distortion correction methods in the prior art, there is no need to segment the foreground object, nor to judge the distortion state of the foreground object, which improves the performance of image distortion correction, increases the speed and efficiency of image distortion correction, can perform image distortion correction in real time, thereby timely preventing the stretching deformation of the foreground object, ensuring that the foreground object has a good presentation effect at different field angles, highly restoring the true state of the foreground object, and giving users a good visual experience.

[0046] Other features and advantages of the present invention will be described in the following description, and part of them will become obvious from the description, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in the written description, claims, and drawings.

[0047] The following further describes the technical solutions of the present invention in detail through the drawings and embodiments. Description of the Drawings

[0048] The accompanying drawings are used to provide a further understanding of the present invention and form a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the accompanying drawings:

[0049] Figure 1 It is a schematic flowchart of the image distortion correction method provided in the embodiment of the present invention;

[0050] Figure 2 It is a schematic structural diagram of the preset distortion correction grid provided in the embodiment of the present invention;

[0051] Figure 3 It is a schematic flowchart of another image distortion correction method provided in the embodiment of the present invention;

[0052] Figure 4 It is a schematic structural diagram of the image distortion correction device provided in the embodiment of the present invention. Detailed implementation manners

[0053] Hereinafter, exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be completely conveyed to those skilled in the art.

[0054] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0055] Research in this application shows that the main reason for image distortion is that the large field of view and short focal length optical system does not satisfy the pinhole model. The focal height of the image plane after the light passes through the camera module is inconsistent with the ideal image height, resulting in a deviation between the actual imaging point and the ideal imaging point. As a result, the imaging content will have geometric shape deformations such as stretching and distortion. For example, when using a mobile phone camera, especially a wide-angle camera, to take pictures of multiple people, the faces at the edge of the picture, where the field of view angle is larger, will have an obvious stretching phenomenon compared with the faces in the center of the picture, or the legs and arms of people at the edge will become significantly thicker, and the imaging effect is not good. Therefore, it is necessary to correct the image or video distortion to maximize the restoration of the true state of the portrait, make the image or video more in line with the human eye visual habit, and enable users to have a good visual experience.

[0056] There is an image distortion correction method that first needs to identify foreground objects, including human figures, and perform algorithmic constraint correction processing on the foreground object area and the background area. In the constraint on the foreground object area, shape correction is performed on the foreground object based on different projection algorithms. Taking the foreground object as a human figure as an example, different projection algorithms can be used to perform shape correction on the head and body areas of the human figure respectively; and, through the constraint of the background area, it is ensured that the image content of the background area will not cause background distortion or tomography due to foreground correction.

[0057] The present invention has found through research that although the above image distortion correction method can solve the problem of deformation or distortion of foreground objects and can achieve a natural, coherent, and coordinated effect between the foreground and the background to a certain extent, however, this method needs to first identify and separate the foreground and the background, and then perform correction processing based on the distortion state of the extracted foreground object, so the efficiency is not high; and, this method needs to establish different image distortion correction models for each group of images respectively, which is not universal, resulting in the weak robustness of this image distortion correction method.

[0058] The present invention provides an image distortion correction method, device, and equipment. This method can directly complete the correction process for foreground objects of different forms by performing image distortion correction in a preset distortion correction grid. The preset correction model does not need to be reset according to the change in the form of the foreground object, so it has strong robustness and universality.

[0059] Referring to Figure 1 As shown, an image distortion correction method provided by an embodiment of the present invention includes the following steps:

[0060] S101: Obtain the pixel coordinates of the foreground object in the image to be corrected.

[0061] In the embodiments of the present invention, the image to be corrected may be an image directly captured by a camera or an image in a video captured by a camera. Of course, it may also be an image obtained from other original fixed carriers, such as an optical disc, a hard disk, or the cloud. The size of the image to be corrected may be a preset size. The image to be corrected may include a foreground object area and a background area, where the image of the foreground object area is the foreground object. The number of foreground objects in the image to be corrected may be one or multiple. The foreground objects in the image to be corrected may be obtained by means in the prior art. For example, a neural network detection technology may be used to determine the foreground objects in the image to be corrected. The foreground objects in the image to be corrected may include human portraits, animal images, plant images, or other detectable objects. Taking a human portrait as an example of the foreground object, a multi-task convolutional deep neural network model (Multi-task Convolutional Deep Neural Network) may be used to detect the face area and detect the face key points of the image to be corrected to identify the foreground object. In the image to be corrected, the part other than the foreground object is the background, such as distant mountains, the sky, buildings, indoor or outdoor environments, etc. Compared with the foreground object, the background is usually farther from the camera in the object space. Correspondingly, compared with the background, the foreground object is usually closer to the camera in the object space.

[0062] In the embodiments of the present invention, the pixel coordinates in the image to be corrected are the pixel coordinates of each pixel point in the image to be corrected. Then, the pixel coordinates of the foreground object are the coordinates of the pixel points corresponding to the foreground object in the image to be corrected.

[0063] S102: Determine at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in a preset distortion correction network.

[0064] In the embodiments of the present invention, before performing the image distortion correction method, a distortion correction network may be preset. The size of the preset distortion correction network may be determined in advance according to the size of the image to be corrected. Assuming the size of the image to be corrected is W*H, the size of the preset distortion correction network may also be W*H, where W represents the width of the image to be corrected and H represents the length of the image to be corrected. A plurality of distortion correction grids may be preset in the preset distortion correction network, and the plurality of distortion correction grids overlap and combine. Among them, the shape of the distortion correction grid may be oval, circular, square, rectangular, or other known shapes in the prior art. Refer to Figure 2As shown, the distortion correction network includes multiple overlapping elliptical distortion correction grids. The smaller the size of the distortion correction grids in the distortion correction network, the higher the precision of distortion correction can be improved. However, the smaller the size of the distortion correction grids, the more the number of distortion correction grids in the distortion correction network will increase, which will increase the performance consumption during the distortion correction process. Therefore, the size and number of multiple distortion correction grids in the distortion correction network can be set according to the actual requirements of distortion correction.

[0065] After setting the distortion correction network and its multiple distortion correction grids, the pixel coordinates of each distortion correction grid in the preset distortion correction network can be obtained according to the pixel coordinate determination method in the above-mentioned prior art, so that by comparing the pixel coordinates of each foreground object with the pixel coordinates in each distortion correction grid, at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in the preset distortion correction network can be determined. In the embodiment of the present invention, since the distortion correction grids in the preset distortion correction network are set in an overlapping combination, the pixel coordinates of the foreground object may correspond to one or more distortion correction grids. Of course, due to the limitations of the size and overlapping layers of the distortion correction grids in the distortion correction network, one or more pixel coordinates of the foreground object may not be within any distortion correction grid, which will not affect the execution of the image distortion correction method proposed by the present invention.

[0066] S103: In the corresponding distortion correction grid, correct the pixel coordinates of the foreground object according to a preset correction model to obtain the corrected pixel coordinates of the foreground object.

[0067] After determining the distortion correction grid corresponding to the pixel coordinates of the foreground object through the above step S102, the pixel coordinates of the foreground object can be corrected according to the preset correction model of each distortion correction grid in each distortion correction grid respectively, so as to obtain the corrected pixel coordinates of the foreground object. When the corrected pixel coordinates of the foreground object in all distortion correction grids are obtained, the image distortion correction process of the image to be corrected is completed, and the corrected image is obtained.

[0068] The image distortion correction method provided by the present invention can be applied to any electronic device, including but not limited to various smart phones, digital cameras, personal computers, laptop computers, tablet computers, etc. The electronic device is equipped with a camera, and the electronic device obtains the image to be corrected by real-time shooting with the camera, and executes the image distortion correction method of the embodiment of the present application on the image to be corrected to correct the distortion of the image to be corrected and obtain the corrected image.

[0069] In the embodiments of the present invention, since the distortion correction network and each distortion correction grid are preset, after obtaining the foreground image, there is no need to segment the foreground image, that is, there is no need to separately identify each different object in the foreground object. Only the pixel coordinates need to be corrected in the distortion correction grid of each pixel point including the foreground object, and then the distortion correction of the foreground object can be completed to obtain the corrected image. For example, assuming that the image to be corrected is a group photo portrait image, it is only necessary to identify multiple portraits in the foreground object and determine the corresponding different distortion correction grids of the foreground object, without the need to segment different portraits based on the portrait segmentation technology. As long as the pixel coordinates of the foreground object including multiple portraits are obtained, the pixel coordinates can be corrected in the corresponding distortion correction grid, and finally the corrected portrait image can be obtained. In this way, it not only avoids the problem of inaccurate image correction caused by incorrect segmentation during object segmentation, but also reduces the performance loss of distortion correction.

[0070] The image distortion correction method provided by the present invention identifies the pixel coordinates of the foreground object in the image to be corrected, determines the distortion correction grid corresponding to the pixel coordinates of the foreground object, and completes the correction of the pixel coordinates of the foreground object according to the preset correction model in the corresponding distortion correction grid, and finally obtains the complete corrected image. By performing image distortion correction in the preset distortion correction grid, the correction process can be directly completed for foreground objects of different shapes, and the preset correction model does not need to be reset according to the change of the shape of the foreground object, so it has strong robustness and universality. Moreover, compared with the image distortion correction methods in the prior art, there is no need to segment the foreground object or judge the distortion state of the foreground object, which improves the performance of image distortion correction, increases the speed and efficiency of image distortion correction, can perform image distortion correction in real time, thus timely preventing the stretching deformation of the foreground object, ensuring that the foreground object has a good presentation effect at different field angles of view, highly restoring the true state of the foreground object, and giving users a good visual experience.

[0071] In a specific embodiment, in the above step S101, the pixel coordinates of the foreground object in the image to be corrected can be obtained in the following manner:

[0072] Based on the following formula 1, the original pixel coordinates of the foreground object in the image to be corrected are subjected to coordinate transformation to obtain the pixel coordinates of the foreground object in the image:

[0073]

[0074] Where: xi represents the row coordinate of the original pixel coordinates of the foreground object, and yi represents the column coordinate of the original pixel coordinates of the foreground object;

[0075] w represents the width of the distortion correction grid, and h represents the height of the distortion correction grid;

[0076] U0i represents the row coordinate of the pixel coordinates of the foreground object, and v0i represents the column coordinate of the pixel coordinates of the foreground object.

[0077] In the embodiments of the present invention, the method for determining the original pixel coordinates of the image to be corrected can adopt the method in the prior art. For example, a certain corner point or the center point in the image to be corrected can be used as the origin, and the horizontal coordinate axis and the vertical coordinate axis are set to obtain the row coordinate and the column coordinate of the original pixel coordinates, so as to obtain the original pixel coordinates. For example, if the coordinates of the top-leftmost corner point in the image to be corrected are (0, 0), then the original pixel coordinates of the pixel point adjacent to the right of this corner point are (1, 0), and the original pixel coordinates of the pixel point adjacent to the bottom of this corner point are (0, 1), and so on. Since all the foreground objects in the image to be corrected have been determined previously, the original pixel coordinates of each foreground object can be obtained.

[0078] In the embodiments of the present invention, before performing the above step of coordinate transformation to obtain the pixel coordinates of the foreground object, the above-mentioned preset distortion correction network and each distortion correction grid therein have been obtained in advance. Therefore, the step of coordinate transformation can be performed based on the above formula 1. By performing coordinate transformation on the original pixel coordinates of the foreground object, it is convenient to perform the subsequent step of coordinate correction.

[0079] Correspondingly, the specific process of performing the above step S102 to determine at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in the preset distortion correction network may include the following steps:

[0080] Obtain the set of pixel coordinates of each distortion correction grid in the preset distortion correction network;

[0081] For each distortion correction grid, based on the fact that the pixel coordinates of the foreground object belong to the set of pixel coordinates of the corresponding distortion correction grid, determine that the pixel coordinates of the foreground object correspond to the distortion correction grid.

[0082] Among them, the following method can be used to determine that the pixel coordinates of the foreground object belong to the set of pixel coordinates of the corresponding distortion correction grid:

[0083] Judge whether the pixel coordinates of the foreground object belong to the set of pixel coordinates of the distortion correction grid. If so, determine that the pixel coordinates of the foreground object belong to the set of pixel coordinates of the corresponding distortion correction grid.

[0084] In the embodiments of the present invention, after obtaining the preset distortion correction network and each of its distortion correction grids, the set of pixel coordinates of each distortion correction grid can be determined in advance by the following method:

[0085] Obtain the original pixel coordinates of each pixel point in the preset distortion correction network;

[0086] For each distortion correction grid in the preset distortion correction network:

[0087] According to the above formula 1, transform the original pixel coordinates of each pixel point in the distortion correction grid to determine the transformed pixel coordinates of each pixel point, and obtain the pixel coordinate set of the distortion correction grid.

[0088] In the embodiments of the present invention, the method for determining the original pixel coordinates of each pixel point in the preset distortion correction network is similar to the method for determining the original pixel coordinates of the to-be-corrected image described above. The specific implementation process can refer to the detailed description of the method for determining the original pixel coordinates of the to-be-corrected image above, and will not be elaborated here.

[0089] In the embodiments of the present invention, assume that there are N distortion correction grids in the above preset distortion correction network, then the pixel coordinate set Q0 of the i-th distortion correction grid among them Ni (x i , y i ) = [u0 i (x i , y i ), v0 i (x i , y i )] T , where U0i(x i , y i ) represents the row coordinate of the pixel point with the original pixel coordinate (xi, yi) in the distortion correction grid, and v0i(x i , y i ) represents the column coordinate of the pixel point with the original pixel coordinate (xi, yi) in the distortion correction grid.

[0090] Since the positions of the distortion correction grids in the preset distortion correction network are determined, after obtaining the original pixel coordinates of each pixel point in the preset distortion correction network, all pixel points and their original pixel coordinates in each distortion correction grid can be determined according to the positions of the pixel points.

[0091] The implementation process of performing coordinate transformation on the pixel points in each distortion correction grid to obtain the pixel coordinates of each pixel point in the distortion correction grid is similar to the process of determining the pixel coordinates of the foreground object described above, and will not be elaborated here. When the pixel coordinates of each pixel point in the distortion correction grid are obtained according to the above formula 1, the pixel coordinate set of the distortion correction grid is obtained.

[0092] In an embodiment of the present invention, the above-mentioned preset correction model may be a spherical projection model; correspondingly, the specific implementation process of correcting the pixel coordinates of the foreground object according to the preset correction model within the corresponding distortion correction grid described in step S103 above to obtain the pixel coordinates of the corrected foreground object may include the following steps:

[0093] In the distortion correction grid, substitute the pixel coordinates of the foreground object into the spherical mapping relation formula 2 of the spherical projection model to obtain the pixel coordinates of the corrected foreground object:

[0094]

[0095] Wherein, θ0 = atan(v0 i / u0 i ), u0i represents the row coordinate of the pixel coordinates of the foreground object, v0i represents the column coordinate of the pixel coordinates of the foreground object, u1i represents the row coordinate of the pixel coordinates of the corrected foreground object, v1i represents the column coordinate of the pixel coordinates of the corrected foreground object, and γ i is the scaling coefficient of the i-th distortion correction grid in the preset distortion correction network.

[0096] Wherein, the scaling coefficients γ of each distortion correction grid in the preset distortion correction network i can be obtained specifically through the following method:

[0097] Obtain the distortion parameters of the camera corresponding to the preset distortion correction network;

[0098] Map the distortion parameters of the camera to the preset distortion correction network to obtain the distortion parameter values at the center positions of each distortion correction grid in the preset distortion correction network;

[0099] According to the distortion parameter values at the center positions of each distortion correction grid and the preset correction coefficient values, obtain the scaling coefficients γ of each distortion correction grid i .

[0100] In an embodiment of the present invention, the distortion parameters of the camera may be directly obtained from the camera manufacturer or obtained by calibrating the camera according to the camera calibration method in the prior art. Among them, the above-mentioned camera calibration method may be a linear calibration method, a non-linear optimization calibration method, a Zhang Zhengyou calibration method or other common calibration methods. In the embodiment of the present invention, the camera calibration method may not be specifically limited as long as the distortion parameters of the camera can be obtained.

[0101] In the embodiment of the present invention, since the distortion rates of different positions of the image captured by the camera are different, and the distortion rate is smaller at the position closer to the optical center of the camera, the distortion parameters of the camera corresponding to the preset distortion correction network include the distortion parameter values of each pixel position in the preset distortion correction network. By obtaining the distortion parameters of the camera, the distortion parameter values of different pixel positions can be obtained, and further, the distortion parameter values of the center positions of each distortion correction grid can be obtained. By performing a multiplication operation on the distortion parameter values of the center positions of the distortion correction grids and the preset correction coefficient values, the scaling coefficients of each distortion correction grid can be obtained. Since the distortion rate is smaller at the position closer to the optical center of the camera, the scaling coefficient of the distortion correction grid closer to the center position of the preset distortion correction network is smaller, that is, the scaling coefficient of the distortion correction grid closer to the edge of the preset distortion correction network is greater than the scaling coefficient of the distortion correction grid closer to the center of the preset distortion correction network.

[0102] In the embodiment of the present invention, since the scaling coefficient of each distortion correction grid in the preset distortion correction network is determined according to the distortion parameters of the camera, the difference in the scaling coefficients of adjacent or overlapping distortion correction grids approaches the difference in the distortion parameter values of adjacent regions in the distortion correction network. Therefore, by correcting the pixel coordinates of the foreground object in each distortion correction grid, a smooth transition can be achieved in the adjacent regions of the foreground object in the corrected image, avoiding problems such as burrs or unconnected images after correction.

[0103] In one embodiment, the above preset correction model can also adopt other stereoscopic projection models in the prior art. For example, a cylindrical projection model or a perspective projection model. Those skilled in the art can, based on the above detailed description of the spherical projection model, use the cylindrical projection model or the perspective projection model to correct the pixel coordinates of the foreground object in the distortion correction grid to obtain the corrected pixel coordinates of the foreground object, thereby obtaining the corrected image. The specific implementation manners of the cylindrical projection model or the perspective projection model can be combined with the above spherical projection model and the relevant descriptions in the prior art, and will not be elaborated here.

[0104] In one embodiment, referring to Figure 3 As shown, before performing the above step S101, the image distortion correction method may further include:

[0105] S100: Determine whether the image to be corrected includes a foreground object;

[0106] If so, execute step S101; if not, end the image distortion correction.

[0107] Since there may be no foreground objects in some images and only the background is included, when it is determined that the image to be corrected does not include foreground objects, the above-mentioned distortion correction process is stopped. Thus, the time for image correction processing can be saved and system resources can be conserved. Of course, when the image to be corrected does not include foreground objects, an image distortion correction method in the prior art can also be used to correct the background in the image to be corrected to obtain an image with a better visual effect. The specific implementation process can refer to the detailed description in the prior art and will not be elaborated here.

[0108] Based on the same inventive concept, an embodiment of the present invention further provides an image distortion correction device. Since the principle of the problem solved by this device is similar to that of the foregoing image distortion correction method, the implementation of this device can refer to the implementation of the foregoing image distortion correction method, and the repeated parts will not be elaborated.

[0109] Refer to Figure 4 As shown, an image distortion correction device provided by an embodiment of the present invention includes: a coordinate determination module 100, a matching module 200, and a correction module 300; where:

[0110] The coordinate determination module 100 is configured to obtain the pixel coordinates of the foreground object in the image to be corrected;

[0111] The matching module 200 is configured to determine at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in a preset distortion correction network; wherein, the preset distortion correction network has the same size as the image to be corrected, and the distortion correction network includes a plurality of distortion correction grids, and the plurality of distortion correction grids overlap and combine;

[0112] The correction module 300 is configured to correct the pixel coordinates of the foreground object according to a preset correction model within the corresponding correction grid to obtain the corrected pixel coordinates of the foreground object.

[0113] In one or some alternative embodiments, the matching module 200 is specifically configured to obtain the pixel coordinate sets of each distortion correction grid in the preset distortion correction network;

[0114] For each distortion correction network, based on the fact that the pixel coordinates of the foreground object belong to the pixel coordinate set of the corresponding distortion correction grid, it is determined that the pixel coordinates of the foreground object correspond to the distortion correction grid.

[0115] In one or some alternative embodiments, the matching module 200 is specifically configured to obtain the original pixel coordinates of each pixel point in the preset distortion correction network;

[0116] For each distortion correction grid in the preset distortion correction network:

[0117] Transform the original pixel coordinates of each pixel point in the distortion correction grid to determine the transformed pixel coordinates of each pixel point, and obtain the pixel coordinate set of the distortion correction grid.

[0118] In one or some alternative embodiments, the coordinate determination module 100 is specifically configured to:

[0119] Determine the original pixel coordinates of the foreground object in the image to be corrected;

[0120] Perform coordinate transformation on the original pixel coordinates of the foreground object in the image to obtain the pixel coordinates of the foreground object in the image.

[0121] In one or some alternative embodiments, the coordinate determination module 100 is specifically configured to perform coordinate transformation on the original pixel coordinates by using the following formula 1:

[0122]

[0123] where xi represents the row coordinate of the original pixel coordinate, and yi represents the column coordinate of the original pixel coordinate;

[0124] w represents the width of the distortion correction grid, and h represents the height of the distortion correction grid;

[0125] u0i represents the row coordinate of the pixel coordinate of the foreground object, and v0i represents the column coordinate of the pixel coordinate of the foreground object.

[0126] In one or some alternative embodiments, the preset correction model is a spherical projection model; the correction module 300 is specifically configured to:

[0127] Substitute the pixel coordinates of the foreground object into the spherical mapping relationship formula of the spherical projection model to obtain the pixel coordinates of the corrected foreground object;

[0128] The spherical mapping relationship formula of the spherical projection model is:

[0129]

[0130] where θ0 = atan(v0 i / u0 i ), u0i represents the row coordinate of the pixel coordinate of the foreground object, v0i represents the column coordinate of the pixel coordinate of the foreground object, u1i represents the row coordinate of the pixel coordinate of the corrected foreground object, v1i represents the column coordinate of the pixel coordinate of the corrected foreground object, and γ i is the scaling factor of the distortion correction grid.

[0131] In one or some alternative embodiments, the correction module 300 is further configured to obtain the scaling factor γ of each distortion correction network in the preset distortion correction network in the following manner i :

[0132] Obtain the distortion parameters of the camera corresponding to the preset distortion correction network;

[0133] Map the distortion parameters of the camera to the preset distortion correction network to obtain the distortion parameter values at the center positions of each distortion correction grid in the preset distortion correction network;

[0134] According to the distortion parameter values at the center positions of each distortion correction grid and the preset correction coefficient values, obtain the scaling factor γ of each distortion correction network i 。

[0135] In one or some alternative embodiments, the image distortion correction device further includes a foreground object determination module, configured to determine whether a foreground object is included in the to-be-corrected image before obtaining the pixel coordinates of the foreground object in the to-be-corrected image;

[0136] If so, determine the pixel coordinates of the foreground object in the to-be-corrected image.

[0137] In one or some alternative embodiments, the shapes of the distortion correction grids in the preset distortion correction network are ellipses, circles, squares, or rectangles.

[0138] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the image distortion correction method as described above.

[0139] An embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the image distortion correction method as described above.

[0140] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) containing computer-usable program code.

[0141] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device generate means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or in multiple blocks.

[0142] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or in multiple blocks.

[0143] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or in multiple blocks.

[0144] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. An image distortion correction method, characterized in that, Including: Obtain the pixel coordinates of the foreground object in the image to be corrected; Determine at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in a preset distortion correction network; wherein, the preset distortion correction network has the same size as the image to be corrected, and the distortion correction network includes a plurality of distortion correction grids, and the plurality of distortion correction grids overlap and combine; Within the corresponding distortion correction grid, correct the pixel coordinates of the foreground object according to a preset correction model to obtain the corrected pixel coordinates of the foreground object; The determining at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in the preset distortion correction network includes: Obtain the pixel coordinate sets of each distortion correction grid in the preset distortion correction network; For each distortion correction grid, based on the fact that the pixel coordinates of the foreground object belong to the pixel coordinate set of the corresponding distortion correction grid, determine that the pixel coordinates of the foreground object correspond to the distortion correction grid.

2. The image distortion correction method according to claim 1, characterized in that, The obtaining the pixel coordinate sets of each distortion correction grid in the preset distortion correction network includes: Obtain the original pixel coordinates of each pixel point in the preset distortion correction network; For each distortion correction grid in the preset distortion correction network: Transform the original pixel coordinates of each pixel point in the distortion correction grid to determine the transformed pixel coordinates of each pixel point, and obtain the pixel coordinate set of the distortion correction grid.

3. The image distortion correction method according to claim 1, characterized in that, The obtaining the pixel coordinates of the foreground object in the image to be corrected includes: Determine the original pixel coordinates of the foreground object in the image to be corrected; Perform coordinate transformation on the original pixel coordinates of the foreground object in the image to obtain the pixel coordinates of the foreground object in the image.

4. The image distortion correction method according to any one of claims 1-3, characterized in that, Perform coordinate transformation on the original pixel coordinates using the following formula 1: where x i represents the row coordinate of the original pixel coordinate, and y i represents the column coordinate of the original pixel coordinate; w represents the width of the distortion correction grid, and h represents the height of the distortion correction grid; u0 i represents the row coordinate of the pixel coordinates of the foreground object, v0 i represents the column coordinate of the pixel coordinates of the foreground object.

5. The image distortion correction method according to any one of claims 1 to 3, characterized in that, The preset correction model is a spherical projection model; the correcting the pixel coordinates of the foreground object according to the preset correction model to obtain the corrected pixel coordinates of the foreground object includes: Substitute the pixel coordinates of the foreground object into the spherical mapping relationship formula of the spherical projection model to obtain the corrected pixel coordinates of the foreground object; The spherical mapping relationship formula of the spherical projection model is: Among them, θ0 = atan(v0 i / u0 i ), u0 i represents the row coordinate of the pixel coordinates of the foreground object, v0 i represents the column coordinate of the pixel coordinates of the foreground object, u1 i represents the row coordinate of the pixel coordinates of the foreground object after correction, v1 i represents the column coordinate of the pixel coordinates of the foreground object after correction, γ i is the scaling factor of the distortion correction grid.

6. The image distortion correction method according to claim 5, wherein It also includes: The scaling factor γ of each distortion correction network in the preset distortion correction network is obtained in the following manner i : Obtain the distortion parameters of the camera corresponding to the preset distortion correction network; Map the distortion parameters of the camera to the preset distortion correction network to obtain the distortion parameter values at the center positions of each distortion correction grid in the preset distortion correction network; According to the distortion parameter values and the preset correction coefficient values at the center positions of each distortion correction grid, the scaling coefficient γ of each distortion correction network is obtained i .

7. An image distortion correction device, characterized in that, Including: A coordinate determination module, configured to obtain the pixel coordinates of the foreground object in the image to be corrected; A matching module, configured to determine at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in a preset distortion correction network; wherein, the preset distortion correction network has the same size as the image to be corrected, and the distortion correction network includes a plurality of distortion correction grids, and the plurality of distortion correction grids overlap and combine; the determining at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in the preset distortion correction network includes: Obtain the set of pixel coordinates of each distortion correction grid in the preset distortion correction network; For each distortion correction network, based on the fact that the pixel coordinates of the foreground object belong to the set of pixel coordinates of the corresponding distortion correction grid, determine that the pixel coordinates of the foreground object correspond to the distortion correction grid; A correction module, configured to correct the pixel coordinates of the foreground object according to a preset correction model within the corresponding correction grid to obtain the corrected pixel coordinates of the foreground object.

8. A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the image distortion correction method according to any one of claims 1-6.

9. An electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, it implements the image distortion correction method according to any one of claims 1-6.

Images