Image correction method and device, electronic device and storage medium

Abstract

The present disclosure relates to an image correction method and device, an electronic device and a storage medium. The image correction method comprises: obtaining a first image captured by an image acquisition module; obtaining a mapping diagram generated based on the corresponding relationship between the field of view and the distortion rate of the image acquisition module; wherein the mapping diagram comprises: a mapping relationship between the original pixel coordinates of each pixel value in the original image captured by the image acquisition module and the corrected target pixel coordinates; based on the mapping diagram, correcting the first image to obtain a corrected second image.

Description

Technical Field

[0001] The present disclosure relates to the field of image technology, and in particular to an image correction method and device, an electronic device, and a storage medium. Background Art

[0002] Image acquisition is widely used in transportation, monitoring, photography and other technical fields. However, due to the structure of the camera, the captured image will be distorted relative to the captured object. Especially for the wide-angle lens, this distortion is more serious due to the structural characteristics of the wide-angle lens itself.

[0003] Related technologies have proposed methods for correcting this distortion. For example, distortion correction is performed using a polynomial model. On the one hand, this method requires calibration of the camera based on a checkerboard, and requires obtaining the internal and external parameters of the camera, which is computationally intensive and time-consuming. On the other hand, the distortion correction of this polynomial model has a wide range of applications. In some scenarios, the correction effect of the captured images is not obvious, and even more image distortion may be introduced due to the correction. In addition, even if the image is corrected within the scope of use of the polynomial model, the image correction effect is not ideal. Summary of the invention

[0004] Embodiments of the present disclosure provide an image correction method and device, an electronic device, and a storage medium.

[0005] A first aspect of the present disclosure provides an image correction method, the method comprising:

[0006] Acquire a first image captured by an image acquisition module;

[0007] Acquire a mapping diagram generated based on the corresponding relationship between the field of view and the distortion rate of the image acquisition module; wherein the mapping diagram includes: a mapping relationship between the original pixel coordinates of each pixel value in the original image acquired by the image acquisition module and the corrected target pixel coordinates;

[0008] Based on the mapping diagram, the first image is corrected to obtain a corrected second image.

[0009] Based on the above solution, the step of obtaining a mapping diagram generated based on the corresponding relationship between the field of view and the distortion rate of the image acquisition module includes:

[0010] Read a mapping diagram pre-generated based on the correspondence between the field of view and the distortion rate of the image acquisition module.

[0011] Based on the above solution, the generation of the mapping diagram includes:

[0012] Acquire the corresponding relationship; wherein the corresponding relationship includes: a mapping relationship between N discretely distributed fields of view of the image acquisition module and the distortion rate, wherein N is a positive integer greater than or equal to 2;

[0013] Based on the interpolation algorithm and the corresponding relationship, an FD curve is generated; wherein the FD curve is used to determine the distortion rate corresponding to the field of view where each original pixel coordinate in the image captured by the image acquisition module is located;

[0014] The mapping diagram is generated based on the FD curve, the first distance and the second distance; wherein the first distance is: the distance between the original pixel coordinates and the center point of the image; the second distance is: the distance between the target pixel coordinates and the center point.

[0015] Based on the above solution, generating the mapping diagram based on the FD curve, the first distance and the second distance includes:

[0016] Determining the field of view of each original pixel coordinate of the image acquisition module;

[0017] Querying the FD curve based on the field of view to determine the distortion rate of each original pixel coordinate;

[0018] Determining a target pixel coordinate corresponding to each original pixel coordinate of the image sensor based on a functional relationship among the distortion rate, the first distance, and the second distance;

[0019] The mapping diagram is generated based on each of the original pixel coordinates and the target pixel coordinates corresponding to each of the original pixel coordinates.

[0020] Based on the above solution, determining the field of view of each original pixel coordinate of the image acquisition module includes:

[0021] Determine the pixel coordinates and the maximum acquisition radius of the center point of the image sensor; wherein the field of view of the pixel coordinates of the center point is a first preset value; the field of view of the predetermined pixel coordinates is a second preset value, wherein the predetermined pixel coordinates are: the pixel coordinates whose distance from the pixel coordinates of the center point is equal to the maximum acquisition radius; the difference between the second preset value and the first preset value is the field of view range value of the image sensor;

[0022] Determine the distance between each original pixel coordinate collected by the image sensor and the pixel coordinate of the center point;

[0023] Determine the field of view ratio of each original pixel coordinate according to the distance between each original pixel coordinate and the center point and the maximum acquisition radius;

[0024] The field of view of each original pixel coordinate is determined based on the field of view ratio and the field of view range value.

[0025] A second aspect of the present disclosure provides an image correction device, the device comprising:

[0026] A first acquisition module, used to acquire a first image acquired by the image acquisition module;

[0027] A second acquisition module is used to acquire a mapping diagram generated based on the corresponding relationship between the field of view and the distortion rate of the image acquisition module; wherein the mapping diagram includes: a mapping relationship between the original pixel coordinates of each pixel value in the original image acquired by the image acquisition module and the corrected target pixel coordinates;

[0028] A generating module is used to correct the first image based on the mapping diagram to obtain a corrected second image.

[0029] Based on the above solution, the second acquisition module is specifically used to read a mapping map pre-generated based on the correspondence between the field of view and the distortion rate of the image acquisition module.

[0030] Based on the above scheme, the device includes:

[0031] A third acquisition module is used to acquire the corresponding relationship; wherein the corresponding relationship includes: a mapping relationship between N discretely distributed fields of view of the image acquisition module and the distortion rate, wherein N is a positive integer greater than or equal to 2;

[0032] An FD curve module, used to generate an FD curve based on an interpolation algorithm and the corresponding relationship; wherein the FD curve is used to determine the distortion rate corresponding to the field of view where each original pixel coordinate in the image captured by the image acquisition module is located;

[0033] A mapping module is used to generate the mapping based on the FD curve, a first distance and a second distance; wherein the first distance is: the distance between the original pixel coordinates and the center point of the image; and the second distance is: the distance between the target pixel coordinates and the center point.

[0034] Based on the above scheme, the mapping module is specifically used to determine the field of view of each original pixel coordinate of the image acquisition module; query the FD curve based on the field of view to determine the distortion rate of each original pixel coordinate; determine the target pixel coordinate corresponding to each original pixel coordinate of the image sensor based on the functional relationship between the distortion rate, the first distance and the second distance; generate the mapping based on each original pixel coordinate and the target pixel coordinate corresponding to each original pixel coordinate.

[0035] Based on the above scheme, the mapping module is specifically used to determine the pixel coordinates and the maximum acquisition radius of the center point of the image sensor; wherein the field of view of the pixel coordinates of the center point is a first preset value; the field of view of the predetermined pixel coordinates is a second preset value, wherein the predetermined pixel coordinates are: the pixel coordinates whose distance from the pixel coordinates of the center point is equal to the maximum acquisition radius; the difference between the second preset value and the first preset value is the field of view range value of the image sensor; determine the distance between each original pixel coordinate collected by the image sensor and the pixel coordinates of the center point; determine the field of view ratio of each original pixel coordinate based on the distance between each original pixel coordinate and the center point and the maximum acquisition radius; determine the field of view of each original pixel coordinate based on the field of view ratio and the field of view range value.

[0036] A third aspect of the present disclosure provides an electronic device, including:

[0037] a memory for storing processor-executable instructions;

[0038] A processor connected to the memory;

[0039] Wherein, the processor is configured to execute the image correction method provided by any one of the technical solutions in the first aspect.

[0040] A fourth aspect of an embodiment of the present disclosure provides a non-temporary computer-readable storage medium. When instructions in the storage medium are executed by a processor of a computer, the computer can execute an image correction method provided by any one of the technical solutions of the first aspect.

[0041] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0042] The technical solution provided by the disclosed embodiment performs correction on the basis of a mapping diagram when correcting an image. Since the original pixel coordinates of each pixel value in the mapping diagram correspond to the target pixel coordinates, a pixel-by-pixel correction of the first image is performed, which has the characteristic of good correction effect. In addition, in the disclosed embodiment, the mapping diagram generated based on the correspondence between the field of view and the distortion rate of the image acquisition module is applicable to any image acquired by the image acquisition module, thus having the characteristic of a wide range of applications. At the same time, there is no need for a large amount of complex calculations, and only the coordinate conversion of the acquired pixel values ​​needs to be performed according to the mapping diagram, which has the characteristics of simple calculation and high correction efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0044] Figure 1 is a flowchart of an image correction method according to an exemplary embodiment;

[0045] Figure 2 is a schematic diagram showing a comparison between original pixel coordinates and target pixel coordinates according to an exemplary embodiment;

[0046] Figure 3 is a schematic flow chart of a method for generating a mapping diagram according to an exemplary embodiment;

[0047] Figure 4 is a schematic diagram showing how to obtain target pixel coordinates based on the principle of similar triangles according to an exemplary embodiment;

[0048] Figure 5 is a schematic diagram of a field of view of an image acquisition module according to an exemplary embodiment;

[0049] Figure 6 is a schematic structural diagram of an image correction device according to an exemplary embodiment;

[0050] Figure 7 The figure is a schematic diagram of the structure of a mobile terminal according to an exemplary embodiment. DETAILED DESCRIPTION

[0051] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0052] like Figure 1 As shown, the present disclosure provides an image correction method, which may include:

[0053] S110: Acquire a first image captured by an image acquisition module;

[0054] S120: Obtain a mapping diagram generated based on the correspondence between the field of view and the distortion rate of the image acquisition module; wherein the mapping diagram includes: a mapping relationship between the original pixel coordinates of each pixel value in the original image acquired by the image acquisition module and the corrected target pixel coordinates;

[0055] S130: Based on the mapping image, correct the first image to obtain a corrected second image.

[0056] In the disclosed embodiments, the image correction method can be used in various electronic devices. The electronic devices may include various terminal devices and servers. The terminal devices may include mobile terminals and / or fixed terminals. The mobile terminals include but are not limited to: mobile phones, tablet computers, tablet and notebook two-in-one devices, e-readers, wearable devices or vehicle-mounted devices, etc.

[0057] The image acquisition module may be various types of lenses with image acquisition functions, such as a wide-angle lens, a telephoto lens, or a 3D lens.

[0058] There may be no specific order relationship between step S110 and step S120. When the image acquisition module acquires the first image, the mapping map of the image acquisition module may be read synchronously. The mapping map may exist in a configuration file of the image acquisition module, etc. The electronic device may also first acquire the first image, and then acquire the mapping map corresponding to the image acquisition module. Alternatively, the mapping map may be acquired immediately after the image acquisition module is started, so that image correction can be performed on each acquired first image according to the mapping map, thereby quickly obtaining the second image.

[0059] The mapping diagram is first generated based on the correspondence between the field of view and the distortion rate of the image acquisition module. The following is an example of this correspondence. As shown in Table 1, Field represents different fields of view, and Distortion represents the distortion rate. The FD tables of different lenses are different. Table 1 is only a reference table. The calculation formula is: Distortion = (real image height - ideal image height) / ideal image height.

[0060] Field Distortion 0.0 0.00％ 0.1 0.21％ 0.2 0.52％ 0.3 0.75％ 0.4 1.06％ 0.5 1.50％ 0.6 1.81％ 0.7 1.86％ 0.8 1.74％ 0.9 1.61％ 1.0 1.58％

[0061] Table 1

[0062] like Figure 2 As shown, the solid origin marks the original pixel coordinates, and the hollow circle marks the target pixel coordinates. The mapping diagram may include: the target pixel coordinates corresponding to the original pixel coordinates of each pixel value.

[0063] For example, the file format of the mapping map is an image format, and the pixel coordinates of each pixel in the image format correspond to the original pixel coordinates of the image captured by the image acquisition module; the target pixel coordinates are stored in each original pixel coordinate. In this way, when the first image is corrected to generate the second image, the mapping map is searched to obtain the target pixel coordinates according to the current original pixel coordinates in the currently traversed first image, and the pixel coordinates of the pixel value of the current original pixel coordinates are changed to the target pixel coordinates according to the target pixel coordinates, thereby obtaining the second image.

[0064] The corresponding relationship between the original pixel coordinates and the target pixel coordinates is stored in an image format, which has the characteristic of convenient storage.

[0065] In the embodiment of the present disclosure, the mapping diagram may be: information in any format that stores the correspondence between the original pixel coordinates and the target pixel coordinates of each pixel value collected by the image sensor, and is not limited to the example of the pixel diagram in the above image format.

[0066] In summary, in the disclosed embodiment, the image correction is performed based on the mapping diagram. Since the original pixel coordinates of each pixel value in the mapping diagram correspond to the target pixel coordinates, the image correction process of the disclosed embodiment performs pixel-by-pixel correction of the first image, which has the characteristics of good correction effect. In addition, in the disclosed embodiment, the image correction is a mapping diagram generated directly based on the correspondence between the field of view and the distortion rate of the image acquisition module, which is applicable to any image acquired by the image acquisition module, and thus has the characteristics of a wide range of applications. At the same time, there is no need for a large number of complex calculations, and only the coordinate conversion of the acquired pixel values ​​needs to be performed according to the mapping diagram, which has the characteristics of simple calculation and high correction efficiency.

[0067] In some embodiments, the mapping map can be generated dynamically or in advance. If it is generated in advance, when it is used, it can be directly read from the pre-generated mapping map, thereby saving the time consumed by image correction and improving the efficiency of image correction. Therefore, in some embodiments, S120 may include: reading a mapping map pre-generated based on the correspondence between the field of view and the distortion rate of the image acquisition module.

[0068] Regardless of whether the mapping map is dynamically generated, pre-generated, or generated when the image acquisition module is first used, the mapping map is generated. The following provides a method for generating a mapping map, which can be combined with Figure 1 The image correction method shown can be used together or alone.

[0069] like Figure 3 As shown, a method for generating a mapping diagram may include:

[0070] S210: Acquire a corresponding relationship; wherein the corresponding relationship may include: a mapping relationship between N discretely distributed fields of view of the image acquisition module and the distortion rate, wherein N is a positive integer greater than or equal to 2;

[0071] S220: Generate an FD curve based on the interpolation algorithm and the corresponding relationship; wherein the FD curve is used to determine the distortion rate corresponding to the field of view where each original pixel coordinate in the image captured by the image acquisition module is located;

[0072] S230: Generate the mapping diagram based on the FD curve, the first distance and the second distance, wherein the first distance is: the distance between the original pixel coordinates and the center point of the image; the second distance is: the distance between the target pixel coordinates and the center point.

[0073] The interpolation algorithm used here may include at least one of the following:

[0074] Linear interpolation algorithm;

[0075] Spline interpolation algorithm.

[0076] The above two interpolation algorithms are simple to use and have a close correspondence with the field of view and distortion rate of the image sensor. Therefore, the FD curve obtained by using the above interpolation algorithm has a high degree of conformity with the actual situation of the image acquisition module.

[0077] The generation of the FD curve can query the corresponding distortion rate based on each field of view corresponding to the original pixel coordinates, thereby realizing the correction of each original pixel coordinate on the image acquisition module.

[0078] Referring to Formula 1, the target pixel coordinates corresponding to each original pixel coordinate can be calculated:

[0079] Distortion=(AD-PD) / PD, formula (1).

[0080] Wherein, Distortion represents the distortion rate. AD is the first distance between the original pixel coordinates and the center point of the image. PD is the second distance between the corrected target pixel coordinates and the center point of the image.

[0081] Since the distortion rate and the first distance are known, the second distance can be obtained.

[0082] According to the second distance, the target pixel coordinates can be further calculated.

[0083] In one implementation, when a plurality of candidate pixel coordinates having pixel coordinates equal to the second distance from the center point are obtained according to the second distance, the target pixel coordinates closest to the original pixel coordinates may be selected.

[0084] In another implementation, according to the principle of similar triangles, based on the first distance and the second distance, the original pixel coordinates, and the pixel coordinates of the center point, a unique target pixel coordinate can be solved.

[0085] In some embodiments, Figure 4As shown, when the target pixel coordinate of any original pixel coordinate is determined based on the similar triangle principle, the target pixel coordinate of any original pixel coordinate can be quickly solved based on the formula (2) corresponding to the similar triangle principle.

[0086]

[0087] In formula (2), AD is the first distance, PD is the second distance, (C X ,C Y ) is the pixel coordinate of the center point, (X AD ,Y AD ) is the original pixel coordinate, and they are all known, then the target pixel coordinate (X PD ,Y PD ).

[0088] In other embodiments, the target pixel coordinates may be determined based on a drawing method based on the principle of similar triangles.

[0089] Drawing method 1:

[0090] A first triangle is established based on the original pixel coordinates, the coordinates of the center point, and the coordinate axis passing through the center point;

[0091] Based on the pixel coordinates of the center point as a vertex of the triangle, and with the straight line connecting the original pixel coordinates and the center point as a side of the second triangle, a second triangle similar to the first triangle is constructed; the vertex of the second triangle located on the straight line connecting the original pixel coordinates and the center point is the point where the target pixel coordinates are located.

[0092] Drawing method 2:

[0093] On the straight line connecting the original pixel coordinates and the center point, a length equal to the length from the center point and equal to the second distance is intercepted, and the intercepted pixel coordinates are the target pixel coordinates; and the interception direction is: the direction from the center point to the original pixel coordinates.

[0094] In some embodiments, the S240 may include:

[0095] Determining the field of view of each original pixel coordinate of the image acquisition module;

[0096] Querying the FD curve based on the field of view to determine the distortion rate of each original pixel coordinate;

[0097] Determining a target pixel coordinate corresponding to each original pixel coordinate of the image sensor based on a functional relationship among the distortion rate, the first distance, and the second distance;

[0098] The mapping diagram is generated based on each of the original pixel coordinates and the target pixel coordinates corresponding to each of the original pixel coordinates.

[0099] In an image captured by an image acquisition module, the fields of view of different pixel coordinates are different. In the disclosed embodiment, the field of view of each pixel coordinate in the captured image is first determined, and then the distortion rate of each original pixel coordinate is determined based on the generated FD curve. Then, the target pixel coordinate is obtained in combination with formula (1), and then the mapping diagram is generated. That is, formula (1) is an example of the functional relationship between the distortion rate, the first distance and the second distance.

[0100] Determining the field of view of each original pixel coordinate of the image acquisition module includes:

[0101] Determine the pixel coordinates and the maximum acquisition radius of the center point of the image sensor; wherein the field of view of the pixel coordinates of the center point is a first preset value; the field of view of the original pixel coordinates whose distance from the pixel coordinates of the center point is equal to the maximum acquisition radius is a second preset value; and the difference between the second preset value and the first preset value is the field of view range value of the image sensor;

[0102] Determine the distance between each original pixel coordinate collected by the image sensor and the pixel coordinate of the center point;

[0103] Determine the field of view ratio of each original pixel coordinate according to the distance between each original pixel coordinate and the center point and the maximum acquisition radius;

[0104] The field of view of each original pixel coordinate is determined based on the field of view ratio and the field of view range value.

[0105] In some embodiments, the maximum collection radius is calculated using formula (3).

[0106] refer to Figure 5 As shown, the viewfinder lens of the image acquisition module is circular, and the inscribed rectangle of the image acquisition module is the maximum image size that can be captured by the image sensor.

[0107] The maximum image size and the pixel coordinates of the center point of the image can be known when the image acquisition module leaves the factory. In view of this, the maximum acquisition radius can be R = sqrt((Max(X)-C(X))*(Max(X)-C(X))+(Max(Y)-C(Y))*(Max(Y)-C(Y))).

[0108] The coordinate of the maximum image size on the X-axis may be Max(X), and the maximum coordinate on the Y-axis may be Max(Y); the Y-axis is perpendicular to the X-axis; the coordinate of the center point of the image on the X-axis may be C(X), and the coordinate on the Y-axis may be C(Y); the center point of the image corresponds to the position of the optical center of the image acquisition module.

[0109] For example, with a maximum image size of 4000*3000, the default lens center and the physical center of the image sensor coincide with each other; then the maximum acquisition radius is R=sqrt((4000-2000)*(4000-2000)+(3000-1500)*(3000-1500))=2500.

[0110] In some embodiments, the first preset value may be 0.0 F, and the second preset value may be 1.0 F. Thus, the second preset value minus the first preset value is the field of view range value, which may be 1.0 F. Wherein, F represents the field of view.

[0111] In some embodiments, the field of view ratio of each original pixel coordinate may be calculated based on formula (4).

[0112] ratio=sqrt((x-2000)*(x-Cx)+(y-1500)*(y-Cy)) / R formula (4)

[0113] (x, y) represents any original pixel coordinate in the image; (Cx, Cy) represents the pixel coordinate of the center point of the image. Ratio represents the field of view ratio of the original pixel coordinate (x, y).

[0114] Thus, the field of view of each original pixel coordinate is field_new=ratio*1.0, and the formula is (5). field_new is the original pixel coordinate.

[0115] The embodiment of the present disclosure provides a method for performing image correction based on the lens distortion coefficient of a lens.

[0116] The general distortion model needs to use a calibration algorithm to obtain the parameters of the camera to achieve distortion correction. The method of the embodiment of the present disclosure does not require any manual intervention or calibration plates and calibration cloths required by the calibration algorithm. It only needs the FD table (full name: Field Distortion table, which reflects the curvature information of the lens) provided by the lens manufacturer.

[0117] The general correction method cannot solve irregular distortion, that is, it will fail when the distortion state of different fields of view is different. This method can perform distortion correction according to different fields of view.

[0118] The method disclosed in the present invention can quickly and accurately establish a mapping relationship between the distortion domain and the correction domain, and can remove distortion in real time. Taking an image of 2000*1500 as an example, it only takes 20ms to complete the dedistortion processing using the method provided in the embodiment of the present invention.

[0119] As shown in Table 1, Field represents different fields of view, and Distortion represents distortion rate. Different lenses have different FD tables, and Table 1 is just a reference table. The calculation formula is: Distortion = (real image height - ideal image height) / ideal image height.

[0120] Generally, the center of the lens is considered to be the center point of the rectangular image. The four corner edge areas are black shadows without information because they are not actually in the field of view. The actual imaging range is a circle with the center point as the center. The radius of the actual imaging area can be obtained as Figure 5 shown.

[0121] When R is the radius, the corresponding field of view is 1.0, and the center position is 0 field of view; other Field fields of view (such as 0.1F, 0.2F) are the ratio of the distance of a pixel in the image from the center of the image to the maximum acquisition radius R.

[0122] Fitting generates FD curve (FD curve, i.e. Field Distortion curve, establishes a rectangular coordinate system with the x-axis as Field and the y-axis as Distortion)

[0123] Taking the image resolution of 4000*3000 as an example, the default lens center and the physical center of the image sensor coincide with each other; the pixel coordinates (pixel coordinates can be referred to as coordinates) of the center point of the image are: (2000,1500).

[0124] The maximum collection radius is R=sqrt((4000-2000)*(4000-2000)+(3000-1500)*(3000-1500))=2500, the corresponding Field is 1.0F, Distortion=1.58%.

[0125] For any pixel coordinate (x, y), the following formula can be used to find out which field of view the pixel is in:

[0126] ratio=sqrt((x-2000)*(x-2000)+(y-1500)*(y-1500)) / 2500;

[0127] The field of view is recorded as: field_new=ratio*1.0;

[0128] Then, according to Table 1 (FD table), the corresponding Distortion value under the new field of view field_new is found and recorded as k_new.

[0129] Since only 11 pairs of discrete points can be obtained from the lens manufacturer, when calculating a pixel located at the 0.35F field of view, it will be found that the corresponding Distortion value cannot be found in the FD table. At this time, it is necessary to use a linear interpolation algorithm to fit an FD curve, so that the relationship between field and distortion can be found directly through the lookup table.

[0130] A mapping diagram is generated, which is a mapping conversion relationship diagram between the distortion map and the correction map.

[0131] (1) Original image: the unprocessed image captured by the camera, i.e. the above-mentioned distorted image;

[0132] (2) Correction image: The image generated by correcting the distortion of the original image.

[0133] like Figure 2 As shown in the figure, the black dots in the figure can be regarded as the original pixel coordinates of the pixels in the original image without any processing, so there is distortion, especially the pixels at the edge are severely bent, and the circles can be regarded as the pixel coordinates of the pixels in the correction image after the distortion is corrected. The "cross" seen in the image represents the center point of the image, AD represents the distortion point (point A, with coordinates (X AD , Y AD )) to the center of the image, referred to as the distortion distance; PD represents the correction point (point P, with coordinates (X PD ,Y PD )) is the distance from the image center, referred to as the correction distance; the relationship between AD and PD can be calculated using the Distortion mentioned in Step 1, then: Distortion = (AD-PD) / PD.

[0134] In the above description, for an unprocessed image captured by a camera, when accessing any distorted pixel (taking pixel A as an example), the mapping process of solving its corresponding corrected pixel is as follows:

[0135] The known parameters are:

[0136] a. Distortion distance AD

[0137] b. The field of view Field_new where the distortion point A is located: the calculation formula is Field_new = AD / R. Where R is the radius of the image.

[0138] c. According to the FD curve, you can directly find the Distortion value corresponding to Fiele_new;

[0139] d. According to the formula Distortion = (AD-PD) / PD, PD can be calculated;

[0140] e. The coordinates (x, y) of point P can be obtained according to PD. The calculation method is as follows:

[0141] The destination pixel coordinates of the coordinates.

[0142]

[0143] In formula (2), AD is the first distance, PD is the second distance, (C X ,C Y ) is the pixel coordinate of the center point, (X AD ,Y AD ) is the original pixel coordinate, and they are all known, then the target pixel coordinate (X PD ,Y PD ).

[0144] When traversing the complete secondary image, the mapping map between each distorted pixel and the correction point can be stored.

[0145] By reading the original image and the distortion correction map, the correction image can be generated and output in real time.

[0146] Using the above method, a 2000*1500 image can be dedistorted in just 20ms.

[0147] The method provided in the embodiment of the present disclosure has the following characteristics:

[0148] 1) The general distortion model needs to use the calibration algorithm to obtain the camera parameters to achieve distortion correction. This method does not require any manual intervention or the calibration plate and calibration cloth required by the calibration algorithm. It only needs the factory FD table provided by the lens manufacturer.

[0149] 2) The general correction method cannot solve irregular distortion, that is, it will fail when the distortion state of different fields of view is different. This method can perform adaptive distortion correction according to different fields of view, and the experimental results show that the correction effect is better.

[0150] 3) This method can quickly and accurately establish the mapping relationship between the distortion domain and the correction domain, and can remove distortion in real time (for example, a 2000*1500 image can be dedistorted in only 20ms).

[0151] like Figure 6As shown, the embodiment of the present disclosure provides an image correction device, which may include:

[0152] A first acquisition module 610 is used to acquire a first image acquired by an image acquisition module;

[0153] The second acquisition module 620 is used to acquire a mapping diagram generated based on the corresponding relationship between the field of view and the distortion rate of the image acquisition module; wherein the mapping diagram includes: a mapping relationship between the original pixel coordinates of each pixel value in the original image acquired by the image acquisition module and the corrected target pixel coordinates;

[0154] The generating module 630 is configured to correct the first image based on the mapping image to obtain a corrected second image.

[0155] In some embodiments, the first acquisition module 610, the second acquisition module 620, and the generation module 630 may include a program module; after the program module is executed by the processor, the functions of any of the above modules can be realized.

[0156] In some embodiments, the first acquisition module 610, the second acquisition module 620, and the generation module 630; the soft-hard combination module includes but is not limited to a programmable array; the programmable array includes but is not limited to: a complex programmable array and / or a field programmable array.

[0157] In some embodiments, the first acquisition module 610, the second acquisition module 620, and the generation module 630; the pure hardware module includes but is not limited to a dedicated integrated circuit.

[0158] In some embodiments, the second acquisition module 620 is specifically used to read a mapping map pre-generated based on the correspondence between the field of view and the distortion rate of the image acquisition module.

[0159] In some embodiments, the device comprises:

[0160] A third acquisition module is used to acquire the corresponding relationship; wherein the corresponding relationship includes: a mapping relationship between N discretely distributed fields of view of the image acquisition module and the distortion rate, wherein N is a positive integer greater than or equal to 2;

[0161] An FD curve module, used to generate an FD curve based on an interpolation algorithm and the corresponding relationship; wherein the FD curve is used to determine the distortion rate corresponding to the field of view where each original pixel coordinate in the image captured by the image acquisition module is located;

[0162] A mapping module is used to generate the mapping based on the FD curve, a first distance and a second distance; wherein the first distance is: the distance between the original pixel coordinates and the center point of the image; and the second distance is: the distance between the target pixel coordinates and the center point.

[0163] In some embodiments, the mapping module is specifically used to determine the field of view of each original pixel coordinate of the image acquisition module; query the FD curve based on the field of view to determine the distortion rate of each original pixel coordinate; determine the target pixel coordinate corresponding to each original pixel coordinate of the image sensor based on the functional relationship between the distortion rate, the first distance and the second distance; generate the mapping based on each original pixel coordinate and the target pixel coordinate corresponding to each original pixel coordinate.

[0164] In some embodiments, the mapping module is specifically used to determine the pixel coordinates and the maximum acquisition radius of the center point of the image sensor; wherein the field of view of the pixel coordinates of the center point is a first preset value; the field of view of the predetermined pixel coordinates is a second preset value, wherein the predetermined pixel coordinates are: the pixel coordinates whose distance from the pixel coordinates of the center point is equal to the maximum acquisition radius; the difference between the second preset value and the first preset value is the field of view range value of the image sensor; determine the distance between each original pixel coordinate collected by the image sensor and the pixel coordinates of the center point; determine the field of view ratio of each original pixel coordinate based on the distance between each original pixel coordinate and the center point and the maximum acquisition radius; determine the field of view of each original pixel coordinate based on the field of view ratio and the field of view range value.

[0165] The present disclosure provides a mobile electronic device, which may include:

[0166] a memory for storing processor-executable instructions;

[0167] A processor connected to the memory;

[0168] The processor is configured to execute the image correction method and / or mapping image generation method provided by any of the aforementioned technical solutions.

[0169] The processor may include various types of storage media, which are non-transitory computer storage media that can continue to retain information stored thereon after the communication device loses power.

[0170] The mobile terminal may include, but is not limited to, various mobile terminals such as a mobile phone, a tablet, or a projection device, or electronic devices such as a fixed terminal.

[0171] The processor can be connected to the memory through a bus or the like, and is used to read the executable program stored in the memory, for example, to execute the following Figure 1 and / or Figure 3 At least one of any of the methods shown.

[0172] Figure 7 8 is a block diagram of a mobile terminal 800 according to an exemplary embodiment. For example, the mobile terminal 800 may be a mobile phone or a mobile computer.

[0173] Reference Figure 7 The mobile terminal 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0174] The processing component 802 generally controls the overall operation of the mobile terminal 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0175] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the mobile terminal 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0176] The power component 806 provides power to various components of the mobile terminal 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the mobile terminal 800.

[0177] The multimedia component 808 includes a screen that provides an output interface between the mobile terminal 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating state, such as a shooting state or a video state, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0178] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the mobile terminal 800 is in an operating state, such as a call state, a recording state, and a voice recognition state, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0179] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0180] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the mobile terminal 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the mobile terminal 800, the sensor assembly 814 can also detect the position change of the mobile terminal 800 or a component of the mobile terminal 800, the presence or absence of user contact with the mobile terminal 800, the orientation or acceleration / deceleration of the mobile terminal 800, and the temperature change of the mobile terminal 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications.

[0181] In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0182] The communication component 816 is configured to facilitate wired or wireless communication between the mobile terminal 800 and other devices. The mobile terminal 800 can access a wireless network based on a communication standard, such as Wi-Fi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0183] In an exemplary embodiment, the mobile terminal 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.

[0184] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the mobile terminal 800 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0185] The present disclosure provides a non-transitory computer-readable storage medium. When the instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal can execute the image correction method provided by any of the above embodiments, for example, it can execute the following steps: Figure 1 and / or Figure 3 At least one of any of the methods shown.

[0186] The image correction method may include: obtaining a first image captured by an image acquisition module; obtaining a mapping diagram generated based on the correspondence between the field of view and the distortion rate of the image acquisition module; wherein the mapping diagram includes: a mapping relationship between the original pixel coordinates of each pixel value in the original image captured by the image acquisition module and the corrected target pixel coordinates; based on the mapping diagram, correcting the first image to obtain a corrected second image.

[0187] It can be understood that the acquiring of the mapping map generated based on the correspondence between the field of view and the distortion rate of the image acquisition module includes: reading a mapping map generated in advance based on the correspondence between the field of view and the distortion rate of the image acquisition module.

[0188] It can be understood that the generation of the mapping diagram includes: obtaining the corresponding relationship; wherein the corresponding relationship includes: the mapping relationship between N discretely distributed fields of view of the image acquisition module and the distortion rate, wherein N is a positive integer greater than or equal to 2; generating an FD curve based on an interpolation algorithm and the corresponding relationship; wherein the FD curve is used to determine the distortion rate corresponding to the field of view of each original pixel coordinate in the image captured by the image acquisition module; generating the mapping diagram based on the FD curve, a first distance and a second distance; wherein the first distance is: the distance between the original pixel coordinate and the center point of the image; the second distance is: the distance between the target pixel coordinate and the center point.

[0189] It can be understood that the generation of the mapping diagram based on the FD curve, the first distance and the second distance includes: determining the field of view of each original pixel coordinate of the image acquisition module; querying the FD curve based on the field of view to determine the distortion rate of each original pixel coordinate; determining the target pixel coordinate corresponding to each original pixel coordinate of the image sensor based on the functional relationship between the distortion rate, the first distance and the second distance; and generating the mapping diagram based on each original pixel coordinate and the target pixel coordinate corresponding to each original pixel coordinate.

[0190] It can be understood that determining the field of view of each original pixel coordinate of the image acquisition module includes: determining the pixel coordinates of the center point of the image sensor and the maximum acquisition radius; wherein the field of view of the pixel coordinates of the center point is a first preset value; the field of view of the predetermined pixel coordinates is a second preset value, wherein the predetermined pixel coordinates are: the pixel coordinates whose distance from the pixel coordinates of the center point is equal to the maximum acquisition radius; the difference between the second preset value and the first preset value is the field of view range value of the image sensor; determining the distance between each original pixel coordinate acquired by the image sensor and the pixel coordinates of the center point; determining the field of view ratio of each original pixel coordinate based on the distance between each original pixel coordinate and the center point and the maximum acquisition radius; determining the field of view of each original pixel coordinate based on the field of view ratio and the field of view range value.

[0191] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0192] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image correction method, characterized in that: The method comprises: Acquire a first image captured by an image acquisition module; Determining the field of view of each original pixel coordinate of the image acquisition module; Acquire a mapping diagram generated based on the correspondence between the field of view and the distortion rate of the image acquisition module; wherein the mapping diagram includes: a mapping relationship between each original pixel coordinate in the original image acquired by the image acquisition module and the corrected target pixel coordinate; the correspondence includes a field of view distortion FD curve reflecting the curvature information of the image acquisition module; the FD curve is used to characterize the correspondence between the field of view and the distortion rate; the distortion rate is the quotient of the difference between the first distance and the second distance and the second distance; the first distance is the distance between the corresponding original pixel coordinate and the center point of the image; the second distance is the distance between the target pixel coordinate and the center point of the image, and the center point of the image corresponds to the optical center of the image acquisition module; Based on the mapping diagram, correcting the first image to obtain a corrected second image; Wherein, determining the field of view of each original pixel coordinate of the image acquisition module includes: Determine the pixel coordinates and the maximum acquisition radius of the center point of the image; wherein the field of view of the pixel coordinates of the center point of the image is a first preset value; the field of view of the predetermined pixel coordinates is a second preset value, wherein the predetermined pixel coordinates are: the pixel coordinates whose distance from the pixel coordinates of the center point of the image is equal to the maximum acquisition radius; the difference between the second preset value and the first preset value is the distance between each original pixel coordinate acquired by the image acquisition module and the pixel coordinates of the center point of the image, which is determined based on the field of view range value of the image acquisition module; Determine the field of view ratio of each original pixel coordinate according to the distance between each original pixel coordinate and the center point of the image and the maximum acquisition radius; The field of view of each original pixel coordinate is determined based on the field of view ratio and the field of view range value.

2. The method according to claim 1, characterized in that: The acquiring of a mapping diagram generated based on a correspondence between the field of view of the image acquisition module and the distortion rate includes: A mapping map generated in advance based on the correspondence between the field of view and the distortion rate of each original pixel coordinate is read.

3. The method according to claim 1 or 2, characterized in that: The generation of the mapping diagram includes: Acquire the corresponding relationship; wherein the corresponding relationship includes: a mapping relationship between N discretely distributed fields of view of the image acquisition module and the distortion rate, wherein N is a positive integer greater than or equal to 2; Based on the interpolation algorithm and the corresponding relationship, the FD curve is generated; wherein the FD curve is used to determine the distortion rate corresponding to the field of view where each original pixel coordinate in the image captured by the image acquisition module is located; The map is generated based on the FD curve, the first distance, and the second distance.

4. The method according to claim 3, characterized in that The generating the mapping diagram based on the FD curve, the first distance and the second distance comprises: Querying the FD curve based on the field of view to determine the distortion rate of each original pixel coordinate; Determining the target pixel coordinates corresponding to each original pixel coordinate of the image acquisition module based on a functional relationship among the distortion rate, the first distance, and the second distance; The mapping diagram is generated based on each of the original pixel coordinates and the target pixel coordinates corresponding to each of the original pixel coordinates.

5. An image correction device, characterized in that: The device comprises: A first acquisition module, used to acquire a first image acquired by the image acquisition module; A mapping module, used to determine the field of view of each original pixel coordinate of the image acquisition module; A second acquisition module is used to acquire a mapping diagram generated based on the correspondence between the field of view and the distortion rate of the image acquisition module; wherein the mapping diagram includes: a mapping relationship between each original pixel coordinate in the original image acquired by the image acquisition module and the corrected target pixel coordinate; the corresponding relationship includes a field of view distortion FD curve reflecting the curvature information of the image acquisition module; the FD curve is used to characterize the correspondence between the field of view and the distortion rate; the distortion rate is the quotient of the difference between the first distance and the second distance and the second distance; the first distance is the distance between the corresponding original pixel coordinate and the center point of the image; the second distance is the distance between the target pixel coordinate and the center point of the image, and the center point of the image corresponds to the optical center of the image acquisition module; A generating module, configured to correct the first image based on the mapping diagram to obtain a corrected second image; Among them, the mapping module is specifically used to determine the pixel coordinates of the center point of the image and the maximum acquisition radius; wherein the field of view of the pixel coordinates of the center point of the image is a first preset value; the field of view of the predetermined pixel coordinates is a second preset value, wherein the predetermined pixel coordinates are: the pixel coordinates whose distance from the pixel coordinates of the center point of the image is equal to the maximum acquisition radius; the difference between the second preset value and the first preset value is the distance between each original pixel coordinate collected by the image acquisition module and the pixel coordinates of the center point of the image determined based on the field of view range value of the image acquisition module; according to the distance between each original pixel coordinate and the center point of the image and the maximum acquisition radius, the field of view ratio of each original pixel coordinate is determined; based on the field of view ratio and the field of view range value, the field of view of each original pixel coordinate is determined.

6. The device according to claim 5, characterized in that The second acquisition module is used to read a mapping map generated in advance based on the correspondence between the field of view and the distortion rate of each original pixel coordinate.

7. The device according to claim 5 or 6, characterized in that The device comprises: A third acquisition module is used to acquire the corresponding relationship; wherein the corresponding relationship includes: a mapping relationship between N discretely distributed fields of view of the image acquisition module and the distortion rate, wherein N is a positive integer greater than or equal to 2; An FD curve module, used to generate the FD curve based on an interpolation algorithm and the corresponding relationship; wherein the FD curve is used to determine the distortion rate corresponding to the field of view where each original pixel coordinate in the image captured by the image acquisition module is located; The mapping module is specifically configured to generate the mapping based on the FD curve, the first distance, and the second distance.

8. The device according to claim 7, characterized in that The mapping module is specifically used to query the FD curve based on the field of view to determine the distortion rate of each original pixel coordinate; based on the functional relationship between the distortion rate, the first distance and the second distance, determine the target pixel coordinate corresponding to each original pixel coordinate of the image acquisition module; The mapping diagram is generated based on each of the original pixel coordinates and the target pixel coordinates corresponding to each of the original pixel coordinates.

9. An electronic device, characterized in that: include: a memory for storing processor-executable instructions; A processor connected to the memory; The processor is configured to execute the image correction method as claimed in any one of claims 1 to 4.

10. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of a computer, the computer is enabled to perform the image correction method according to any one of claims 1 to 4.

Images