Image distortion correction method and device, camera module and storage medium

Abstract

Embodiments of the present application disclose a kind of image distortion correction method, device, camera module and storage medium, the method includes obtaining the parameter of camera module and the image to be corrected, according to the parameter of camera module, determine the ideal mapping point corresponding to actual mapping point in the image to be corrected, according to the distortion offset of actual mapping point and ideal mapping point, determine the distortion correction intensity of actual mapping point, according to the correction coordinate of ideal mapping point, according to the correction coordinate of actual mapping point, to be corrected image is corrected by assigning value to actual mapping point.Therefore, by determining the distortion offset and corresponding distortion correction intensity of different actual mapping points in the image to be corrected, the corresponding actual mapping point is assigned value according to the correction coordinate under different distortion correction intensity, adaptive correction of different actual mapping points is realized, the loss rate of field angle can be reduced, and the distortion image correction effect is optimized.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and specifically to an image distortion correction method, apparatus, camera module, and storage medium. Background Technology

[0002] With the development and advancement of technology, many scenarios now require short-range, wide-angle imaging applications. This necessitates shorter focal lengths for camera lenses to capture a wider field of view. However, this also reduces the optical path in the peripheral areas outside the central imaging region, increasing the difficulty of designing image distortion correction. Consequently, the image may not be displayed according to the shape of the illuminated object, especially around the edges, resembling a barrel shape. Objects around the edges of the captured image also show distortion, resulting in a decrease in image quality and a significant negative impact on the product.

[0003] In existing technologies, camera calibration methods can be used to correct distortion in wide-angle cameras. However, the data obtained using camera calibration methods has significant errors when the image exhibits uneven distortion, resulting in poor distortion correction and a large loss of field of view, thus failing to allow users to experience the ultra-wide vision of a wide-angle camera. Summary of the Invention

[0004] In view of this, the present invention aims to provide an image distortion correction method, apparatus, camera module and storage medium to reduce the image distortion field of view loss rate and optimize the distortion correction effect.

[0005] In a first aspect, embodiments of the present invention aim to provide an image distortion correction method, the method comprising:

[0006] Obtain the parameters of the camera module and the image to be corrected, wherein the image to be corrected is an image captured by the camera module and includes at least one actual mapping point;

[0007] Determine the ideal mapping point corresponding to the actual mapping point based on the parameters;

[0008] The distortion correction intensity of the actual mapping point is determined based on the distortion offset between the actual mapping point and the ideal mapping point.

[0009] The correction coordinates of the ideal mapping point are determined based on the distortion correction intensity.

[0010] The actual mapping points are assigned values ​​based on the correction coordinates to correct the image to be corrected.

[0011] Further, determining the distortion correction intensity of the actual mapping point based on the distortion offset between the actual mapping point and the ideal mapping point includes:

[0012] The distortion offset of the actual mapping point is determined based on the image coordinates of the actual mapping point and the image coordinates of the ideal mapping point.

[0013] The distortion correction intensity of the actual mapping point is determined based on the distortion offset.

[0014] Further, determining the distortion offset of the actual mapping point based on the image coordinates of the actual mapping point and the image coordinates of the ideal mapping point includes:

[0015] The Eulerian distance between the actual mapping point and the ideal mapping point is determined based on the image coordinates of the actual mapping point and the ideal mapping point.

[0016] The Euler distance is determined as the distortion offset.

[0017] Further, determining the distortion correction intensity of the actual mapping point based on the distortion offset between the actual mapping point and the ideal mapping point includes:

[0018] The pixel attributes of the actual mapping point are determined based on the distortion offset between the actual mapping point and the ideal mapping point. The pixel attributes are used to characterize whether the actual mapping point is an edge point or a non-edge point.

[0019] The distortion correction intensity of the corresponding actual mapping point is determined based on the pixel attributes.

[0020] Further, determining the pixel attributes of the actual mapped point based on the distortion offset between the actual mapped point and the ideal mapped point includes:

[0021] When the distortion offset is less than or equal to a preset distance threshold, the corresponding actual mapping point is determined to be a non-edge point;

[0022] When the distortion offset is greater than a preset distance threshold, the corresponding actual mapping point is determined as an edge point.

[0023] Further, the distortion correction intensity includes a first component correction intensity and a second component correction intensity, and the step of determining the distortion correction intensity of the corresponding actual mapping point based on the pixel attributes includes:

[0024] When the actual mapping point is a non-edge point, the first component correction intensity and the second component correction intensity are determined according to the first preset intensity and the second preset intensity;

[0025] When the actual mapping point is an edge point, the first component correction intensity is determined based on the image coordinates of the actual mapping point and the first preset intensity, and the second component correction intensity is determined based on the image coordinates of the actual mapping point and the second preset intensity.

[0026] Furthermore, the image coordinates of the actual mapping point include a first coordinate and a second coordinate. The smaller the distance from the first coordinate to the image edge, the smaller the first correction intensity; the smaller the distance from the second coordinate to the image edge, the smaller the second correction intensity.

[0027] Further, determining the first component correction intensity based on the image coordinates of the actual mapping point and the first preset intensity includes:

[0028] When the first coordinate is greater than zero and less than the first preset state value, the first difference between the first preset state value and the first coordinate, the first ratio between the first preset state value and the first difference are determined sequentially, and the product of the first ratio and the first preset intensity is determined as the first component correction intensity.

[0029] When the first coordinate is greater than the difference between the first boundary value and the first preset state value but less than the first boundary value, a second difference between the first boundary value and the first preset state value, a third difference between the first coordinate and the second difference, and a second ratio between the first preset state value and the third difference are determined sequentially. The product of the second ratio and the first preset intensity is then determined as the first component correction intensity.

[0030] Determining the second component correction intensity based on the image coordinates of the actual mapping point and the second preset intensity includes:

[0031] When the second coordinate is greater than zero and less than the second preset state value, the fourth difference between the second preset state value and the second coordinate, the third ratio between the second preset state value and the fourth difference are determined sequentially, and the product of the third ratio and the second preset intensity is determined as the second component correction intensity.

[0032] When the second coordinate is greater than the difference between the second boundary value and the second preset state value but less than the second boundary value, the fifth difference between the second boundary value and the second preset state value, the sixth difference between the second coordinate and the fifth difference, and the fourth ratio between the second preset state value and the sixth difference are determined in sequence, and the product of the fourth ratio and the second preset intensity is determined as the second component correction intensity.

[0033] Furthermore, the method also includes:

[0034] When the image to be corrected is a non-infrared image, the image to be corrected is corrected.

[0035] Furthermore, the acquisition of camera module parameters includes:

[0036] The camera module is calibrated based on the calibration image captured by the calibration board to obtain the parameters of the camera module.

[0037] Further, determining the ideal mapping point corresponding to the actual mapping point based on the parameters includes:

[0038] The product of the actual mapping point and the parameter is determined to determine the ideal mapping point corresponding to the actual mapping point.

[0039] Secondly, embodiments of the present invention aim to provide an image distortion correction device, the device comprising:

[0040] A parameter acquisition unit is used to acquire parameters of the camera module and an image to be corrected, wherein the image to be corrected is an image captured by the camera module and includes at least one actual mapping point.

[0041] A mapping point determination unit is used to determine the ideal mapping point corresponding to the actual mapping point based on the parameters.

[0042] The correction intensity determination unit is used to determine the distortion correction intensity of the actual mapping point based on the distortion offset between the actual mapping point and the ideal mapping point;

[0043] The correction coordinate determination unit is used to determine the correction coordinates of the ideal mapping point based on the distortion correction intensity.

[0044] The correction unit is used to assign values ​​to the actual mapping points according to the correction coordinates in order to correct the image to be corrected.

[0045] Thirdly, embodiments of the present invention aim to provide a camera module, the camera module comprising:

[0046] The image acquisition unit is configured to acquire the image to be corrected.

[0047] A controller is configured to execute computer program instructions, wherein the computer program instructions are executed by the controller to implement the method as described in any of the preceding methods.

[0048] Fourthly, embodiments of the present invention aim to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0049] The technical solution of this invention obtains the parameters of a camera module and the image to be corrected. Based on the camera module parameters, it determines the ideal mapping point corresponding to the actual mapping point in the image to be corrected. Based on the distortion offset between the actual and ideal mapping points, it determines the distortion correction intensity of the actual mapping point. Based on the distortion correction intensity, it determines the correction coordinates of the ideal mapping point. Finally, it assigns values ​​to the actual mapping points based on the correction coordinates to correct the image. Therefore, by determining the distortion offset and corresponding distortion correction intensity for different actual mapping points in the image to be corrected, and then assigning values ​​to the corresponding actual mapping points based on the correction coordinates under different distortion correction intensities, adaptive correction of different actual mapping points is achieved. This reduces the field-of-view loss rate and optimizes the distortion correction effect. Attached Figure Description

[0050] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0051] Figure 1 This is a flowchart of the image distortion correction method according to an embodiment of the present invention;

[0052] Figure 2 This is a flowchart illustrating the determination of distortion correction intensity according to an embodiment of the present invention;

[0053] Figure 3 This is a flowchart illustrating the determination of distortion correction intensity based on distortion offset according to an embodiment of the present invention;

[0054] Figure 4 This is another flowchart of the image distortion correction method according to an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of an image distortion correction device according to an embodiment of the present invention;

[0056] Figure 6 This is a schematic diagram of a camera module according to an embodiment of the present invention. Detailed Implementation

[0057] The present invention is described below based on embodiments, but the invention is not limited to these embodiments. In the detailed description of the invention below, certain specific details are described in detail. Those skilled in the art will fully understand the invention even without these details. To avoid obscuring the essence of the invention, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0058] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0059] Unless the context explicitly requires it, words such as "including" or "contains" in the instruction manual should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0060] In the description of this invention, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0061] The solutions described in this specification and embodiments, if involving the processing of personal information, will be processed only under the premise of having a legal basis (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be processed within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0062] Existing methods for correcting distortion in wide-angle cameras using camera calibration suffer from significant errors when uneven image distortion occurs, resulting in a large field-of-view loss and poor image distortion correction. Therefore, this invention aims to provide an image distortion correction method to reduce field-of-view loss during image distortion and optimize image distortion correction.

[0063] Figure 1 This is a flowchart of an image distortion correction method according to an embodiment of the present invention. Figure 1 As shown, the image distortion correction method in this embodiment includes the following steps:

[0064] In step S110, the parameters of the camera module and the image to be corrected are obtained.

[0065] In this embodiment, the camera module parameters are inherent parameters of the camera module. The image to be corrected is an image captured by the camera module, including at least one actual mapping point.

[0066] Optionally, in this embodiment, when obtaining the parameters of the camera module, the camera module is calibrated based on the calibration image captured by the camera module on the calibration board to obtain the parameters of the camera module. The calibration method can employ existing camera calibration methods, such as Zhang's calibration method. The parameters of the camera module include intrinsic and extrinsic parameters. Intrinsic parameters can be represented by a 3x3 parameter matrix, while extrinsic parameters can include rotation and translation matrices, represented by 3x3 and 3x1 parameter matrices respectively. Intrinsic parameters can be used to convert data from camera coordinates to image coordinates. Extrinsic parameters can be used to convert data from world coordinates to camera coordinates. The world coordinate system refers to the user-defined coordinate system of the three-dimensional world, which describes the position of the target object in the real world. The camera coordinate system refers to the coordinate system established on the camera, used to describe the position of the object from the camera's perspective. The image coordinate system, also known as the pixel coordinate system, is used to describe the projection relationship of the object from the camera coordinate system to the image coordinate system during the imaging process.

[0067] Furthermore, to facilitate image output, the image to be corrected in this embodiment adopts the NV12 format. NV12 is an image format supported by the Internet, and the corresponding image data includes Y component data and UV component data. The Y component data is used to store the image's brightness information, and the UV component data is used to store the image's color information, where U represents blue projection information and V represents red projection information. Therefore, by using the above-mentioned format for the image to be corrected, during image transmission, it is unnecessary to set up three separate video transmission signals for the YUV components; instead, different component data can be transmitted simultaneously, occupying very little bandwidth during image transmission and achieving higher transmission efficiency.

[0068] It should be understood that, due to the relationships between different component data in an image of the same format, and the ability to convert between different component data in images of different formats, the image to be corrected in this embodiment can also be an RGB image or an HSV image. Taking an RGB image as an example, the R component data can be compared to the U component data, and the GB component data can be compared to the UV component data. Below, the distortion correction process of the NV12 format image to be corrected will continue to be explained as an example, providing a reference for image correction methods of other formats.

[0069] It should be noted that NV12 format images acquired by the camera module in non-infrared mode include three component data: Y, U, and V. NV12 format images acquired in infrared mode only include the Y component data. Therefore, in this embodiment, when performing distortion correction on the image to be corrected, the Y component data in the image to be corrected is corrected first.

[0070] In step S120, the ideal mapping point corresponding to the actual mapping point is determined based on the parameters.

[0071] In this embodiment, the actual mapping point is a pixel in the image to be corrected that may be distorted, and the ideal mapping point is the pixel corresponding to the actual mapping point that is not distorted. The actual mapping point and the ideal mapping point are represented by image coordinates (i.e., pixel coordinates) in the image. After determining the parameters of the camera module, this embodiment determines the ideal mapping point corresponding to the actual mapping point based on the intrinsic and / or extrinsic parameters in the camera module parameters.

[0072] Optionally, in this embodiment, the Y component data of the image to be corrected is extracted, the product of the actual mapping point and the camera module parameters is determined based on the Y component data, and the ideal mapping point corresponding to the actual mapping point is determined based on the product. The specific calculation formula is as follows:

[0073] Ideal mapping point of Y component = Actual mapping point of Y component × Camera parameters

[0074] In step S130, the distortion correction intensity of the actual mapping point is determined based on the distortion offset between the actual mapping point and the ideal mapping point.

[0075] Figure 2 This is a flowchart illustrating the determination of distortion correction intensity according to an embodiment of the present invention. For example... Figure 2 As shown, this embodiment includes the following steps when determining the distortion correction intensity of the actual mapping point.

[0076] In step S210, the distortion offset of the actual mapping point is determined based on the image coordinates of the actual mapping point and the image coordinates of the ideal mapping point.

[0077] In this embodiment, the distortion offset of the actual mapping point is determined based on the image coordinates of the actual mapping point and the ideal mapping point in the calibration image. Furthermore, when determining the distortion offset, the Eulerian distance between the actual mapping point and the ideal mapping point is determined based on their image coordinates, and this Eulerian distance is used as the distortion offset.

[0078] In step S220, the distortion correction intensity of the actual mapping point is determined based on the distortion offset.

[0079] In this embodiment, since the actual mapping points at different locations exhibit varying degrees of distortion, applying the same distortion correction intensity would result in overcorrection of pixels with minimal distortion or undercorrection of pixels with severe distortion. Therefore, this embodiment characterizes the degree of distortion at different actual mapping points based on the distortion offset, determines the corresponding distortion correction intensity based on the different degrees of distortion, and applies the appropriate correction intensity based on the distortion correction intensity of the actual mapping point, thereby improving correction accuracy.

[0080] Figure 3 This is a flowchart illustrating the determination of distortion correction intensity based on distortion offset, according to an embodiment of the present invention. Figure 3 As shown, in this embodiment, the distortion correction intensity of the actual mapping point is determined through the following steps, which can improve the accuracy of the distortion correction intensity at different locations.

[0081] In step S310, the pixel attributes of the actual mapping point are determined based on the distortion offset between the actual mapping point and the ideal mapping point.

[0082] In this embodiment, pixel attributes are used to characterize whether an actual edge point is an edge point or a non-edge point. Edge points represent locations at the image edge, while non-edge points represent other locations in the image besides edge points. Based on the barrel-like shape formed under the wide-angle shooting principle, the larger the distortion offset, the closer the corresponding actual mapped point is to the image boundary, and the greater the probability that the actual mapped point is an edge point. Conversely, assuming a smaller distortion offset, the corresponding actual mapped point is farther from the image boundary, and the greater the probability that the actual mapped point is a non-edge point.

[0083] Optionally, in this embodiment, a preset distance threshold DisThr0 is pre-set, and the pixel attributes of the actual mapping point are determined based on the relationship between the distortion offset CurrDistance and the preset distance threshold DisThr0. Further, in this embodiment, when the distortion offset is less than or equal to the preset distance threshold, the corresponding actual mapping point is determined to be a non-edge point; or when the distortion offset is greater than the preset distance threshold, the corresponding actual mapping point is determined to be an edge point. Therefore, by processing edge points and non-edge points separately, the distortion correction intensity of the actual mapping points at different positions in the image can be determined. This avoids pixels at the edge of the image to be corrected (i.e., edge points) being replaced by pixels not close to the edge (i.e., non-edge points), thereby reducing the reduction in the field of view of the corrected image, which is beneficial for optimizing the distortion correction effect and improving the ultra-wide visual experience when using a wide-angle camera module.

[0084] It should be understood that the preset distance threshold in this embodiment can be set according to actual shooting experience and the imaging principle of the camera module, so as to quickly and accurately determine the pixel attributes of the actual mapping point according to the relationship between the distortion offset of the actual mapping point and the preset distance threshold, thereby improving the efficiency of subsequent image distortion correction.

[0085] In step S320, the distortion correction intensity of the corresponding actual mapping point is determined based on the pixel attributes.

[0086] In this embodiment, to improve the accuracy of distortion correction for pixels at different locations, after determining the pixel attributes of the actual mapped point based on the Y component data, the embodiment determines the corresponding distortion correction intensity based on the pixel attributes. The distortion correction intensity includes a first component correction intensity and a second component correction intensity. In some embodiments, the first component correction intensity can be the correction intensity in the image width direction, and the second component correction intensity can be the correction intensity in the image height direction; or, in other embodiments, the first component correction intensity can be the correction intensity in the image height direction, and the second component correction intensity can be the correction intensity in the image width direction, without specific limitations. Therefore, by determining the first component correction intensity and the second component correction intensity of the actual mapped point with different pixel attributes, the width and height directions of the actual mapped point can be corrected separately, ensuring the accuracy and completeness of image distortion correction.

[0087] For ease of description, the following explanation will use the first component correction intensity as the correction intensity in the image width direction and the second component correction intensity as the correction intensity in the image height direction.

[0088] Optionally, when the actual mapped point is a non-edge point (CurrDistance≤DisThr0), the intensity is determined according to the first preset intensity K. w0 Second preset strength K h0 Determine the first component correction intensity K w Second component correction intensity K h When the actual mapping point is an edge point, the first component correction intensity is determined based on the distance from the actual mapping point to the first image edge and a first preset intensity, and the second component correction intensity is determined based on the distance from the actual mapping point to the second image edge and a second preset intensity. The first image edge is the image edge closest to the actual mapping point in a first direction, and the second image edge is the image edge closest to the actual mapping point in a second direction.

[0089] Assuming the actual mapping point coordinates are (u0, v0), then based on the image coordinates (u0, v0) of the actual mapping point and the first preset intensity K... w0 Determine the first component correction intensity K w Based on the image coordinates (u0, v0) and the second preset intensity K h0 Determine the second component correction intensity K h .

[0090] Furthermore, for non-edge points (CurrDistance≤DisThr0), the first preset intensity K is respectively... w0 Second preset strength K h0 The first component correction intensity K was determined to be... wSecond component correction intensity K h That is, when the actual mapping point is a non-edge point, K w =K w0 K h =K w0 .

[0091] For an edge point (CurrDistance>DisThr0), the first component correction intensity K is determined at the edge point. w In this embodiment, the distance from the actual mapping point to the image edge is determined based on the image coordinates of the actual mapping point, and the first component correction intensity is determined based on the distance. The first component correction intensity is positively correlated with the distance; the smaller the distance, the closer to the boundary, and the smaller the corresponding first component correction intensity.

[0092] Optionally, in this embodiment, the image coordinates of the actual mapped point include a first coordinate in the first direction, and the coordinate values ​​in the first direction corresponding to the two edges of the image in the second direction are 0 and a first boundary value Width, respectively. The span of the first edge region in the first direction is determined by a preset first preset state value buff. w Representation. The distance between the actual mapped point and the edge of the first image can be represented by the first preset state value buff. w The first difference buff with the first coordinate u0 w -u0 can be used to reflect this, or it can be used to determine the first boundary value Width and the first preset state value buff. w The second difference Width-buff w Then, the first coordinate u0 and the second difference Width-buff are used to... w The third difference u0-(Width-buff) w This is reflected in the first difference buff. w A larger -u0 indicates a smaller distance between the actual mapped point and the image edge on the side with image coordinates of 0 in the first direction, resulting in a smaller correction intensity for the corresponding first component. Similarly, when the third difference u0-(Width-buff) is larger... w The larger the value, the smaller the distance between the actual mapping point and the image edge on the Width side of the first direction, and the smaller the corresponding first component correction intensity.

[0093] Furthermore, in this embodiment, the first component correction intensity K is determined through the following steps. w In the actual mapped point, the first coordinate u0 in the image coordinates (u0, v0) is less than the first preset state value buff. w At that time, the first preset state value buff is determined sequentially. wThe first difference buff with the first coordinate u0 w -u0, and the first preset state value buff w Difference buff with the first value w -u0's first ratio buff w / (buff w -u0), and buff the first ratio. w / (buff w -u0) and the first preset intensity K w0 The product K w0 [buff w / (buff w -u0)] is determined as the first component correction intensity K w Alternatively, if the first coordinate u0 is greater than the first boundary value Width and the first preset state value buff... w The difference between Width-buff w When the value is less than the first boundary value Width, the first boundary value Width and the first preset state value buff are determined sequentially. w The second difference Width-buff w The first coordinate u0 and the second difference Width-buff w The third difference u0-(Width-buff) w ), and the first preset state value buff w The difference with the third value u0-(Width-buff) w The second ratio buff w / [u0-(Width-buff w )], and buff the second ratio. w / [u0-(Width-buff w )] and the first preset intensity K w0 The product of these factors is determined as the first component correction intensity. Therefore, by segmenting the process, the distortion correction intensity of different edge points in the width direction is determined, making the distortion correction intensity of edge points at different locations more realistic. This is beneficial for further improving the accuracy of distorted image correction and optimizing the distorted image correction effect.

[0094] For ease of explanation, the above determination rules are expressed by the following formula.

[0095] When 0 <u0<buff w At that time, K w =K w0 [buff w / (buff w -u0)];

[0096] When Width - buff w <u0 < Width, K w = K w0 {buff w / [u0 - (Width - buff w )]};

[0097] Where, u0 is the coordinate of the pixel point in the width direction of the image, corresponding to the first coordinate; buff w is the preset state value in the width direction, corresponding to the first preset state value; Width is the boundary value in the width direction, corresponding to the first boundary value; the width coordinate intervals [0, buff w and [Width - buff w , Width] are respectively used to represent the edge regions near the two sides of the image; buff w <Width - buff w ; K w is the correction intensity in the width direction, corresponding to the first component correction intensity; K w0 is the preset correction intensity in the width direction, corresponding to the first preset intensity.

[0098] Optionally, the image coordinates in this embodiment further include a second coordinate in the height direction. The coordinate values corresponding to the two sides of the image in the second direction are 0 and the second boundary value Height respectively. The span of the second edge region in the second direction is represented by a preset second preset state value buff h . The distance between the actual mapping point and the image edge can be reflected by the fourth difference buff h - v0 between the second preset state value buff h and the second coordinate v0, or after determining the fifth difference Height - buff h between the second boundary value Height and the second preset state value buff h , it can be reflected by the sixth difference v / (Height - buff h ) corresponding to the second coordinate v0 and the fifth difference Height - buff h . When the fourth difference buff h - v0 is larger, it indicates that the actual mapping point is closer to the image edge on the side where the image coordinate in the second direction is 0, and the corresponding second component correction intensity is smaller. Similarly, when the sixth difference v0 - (Height - buff h ) is larger, it indicates that the actual mapping point is closer to the image edge on the side where the image coordinate in the second direction is Height, and the corresponding second component correction intensity is smaller.

[0099] Furthermore, in this embodiment, the second component correction intensity K is determined through the following steps. h In the actual mapped point, the second coordinate v0 in the image coordinates (u0, v0) is greater than or less than the second preset state value buff. h At that time, the second preset state value buff is determined sequentially. h The fourth difference buff with the second coordinate v0 h -v0, the second preset state value buff h Difference with the fourth (buff) h The third ratio buff of -v0) h / (buff h -v0), and buff the third ratio. h / (buff h -v0) and the second preset intensity K h0 The product K h0 [buff h / (buff h -v0)] is determined as the second component correction intensity K h Alternatively, if the second coordinate v0 is greater than the second boundary value Height and the second preset state value buff... h The difference Height-buff h When the value is less than the second boundary value Height, the second boundary value Height and the second preset state value buff are determined sequentially. h The fifth difference Height-buff h The difference between the second coordinate v0 and the fifth value is Height-buff. h The sixth difference v0-(Height-buff) h ), and the second preset state value buff h The difference with the sixth value v0-(Height-buff) h The fourth ratio buff h / [v0-(Height-buff h )], and buff the fourth ratio. h / [v0-(Height-buff h )] and the second preset intensity K h0 The product K h0 {buff h / [v0-(Height-buff h The second component correction intensity K was determined. hThus, by means of segmented processing, the distortion correction intensity of different edge points in the height direction is determined, making the distortion correction intensity of edge points at different positions more in line with the actual situation, which is beneficial to further improving the correction accuracy of distorted images and optimizing the distortion image correction effect.

[0100] Meanwhile, for the sake of convenience of explanation, the above determination rule is expressed by the following formula.

[0101] When 0 < v0 < buff h , K h = K h0 [buff h / (buff h - v0)];

[0102] When Height - buff h < v0 < Height, K h = K h0 {buff h / [v0 - (Height - buff h )]};

[0103] Among them, v0 is the coordinate of the pixel point in the height direction of the image, corresponding to the second coordinate; buff h is the preset state value in the height direction, corresponding to the second preset state value; Height is the boundary value in the height direction, corresponding to the second boundary value; the height coordinate intervals [0, buff h and [Height - buff h , Height] are respectively used to represent the edge regions near the two sides of the image, buff h < Height - buff h ; K h is the correction intensity in the height direction, corresponding to the second component correction intensity; K h0 is the preset correction intensity in the height direction, corresponding to the second preset intensity.

[0104] It should be noted that the first preset state value buff w and the second preset state value buff h are respectively the preset coordinate thresholds in the width direction and height direction of the image, the first preset intensity K w0 and the second preset intensity K h0These are the preset distortion correction intensities in the width and height directions of the image, respectively. The first boundary value Width and the second boundary value Height are the boundary coordinate values ​​of the image to be corrected in the width and height directions, respectively. The preset coordinate thresholds and preset distortion correction intensities can be determined based on actual correction experience to better meet correction requirements and further optimize the image distortion correction effect.

[0105] Furthermore, in some other possible embodiments, the first component correction intensity is the correction intensity in the image height direction, and the second component correction intensity is the correction intensity in the image width direction. In this case, the first coordinate is the first coordinate of the image height, and the second coordinate is the coordinate in the image width direction. The method for determining the correction intensity in the image height direction is the method for determining the first component correction intensity, and the method for determining the correction intensity in the image width direction is the method for determining the second component correction intensity.

[0106] In step S140, the correction coordinates of the ideal mapping point are determined based on the distortion correction intensity.

[0107] In this embodiment, the first component correction intensity K of the actual mapping point is determined. w Second component correction intensity K h Then, based on the first component correction intensity K w Second component correction intensity K h The image coordinates of the ideal mapping point (u1, v1) corresponding to the actual mapping point are updated. The updated ideal mapping point coordinates are (u1', v1'), and u1' = K. w *u1, v1' = K h *v1. That is, the corrected coordinates of the ideal mapping point (u1,v1) are (K w *u1,K h *v1).

[0108] In step S150, the actual mapping points are assigned values ​​according to the correction coordinates in order to correct the image to be corrected.

[0109] In this embodiment, after determining the corrected coordinates (u1', v1') of the ideal mapping point corresponding to the actual mapping point, the corrected coordinates are assigned to the actual mapping point to correct the image to be corrected. It should be noted that u and v here are only used to represent the pixel coordinates corresponding to the pixel point, and are not the data corresponding to the U component and V component.

[0110] To ensure the effectiveness of the correction, the image distortion correction method in this embodiment determines whether the image to be corrected is a non-infrared image after determining the correction coordinates of the ideal mapping point corresponding to the actual mapping point. If the image to be corrected is a non-infrared image, other component data of the image to be corrected are corrected.

[0111] Optionally, in this embodiment, the image to be corrected can be determined to be a non-infrared or infrared image based on the number of pixels in the image to be corrected, or it can be determined based on the operating mode of the camera module (infrared mode or non-infrared mode). Further, in this embodiment, when determining whether the image to be corrected is a non-infrared or infrared image based on the number of pixels in the image to be corrected, the number of pixels in the image to be corrected is determined, and the number of pixels in the image to be corrected is judged. Based on the judgment result of the number of pixels, the correction of other component data is determined. Specifically, when the image to be corrected uses the aforementioned image format (NV12), in this embodiment, when the number of pixels in the image to be corrected is a first preset value, the image to be corrected after being assigned correction coordinates is corrected again; or, when the number of pixels in the image to be corrected is a second preset value, the correction operation of the image to be corrected after being assigned correction coordinates is ended.

[0112] Specifically, when the number of pixels in the image to be corrected is a first preset value, the corresponding image to be corrected is a non-infrared image, which simultaneously contains Y component data, U component data, and V component data. In this case, the UV component pixels need to be processed. Further, in this embodiment, after assigning values ​​to the Y component data of the actual mapping points according to the correction coordinates, the image to be corrected is then re-corrected on the U and V components. The U and V component data of the coordinate points in the image before distortion correction that are consistent with the correction coordinates are used as the corrected U and V component data of the actual mapping points to complete the correction of the image to be corrected. Here, it is assumed that the first preset value is n1, where n1 = Width * Height * 3 / 2, and Width and Height are the first boundary value and the second boundary value, respectively, used to characterize the maximum width and maximum height of the image to be corrected. Furthermore, assuming the image coordinates of the actual mapping point P are (u0, v0) and the corresponding corrected image coordinates are (u1', v1'), then the U and V components of the corrected point P correspond to the U and V component data of the pixel at coordinates (u1', v1'), respectively. Thus, the above processing method achieves image distortion correction of the image to be corrected in non-infrared mode, thereby determining a color image with the same format as the image to be corrected after distortion correction.

[0113] Alternatively, when the number of pixels in the image to be corrected is a second preset value, the corresponding image to be corrected is an infrared image. This image only contains Y component data and not U or V component data, so there is no need to process the UV component pixels. That is, after assigning values ​​to the Y component data of the actual mapping points according to the correction coordinates, the correction of the image is achieved. Here, the second preset value is assumed to be n2, where n2 = Width * Height. Width and Height are the first and second boundary values, respectively, used to characterize the maximum width and maximum height of the image to be corrected. Thus, the above processing method achieves image distortion correction of the image to be corrected in infrared mode, thereby determining an image with the same format as the original image after distortion correction.

[0114] The technical solution of this embodiment determines the distortion offset and corresponding distortion correction intensity corresponding to different actual mapping points in the image to be corrected. It then assigns values ​​to the corresponding actual mapping points based on the correction coordinates under different distortion correction intensities, achieving adaptive correction for different actual mapping points. This reduces the field-of-view loss rate and optimizes the distortion correction effect. Simultaneously, by judging the pixels in the image to be corrected and performing distortion correction on different component data, the accuracy of image distortion correction can be improved, further enhancing the image correction effect.

[0115] Figure 4 This is another flowchart of the image distortion correction method according to an embodiment of the present invention. For example... Figure 4 As shown, in this embodiment, distortion correction of the image to be corrected is achieved through the following steps.

[0116] In step S410, the parameters of the camera module are obtained.

[0117] In this embodiment, the parameters of the camera module are determined by capturing calibration images of the calibration board using the camera module.

[0118] In step S420, the number of pixels in the image to be corrected is obtained.

[0119] In this embodiment, after the corresponding image to be corrected is acquired by the camera module, the number of pixels in the image to be corrected is obtained, it is determined whether the number of pixels in the image to be corrected is a first preset value or a second preset value, and the corresponding correction strategy is determined according to the number of pixels in the image to be corrected.

[0120] Furthermore, in this embodiment, the first preset value is Width*Height*3 / 2, and the second preset value is Width*Height, where Width and Height represent the maximum width and maximum height of the image to be corrected, respectively. When the number of pixels is the first preset value, it indicates that the image to be corrected is a non-infrared image, which includes data in both the Y and UV components, requiring distortion correction for the Y and UV components separately. When the number of pixels is the second preset value, it indicates that the image to be corrected is an infrared image, which only includes Y component data, requiring distortion correction for only the Y component data.

[0121] In step S430, the ideal mapping point corresponding to the actual mapping point in the image to be corrected is determined based on the camera module parameters.

[0122] In this embodiment, when performing distortion correction on the Y component data, the ideal mapping point corresponding to the actual mapping point in the image to be corrected is determined based on the intrinsic and extrinsic parameters in the camera module parameters. The method for determining the ideal mapping point has been described above and will not be repeated here.

[0123] In step S440, the distortion offset of the actual mapping point is determined based on the image coordinates of the actual mapping point and the ideal mapping point.

[0124] In this embodiment, the Euler distance between the actual mapping point and the ideal mapping point is determined based on the image coordinates of the actual mapping point and the ideal mapping point, and the Euler distance is determined as the distortion offset.

[0125] In step S450, the pixel attributes of the corresponding actual mapping point are determined based on the distortion offset.

[0126] In this embodiment, pixel attributes are used to characterize whether an actual edge point is an edge point or a non-edge point. When determining pixel attributes, the pixel attributes of the actual mapped point are determined based on the relationship between the distortion offset and a preset distance threshold. When the distortion offset is less than or equal to the preset distance threshold, the corresponding actual mapped point is determined to be a non-edge point; when the distortion offset is greater than the preset distance threshold, the corresponding actual mapped point is determined to be an edge point.

[0127] In step S460, the distortion correction intensity is determined based on the pixel attributes.

[0128] In this embodiment, different strategies are used to determine the corresponding distortion correction intensity for edge points and non-edge points.

[0129] In step S461, when the actual mapping point is a non-edge point, the first component correction intensity and the second component correction intensity are determined according to the first preset intensity and the second preset intensity.

[0130] In this embodiment, for non-edge points, according to the first preset intensity K w0 Second preset strength K h0 Determine the first component correction intensity K w Second component correction intensity K h The specific rules for determining this can be found in the aforementioned content.

[0131] In step S462, when the actual mapping point is an edge point, the first component correction intensity is determined based on the image coordinates of the actual mapping point and the first preset intensity, and the second component correction intensity is determined based on the image coordinates of the actual mapping point and the second preset intensity.

[0132] In this embodiment, for edge points, the first coordinate u0 of the actual mapped point and the first preset intensity K are used. w0 Determine the first component correction intensity K w According to the second coordinate v0 and the second preset intensity K h0 Determine the second component correction intensity K h The specific rules for determining this can be found in the aforementioned content.

[0133] In step S470, the correction coordinates of the ideal mapping point are determined based on the correction intensity of the first component and the correction intensity of the second component.

[0134] In this embodiment, the correction intensity K of the first component is used. w Second component correction intensity K h The image coordinates of the ideal mapping point (u1, v1) corresponding to the actual mapping point are redefined, and the updated ideal mapping point coordinates are (u1', v1'), where u1' = K. w *u1, v1' = K h *v1. That is, the corrected coordinates of the ideal mapping point (u1,v1) are (K w *u1,K h *v1).

[0135] In step S480, the actual mapping points are assigned values ​​according to the correction coordinates in order to correct the image to be corrected.

[0136] In this embodiment, in one optional implementation, when the number of pixels in the image to be corrected is a first preset value (Width*Height*3 / 2), after assigning values ​​to the actual mapping points according to the correction coordinates, the U and V component data of the coordinate points in the image before distortion correction that are consistent with the correction coordinates are used as the U and V component data of the actual mapping points after correction. For example, assuming the actual mapping point image coordinates of pixel P are (u0, v0), and after reconfirmation, the ideal mapping point image coordinates of P are (u1', v1'), then the U component coordinates of P are the U component of the pixel point with coordinates (u1', v1') in the image before distortion correction; similarly, the V component of P is the V component of the pixel point with coordinates (u1', v1') in the image before distortion correction. Thus, by assigning coordinate values ​​to the Y component of the pixel point and correcting the UV components through the above-mentioned processing of the method components, image distortion correction on the YUV components of the image to be corrected is achieved.

[0137] In another optional implementation, when the number of pixels in the image to be corrected is a second preset value (Width*Height), this embodiment assigns values ​​to the actual mapping points according to the correction coordinates, and realizes the distortion correction of the image to be corrected after the assignment is completed.

[0138] The technical solution of this embodiment determines the distortion offset and corresponding distortion correction intensity corresponding to different actual mapping points in the image to be corrected. It then assigns values ​​to the corresponding actual mapping points based on the correction coordinates under different distortion correction intensities, achieving adaptive correction for different actual mapping points. This reduces the field-of-view loss rate and optimizes the distortion correction effect. Simultaneously, by judging the pixels in the image to be corrected and performing distortion correction on different component data, the accuracy of image distortion correction can be improved, further enhancing the image correction effect.

[0139] Figure 5 This is a schematic diagram of an image distortion correction device according to an embodiment of the present invention. Figure 5 As shown, the image distortion correction device in this embodiment includes a parameter acquisition unit 11, a mapping point determination unit 12, a correction intensity determination unit 13, a correction coordinate determination unit 14, and a correction unit 15. The parameter acquisition unit 11 acquires the parameters of the camera module and the image to be corrected. The image to be corrected is an image captured by the camera module and includes at least one actual mapping point. The mapping point determination unit 12 determines the ideal mapping point corresponding to the actual mapping point based on the parameters of the camera module. The correction intensity determination unit 13 determines the distortion correction intensity of the actual mapping point based on the distortion offset between the actual mapping point and the ideal mapping point. The correction coordinate determination unit 14 determines the correction coordinates of the ideal mapping point based on the distortion correction intensity. The correction unit 15 assigns values ​​to the actual mapping point according to the correction coordinates to correct the image to be corrected.

[0140] Optionally, the image distortion correction device in this embodiment further includes a correction determination unit 16. The correction determination unit 16 is used to correct the image to be corrected when the image to be corrected is a non-infrared image.

[0141] Figure 6 This is a schematic diagram of a camera module according to an embodiment of the present invention. Figure 6 As shown, the camera module in this embodiment includes an image acquisition unit 21 and a controller 22. The image acquisition unit 21 is configured to acquire an image to be corrected. The controller 22 is configured to execute computer program instructions, which are executed by the controller to implement the image distortion correction method described in any of the preceding claims.

[0142] The technical solution of this embodiment, after the image acquisition unit in the camera module acquires the image to be corrected, determines the distortion offset and corresponding distortion correction intensity corresponding to different actual mapping points in the image to be corrected. Based on the correction coordinates under different distortion correction intensities, values ​​are assigned to the corresponding actual mapping points, achieving adaptive correction for different actual mapping points. This reduces the field-of-view loss rate and optimizes the distortion correction effect. Simultaneously, by judging the pixels in the image to be corrected and performing distortion correction on different component data, the accuracy of image distortion correction can be improved, further enhancing the image correction effect. This allows the camera module to output high-quality distortion-corrected wide-angle images, improving the overall performance of the camera module.

[0143] Furthermore, the image distortion correction device or camera module disclosed in this embodiment can be applied to devices or equipment such as surveillance cameras and vehicles, without any specific limitations.

[0144] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] This application is described with reference to flowchart illustrations of methods, apparatus (devices), and computer program products according to embodiments of this application. It should be understood that each step in the flowchart can be implemented by computer program instructions.

[0146] These computer program instructions may be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction means, the implementation process of which is described in the instruction means. Figure 1 The function specified in one or more processes.

[0147] These computer program instructions may also be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, produce instructions for implementing processes. Figure 1 A device for a function specified in one or more processes.

[0148] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program for use by a computer to execute some or all of the above-described method embodiments.

[0149] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program specifying the relevant hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0150] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can be modified and varied in various ways. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image distortion correction method, characterized in that, The method includes: Obtain the parameters of the camera module and the image to be corrected, wherein the image to be corrected is an image captured by the camera module and includes at least one actual mapping point; Determine the ideal mapping point corresponding to the actual mapping point based on the parameters; The distortion correction intensity of the actual mapping point is determined based on the distortion offset between the actual mapping point and the ideal mapping point. The correction coordinates of the ideal mapping point are determined based on the distortion correction intensity. The actual mapping points are assigned values ​​according to the correction coordinates in order to correct the image to be corrected. The step of determining the distortion correction intensity of the actual mapping point based on the distortion offset between the actual mapping point and the ideal mapping point includes: The pixel attributes of the actual mapping point are determined based on the distortion offset between the actual mapping point and the ideal mapping point. The pixel attributes are used to characterize whether the actual mapping point is an edge point or a non-edge point. The distortion correction intensity of the corresponding actual mapping point is determined based on the pixel attributes. Determining the pixel attributes of the actual mapped point based on the distortion offset between the actual mapped point and the ideal mapped point includes: When the distortion offset is less than or equal to a preset distance threshold, the corresponding actual mapping point is determined to be a non-edge point; when the distortion offset is greater than the preset distance threshold, the corresponding actual mapping point is determined to be an edge point. The distortion correction intensity includes a first component correction intensity and a second component correction intensity. Determining the distortion correction intensity of the corresponding actual mapped point based on the pixel attributes includes: When the actual mapping point is a non-edge point, the first component correction intensity and the second component correction intensity are determined according to the first preset intensity and the second preset intensity, where the first preset intensity and the second preset intensity are preset distortion correction in the image width direction and the height direction, respectively; when the actual mapping point is an edge point, the first component correction intensity is determined according to the image coordinates of the actual mapping point and the first preset intensity, and the second component correction intensity is determined according to the image coordinates of the actual mapping point and the second preset intensity.

2. The method according to claim 1, characterized in that, The image coordinates of the actual mapping point include a first coordinate and a second coordinate. The smaller the distance from the first coordinate to the image edge, the smaller the correction intensity of the first component; the smaller the distance from the second coordinate to the image edge, the smaller the correction intensity of the second component.

3. The method according to claim 2, characterized in that, Determining the first component correction intensity based on the image coordinates of the actual mapping point and the first preset intensity includes: When the first coordinate is greater than zero and less than the first preset state value, the first difference between the first preset state value and the first coordinate, the first ratio between the first preset state value and the first difference are determined sequentially, and the product of the first ratio and the first preset intensity is determined as the first component correction intensity. When the first coordinate is greater than the difference between the first boundary value and the first preset state value but less than the first boundary value, a second difference between the first boundary value and the first preset state value, a third difference between the first coordinate and the second difference, and a second ratio between the first preset state value and the third difference are determined sequentially. The product of the second ratio and the first preset intensity is then determined as the first component correction intensity. Determining the second component correction intensity based on the image coordinates of the actual mapping point and the second preset intensity includes: When the second coordinate is less than the second preset state value, the fourth difference between the second preset state value and the second coordinate, the third ratio between the second preset state value and the fourth difference are determined in sequence, and the product of the third ratio and the second preset intensity is determined as the second component correction intensity. When the second coordinate is greater than the difference between the second boundary value and the second preset state value but less than the second boundary value, the fifth difference between the second boundary value and the second preset state value, the sixth difference between the second coordinate and the fifth difference, and the fourth ratio between the second preset state value and the sixth difference are determined in sequence, and the product of the fourth ratio and the second preset intensity is determined as the second component correction intensity.

4. The method according to claim 1, characterized in that, The method further includes: When the image to be corrected is a non-infrared image, the image to be corrected is corrected.

5. An image distortion correction device, characterized in that, The device includes: A parameter acquisition unit is used to acquire parameters of the camera module and an image to be corrected, wherein the image to be corrected is an image captured by the camera module and includes at least one actual mapping point. A mapping point determination unit is used to determine the ideal mapping point corresponding to the actual mapping point based on the parameters. The correction intensity determination unit is used to determine the distortion correction intensity of the actual mapping point based on the distortion offset between the actual mapping point and the ideal mapping point; The correction coordinate determination unit is used to determine the correction coordinates of the ideal mapping point based on the distortion correction intensity. A correction unit is used to assign values ​​to the actual mapping points according to the correction coordinates in order to correct the image to be corrected. The correction intensity determination unit is further configured to determine the pixel attributes of the actual mapping point based on the distortion offset between the actual mapping point and the ideal mapping point, wherein the pixel attributes are used to characterize whether the actual mapping point is an edge point or a non-edge point; and to determine the distortion correction intensity of the corresponding actual mapping point based on the pixel attributes. When determining the distortion correction intensity of the corresponding actual mapping point based on the pixel attributes, the correction intensity determination unit is further configured to determine the corresponding actual mapping point as a non-edge point when the distortion offset is less than or equal to a preset distance threshold; and to determine the corresponding actual mapping point as an edge point when the distortion offset is greater than the preset distance threshold. The distortion correction intensity includes a first component correction intensity and a second component correction intensity. The correction intensity determining unit is further configured to determine the first component correction intensity and the second component correction intensity based on a first preset intensity and a second preset intensity when the actual mapping point is a non-edge point, wherein the first preset intensity and the second preset intensity are preset distortion correction in the image width direction and the image height direction, respectively; and to determine the first component correction intensity based on the image coordinates of the actual mapping point and the first preset intensity when the actual mapping point is an edge point, and to determine the second component correction intensity based on the image coordinates of the actual mapping point and the second preset intensity.

6. A camera module, characterized in that, The camera module includes: The image acquisition unit is configured to acquire the image to be corrected. A controller is configured to execute computer program instructions, wherein the computer program instructions are executed by the controller to implement the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method steps of any one of claims 1-4.

Images