Image distortion correction method, device, equipment and medium

Abstract

The embodiments of the present application provide an image distortion correction method, device, equipment and medium, which are used to solve the problem that the image cannot be accurately and effectively corrected for distortion in the related art. In the embodiments of the present application, the electronic device obtains an image of a calibration plate set at a preset position and collected by a collection device; identifies the target corner point in the image, and identifies the corner point in the calibration plate that matches the target corner point, obtains the observed coordinates of the target corner point in the image, and based on the parameters of the collection device and the pre-saved coordinates of the corner point in the world coordinate system, determines the undistorted coordinates corresponding to the collection device when no distortion occurs; based on the observed coordinates and the undistorted coordinates, determines the distortion parameters of the collection device; and performs distortion correction on the image collected by the collection device based on the distortion parameters. Thus, the electronic device can accurately determine the distortion parameters of the collection device and accurately and effectively realize the correction of the image collected by the collection device.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image distortion correction method, device, equipment and medium. Background Art

[0002] Image distortion refers to the phenomenon that an image captured by an optical system (such as a camera lens) has deviations in geometric shape. This deviation may cause straight lines in the image to appear as curves, resulting in the shape of objects in the image being inconsistent with the actual scene. Common types of distortion include barrel distortion, pincushion distortion, and radial distortion. These distortions will affect the accuracy of images in the fields of photography, computer vision, and 3D reconstruction, so the correction of image distortion is an important research direction.

[0003] Researchers have proposed a variety of methods to correct image distortion, which usually restore the true geometric structure of the image through mathematical modeling and optimization algorithms. The polynomial distortion model is the most commonly used method, which describes the distortion in the image by introducing a series of distortion parameters. Usually, a radial distortion model is used to correct the distortion near the center of the lens. However, the polynomial distortion model may require higher-order polynomials when dealing with complex distortions, which increases the computational complexity. In addition, for severe distortions, the correction effect of this method may be limited.

[0004] In recent years, image distortion correction methods based on deep learning have gradually emerged. By training a large amount of calibration data, a neural network model that can automatically identify and correct image distortion is constructed. This method can perform correction when the lens parameters are unknown and has good versatility. However, this method relies on a large amount of calibration data for training. If the data is insufficient or of low quality, the correction effect will be greatly reduced. Summary of the invention

[0005] The embodiments of the present application provide an image distortion correction method, device, equipment and medium, which are used to solve the problem that the image distortion correction cannot be performed accurately and effectively in the related art.

[0006] In a first aspect, an embodiment of the present application provides a distortion correction method, the method comprising:

[0007] Acquire an image of a calibration plate set at a preset position acquired by an acquisition device;

[0008] Identify the target corner point in the image; obtain the observed coordinates of the target corner point in the image, identify the corner point in the calibration plate that matches the target corner point, and determine the undistorted coordinates corresponding to the corner point when the acquisition device is not distorted based on the parameters of the acquisition device and the pre-saved coordinates of the corner point in the world coordinate system;

[0009] Based on the observed coordinates and the undistorted coordinates, distortion parameters of the acquisition device are determined; and distortion correction is performed on the image acquired by the acquisition device based on the distortion parameters.

[0010] Further, the identifying target corner points in the image includes:

[0011] identifying each corner point in the image;

[0012] The Harris response value of each corner point is determined, and the corner point corresponding to the maximum response value is determined as the target corner point.

[0013] Further, identifying each corner point in the image includes:

[0014] For each pixel in the image, a circular area of ​​a preset size is determined with the pixel as the center, and for each other pixel in the circular area other than the pixel, the difference between the intensity value of the other pixel and the pixel is determined. If the difference exceeds a preset threshold, the other pixel is marked as a bright spot, otherwise, the other pixel is marked as a dark spot; according to the position of each other pixel in the image, each other pixel is sorted, and if there are a continuous preset number of other pixel points that are all marked as bright spots or all marked as dark spots, the pixel is determined as a corner point.

[0015] Further, the identifying the corner point in the calibration plate that matches the target corner point includes:

[0016] Determine an object descriptor of the object corner point in the image;

[0017] And determine the descriptor of each corner point of the calibration plate; determine the similarity between the descriptor of each corner point and the target descriptor; and determine the corner point with the highest similarity as the corner point in the calibration plate that matches the target corner point.

[0018] Further, the determining of the target descriptor of the target corner point in the image includes:

[0019] Determine a window of a preset size with the target corner point as the center; determine the centroid in the window, and determine the vector direction from the target corner point to the centroid as the main direction;

[0020] Constructing a rotation matrix corresponding to the angle of the main direction; rotating each pixel point in the window based on the rotation matrix; so as to rotate the pixel points in the window to a direction consistent with the main direction;

[0021] A preset number of pixel pairs in the rotated window are obtained. For each pixel pair, if the intensity value of the first pixel in the pixel pair is greater than the intensity value of the second pixel, the corresponding bit of the pixel pair in the target descriptor is recorded as the first identifier; otherwise, the corresponding bit of the pixel pair is recorded as the second identifier.

[0022] Further, determining the centroid in the window includes:

[0023] For each pixel point in the window, determine a first product of the horizontal coordinate of the pixel point and the intensity value of the pixel point, and determine a second product of the vertical coordinate of the pixel point and the intensity value of the pixel point;

[0024] Determine a first sum of each first product, a second sum of each second product, and determine a third sum of intensity values ​​of each pixel in the window;

[0025] The ratio of the first sum value to the third sum value is determined as the horizontal coordinate of the center of mass, and the ratio of the second sum value to the third sum value is determined as the vertical coordinate of the center of mass.

[0026] Furthermore, after acquiring the image of the calibration plate set at the preset position acquired by the acquisition device, and before identifying the observation coordinates of the preset corner points of the calibration plate in the image, the method further includes:

[0027] Perform illumination normalization processing on the image.

[0028] Furthermore, performing illumination normalization processing on the image includes:

[0029] Dividing the image into sub-blocks of a preset size;

[0030] Using histogram equalization to evenly distribute the intensity range of each sub-block;

[0031] The boundary of each sub-block after uniform distribution is processed by bilinear interpolation.

[0032] Furthermore, the uniformly distributing the intensity range of each sub-block by using histogram equalization includes:

[0033] For each gray level of each sub-block, determine the probability sum of the gray level and all gray levels below the gray level in the sub-block; for each pixel in the sub-block, determine the adjusted gray level of the pixel according to the product of the gray level corresponding to the pixel and the probability sum determined corresponding to the gray level, and adjust the pixel based on the adjusted gray level.

[0034] Further, determining the distortion parameters of the acquisition device based on the observed coordinates and the undistorted coordinates includes:

[0035] Determine the distorted coordinates corresponding to the undistorted coordinates based on the pre-stored distortion function, the initial distortion parameters and the undistorted coordinates;

[0036] Adjusting the distortion parameter based on the Gauss-Newton method until the error between the distortion coordinate determined based on the adjusted distortion parameter and the observation coordinate is minimized;

[0037] The distortion parameter corresponding to the minimum error is determined as the distortion parameter of the acquisition device.

[0038] In a second aspect, an embodiment of the present application further provides an image distortion correction device, the device comprising:

[0039] An acquisition module, used to acquire an image of a calibration plate set at a preset position acquired by an acquisition device;

[0040] A processing module, configured to identify a target corner point in the image; obtain the observed coordinates of the target corner point in the image, identify a corner point in the calibration plate that matches the target corner point, and determine the undistorted coordinates corresponding to the corner point when the acquisition device is not distorted based on the parameters of the acquisition device and the pre-saved coordinates of the corner point in the world coordinate system;

[0041] A determination module is used to determine the distortion parameters of the acquisition device based on the observed coordinates and the undistorted coordinates; and to perform distortion correction on the image acquired by the acquisition device based on the distortion parameters.

[0042] Furthermore, the processing module is specifically used to identify each corner point in the image; determine the Harris response value of each corner point, and determine the corner point corresponding to the maximum response value as the target corner point.

[0043] Furthermore, the processing module is specifically used to determine, for each pixel point in the image, a circular area of ​​a preset size with the pixel point as the center, and for each other pixel point other than the pixel point in the circular area, determine the difference between the intensity value of the other pixel point and the pixel point; if the difference exceeds a preset threshold, the other pixel point is marked as a bright spot; otherwise, the other pixel point is marked as a dark spot; each other pixel point is sorted according to its position in the image; if there are a continuous preset number of other pixel points that are all marked as bright spots or all marked as dark spots, the pixel point is determined as a corner point.

[0044] Furthermore, the processing module is specifically used to determine the target descriptor of the target corner point in the image; and determine the descriptor of each corner point of the calibration plate; determine the similarity between the descriptor of each corner point and the target descriptor; and determine the corner point with the highest similarity as the corner point in the calibration plate that matches the target corner point.

[0045] Furthermore, the processing module is specifically used to determine a window of a preset size with the target corner point as the center; determine the center of mass in the window, and determine the vector direction from the target corner point to the center of mass as the main direction; construct a rotation matrix corresponding to the angle of the main direction; rotate each pixel point in the window based on the rotation matrix; so as to rotate the pixel points in the window to a direction consistent with the main direction; obtain a preset number of pixel pairs in the rotated window, and for each pixel pair, if the intensity value of the first pixel point in the pixel pair is greater than the intensity value of the second pixel point, then the corresponding bit of the pixel pair in the target descriptor is recorded as the first identifier, otherwise, the corresponding bit of the pixel pair is recorded as the second identifier.

[0046] Furthermore, the processing module is specifically used to determine, for each pixel point in the window, a first product of the horizontal coordinate of the pixel point and the intensity value of the pixel point, and a second product of the vertical coordinate of the pixel point and the intensity value of the pixel point; determine a first sum of each first product, and a second sum of each second product, and determine a third sum of the intensity values ​​of each pixel point in the window; determine the ratio of the first sum to the third sum as the horizontal coordinate of the center of mass, and determine the ratio of the second sum to the third sum as the vertical coordinate of the center of mass.

[0047] Furthermore, the processing module is also used to perform illumination normalization processing on the image.

[0048] Furthermore, the processing module is specifically used to divide the image into sub-blocks of preset size; use histogram equalization to evenly distribute the intensity range of each sub-block; and perform bilinear difference processing on the boundary of each sub-block after even distribution.

[0049] Furthermore, the processing module is specifically used to determine, for each gray level of each sub-block, the probability sum of the gray level and all gray levels below the gray level in the sub-block; for each pixel in the sub-block, determine the adjusted gray level of the pixel according to the product of the gray level corresponding to the pixel and the probability sum determined corresponding to the gray level, and adjust the pixel based on the adjusted gray level.

[0050] Furthermore, the determination module is specifically used to determine the distorted coordinates corresponding to the undistorted coordinates based on a pre-saved distortion function, initial distortion parameters and the undistorted coordinates; adjust the distortion parameters based on the Gauss-Newton method until the error between the distorted coordinates determined based on the adjusted distortion parameters and the observed coordinates is minimized; and determine the distortion parameters corresponding to the minimum error as the distortion parameters of the acquisition device.

[0051] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0052] A memory for storing program instructions;

[0053] The processor is used to call the program instructions stored in the memory, and execute the steps included in any one of the above-mentioned methods for correcting image distortion according to the obtained program instructions.

[0054] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes a method for image distortion correction as described in any one of the above items.

[0055] In the embodiment of the present application, the electronic device acquires an image of a calibration plate set at a preset position acquired by an acquisition device; identifies a target corner point in the image, and identifies a corner point in the calibration plate that matches the target corner point, acquires the observed coordinates of the target corner point in the image, and determines the undistorted coordinates corresponding to the acquisition device when no distortion occurs based on the parameters of the acquisition device and the pre-saved coordinates of the corner point in the world coordinate system; determines the distortion parameters of the acquisition device based on the observed coordinates and the undistorted coordinates; and performs distortion correction on the image acquired by the acquisition device based on the distortion parameters. Thus, the electronic device can accurately determine the distortion parameters of the acquisition device and accurately and effectively correct the image acquired by the acquisition device. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A schematic diagram of a process of an image distortion correction method provided in an embodiment of the present application;

[0058] Figure 2 A detailed process diagram of image distortion correction provided by an embodiment of the present application;

[0059] Figure 3 A structural diagram of an image distortion correction device provided in an embodiment of the present application;

[0060] Figure 4 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] In order to solve the problem in the related art that it is impossible to accurately and effectively correct image distortion, the embodiments of the present application provide an image distortion correction method, device, equipment and medium.

[0063] The image distortion correction method comprises: acquiring an image of a calibration plate set at a preset position acquired by an acquisition device; identifying a target corner point in the image; acquiring the observed coordinates of the target corner point in the image, and identifying a corner point in the calibration plate that matches the target corner point, and determining the undistorted coordinates corresponding to the corner point when the acquisition device is not distorted based on the parameters of the acquisition device and the pre-saved coordinates of the corner point in the world coordinate system; determining the distortion parameters of the acquisition device based on the observed coordinates and the undistorted coordinates; and performing distortion correction on the image acquired by the acquisition device based on the distortion parameters. Example 1

[0064] Figure 1 A schematic diagram of a process of an image distortion correction method provided in an embodiment of the present application, the process includes the following steps:

[0065] S101: Acquire an image of a calibration plate set at a preset position acquired by an acquisition device.

[0066] The image distortion correction method provided in the embodiment of the present application is applied to an electronic device, which may be an intelligent device such as an acquisition device, a PC or a server.

[0067] In order to perform distortion correction on the image captured by the acquisition device, when image correction is required, the calibration plate can be set at a preset position, and the image containing the calibration plate can be captured by a high-precision acquisition device. The calibration plate can include a black square frame and a fixed-pitch pattern array contained therein, such as multiple rows and columns of solid circles or other geometric shapes.

[0068] S102: Identify the target corner point in the image; obtain the observed coordinates of the target corner point in the image, and identify the corner point in the calibration plate that matches the target corner point, and determine the undistorted coordinates corresponding to the corner point when the acquisition device is not distorted based on the parameters of the acquisition device and the pre-saved coordinates of the corner point in the world coordinate system.

[0069] The electronic device can identify the target corner points in the image. In a possible implementation, the electronic device can use the Kitchen-Rosenfeld corner detection algorithm based on the local gradient amplitude value and the gradient direction change rate on the boundary, and the Smallest Univalue Segment Assimilatating Nucleus (SUSAN) corner detection algorithm to identify each corner point in the image and obtain the observed coordinates of each corner point in the image. In a possible implementation, the electronic device can determine a randomly preset number of corner points in each corner point as the target corner point.

[0070] The electronic device also identifies corner points in the calibration plate that match the target corner points. In one possible implementation, the electronic device can input the image and calibration plate marked with the target corner points into a pre-trained model to obtain corner points in the calibration plate output by the model that match the target corner points.

[0071] When placing the calibration plate, business personnel can record the coordinates of each corner point in the world coordinate system. The electronic device can obtain the coordinates of the corner point that matches the target corner point in the world coordinate system, and based on the parameters of the acquisition device and the acquired coordinates, determine the undistorted coordinates corresponding to the corner point when the acquisition device is not distorted.

[0072] S103: Determine distortion parameters of the acquisition device based on the observed coordinates and the undistorted coordinates; and perform distortion correction on the image acquired by the acquisition device based on the distortion parameters.

[0073] After acquiring the observed coordinates and the undistorted coordinates, the electronic device may determine the distortion parameters of the acquisition device based on the observed coordinates and the undistorted coordinates.

[0074] Specifically, the observed coordinates and undistorted coordinates satisfy the following formula:

[0075]

[0076] Among them, x ， ,y ， are the horizontal and vertical coordinates of the observed coordinates, x and y are the horizontal and vertical coordinates of the undistorted coordinates, and r is the distance from the image pixel to the center of the image. Specifically, r 2=x 2 +y 2 , k 1 , k 2 , k 3 is the radial distortion parameter, p 1 、p 2 is the tangential distortion parameter.

[0077] In a possible implementation, the electronic device may store multiple distortion parameter groups, and the electronic device may determine whether there is a distortion parameter satisfying the above formula in each distortion parameter group. If so, the distortion parameter is determined as the distortion parameter of the acquisition device.

[0078] After determining the distortion parameters of the acquisition device, the electronic device can subsequently use the distortion parameters to perform distortion correction on the image acquired by the acquisition device. Specifically, the electronic device can apply the distortion correction formula based on the calculated distortion parameters to correct each pixel in the image to obtain a corrected image.

[0079] In the embodiment of the present application, the electronic device acquires an image of a calibration plate set at a preset position acquired by an acquisition device; identifies a target corner point in the image, and identifies a corner point in the calibration plate that matches the target corner point, acquires the observed coordinates of the target corner point in the image, and determines the undistorted coordinates corresponding to the acquisition device when no distortion occurs based on the parameters of the acquisition device and the pre-saved coordinates of the corner point in the world coordinate system; determines the distortion parameters of the acquisition device based on the observed coordinates and the undistorted coordinates; and performs distortion correction on the image acquired by the acquisition device based on the distortion parameters. Thus, the electronic device can accurately determine the distortion parameters of the acquisition device and accurately and effectively correct the image acquired by the acquisition device. Example 2

[0080] In order to identify the target corner points in the image, based on the above embodiment, in the embodiment of the present application, the identifying the target corner points in the image includes:

[0081] identifying each corner point in the image;

[0082] The Harris response value of each corner point is determined, and the corner point corresponding to the maximum response value is determined as the target corner point.

[0083] In order to identify the target corner points in the image, the electronic device may first identify each corner point in the image. In a possible implementation, the electronic device may use the Kitchen-Rosenfeld corner detection algorithm or the KLT corner detection algorithm or the SUSAN corner detection algorithm to identify each corner point in the image.

[0084] After acquiring each corner point in the image, the electronic device can perform non-maximum suppression on the detected corner points and calculate the Harris response value of each corner point. Specifically, how to calculate the Harris response value of the corner point is an existing technology and will not be repeated here. The electronic device can determine the corner point corresponding to the maximum response value as the target corner point.

[0085] In the embodiment of the present application, the electronic device performs non-maximum suppression on the detected corner points, which can effectively remove redundant corner points with low response values, ensure that only the most significant corner points are retained, and improve the accuracy and reliability of the feature points. Calculating the Harris response value and retaining the corner point with the maximum response value can further filter out the most distinguishing feature points. Example 3

[0086] In order to identify each corner point in the image, based on the above embodiments, in the embodiment of the present application, identifying each corner point in the image includes:

[0087] For each pixel in the image, a circular area of ​​a preset size is determined with the pixel as the center, and for each other pixel in the circular area other than the pixel, the difference in intensity value between the other pixel and the pixel is determined; if the difference exceeds a preset first threshold, the other pixel is marked as a bright spot; if the difference is lower than a preset second threshold, the other pixel is marked as a dark spot; wherein the first threshold is greater than the second threshold; each other pixel is sorted according to its position in the image, and if there are a continuous preset number of other pixel points that are all marked as bright spots or all marked as dark spots, the pixel is determined as a corner point.

[0088] In order to identify each corner point in the image, the electronic device can determine a circular area of ​​a preset size with the pixel point as the center of the circle for each pixel point in the image. Specifically, the circular area can also be called a circular neighborhood, and the radius of the circular neighborhood can be , the circular neighborhood contains a total of 16 pixels.

[0089] The electronic device can determine the difference in intensity value between each other pixel point other than the pixel point in the circular area and the other pixel point, and if the difference exceeds a preset first threshold, where the first threshold value can be 20, the other pixel point is marked as a bright spot; if the difference does not exceed a preset second threshold value, the other pixel point is marked as a dark spot, where the second threshold value can be -20.

[0090] The electronic device may sort each other pixel point according to the position of each other pixel point in the image, and if there are a continuous preset number of other pixel points that are all marked as bright spots or all marked as dark spots, the pixel point is determined as a corner point. The preset number may be 12. That is, if any group of 12 consecutive pixel points in the circular neighborhood is detected, and if these pixel points are all bright spots or all dark spots, the central pixel point is considered to be a corner point. Example 4

[0091] In order to identify the corner points in the calibration plate that match the target corner points, based on the above embodiments, in an embodiment of the present application, the identifying the corner points in the calibration plate that match the target corner points includes:

[0092] Determine an object descriptor of the object corner point in the image;

[0093] And determine the descriptor of each corner point of the calibration plate; determine the similarity between the descriptor of each corner point and the target descriptor; and determine the corner point with the highest similarity as the corner point in the calibration plate that matches the target corner point.

[0094] In order to identify the corner points in the calibration plate that match the target corner points, the electronic device may determine the target descriptor of the target corner points after determining the target corner points in the image.

[0095] The electronic device can also determine the descriptor of each corner point of the calibration plate; for each corner point, determine the similarity between the descriptor of the corner point and the target descriptor of the target corner point, thereby determining the similarity between each corner point and the target corner point; and determine the corner point with the highest similarity as the corner point in the calibration plate that matches the target corner point.

[0096] In order to identify the corner points in the calibration plate that match the target corner points, based on the above embodiments, in the embodiment of the present application, the target descriptor of the target corner points in the image is determined, including:

[0097] Determine a window of a preset size with the target corner point as the center; determine the centroid in the window, and determine the vector direction from the target corner point to the centroid as the main direction;

[0098] Constructing a rotation matrix corresponding to the angle of the main direction; rotating each pixel point in the window based on the rotation matrix; so as to rotate the pixel points in the window to a direction consistent with the main direction;

[0099] A preset number of pixel pairs in the rotated window are obtained. For each pixel pair, if the intensity value of the first pixel in the pixel pair is greater than the intensity value of the second pixel, the corresponding bit of the pixel pair in the target descriptor is recorded as the first identifier; otherwise, the corresponding bit of the pixel pair is recorded as the second identifier.

[0100] In order to ensure that the descriptor is invariant to the rotation of the image, the descriptor determined in the embodiment of the present application may be a Rotated Binary Robust Independent Basic Feature (ORB) descriptor.

[0101] Specifically, the electronic device may take a window of a preset size with the target corner point as the center, and calculate the centroid within the window, wherein the preset size is 31*31.

[0102] After determining the centroid, the electronic device may determine the direction vector from the target corner point to the centroid as the main direction, wherein when calculating the angle of the main direction, the slope corresponding to the straight line from the target corner point to the centroid may be determined based on the coordinates of the centroid and the coordinates of the target corner point, and the angle of the main direction may be determined based on the slope. Specifically, the electronic device may calculate the direction angle by the following formula:

[0103]

[0104] in, is the direction angle, is the horizontal coordinate of the center of mass, is the ordinate of the center of mass, is the horizontal coordinate of the target corner point, is the ordinate of the target corner point.

[0105] After determining the main direction, the electronic device can construct a rotation matrix corresponding to the angle of the main direction. The rotation matrix is ​​a two-dimensional rotation matrix used to describe the transformation of rotating the angle counterclockwise around the origin in a two-dimensional plane. Specifically, the rotation matrix can be: .

[0106] After determining the rotation matrix, the electronic device may rotate each pixel point in the window based on the rotation matrix, so that the pixel points in the window are all rotated to a direction consistent with the main direction.

[0107] Specifically, the electronic device can determine the coordinates of the rotated pixel point by the following formula:

[0108] ;

[0109] ;

[0110] in, is the pixel in the i-th pair of pixels The rotated coordinates, is the direction angle, Pixel The horizontal axis of Pixel The vertical coordinate of is the pixel in the i-th pair of pixels The rotated coordinates, Pixel The horizontal axis of Pixel The vertical coordinate of .

[0111] The electronic device can determine a preset number of pixel pairs based on the rotated window, wherein the preset number can be 256, and determine the descriptor based on the intensity value of the pixel of each pixel pair. Specifically, for each pixel pair, if the intensity value of the first pixel in the pixel pair is greater than the intensity value of the second pixel, the electronic device can record the corresponding bit of the pixel pair in the target descriptor as the first identifier, otherwise, record the corresponding bit of the pixel pair as the second identifier. The first identifier can be 1, and the second identifier can be 0.

[0112] For example, the rotated pixel pair ,if Then the descriptor The bit is 1, otherwise it is 0. Specifically, how to select pixel pairs is a prior art, which will not be described in detail here. Specifically, how to calculate the ORB descriptor of feature points is a prior art, which will not be described in detail here.

[0113] In the embodiment of the present application, the target corner point can also be called a feature point, and the electronic device determines the main direction of the feature point by calculating the centroid in the neighborhood of the feature point. This makes the feature point not only dependent on the position feature, but also contains direction information, which helps to maintain the consistency of the feature point under the condition of rotation transformation. And by calculating the descriptor using the main direction of the feature point, the descriptor can be made rotationally invariant. Thus, it has better robustness to rotation transformation during the matching process.

[0114] In order to identify the corner points in the calibration plate that match the target corner points, based on the above embodiments, in the embodiment of the present application, determining the centroid in the window includes:

[0115] For each pixel point in the window, determine a first product of the horizontal coordinate of the pixel point and the intensity value of the pixel point, and determine a second product of the vertical coordinate of the pixel point and the intensity value of the pixel point;

[0116] Determine a first sum of each first product, a second sum of each second product, and determine a third sum of intensity values ​​of each pixel in the window;

[0117] The ratio of the first sum value to the third sum value is determined as the horizontal coordinate of the center of mass, and the ratio of the second sum value to the third sum value is determined as the vertical coordinate of the center of mass.

[0118] In an embodiment of the present application, when determining the centroid, the electronic device may determine the product of the horizontal coordinate of each pixel point and the intensity value of the pixel point for each pixel point in the window, and for the sake of distinction, the product may be determined as the first product, and the product of the vertical coordinate of the pixel point and the intensity value of the pixel point may be determined, and for the sake of distinction, the product may be determined as the second product. The electronic device may determine the first sum of each first product and the second sum of each second product, and determine the third sum of the intensity values ​​of each pixel point in the window. The electronic device may determine the ratio of the first sum to the third sum as the horizontal coordinate of the centroid, and determine the ratio of the second sum to the third sum as the vertical coordinate of the centroid.

[0119] Specifically, the electronic device can determine the coordinates of the center of mass by the following formula:

[0120] ;

[0121] ;

[0122] in, is the horizontal coordinate of the centroid, and x is the horizontal coordinate of a pixel point in the window. is the intensity value of a pixel in the window, is the ordinate of the centroid, and y is the ordinate of a pixel point in the window. Example 5

[0123] In order to improve the accuracy of image correction, based on the above embodiments, in the embodiment of the present application, after acquiring the image of the calibration plate set at the preset position acquired by the acquisition device, and before identifying the observation coordinates of the preset corner points of the calibration plate in the image, the method further includes:

[0124] Perform illumination normalization processing on the image.

[0125] Since changes in ambient light often lead to inconsistent image lighting conditions during the image acquisition process, this phenomenon may seriously interfere with the subsequent precise matching of the calibration plate and the corner points in the image, thereby affecting the accuracy and reliability of image correction. In order to effectively improve the accuracy of corner point matching and the quality of the final image correction, the electronic device can perform lighting normalization on the image after receiving the image transmitted by the acquisition device. This processing aims to adjust the overall intensity and contrast of the image through an algorithm so that the lighting conditions in the image tend to be consistent, thereby significantly reducing the corner point recognition error caused by uneven lighting and ensuring that the calibration plate matches the corner points in the image more accurately and correctly.

[0126] In order to improve the accuracy of image correction, on the basis of the above embodiments, in the embodiment of the present application, the illumination normalization processing of the image includes:

[0127] Dividing the image into sub-blocks of a preset size;

[0128] Using histogram equalization to evenly distribute the grayscale range of each sub-block;

[0129] The boundary of each sub-block after uniform distribution is processed by bilinear interpolation.

[0130] In the embodiment of the present application, the electronic device can divide the image into sub-blocks of a preset size, wherein each sub-block does not overlap and the size of each sub-block is .

[0131] Furthermore, the electronic device may use histogram equalization to evenly distribute the grayscale range of each sub-block.

[0132] After adjusting the grayscale of each sub-block, the electronic device can perform bilinear interpolation on the sub-block boundary to smooth the transition and avoid the block effect, wherein the adjustment can be based on the bilinear difference weight of each pixel and the pixel of the equalized image. Specifically, the following formula can be used to perform bilinear interpolation on the sub-block boundary:

[0133]

[0134] in, is the equalized image of the sub-block, is the bilinear interpolation weight. Specifically, how to determine the bilinear difference weight is a prior art and will not be described in detail here.

[0135] In an embodiment of the present application, the image is subjected to illumination normalization processing before the target corner points are acquired. Illumination normalization enhances contrast by adjusting the histogram distribution of the image, reduces the impact of illumination changes on the image processing algorithm, and improves the performance of the descriptor in scenes with large illumination changes, thereby achieving high-quality image distortion correction. This method can effectively solve the problems of limited application scenarios in the prior art, and insufficient accuracy and stability of distortion correction. The method of the present invention performs illumination normalization processing on the image before feature point extraction. Illumination normalization enhances contrast by adjusting the histogram distribution of the image, and reduces the impact of illumination changes on the image processing algorithm. By calculating the center of mass within the neighborhood of the feature point, the main direction of the feature point is determined, so that the matching process has better robustness to rotation transformation.

[0136] In order to improve the accuracy of image correction, based on the above embodiments, in the embodiment of the present application, the use of histogram equalization to uniformly distribute the intensity range of each sub-block includes:

[0137] For each gray level of each sub-block, determine the probability sum of the gray level and all gray levels below the gray level in the sub-block; for each pixel in the sub-block, determine the adjusted gray level of the pixel according to the product of the gray level corresponding to the pixel and the probability sum determined corresponding to the gray level, and adjust the pixel based on the adjusted gray level.

[0138] In the embodiment of the present application, the grayscale value range of each sub-block is assumed to be [0, L-1], is the number of gray levels. For each sub-block, the electronic device can determine the probability distribution of a gray level of the sub-block as the ratio of the number of pixels with the gray value of the gray level in the sub-block to the total number of pixels in the image. Specifically, the electronic device can determine the probability distribution of the gray level k of the sub-block by the following formula:

[0139]

[0140] in, is the probability distribution of gray level k, is the number of pixels of gray level k, is the total number of pixels in the image.

[0141] For each sub-block, after obtaining the probability distribution of each gray level in the sub-block, the electronic device can determine the cumulative distribution function of the sub-block according to the probability distribution of each gray level, wherein the electronic device can determine the sum of the probability distributions of a certain gray level and all gray levels below the gray level. Specifically, the electronic device can calculate the cumulative distribution of each gray level in the sub-block by the following formula:

[0142]

[0143] in, Grayscale And the probability and of all gray levels below it, is the probability distribution of gray level j, and k is the gray level k.

[0144] The electronic device may determine the adjusted grayscale level of each pixel in each sub-block according to the grayscale level of the pixel and the cumulative distribution of the sub-block at the grayscale level, wherein the electronic device may determine the product of the cumulative distribution of the grayscale level of the pixel and the maximum grayscale level in the image, and round down the product to an integer value to determine the adjusted grayscale level of the pixel. Specifically, the electronic device may determine the adjusted grayscale level of the pixel by the following formula:

[0145]

[0146] in, is the grayscale after adjustment, Indicates rounding down. is the number of gray levels, is the gray level in the sub-block and the probability and of all gray levels below it. Example 6

[0147] In order to accurately determine the distortion parameters, based on the above embodiments, in the embodiment of the present application, the distortion parameters of the acquisition device are determined based on the observed coordinates and the undistorted coordinates, including:

[0148] Determine the distorted coordinates corresponding to the undistorted coordinates based on the pre-stored distortion function, the initial distortion parameters and the undistorted coordinates;

[0149] Adjusting the distortion parameter based on the Gauss-Newton method until the error between the distortion coordinate determined based on the adjusted distortion parameter and the observation coordinate is minimized;

[0150] The distortion parameter corresponding to the minimum error is determined as the distortion parameter of the acquisition device.

[0151] In order to accurately determine the distortion parameters, the electronic device may determine the distorted coordinates corresponding to the undistorted coordinates based on the pre-stored distortion function, the initial distortion parameters, and the acquired undistorted coordinates;

[0152] Specifically, the distortion function is:

[0153]

[0154] in, is the abscissa of the distortion coordinate, is the ordinate of the distorted coordinate, x is the abscissa of the undistorted coordinate, y is the ordinate of the undistorted coordinate, and r is the distance from the image pixel to the center of the image. Specifically, r 2 =x 2 +y 2 , k 1 , k 2 , k 3 is the radial distortion parameter, p 1 、p 2 is the tangential distortion parameter.

[0155] The electronic device may adjust the distortion parameters based on the Gauss-Newton method until the error between the distortion coordinates determined based on the adjusted distortion parameters and the observed coordinates is minimized. In a possible implementation, the error between the projected distortion coordinates corresponding to the distortion coordinates determined based on the adjusted distortion parameters and the projected observed coordinates corresponding to the observed coordinates is minimized.

[0156] Specifically, the projection error can be defined as the difference between the observed coordinates and the coordinates predicted by the distortion model. i The observation coordinates, the projection error can be expressed as:

[0157]

[0158] in, is the projection error, is the projection distortion coordinate calculated by the distortion function, is the projected observation coordinate.

[0159] Next, to quantify the overall error, the electronic device can define the error function as the sum of the squares of the projection errors of all observation points corresponding to the observation coordinates:

[0160]

[0161] This error function measures the overall difference between the distortion function prediction and the actual observations.

[0162] In order to adjust the distortion parameters to minimize this error function, the electronic device uses an iterative optimization method. In each iteration, there is a current distortion parameter vector , which represents the current estimate of the distortion parameters. The goal of this application is to find distortion parameters that are closer to the true values ​​through an optimization process in order to better correct the image or improve the accuracy of the camera model. First, the electronic device can calculate the error vector e, which contains the projection errors of all observation points. These errors are caused by the inaccuracy of the current distortion parameters and measure the difference between the observed values ​​and the theoretical values ​​(i.e., the predicted values ​​calculated based on the current distortion parameters). The Jacobian matrix can be used , which is a matrix containing the partial derivatives of the error function with respect to the distortion parameters. Specifically, each row of the Jacobian matrix corresponds to the partial derivative of the error of an observation point with respect to a distortion parameter. Then, construct the matrix ,in is the transpose of the Jacobian matrix, is the identity matrix, is a regularization parameter. This matrix is ​​one of the key matrices used in the optimization process. It combines the second-order derivative information of the error function and the regularization term. The purpose of the regularization term is to control the stability of the optimization process and prevent the parameter estimation from being too extreme due to data noise or model overfitting.

[0163] Specifically, the following formula can be used to update the distortion parameter vector:

[0164]

[0165] is the current distortion parameter vector, is the updated distortion parameter vector. is the Jacobian matrix, which contains the partial derivative information of the error function with respect to the distortion parameters. e is the current error vector, which consists of the projection errors of all observation points. λ is a regularization parameter used to control the stability of the optimization process and prevent overfitting. I is the identity matrix whose dimensions match the dimensions of the distortion parameter vector.

[0166] It should be noted that in the process of adjusting the distortion parameter vector, both the adjusted distortion parameter vector and the unadjusted distortion parameter vector contain the same type of distortion parameters in the same order and position. This means that for each element in the vector (i.e., each distortion parameter), the position in the two vectors is one-to-one corresponding, allowing each distortion parameter to be directly modified and optimized through the iterative update process.

[0167] parameter λ It plays an important role in controlling the update direction. λ When is large, the update process is more inclined to the gradient descent method, that is, the update is performed in the opposite direction of the error function gradient, but the step size may be limited by the regularization term. λ When is small, the update process is closer to the Gauss-Newton method, which quickly approaches the optimal solution through quadratic approximation in a local area.

[0168] By continuously iterating this process, the electronic device can gradually adjust the distortion parameters until the error function reaches an acceptable minimum value. At this point, the corresponding distortion parameter vector is considered to be the optimal distortion parameter of the acquisition device, which can most accurately reflect the actual distortion of the device.

[0169] This process may be referred to as calculating radial and tangential distortion parameters in the acquisition device using a Levenberg-Marquardt least squares optimization algorithm.

[0170] The method provided in the embodiment of the present application can effectively solve the problem that the image distortion correction method based on deep learning in the prior art depends on training data, and the complexity of the model and the time cost of the training process are also high.

[0171] Figure 2 A detailed process diagram of image distortion correction provided in an embodiment of the present application includes the following steps:

[0172] S201: Acquire an image of a calibration plate set at a preset position captured by a capture device.

[0173] S202: Perform illumination normalization processing on the image.

[0174] S203: Identify each corner point in the image; determine the Harris response value of each corner point, and determine the corner point corresponding to the maximum response value as the target corner point.

[0175] S204: Identify corner points in the calibration plate that match the target corner points.

[0176] S205: Obtain the observed coordinates of the target corner point in the image, and determine the undistorted coordinates corresponding to the corner point when the acquisition device is not distorted based on the parameters of the acquisition device and the pre-saved coordinates of the corner point in the world coordinate system.

[0177] S206: Determine distortion parameters of the acquisition device based on the observed coordinates and the undistorted coordinates. Example 7

[0178] Based on the same inventive concept, the present application embodiment provides an image distortion correction device, please refer to Figure 3 , the device comprises:

[0179] An acquisition module 301 is used to acquire an image of a calibration plate set at a preset position acquired by an acquisition device;

[0180] The processing module 302 is used to identify the target corner point in the image; obtain the observed coordinates of the target corner point in the image, identify the corner point in the calibration plate that matches the target corner point, and determine the undistorted coordinates corresponding to the corner point when the acquisition device is not distorted based on the parameters of the acquisition device and the pre-saved coordinates of the corner point in the world coordinate system;

[0181] The determination module 303 is used to determine the distortion parameters of the acquisition device based on the observed coordinates and the undistorted coordinates, so as to perform distortion correction on the image acquired by the acquisition device based on the distortion parameters.

[0182] In a possible implementation, the processing module 302 is specifically configured to identify each corner point in the image; determine a Harris response value of each corner point, and determine a corner point corresponding to a maximum response value as a target corner point.

[0183] In a possible implementation, the processing module 302 is specifically used to determine, for each pixel in the image, a circular area of ​​a preset size with the pixel as the center, and for each other pixel in the circular area other than the pixel, determine the difference between the intensity value of the other pixel and the pixel; if the difference exceeds a preset threshold, mark the other pixel as a bright spot; otherwise, mark the other pixel as a dark spot; sort each other pixel according to its position in the image; if there are a continuous preset number of other pixel points that are all marked as bright spots or all marked as dark spots, determine the pixel as a corner point.

[0184] In a possible implementation, the processing module 302 is specifically used to determine the target descriptor of the target corner point in the image; and determine the descriptor of each corner point of the calibration plate; determine the similarity between the descriptor of each corner point and the target descriptor; and determine the corner point with the highest similarity as the corner point in the calibration plate that matches the target corner point.

[0185] In a possible implementation, the processing module 302 is specifically used to determine a window of a preset size with the target corner point as the center; determine the center of mass in the window, and determine the vector direction from the target corner point to the center of mass as the main direction; construct a rotation matrix corresponding to the angle of the main direction; rotate each pixel point in the window based on the rotation matrix; so as to rotate the pixel points in the window to a direction consistent with the main direction; obtain a preset number of pixel pairs in the rotated window, and for each pixel pair, if the intensity value of the first pixel point in the pixel pair is greater than the intensity value of the second pixel point, then the corresponding bit of the pixel pair in the target descriptor is recorded as the first identifier, otherwise, the corresponding bit of the pixel pair is recorded as the second identifier.

[0186] In a possible implementation, the processing module 302 is specifically used to determine, for each pixel in the window, a first product of the horizontal coordinate of the pixel and the intensity value of the pixel, and a second product of the vertical coordinate of the pixel and the intensity value of the pixel; determine a first sum of each first product and a second sum of each second product, and determine a third sum of the intensity values ​​of each pixel in the window; determine a ratio of the first sum to the third sum as the horizontal coordinate of the center of mass, and determine a ratio of the second sum to the third sum as the vertical coordinate of the center of mass.

[0187] In a possible implementation, the processing module 302 is further configured to perform illumination normalization processing on the image.

[0188] In a possible implementation, the processing module 302 is specifically used to divide the image into sub-blocks of preset size; use histogram equalization to evenly distribute the intensity range of each sub-block; and perform bilinear interpolation processing on the boundary of each sub-block after even distribution.

[0189] In a possible implementation, the processing module 302 is specifically used to determine, for each gray level of each sub-block, the probability sum of the gray level and all gray levels below the gray level in the sub-block; for each pixel in the sub-block, determine the adjusted gray level of the pixel according to the product of the gray level corresponding to the pixel and the probability sum determined corresponding to the gray level, and adjust the pixel based on the adjusted gray level.

[0190] In a possible implementation, the determination module 303 is specifically used to determine the distorted coordinates corresponding to the undistorted coordinates based on a pre-saved distortion function, initial distortion parameters and the undistorted coordinates; adjust the distortion parameters based on the Gauss-Newton method until the error between the distorted coordinates determined based on the adjusted distortion parameters and the observed coordinates is minimized; and determine the distortion parameters corresponding to the minimum error as the distortion parameters of the acquisition device. Example 8

[0191] Based on the same inventive concept, the present application embodiment provides an electronic device, which can realize the function of the above-mentioned abnormality identification device, please refer to Figure 4 The device includes a processor 401, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.

[0192] The memory 403 stores a computer program. When the program is executed by the processor 401, the processor 401 performs the following steps:

[0193] Acquire an image of a calibration plate set at a preset position acquired by an acquisition device;

[0194] Identify the target corner point in the image; obtain the observed coordinates of the target corner point in the image, identify the corner point in the calibration plate that matches the target corner point, and determine the undistorted coordinates corresponding to the corner point when the acquisition device is not distorted based on the parameters of the acquisition device and the pre-saved coordinates of the corner point in the world coordinate system;

[0195] Based on the observed coordinates and the undistorted coordinates, distortion parameters of the acquisition device are determined; and distortion correction is performed on the image acquired by the acquisition device based on the distortion parameters.

[0196] In a possible implementation, the identifying target corner points in the image includes:

[0197] identifying each corner point in the image;

[0198] The Harris response value of each corner point is determined, and the corner point corresponding to the maximum response value is determined as the target corner point.

[0199] In a possible implementation, identifying each corner point in the image includes:

[0200] For each pixel in the image, a circular area of ​​a preset size is determined with the pixel as the center, and for each other pixel in the circular area other than the pixel, the difference between the intensity value of the other pixel and the pixel is determined. If the difference exceeds a preset threshold, the other pixel is marked as a bright spot, otherwise, the other pixel is marked as a dark spot; according to the position of each other pixel in the image, each other pixel is sorted, and if there are a continuous preset number of other pixel points that are all marked as bright spots or all marked as dark spots, the pixel is determined as a corner point.

[0201] In a possible implementation, the identifying a corner point in the calibration plate that matches the target corner point includes:

[0202] Determine an object descriptor of the object corner point in the image;

[0203] And determine the descriptor of each corner point of the calibration plate; determine the similarity between the descriptor of each corner point and the target descriptor; and determine the corner point with the highest similarity as the corner point in the calibration plate that matches the target corner point.

[0204] In a possible implementation, the determining the target descriptor of the target corner point in the image includes:

[0205] Determine a window of a preset size with the target corner point as the center; determine the centroid in the window, and determine the vector direction from the target corner point to the centroid as the main direction;

[0206] Constructing a rotation matrix corresponding to the angle of the main direction; rotating each pixel point in the window based on the rotation matrix; so as to rotate the pixel points in the window to a direction consistent with the main direction;

[0207] A preset number of pixel pairs in the rotated window are obtained. For each pixel pair, if the intensity value of the first pixel in the pixel pair is greater than the intensity value of the second pixel, the corresponding bit of the pixel pair in the target descriptor is recorded as the first identifier; otherwise, the corresponding bit of the pixel pair is recorded as the second identifier.

[0208] In a possible implementation, determining the center of mass in the window includes:

[0209] For each pixel point in the window, determine a first product of the horizontal coordinate of the pixel point and the intensity value of the pixel point, and determine a second product of the vertical coordinate of the pixel point and the intensity value of the pixel point;

[0210] Determine a first sum of each first product, a second sum of each second product, and determine a third sum of intensity values ​​of each pixel in the window;

[0211] The ratio of the first sum value to the third sum value is determined as the horizontal coordinate of the center of mass, and the ratio of the second sum value to the third sum value is determined as the vertical coordinate of the center of mass.

[0212] In a possible implementation, after acquiring the image of the calibration plate set at the preset position acquired by the acquisition device, and before identifying the observation coordinates of the preset corner points of the calibration plate in the image, the method further includes:

[0213] Perform illumination normalization processing on the image.

[0214] In a possible implementation, performing illumination normalization processing on the image includes:

[0215] Dividing the image into sub-blocks of a preset size;

[0216] Using histogram equalization to evenly distribute the intensity range of each sub-block;

[0217] The boundary of each sub-block after uniform distribution is processed by bilinear interpolation.

[0218] In a possible implementation manner, the uniformly distributing the intensity range of each sub-block by using histogram equalization includes:

[0219] For each gray level of each sub-block, determine the probability sum of the gray level and all gray levels below the gray level in the sub-block; for each pixel in the sub-block, determine the adjusted gray level of the pixel according to the product of the gray level corresponding to the pixel and the probability sum determined corresponding to the gray level, and adjust the pixel based on the adjusted gray level.

[0220] In a possible implementation, determining the distortion parameters of the acquisition device based on the observed coordinates and the undistorted coordinates includes:

[0221] Determine the distorted coordinates corresponding to the undistorted coordinates based on the pre-stored distortion function, the initial distortion parameters and the undistorted coordinates;

[0222] Adjusting the distortion parameter based on the Gauss-Newton method until the error between the distortion coordinate determined based on the adjusted distortion parameter and the observation coordinate is minimized;

[0223] The distortion parameter corresponding to the minimum error is determined as the distortion parameter of the acquisition device.

[0224] The communication bus mentioned in the above server can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0225] The communication interface 402 is used for communication between the electronic device and other devices.

[0226] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0227] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; it can also be a digital signal processing processor (DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. Example 8

[0228] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program executable by an electronic device. When the program runs on the electronic device, the electronic device implements the following steps when executing:

[0229] Acquire an image of a calibration plate set at a preset position acquired by an acquisition device;

[0230] Identify the target corner point in the image; obtain the observed coordinates of the target corner point in the image, identify the corner point in the calibration plate that matches the target corner point, and determine the undistorted coordinates corresponding to the corner point when the acquisition device is not distorted based on the parameters of the acquisition device and the pre-saved coordinates of the corner point in the world coordinate system;

[0231] Based on the observed coordinates and the undistorted coordinates, distortion parameters of the acquisition device are determined; and distortion correction is performed on the image acquired by the acquisition device based on the distortion parameters.

[0232] In a possible implementation, the identifying target corner points in the image includes:

[0233] identifying each corner point in the image;

[0234] The Harris response value of each corner point is determined, and the corner point corresponding to the maximum response value is determined as the target corner point.

[0235] In a possible implementation, identifying each corner point in the image includes:

[0236] For each pixel in the image, a circular area of ​​a preset size is determined with the pixel as the center, and for each other pixel in the circular area other than the pixel, the difference between the intensity value of the other pixel and the pixel is determined. If the difference exceeds a preset threshold, the other pixel is marked as a bright spot, otherwise, the other pixel is marked as a dark spot; according to the position of each other pixel in the image, each other pixel is sorted, and if there are a continuous preset number of other pixel points that are all marked as bright spots or all marked as dark spots, the pixel is determined as a corner point.

[0237] In a possible implementation, the identifying a corner point in the calibration plate that matches the target corner point includes:

[0238] Determine an object descriptor of the object corner point in the image;

[0239] And determine the descriptor of each corner point of the calibration plate; determine the similarity between the descriptor of each corner point and the target descriptor; and determine the corner point with the highest similarity as the corner point in the calibration plate that matches the target corner point.

[0240] In a possible implementation, the determining the target descriptor of the target corner point in the image includes:

[0241] Determine a window of a preset size with the target corner point as the center; determine the centroid in the window, and determine the vector direction from the target corner point to the centroid as the main direction;

[0242] Constructing a rotation matrix corresponding to the angle of the main direction; rotating each pixel point in the window based on the rotation matrix; so as to rotate the pixel points in the window to a direction consistent with the main direction;

[0243] A preset number of pixel pairs in the rotated window are obtained. For each pixel pair, if the intensity value of the first pixel in the pixel pair is greater than the intensity value of the second pixel, the corresponding bit of the pixel pair in the target descriptor is recorded as the first identifier; otherwise, the corresponding bit of the pixel pair is recorded as the second identifier.

[0244] In a possible implementation, determining the center of mass in the window includes:

[0245] For each pixel point in the window, determine a first product of the horizontal coordinate of the pixel point and the intensity value of the pixel point, and determine a second product of the vertical coordinate of the pixel point and the intensity value of the pixel point;

[0246] Determine a first sum of each first product, a second sum of each second product, and determine a third sum of intensity values ​​of each pixel in the window;

[0247] The ratio of the first sum value to the third sum value is determined as the horizontal coordinate of the center of mass, and the ratio of the second sum value to the third sum value is determined as the vertical coordinate of the center of mass.

[0248] In a possible implementation, after acquiring the image of the calibration plate set at the preset position acquired by the acquisition device, and before identifying the observation coordinates of the preset corner points of the calibration plate in the image, the method further includes:

[0249] Perform illumination normalization processing on the image.

[0250] In a possible implementation, performing illumination normalization processing on the image includes:

[0251] Dividing the image into sub-blocks of a preset size;

[0252] Using histogram equalization to evenly distribute the intensity range of each sub-block;

[0253] The boundary of each sub-block after uniform distribution is subjected to bilinear interpolation processing.

[0254] In a possible implementation manner, the uniformly distributing the intensity range of each sub-block by using histogram equalization includes:

[0255] For each gray level of each sub-block, determine the probability sum of the gray level and all gray levels below the gray level in the sub-block; for each pixel in the sub-block, determine the adjusted gray level of the pixel according to the product of the gray level corresponding to the pixel and the probability sum determined corresponding to the gray level, and adjust the pixel based on the adjusted gray level.

[0256] In a possible implementation, determining the distortion parameters of the acquisition device based on the observed coordinates and the undistorted coordinates includes:

[0257] Determine the distorted coordinates corresponding to the undistorted coordinates based on the pre-stored distortion function, the initial distortion parameters and the undistorted coordinates;

[0258] Adjusting the distortion parameter based on the Gauss-Newton method until the error between the distortion coordinate determined based on the adjusted distortion parameter and the observation coordinate is minimized;

[0259] The distortion parameter corresponding to the minimum error is determined as the distortion parameter of the acquisition device.

[0260] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0261] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0262] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0263] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0264] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for image distortion correction, characterized in that: The method comprises: Acquire an image of a calibration plate set at a preset position acquired by an acquisition device; Identify the target corner point in the image; obtain the observed coordinates of the target corner point in the image, and identify the corner point in the calibration plate that matches the target corner point, and determine the undistorted coordinates corresponding to the corner point that matches the target corner point when the acquisition device is not distorted based on the parameters of the acquisition device and the pre-saved coordinates of the corner point that matches the target corner point in the world coordinate system; Determining distortion parameters of the acquisition device based on the observed coordinates and the undistorted coordinates; and performing distortion correction on the image acquired by the acquisition device based on the distortion parameters; The step of identifying the corner point in the calibration plate that matches the target corner point includes: Determine an object descriptor of the object corner point in the image; And determine the descriptor of each corner point of the calibration plate; determine the similarity between the descriptor of each corner point and the target descriptor; and determine the corner point with the highest similarity as the corner point in the calibration plate that matches the target corner point.

2. The method according to claim 1, characterized in that The identifying target corner points in the image comprises: identifying each corner point in the image; The Harris response value of each corner point is determined, and the corner point corresponding to the maximum response value is determined as the target corner point.

3. The method according to claim 2, characterized in that The identifying each corner point in the image comprises: For each pixel in the image, a circular area of ​​a preset size is determined with the pixel as the center, and for each other pixel in the circular area other than the pixel, the difference between the intensity value of the other pixel and the pixel is determined. If the difference exceeds a preset threshold, the other pixel is marked as a bright spot, otherwise, the other pixel is marked as a dark spot; according to the position of each other pixel in the image, each other pixel is sorted, and if there are a continuous preset number of other pixel points that are all marked as bright spots or all marked as dark spots, the pixel is determined as a corner point.

4. The method according to claim 1, characterized in that: The determining of the target descriptor of the target corner point in the image comprises: Determine a window of a preset size with the target corner point as the center; determine the centroid in the window, and determine the vector direction from the target corner point to the centroid as the main direction; Constructing a rotation matrix corresponding to the angle of the main direction; rotating each pixel point in the window based on the rotation matrix; so as to rotate the pixel points in the window to a direction consistent with the main direction; A preset number of pixel pairs in the rotated window are obtained. For each pixel pair, if the intensity value of the first pixel in the pixel pair is greater than the intensity value of the second pixel, the corresponding bit of the pixel pair in the target descriptor is recorded as the first identifier; otherwise, the corresponding bit of the pixel pair is recorded as the second identifier.

5. The method according to claim 4, characterized in that Determining the center of mass in the window includes: For each pixel point in the window, determine a first product of the horizontal coordinate of the pixel point and the intensity value of the pixel point, and determine a second product of the vertical coordinate of the pixel point and the intensity value of the pixel point; Determine a first sum of each first product, a second sum of each second product, and determine a third sum of intensity values ​​of each pixel in the window; The ratio of the first sum value to the third sum value is determined as the horizontal coordinate of the center of mass, and the ratio of the second sum value to the third sum value is determined as the vertical coordinate of the center of mass.

6. The method according to claim 1, characterized in that After acquiring the image of the calibration plate set at the preset position acquired by the acquisition device, and before identifying the observation coordinates of the preset corner points of the calibration plate in the image, the method further includes: Perform illumination normalization processing on the image.

7. The method according to claim 6, characterized in that The performing illumination normalization processing on the image comprises: Dividing the image into sub-blocks of a preset size; Using histogram equalization to evenly distribute the intensity range of each sub-block; The boundary of each sub-block after uniform distribution is subjected to bilinear interpolation processing.

8. The method according to claim 7, characterized in that The uniformly distributing the intensity range of each sub-block by using histogram equalization includes: For each gray level of each sub-block, determine the probability sum of the gray level and all gray levels below the gray level in the sub-block; for each pixel in the sub-block, determine the adjusted gray level of the pixel according to the product of the gray level corresponding to the pixel and the probability sum determined corresponding to the gray level, and adjust the pixel based on the adjusted gray level.

9. The method according to claim 1, characterized in that: The step of determining the distortion parameters of the acquisition device based on the observed coordinates and the undistorted coordinates includes: Determine the distorted coordinates corresponding to the undistorted coordinates based on the pre-stored distortion function, the initial distortion parameters and the undistorted coordinates; Adjusting the distortion parameter based on the Gauss-Newton method until the error between the distortion coordinate determined based on the adjusted distortion parameter and the observation coordinate is minimized; The distortion parameter corresponding to the minimum error is determined as the distortion parameter of the acquisition device.

10. An image distortion correction device, characterized in that: The device comprises: An acquisition module, used to acquire an image of a calibration plate set at a preset position acquired by an acquisition device; A processing module, configured to identify a target corner point in the image; obtain the observed coordinates of the target corner point in the image, identify a corner point in the calibration plate that matches the target corner point, and determine the undistorted coordinates corresponding to the corner point that matches the target corner point when the acquisition device is not distorted, based on the parameters of the acquisition device and the pre-saved coordinates of the corner point that matches the target corner point in the world coordinate system; A determination module, configured to determine the distortion parameters of the acquisition device based on the observed coordinates and the undistorted coordinates; and to perform distortion correction on the image acquired by the acquisition device based on the distortion parameters; Among them, the processing module is specifically used to determine the target descriptor of the target corner point in the image; and determine the descriptor of each corner point of the calibration plate; determine the similarity between the descriptor of each corner point and the target descriptor; and determine the corner point with the highest similarity as the corner point in the calibration plate that matches the target corner point.

11. An electronic device, characterized in that: include: A memory for storing program instructions; A processor is used to call the program instructions stored in the memory, and execute the steps included in the method for image distortion correction according to any one of claims 1 to 9 according to the obtained program instructions.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method for correcting image distortion as claimed in any one of claims 1 to 9.

Images