A method for calibrating distortion of a camera, an electronic device, and a storage medium

Abstract

The application provides a camera distortion calibration method, an electronic device and a storage medium. The camera distortion calibration method comprises: obtaining first coordinates of each grid point of a calibration board image in a pixel coordinate system, second coordinates of each grid point of the calibration board image in a world coordinate system, an initial value of a radial distortion coefficient of a camera, an intrinsic parameter matrix of the camera and an extrinsic parameter of the camera; wherein the calibration board image is collected by the camera; and determining the radial distortion coefficient of the camera based on the first coordinates and the second coordinates of each grid point of the calibration board image, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix of the camera and the extrinsic parameter of the camera, and an improved distortion algorithm; wherein the improved distortion algorithm comprises a penalty term, and the penalty term is used to make a distortion curve corresponding to the improved distortion algorithm have no zero point. In this way, the distortion of the image edge in the de-distortion process can be effectively improved, and the accuracy and stability of the camera calibration can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of camera calibration, in particular to a camera distortion calibration method, an electronic device and a storage medium. BACKGROUND

[0002] Generally speaking, a camera can be roughly divided into three parts: a lens, a photosensitive element and a processing circuit. When light passes through the lens, an "image" of an object is formed on the photosensitive element, which is similar to the principle of pinhole imaging, and then becomes an electronic picture after a series of processing. Of course, certain errors (such as radial distortion of the image) will occur in this imaging process, so in order to eliminate or correct these errors, camera calibration is usually required before the camera is shipped, so as to obtain camera parameters (such as radial distortion coefficients) for eliminating or correcting these errors.

[0003] At present, the calibration of distortion usually uses Brown-Conrady Model algorithm or Rational Model algorithm, and the basic principle is to use a polynomial to fit the deviation of the actual light path relative to the pinhole model. However, the inventors have found in practice that in some scenarios, the radial distortion coefficients calculated by the above algorithms will cause the edge position of the image to appear distorted when the image is de-distorted. As shown in FIG. 1, the image of the calibration board captured by the camera contains four calibration boards, and when the radial distortion coefficients calculated by the Brown-Conrady Model algorithm are used to de-distort the image, the four corners of the image all appear distorted. Figure 1 SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a camera distortion calibration method, an electronic device and a storage medium, so as to improve the situation that the image edge appears distorted in the de-distortion process, and improve the accuracy and stability of camera calibration.

[0005] The present application is implemented as follows:

[0006] In a first aspect, the embodiments of the present application provide a camera distortion calibration method, comprising: obtaining first coordinates of each grid point of a calibration board image in a pixel coordinate system and second coordinates in a world coordinate system, an initial value of a radial distortion coefficient of a camera, an intrinsic matrix and an extrinsic parameter of the camera; wherein the calibration board image is captured by the camera; determining the radial distortion coefficient of the camera based on the first coordinates and the second coordinates of each grid point of the calibration board image, the initial value of the radial distortion coefficient of the camera, the intrinsic matrix and the extrinsic parameter of the camera, and an improved distortion algorithm; wherein the improved distortion algorithm includes a penalty term, and the penalty term is used to make the distortion curve corresponding to the improved distortion algorithm have no zero point. ​

[0007] Since the reason why the edge position of the image appears distorted is that the distortion curve corresponding to the current distortion algorithm has a zero-crossing point, in the embodiments of the present application, the distortion algorithm is improved by adding a penalty term, so that the distortion curve corresponding to the improved distortion algorithm has no zero-crossing point. In this way, the situation that the image edge appears distorted in the de-distortion process can be effectively improved, and the accuracy and stability of camera calibration are improved.

[0008] In combination with the technical solutions provided in the first aspect, in some possible implementation manners, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and the extrinsic parameter of the camera are acquired, including: determining the extrinsic parameter of the camera based on the relationship between the world coordinate system and the camera coordinate system of the camera itself; and / or determining the intrinsic parameter matrix of the camera based on the relationship between the camera coordinate system and the pixel coordinate system; and / or fitting the initial value of the radial distortion coefficient of the camera based on the first coordinates and the second coordinates of each grid point of the calibration board image, the extrinsic parameter of the camera and the intrinsic parameter matrix of the camera. The initial value of the radial distortion coefficient of the camera and the intrinsic parameter matrix and the extrinsic parameter of the camera can be accurately determined through the Zhang algorithm.

[0009] In combination with the technical solutions provided in the first aspect, in some possible implementation manners, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and the extrinsic parameter of the camera, and the improved distortion algorithm are used to determine the radial distortion coefficient of the camera, including: S1: determining the estimated projection point of each grid point based on the second coordinates of each grid point of the calibration board image in the world coordinate system, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and the extrinsic parameter of the camera, and the improved distortion algorithm; wherein the estimated projection point is a coordinate point in the pixel coordinate system; S2: calculating the error value of the estimated projection point of each grid point and the first coordinates of each grid point in the pixel coordinate system; S3: optimizing the error value to obtain the optimized radial distortion coefficient; S4: replacing the initial value of the radial distortion coefficient in steps S1-S3 with the optimized radial distortion coefficient to determine the radial distortion coefficient after re-optimization; and iteratively performing the above steps until a preset iteration condition is met, and outputting the optimized radial distortion coefficient corresponding to the latest error value; wherein the output optimized radial distortion coefficient corresponding to the latest error value is the radial distortion coefficient of the camera. In the embodiments of the present application, the radial distortion coefficient of the camera can be reasonably and effectively calculated through the above iteration process.

[0010] In combination with the technical solutions provided in the first aspect, in some possible implementation manners, the improved distortion algorithm is:

[0011]

[0012]

[0013] wherein x d and y d represent the third coordinates of the grid points in the image coordinate system with distortion; x d and y d are used to determine the estimated projection points; x u and y u represent the fourth coordinates of the grid points in the image coordinate system without distortion; x u and y u are obtained through the second coordinates; k1-k6 represent radial distortion coefficients; Q(r 2 ) represents the penalty term.

[0014] In the embodiments of the present application, by adding a penalty term Q(r 2 ) to the Rational Model algorithm, the improved Rational Model algorithm can correspond to a distortion curve without zero points. In this way, the distortion process through the Rational Model algorithm can effectively improve the situation of image edge distortion.

[0015] In some possible implementation manners, in combination with the technical solutions provided by the first aspect, the expression of the penalty term is:

[0016] Q(r 2 ) = Q1(r 2 ) + Q2(r 2 );

[0017]

[0018]

[0019] wherein Q1(r 2 ) represents a penalty subterm of the function value of F(r 2 ) at the edge field r = r max , F(r 2 ) = 1 + k1r 2 + k2r 4 + k3r 6 ; Q2(r 2 ) represents a penalty subterm of the function value of G(r 2 ) at the edge field r = r max , G(r 2 ) = 1 + k4r 2+k5r 4 +k6r 6 ;q represents the penalty error value; Represents the third coordinate x u and y u The maximum value corresponding to the sum of squares;

[0020] Or, the expression of the penalty term is:

[0021] Q(r 2 )=Q1(r 2 )+Q2(r 2 )+Q3(r 2 )+Q4(r 2 );

[0022]

[0023]

[0024]

[0025]

[0026] Among them, Q1(r 2 ) represents F(r 2 ) at the edge of the field of view r = r max The penalty term of the function value; Q2(r 2 ) represents G(r 2 ) at the edge of the field of view r = r max The penalty sub-item of the function value; Q3(r 2 ) represents F(r 2 ) in the field of view range [0,r max ) Minimum of the inner function The penalty sub-item; Q4(r 2 ) represents G(r 2 ) in the field of view range [0,r max ) minimum value of the internal function G(r1 2 ) penalty sub-item; q represents the penalty error value; F(r 2 )=1+k1r 2 +k2r 4 +k3r 6 ;G(r 2 )=1+k4r 2 +k5r 4 +k6r 6 ; Represents the third coordinate x u and y u The maximum value corresponding to the sum of squares; represents F(r 2a minimum value of G(r 2 ) represents a minimum value of G(r 2 ).

[0027] In an embodiment of the present application, whether to add the penalty error value q is determined by the value of and The specific value of the penalty term Q(r 2 ) can be determined simply and efficiently, and the corresponding cubic curve of the Rational Model algorithm is effectively limited to have no zero point in [0, r max ).In another embodiment of the present application, a minimum value is introduced, and whether to add the penalty error value q is determined by the value of and 1+k4r1 2 +k5r1 4 +k6r1 6 The specific value of the penalty term Q(r 2 ) can be determined more strictly and accurately, and the corresponding cubic curve of the Rational Model algorithm is effectively limited to have no zero point in [0, r max ).

[0028] In combination with the technical solutions of the first aspect, in some possible implementation manners, the improved distortion algorithm is:

[0029] x d =x u (1+k1r 2 +k2r 4 +k3r 6 )+Q(r 2 );

[0030] y d =y u (1+k1r 2 +k2r 4 +k3r 6 )+Q(r 2 );

[0031] wherein x d and y d represent the third coordinates of the grid points in the image coordinate system with distortion; x d and y d are used to determine the estimated projection points; x u and y u represent the fourth coordinates of the grid points in the image coordinate system without distortion; x u and y u ​obtained through the second coordinate; k1-k3 represent radial distortion coefficients; Q(r 2 ) represents the penalty term.

[0032] In the embodiments of the present application, by adding a penalty term Q(r 2 ) to the Brown-Conrady Model algorithm, the improved Brown-Conrady Model algorithm can correspond to a distortion curve without zero points. In this way, the image edge distortion in the Brown-Conrady Model algorithm can be effectively improved.

[0033] In some possible implementation manners, the expression of the penalty term is as follows in combination with the technical solution provided by the first aspect:

[0034]

[0035] Q(r 2 ) represents a penalty subterm of the function value of F(r 2 ) at the edge field r=r max ; F(r 2 ) = 1+k1r 2 +k2r 4 +k3r 6 ; q represents a penalty error value; represents the maximum value corresponding to the sum of squares of the third coordinates x u and y u ;

[0036] Alternatively, the expression of the penalty term is as follows:

[0037] Q(r 2 ) = Q5(r 2 )+Q6(r 2 );

[0038]

[0039]

[0040] Q5(r 2 ) represents a penalty subterm of the function value of F(r 2 ) at the edge field r=r max ; Q6(r 2 ) represents a penalty subterm of the function minimum value F(r0 2 ) of F(r 2 ) in the field range [0,r max ); F(r 2 ) = 1+k1r2 +k2r 4 +k3r 6 ;q represents the penalty error value; Represents the third coordinate x u and y u The maximum value corresponding to the sum of squares; represents F(r 2 ) minimum value; F(r 2 )=1+k1r 2 +k2r 4 +k3r 6 .

[0041] In one embodiment of the present application, by The value of Whether it is necessary to add a penalty error value q can simply and efficiently determine the penalty term Q(r 2 ) specific value, thereby effectively limiting the cubic curve corresponding to the Brown-Conrady Model algorithm, in [0,r max ) to maintain no zero crossing. In another embodiment of the present application, by introducing a minimum value, and by The value of Whether it is necessary to add a penalty error value q can more strictly and accurately determine the penalty term Q(r 2 ) specific value, thereby effectively limiting the cubic curve corresponding to the Brown-Conrady Model algorithm, in [0,r max ) to keep no zero crossing.

[0042] In combination with the technical solution provided in the first aspect above, in some possible implementations, the calibration plate image includes images of the calibration plate at at least three different angles.

[0043] In the embodiment of the present application, at least three calibration plates at different angles are used to facilitate the subsequent direct determination of the radial distortion coefficient of the camera.

[0044] In a second aspect, the embodiments of the present application provide an imaging method of a camera, comprising: obtaining first coordinates of each grid point of a calibration board image in a pixel coordinate system, second coordinates of each grid point of the calibration board image in a world coordinate system, an initial value of a radial distortion coefficient of the camera, an intrinsic matrix and an extrinsic parameter of the camera; wherein the calibration board image is captured by the camera; determining the radial distortion coefficient of the camera based on the first coordinates and the second coordinates of each grid point of the calibration board image, the initial value of the radial distortion coefficient of the camera, the intrinsic matrix and the extrinsic parameter of the camera, and an improved distortion algorithm; wherein the improved distortion algorithm comprises a penalty term, and the penalty term is used to make a distortion curve corresponding to the improved distortion algorithm have no zero point; and correcting distortion of an image captured by the camera based on the radial distortion coefficient of the camera.

[0045] In a third aspect, the embodiments of the present application provide a distortion calibration device of a camera, comprising: a first obtaining module, configured to obtain first coordinates of each grid point of a calibration board image in a pixel coordinate system, second coordinates of each grid point of the calibration board image in a world coordinate system, an initial value of a radial distortion coefficient of the camera, an intrinsic matrix and an extrinsic parameter of the camera; wherein the calibration board image is captured by the camera; and a first processing module, configured to determine the radial distortion coefficient of the camera based on the first coordinates and the second coordinates of each grid point of the calibration board image, the initial value of the radial distortion coefficient of the camera, the intrinsic matrix and the extrinsic parameter of the camera, and an improved distortion algorithm; wherein the improved distortion algorithm comprises a penalty term, and the penalty term is used to make a distortion curve corresponding to the improved distortion algorithm have no zero point.

[0046] In a fourth aspect, the embodiments of the present application provide an imaging device of a camera, comprising: a second obtaining module, configured to obtain first coordinates of each grid point of a calibration board image in a pixel coordinate system, second coordinates of each grid point of the calibration board image in a world coordinate system, an initial value of a radial distortion coefficient of the camera, an intrinsic matrix and an extrinsic parameter of the camera; wherein the calibration board image is captured by the camera; a second processing module, configured to determine the radial distortion coefficient of the camera based on the first coordinates and the second coordinates of each grid point of the calibration board image, the initial value of the radial distortion coefficient of the camera, the intrinsic matrix and the extrinsic parameter of the camera, and an improved distortion algorithm; wherein the improved distortion algorithm comprises a penalty term, and the penalty term is used to make a distortion curve corresponding to the improved distortion algorithm have no zero point; and a distortion correction module, configured to correct distortion of an image captured by the camera based on the radial distortion coefficient of the camera.

[0047] In a fifth aspect, a computer program product is provided, comprising computer program instructions, which, when read and executed by a processor, perform the method provided in the first aspect or the second aspect.

[0048] In a sixth aspect, an electronic device is provided, and the electronic device comprises a processor and a memory, the processor and the memory being connected; the memory is configured to store a program; and the processor is configured to invoke the program stored in the memory and execute the method provided in the first aspect or the second aspect.

[0049] In a seventh aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program, when executed by a processor, performs the method provided in the first aspect or the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0051] Figure 1 An effect diagram after distortion removal in the prior art.

[0052] Figure 2 A distortion curve diagram corresponding to the Rational Model algorithm.

[0053] Figure 3 A module block diagram of an electronic device provided by the embodiments of the present application.

[0054] Figure 4 A flowchart of a camera distortion calibration method provided by the embodiments of the present application.

[0055] Figure 5 A flowchart of a camera imaging method provided by the embodiments of the present application.

[0056] Figure 6 An effect diagram after distortion removal provided by the embodiments of the present application.

[0057] Figure 7 A module block diagram of a camera distortion calibration device provided by the embodiments of the present application.

[0058] Figure 8 A module block diagram of a camera imaging device provided by the embodiments of the present application.

[0059] Icon: 100 - electronic device; 110 - processor; 120 - memory; 130 - camera; 200 - camera distortion calibration apparatus; 210 - first acquisition module; 220 - first processing module; 300 - camera imaging apparatus; 310 - second acquisition module; 320 - second processing module; 330 - de-distortion module. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

[0061] Before the technical solutions in the embodiments of the present application are described in detail, the Brown-Conrady Model algorithm and the Rational Model algorithm commonly used in the prior art for camera distortion calibration are analyzed.

[0062] It should be noted that the process of camera distortion calibration can be understood as the process of determining the distortion coefficients of the camera.

[0063] The expression of the Rational Model algorithm is as follows:

[0064]

[0065]

[0066] In the formulas (1) and (2), x d and y d represent the coordinates with distortion in the image coordinate system; x u and y u represent the coordinates without distortion in the image coordinate system; k1-k6 represent the radial distortion coefficients; p1 and p2 represent the tangential distortion coefficients.

[0067] Since the values of p1 and p2 are usually small, the tangential distortion is not considered for the time being, and then the formulas (1) and (2) can be changed to:

[0068]

[0069]

[0070] As can be seen from the formulas (3) and (4), the radial distortion is only related to r and is the quotient of two polynomials. Then the distortion term containing r can be written as:

[0071] Let R = r 2 , then f(r) = 1 + k1r 2 + k2r 4 + k3r6 = f(r 2 ) = F(R) = 1 + k1R + k2R 2 + k3R 4 , g(r) = 1 + k4r 2 + k5r 4 + k6r 6 = g(r 2 ) = G(R) = 1 + k4R + k5R 2 + k6R 4 .

[0072] It can be seen that F(R) and G(R) are cubic functions with respect to R, where R ∈ [0, ∞). Please refer to Figure 2 , Figure 2 The solid curve of the arc is the curve of G(R) with respect to r, the dotted curve of the arc is the curve of F(R) with respect to r, and the curve marked as A is the curve of with respect to r.

[0073] Due to the absence of constraints on the radial distortion coefficients and the nature of the cubic function, F(R) and G(R) can have a zero-crossing point at a certain r (such as r = 0.79 in Figure 2 ), and their division will cause zero division or division by a decimal, resulting in invalid values or suddenly large values. The inventors found in research that when a mutation occurs as at r = 0.79 in Figure 2 , the edge position shown in the de-distorted image is distorted as shown in Figure 1 .

[0074] The expression of the Brown-Conrady Model algorithm is:

[0075]

[0076]

[0077] It should be noted that the difference between the Brown-Conrady Model algorithm and the Rational Model algorithm is that the Brown-Conrady Model algorithm only includes radial distortion coefficients k1-k3, while the Rational Model algorithm also includes radial distortion coefficients k4-k6, and the other parameters are the same. Therefore, based on the same analysis process, it can be known that the distortion curve corresponding to the Brown-Conrady Model algorithm will also have a zero-crossing point, and thus the radial distortion coefficients calculated by using the Brown-Conrady Model algorithm will cause the edge position of the image to be distorted in the subsequent de-distortion process.

[0078] The defects of the two distortion algorithms described above are the results of the inventors after practice and careful study, therefore, the discovery process of the above problems and the solutions proposed by the embodiments of the present application to the above problems should be the contributions of the inventors to the present application during the process of the present application.

[0079] Please refer to Figure 3 The electronic device 100 provided by the embodiments of the present application can be any device containing a camera, for example, the electronic device 100 can be, but is not limited to, a smart phone, a tablet computer, a single-lens reflex camera, a video camera, etc.

[0080] In structure, the electronic device 100 can include a processor 110, a memory 120 and a camera 130.

[0081] The processor 110 is directly or indirectly electrically connected with the memory 120 and the camera 130 to realize the transmission or interaction of data, for example, these elements can be electrically connected with each other through one or more communication buses or signal lines. The distortion calibration device of the camera or the imaging device of the camera includes at least one software module stored in the memory 120 in the form of software or firmware (Firmware) or solidified in the operating system (Operating System, OS) of the electronic device 100. The processor 110 is used to execute the executable module stored in the memory 120, and the processor 110 can execute the computer program after receiving the execution instruction.

[0082] The processor 110 can be an integrated circuit chip with signal processing capability. The processor 110 can also be a general-purpose processor, for example, can be a central processing unit (Central Processing Unit, CPU), a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a discrete gate or transistor logic device, a discrete hardware component, and can realize or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application. In addition, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0083] The memory 120 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), and an electrically erasable programmable read-only memory (EEPROM). The memory 120 is used to store programs, and the processor 110 executes the programs after receiving execution instructions.

[0084] It should be noted that Figure 3 The structure shown is for illustration only. The electronic device 100 provided in the embodiment of the present application may also have Figure 3 Fewer or more components, or with Figure 3 Different configurations are shown. In addition, Figure 3 The components shown may be implemented by software, hardware or a combination thereof.

[0085] See also Figure 4 , Figure 4 This is a flowchart of the distortion calibration method of the camera of the embodiment of the present application, which is applied to Figure 4 It should be noted that the camera distortion calibration method provided in the embodiment of the present application is not based on Figure 4 The order shown below is limited, and the method includes: step S101-step S102.

[0086] Step S101: obtaining the first coordinates of each grid point of the calibration plate image in the pixel coordinate system and the second coordinates in the world coordinate system, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and the extrinsic parameter of the camera; wherein the calibration plate image is collected by the camera.

[0087] Step S102: Determine the radial distortion coefficient of the camera based on the first coordinate and the second coordinate of each grid point of the calibration plate image, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and the extrinsic parameter of the camera, and the improved distortion algorithm; wherein the improved distortion algorithm includes a penalty term, and the penalty term is used to ensure that the distortion curve corresponding to the improved distortion algorithm has no zero crossing point.

[0088] Since the reason why the edge position of the image appears distorted is that the distortion curve corresponding to the current distortion algorithm has a zero-crossing point, in the embodiment of the application, the distortion algorithm is improved by adding a penalty term, so that the distortion curve corresponding to the improved distortion algorithm has no zero-crossing point. In this way, the situation that the image edge appears distorted in the de-distortion process can be effectively improved, and the accuracy and stability of camera calibration can be improved.

[0089] The above steps will be described in combination with specific examples.

[0090] As an implementation mode, step S101 specifically includes: acquiring a calibration board image; acquiring first coordinates of each grid point in the calibration board image in a pixel coordinate system and second coordinates of each grid point in a world coordinate system, calculating an initial value of a radial distortion coefficient of a camera, an intrinsic parameter matrix and an extrinsic parameter of the camera based on Zhang's algorithm.

[0091] Illustratively, the staff first needs to keep the electronic device and the calibration board in a fixed position and keep still. In order to directly determine the radial distortion coefficient of the camera subsequently, the number of calibration boards can be at least three, and the at least three calibration boards are at different folding angles. For example, the placement form of four calibration boards in the middle can be used. Figure 1 After fixing the calibration board and the electronic device, the calibration board is photographed by the electronic device to obtain the calibration board image. Usually in the operation process, the calibration board can be placed at the center of the figure. In addition, in order to obtain a calibration board image with good quality, correct focusing and reasonable lighting are required when photographing.

[0092] After obtaining the calibration board image, the image can be processed at this time, and the processing mode can include image filtering, gradient image and second derivative image. Among them, the noise can be removed by image filtering, and the first coordinates of each grid point of the calibration board image in the pixel coordinate system can be extracted by the gradient image and the second derivative image. In addition, when determining each grid point of the calibration board image, the grid points of the calibration board image can be determined in sequence from left to right and from top to bottom, and the mis-collected points with a neighboring distance less than a preset value can be removed. Moreover, in order to improve the accuracy of the coordinate points, the coordinates can also be sub-pixelized.

[0093] It should be noted that the second coordinates of each grid point in the calibration board image in the world coordinate system are determined by the placement angle of the calibration board. The first coordinates of each grid point in the calibration board image in the pixel coordinate system are continuous coordinate points in the image, and the second coordinates of each grid point in the calibration board image in the world coordinate system are related to the world coordinate system determined by each calibration board. Specifically, each calibration board takes the upper left corner grid of the board body as the origin of the world coordinate system, the plane on which the board body is located corresponds to the z-axis of the world coordinate system, the x-axis of the world coordinate system is in the direction of the width of the calibration board, and the y-axis of the world coordinate system is in the direction of the height of the calibration board. In addition, the size of the calibration board and the related information of the world coordinate system in the calibration board are prepared in the electronic device by the staff in advance.

[0094] After obtaining the first coordinates of each grid point in the calibration board image in the pixel coordinate system and the second coordinates in the world coordinate system, the electronic device can directly calculate the initial values of the intrinsic matrix, the extrinsic parameters and the radial distortion coefficient of the camera according to the Zhang algorithm. The specific process of the Zhang algorithm is as follows: determining the extrinsic parameters of the camera based on the relationship between the world coordinate system and the camera coordinate system of the camera itself; determining the intrinsic matrix of the camera based on the relationship between the camera coordinate system and the pixel coordinate system; and fitting the initial value of the radial distortion coefficient of the camera based on the first coordinates and the second coordinates of each grid point of the calibration board image, the extrinsic parameters of the camera and the intrinsic matrix of the camera.

[0095] It should be noted that the process of calibration by the Zhang algorithm involves the conversion of four coordinate systems. The four coordinate systems are: the world coordinate system, the camera coordinate system, the image coordinate system and the pixel coordinate system.

[0096] The world coordinate system is a three-dimensional world coordinate system defined by the staff and introduced to describe the position of the target object in the real world. The camera coordinate system is a coordinate system established on the camera itself and defined to describe the position of the object from the perspective of the camera, which serves as an intermediate link between the world coordinate system and the image / pixel coordinate system. The image coordinate system is introduced to describe the projection and transmission relationship of the object from the camera coordinate system to the image coordinate system in the imaging process, which facilitates the further obtaining of the coordinates in the pixel coordinate system. The pixel coordinate system is introduced to describe the coordinates of the image point on the digital image (photograph) after the imaging of the object, and is the coordinate system of the information actually read from the camera.

[0097] The Zhang algorithm can accurately determine the extrinsic parameters, the intrinsic matrix and the initial value of the radial distortion coefficient of the camera.

[0098] Since the specific calculation process of Zhang's algorithm is well known in the art, no further description is given.

[0099] It should be noted that when the Rational Model algorithm is used to calculate the radial distortion coefficients subsequently, the initial values k1-k6 of the radial distortion coefficients obtained by Zhang's algorithm are [k'1, k'2, k'3, 0, 0, 0]. Among them, k'1, k'2, k'3 are the values of the radial distortion coefficients k1, k2, k3 obtained by Zhang's algorithm, and the values of k4, k5, k6 are 0. When the Brown-Conrady Model algorithm is used to calculate the radial distortion coefficients subsequently, the initial values k1, k2, k3 of the radial distortion coefficients obtained by Zhang's algorithm are k'1, k'2, k'3.

[0100] It can be understood that the parameters in step S101 can also be directly input by the staff, and the present application does not limit this.

[0101] As an implementation manner, step S102 can specifically include steps S1-S4.

[0102] S1: determining the estimated projection point of each grid point based on the second coordinates of each grid point of the calibration board image in the world coordinate system, the initial values of the radial distortion coefficients of the camera, the intrinsic matrix and the extrinsic parameters of the camera, and the improved distortion algorithm; wherein the estimated projection point is a coordinate point in the pixel coordinate system.

[0103] S2: calculating the error value of the estimated projection point of each grid point and the first coordinates of each grid point in the pixel coordinate system.

[0104] S3: optimizing the error value to obtain the optimized radial distortion coefficients.

[0105] S4: substituting the optimized radial distortion coefficients into steps S1-S3 to replace the initial values of the radial distortion coefficients, so as to determine the radial distortion coefficients after re-optimization; and iteratively performing the above steps until the preset iteration condition is met, and outputting the optimized radial distortion coefficients corresponding to the latest error value; wherein the output optimized radial distortion coefficients corresponding to the latest error value are the radial distortion coefficients of the camera.

[0106] The above steps will be described below in combination with the Rational Model algorithm and the Brown-Conrady Model algorithm.

[0107] When the distortion algorithm is the Rational Model algorithm, the expression of the improved distortion algorithm based on the Rational Model algorithm is:

[0108]

[0109]

[0110] In formula (7) and formula (8), x d and y d Indicates the distorted third coordinate of the grid point in the image coordinate system; x d and y d Used to determine the above estimated projection point; x u and y u Indicates the fourth coordinate of the grid point without distortion in the image coordinate system; x u and y u Obtained through the second coordinate; k1~k6 represent radial distortion coefficients; Q(r 2 ) represents a penalty term. In the embodiment of the present application, the improved distortion algorithm adds a penalty term Q(r 2 ). By adding a penalty term Q(r 2 ) can ensure that the distortion curve corresponding to the improved Rational Model algorithm has no zero crossing points. In this way, the distortion of image edges during the dedistortion process using the Rational Model algorithm can be effectively improved.

[0111] In one embodiment, the penalty term Q(r 2 ) expression:

[0112] Q(r 2 )=Q1(r 2 )+Q2(r 2 ) (9)

[0113]

[0114]

[0115] In formulas (9) to (11), Q(r 2 ) represents the penalty term, Q1(r 2 ) represents F(r 2 ) at the edge of the field of view r = r max The penalty term of the function value, F(r 2 )=1+k1r 2 +k2r 4 +k3r 6 ;Q2(r 2 ) represents G(r 2 ) at the edge of the field of view r = rmax The penalty term of the function value, G(r 2 )=1+k4r 2 +k5r 4 +k6r 6 ;q represents the penalty error value; Represents the third coordinate x u and y u The maximum value of the sum of the squares of . In the above expression, The value of and Whether it is necessary to add a penalty error value q can simply and efficiently determine the penalty term Q(r 2 ) specific value, thereby effectively limiting the cubic curve corresponding to the Rational Model algorithm, in [0,r max ) to keep no zero crossing.

[0116] It should be noted that the penalty error value q can be set to a larger value, such as 10 or 100, without affecting the calibration effect. Of course, it can also be determined according to the number of grid points. For example, the penalty error value q can be the number of grid points multiplied by 8. This application does not limit this.

[0117] In another implementation, the penalty term Q(r 2 ) expression:

[0118] Q(r 2 )=Q1(r 2 )+Q2(r 2 )+Q3(r 2 )+Q4(r 2 ) (12)

[0119]

[0120]

[0121]

[0122]

[0123] In formulas (12) to (16), Q(r 2 ) represents the penalty term, Q1(r 2 ) represents F(r 2 ) at the edge of the field of view r = r max The penalty term of the function value; Q2(r 2 ) represents G(r 2 ) at the edge of the field of view r = r max The penalty sub-item of the function value; Q3(r 2) represents F(r 2 ) in the field of view range [0, r max ) function minimum value penalty subterm; Q4(r 2 ) represents G(r 2 ) in the field of view range [0, r max ) function minimum value G(r1 2 ) penalty subterm; q represents a penalty error value; represents the maximum value corresponding to the sum of squares of the third coordinates x u and y u ; represents the minimum value of F(r 2 ); F(r 2 ) = 1 + k1r 2 + k2r 4 + k3r 6 ; G(r1 2 ) represents the minimum value of G(r 2 ); G(r 2 ) = 1 + k4r 2 + k5r 4 + k6r 6 . It should be noted that by introducing the minimum value, and by the value of , whether and 1 + k4r1 2 + k5r1 4 + k6r1 6 need to add a penalty error value q can more strictly and accurately determine the specific value of the penalty term Q(r 2 ), and then effectively limit the corresponding cubic curve of the Rational Model algorithm to have no zero point in [0, r max ).

[0124] Correspondingly, the specific process of step S1 is: first, based on the camera's external parameters and the second coordinates of each grid point of the calibration board image in the world coordinate system, the fourth coordinates of each grid point of the calibration board image in the image coordinate system without distortion are determined, and the expression of the fourth coordinates without distortion is calculated as:

[0125]

[0126] In formula (17), [*] represents a matrix, (X, Y, Z) represents the second coordinates of the grid point in the world coordinate system; R and t are the external parameters of the camera, R is specifically a 3*3 vector, and t is specifically a 3*1 vector; s is an arbitrary numerical scaling factor.

[0127] After the fourth coordinate of each grid point in the image coordinate system without distortion is calculated, the third coordinate x d and y d with distortion in the image coordinate system is obtained based on formula (7) and formula (8) d and y d , and the intrinsic matrix of the camera, to obtain the estimated projection point of each grid point. The expression for calculating the estimated projection point is:

[0128]

[0129] In formula (18), u p and v p represent the estimated projection point of the grid point; represents the intrinsic matrix, wherein f represents the focal length, dx represents the pixel size in the x direction, dy represents the pixel size in the y direction, x0 and y0 represent the principal point coordinates.

[0130] Correspondingly, the specific process of step S2 is: calculating the first error value between the estimated projection point of each grid point and the first coordinate of the respective corresponding grid point in the pixel coordinate system, and then stacking all the first error values to obtain the error value.

[0131] The expression for calculating the first error value is:

[0132]

[0133] In formula (19), e x and e y represent a set of first error values, and u and v represent the first coordinate of the grid point in the pixel coordinate system. Since the stacking of the above vectors is well known in the art, it will not be described in detail here.

[0134] Correspondingly, in step S3, the optimizer is called to optimize the error value, and then the optimized radial distortion coefficients are obtained. It should be noted that the optimization algorithm performed by the optimizer can include but is not limited to the LM (Levenberg-Marquardt) algorithm, gradient descent method, etc. Since such optimization algorithms are also well known in the art, they will not be described in detail here.

[0135] Then, the optimized radial distortion coefficients are substituted into steps S1-S3 to replace the initial values of the radial distortion coefficients, and the radial distortion coefficients after re-optimization are determined; and the iteration is continued in this way until the preset iteration condition is met, and the optimized radial distortion coefficients corresponding to the latest error value are output as the radial distortion coefficients k1-k6 of the camera.

[0136] The preset iteration condition can refer to the minimum error value, or the error value no longer changes, or a fixed iteration number, such as ten times, twenty times, one hundred times, and the like, which is not limited in the application.

[0137] When the distortion algorithm is the Brown-Conrady Model algorithm, the expression of the improved distortion algorithm based on the Brown-Conrady Model algorithm is:

[0138] x d =x u (1+k1r 2 +k2r 4 +k3r 6 )+Q(r 2 ) (20)

[0139] y d =y u (1+k1r 2 +k2r 4 +k3r 6 )+Q(r 2 ) (21)

[0140] In the formula (20) and the formula (21), x d and y d represent the third coordinates of the grid points in the image coordinate system with distortion; x d and y d are used to determine the above-mentioned estimated projection points; x u and y u represent the fourth coordinates of the grid points in the image coordinate system without distortion; x u and y u are obtained through the second coordinates; k1-k3 represent the radial distortion coefficients; Q(r 2 ) represents the penalty term. In the embodiments of the application, the improved distortion algorithm adds a penalty term Q(r 2 ) to the original Brown-Conrady Model algorithm. By adding a penalty term Q(r 2 ) to the Brown-Conrady Model algorithm, the improved Brown-Conrady Model algorithm can have a zero-free distortion curve. In this way, the distortion process of the Brown-Conrady Model algorithm can effectively improve the distortion of the image edge.

[0141] In an embodiment, the expression of the penalty term Q(r 2 ) is:

[0142]

[0143] In formula (22), Q(r 2 ) represents F(r 2 ) at the edge of the field of view r = r max The penalty term of the function value, F(r 2 )=1+k1r 2 +k2r 4 +k3r 6 ;q represents the penalty error value; Represents the third coordinate x u and y u The maximum value of the sum of the squares of . In the above expression, The value of Whether it is necessary to add a penalty error value q can simply and efficiently determine the penalty term Q(r 2 ) specific value, thereby effectively limiting the cubic curve corresponding to the Brown-Conrady Model algorithm, in [0,r max ) to keep no zero crossing.

[0144] It should be noted that the penalty error value q can be set to a larger value, such as 10 or 100, without affecting the calibration effect. Of course, it can also be determined according to the number of grid points. For example, the penalty error value q can be the number of grid points multiplied by 8. This application does not limit this.

[0145] In another implementation, the penalty term Q(r 2 ) expression:

[0146] Q(r 2 )=Q5(r 2 )+Q6(r 2 ) (twenty three)

[0147]

[0148]

[0149] In formulas (23) to (25), Q5(r 2 ) represents F(r 2 ) at the edge of the field of view r = r max The penalty sub-item of the function value; Q6(r 2 ) represents F(r 2 ) in the field of view range [0,r max ) Minimum of the inner function The penalty sub-item; F(r 2 )=1+k1r 2 +k2r 4 +k3r6 ;q represents the penalty error value; Represents the third coordinate x u and y u The maximum value corresponding to the sum of squares; represents F(r 2 ) minimum value; F(r 2 )=1+k1r 2 +k2r 4 +k3r 6 It should be noted that by introducing the minimum value and The value of Whether it is necessary to add a penalty error value q can more strictly and accurately determine the penalty term Q(r 2 ) specific value, thereby effectively limiting the cubic curve corresponding to the Brown-Conrady Model algorithm, in [0,r max ) to keep no zero crossing.

[0150] Correspondingly, the specific process of step S1 is as follows: first, based on the extrinsic parameters of the camera and the second coordinates of each grid point of the calibration plate image in the world coordinate system, the fourth coordinate of each grid point of the calibration plate image is determined without distortion in the image coordinate system. The expression for calculating the fourth coordinate without distortion is:

[0151]

[0152] In formula (26), [*] represents a matrix, (X, Y, Z) represents the second coordinate of the grid point in the world coordinate system; R and t are the external parameters of the camera, R is specifically a 3*3 vector, and t is specifically a 3*1 vector; s is a scaling factor of an arbitrary value.

[0153] After calculating the fourth coordinate of each grid point without distortion in the image coordinate system, the third coordinate x with distortion in the image coordinate system is obtained based on formula (20) and formula (21): d and y d , then based on the distorted third coordinate x in the image coordinate system d and y d , the camera's intrinsic parameter matrix can be used to obtain the estimated projection point of each grid point. The expression for calculating the estimated projection point is:

[0154]

[0155] In formula (27), u p and v p Represents the estimated projection point of the grid point; Represents the intrinsic parameter matrix, where f represents the focal length, dx represents the pixel size in the x direction, dy represents the pixel size in the y direction, and x0 and y0 represent the coordinates of the principal point.

[0156] Correspondingly, the specific process of step S2 is: calculating a first error value between the estimated projection point of each grid point and the first coordinate of the respective corresponding grid point in the pixel coordinate system, and then stacking all the first error values to obtain the error value.

[0157] The expression for calculating the first error value is:

[0158]

[0159] In formula (28), e x and e y represent a set of first error values, and u and v represent the first coordinates of the grid points in the pixel coordinate system. Since the stacking of the above vectors is well known in the art, no further description is given here.

[0160] Correspondingly, in step S3, the error value is optimized by calling an optimizer, and then the optimized radial distortion coefficients are obtained. It should be noted that the optimization algorithm performed by the optimizer can include but is not limited to the LM (Levenberg-Marquardt) algorithm, gradient descent method, etc. Since such optimization algorithms are also well known in the art, no further description is given here.

[0161] Then, the optimized radial distortion coefficients are substituted into steps S1-S3 to replace the initial values of the radial distortion coefficients, and the radial distortion coefficients after re-optimization are determined; and the iteration is continued in this way until the preset iteration condition is met, and the optimized radial distortion coefficients corresponding to the latest error value are output as the radial distortion coefficients k1-k3 of the camera.

[0162] The preset iteration condition can mean that the error value is minimum, or that the error value no longer changes, or a fixed number of iterations can be set, such as ten times, twenty times, one hundred times, etc., which is not limited in the present application.

[0163] It should be noted that the above schemes are all for the calculation of the radial distortion coefficients, and if the tangential distortion coefficients also need to be calculated, only the last two tangential distortion terms in formula (1) and formula (2) need to be retained.

[0164] Please refer to Figure 5 , based on the same inventive concept, the present application also provides a camera imaging method, which can also be applied to the electronic device 100 shown in Figure 3 . It should be noted that the camera imaging method provided by the present application is not limited in the order shown in Figure 5 and the following. The method comprises steps S201-S203.

[0165] Step S201: obtaining first coordinates of each grid point of a calibration board image in a pixel coordinate system and second coordinates of each grid point of the calibration board image in a world coordinate system, an initial value of a radial distortion coefficient of a camera, an intrinsic matrix and an extrinsic parameter of the camera; wherein the calibration board image is captured by the camera.

[0166] Step S202: determining the radial distortion coefficient of the camera based on the first coordinates and the second coordinates of each grid point of the calibration board image, the initial value of the radial distortion coefficient of the camera, the intrinsic matrix and the extrinsic parameter of the camera, and an improved distortion algorithm; wherein the improved distortion algorithm comprises a penalty term, and the penalty term is used to make the distortion curve corresponding to the improved distortion algorithm have no zero point.

[0167] Since steps S201-S202 are the same as steps S101-S102 in the foregoing embodiment, to avoid redundancy, details are not repeated here, and the same parts are referred to each other.

[0168] Step S203: correcting the distortion of the image captured by the camera based on the radial distortion coefficient of the camera.

[0169] Finally, the camera can correct the distortion of the image captured by the camera based on the radial distortion coefficient. For example, if the radial distortion coefficients k1-k6 of the camera are obtained by using the improved Rational Model algorithm, the estimated projection point can be obtained according to formula (17) and formula (18), and the estimated projection point is the actual point after distortion correction. If the radial distortion coefficients k1-k3 of the camera are obtained by using the improved Brown-Conrady Model algorithm, the estimated projection point can be obtained according to formula (26) and formula (27), and the estimated projection point is the actual point after distortion correction.

[0170] Since the above two distortion algorithms are complex model algorithms with many parameters, the compensation added in the embodiment of the present application has little effect on the accuracy of the final calibration. The image after distortion correction obtained by the above method can be referred to as Figure 6 It can be seen that the above method can effectively improve the distortion of the image edge in the distortion correction process.

[0171] Please refer to Figure 7 Based on the same inventive concept, the embodiment of the present application also provides a distortion calibration device 200 of a camera, which comprises:

[0172] A first obtaining module 210 is configured to obtain first coordinates of each grid point of a calibration board image in a pixel coordinate system and second coordinates of each grid point of the calibration board image in a world coordinate system, an initial value of a radial distortion coefficient of a camera, an intrinsic matrix and an extrinsic parameter of the camera; wherein the calibration board image is captured by the camera.

[0173] The first processing module 220 is configured to determine the radial distortion coefficient of the camera based on the first coordinates and the second coordinates of each grid point of the calibration board image, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and the extrinsic parameter of the camera, and the improved distortion algorithm; wherein the improved distortion algorithm comprises a penalty term, and the penalty term is used to make the distortion curve corresponding to the improved distortion algorithm have no zero point.

[0174] Optionally, the first obtaining module 210 is specifically configured to obtain the first coordinates of each grid point of the calibration board image in the pixel coordinate system and the second coordinates of each grid point in the world coordinate system; determine the extrinsic parameter of the camera based on the relationship between the world coordinate system and the camera coordinate system of the camera itself; and / or determine the intrinsic parameter matrix of the camera based on the relationship between the camera coordinate system and the pixel coordinate system; and / or fit the initial value of the radial distortion coefficient of the camera based on the first coordinates and the second coordinates of each grid point of the calibration board image, the extrinsic parameter of the camera, and the intrinsic parameter matrix of the camera.

[0175] Optionally, the first processing module 220 is specifically configured to perform the following steps: S1, determine the estimated projection point of each grid point based on the second coordinates of each grid point of the calibration board image in the world coordinate system, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and the extrinsic parameter of the camera, and the improved distortion algorithm; wherein the estimated projection point is a coordinate point in the pixel coordinate system; S2, calculate the error value of the estimated projection point of each grid point and the first coordinates of each grid point in the pixel coordinate system; S3, optimize the error value to obtain the optimized radial distortion coefficient; S4, replace the initial value of the radial distortion coefficient with the optimized radial distortion coefficient in steps S1-S3 to determine the radial distortion coefficient after re-optimization; and iteratively perform the above steps until a preset iteration condition is met, and output the optimized radial distortion coefficient corresponding to the latest error value; wherein the output optimized radial distortion coefficient corresponding to the latest error value is the radial distortion coefficient of the camera.

[0176] Please refer to Figure 8 , based on the same inventive concept, the embodiment of the present application further provides a camera imaging device 300, which comprises:

[0177] The second obtaining module 310 is configured to obtain the first coordinates of each grid point of the calibration board image in the pixel coordinate system and the second coordinates of each grid point in the world coordinate system, the initial value of the radial distortion coefficient of the camera, and the intrinsic parameter matrix and the extrinsic parameter of the camera; wherein the calibration board image is collected by the camera.

[0178] The second processing module 320 is configured to determine the radial distortion coefficient of the camera based on the first coordinate and the second coordinate of each grid point of the calibration board image, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and the extrinsic parameter of the camera, and an improved distortion algorithm; wherein the improved distortion algorithm comprises a penalty term, and the penalty term is used to make the distortion curve corresponding to the improved distortion algorithm have no zero point.

[0179] The de-distortion module 330 is configured to correct the distortion of the image collected by the camera based on the radial distortion coefficient of the camera.

[0180] It should be noted that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be described here.

[0181] Based on the same inventive concept, the embodiments of the present application further provide a computer program product comprising computer program instructions, which, when read and executed by a processor, perform the method provided in the foregoing embodiments.

[0182] Based on the same inventive concept, the embodiments of the present application further provide a computer readable storage medium having a computer program stored thereon, which, when executed, performs the method provided in the foregoing embodiments.

[0183] The storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media sets. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0184] In the embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented by other ways. The device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there can be another division way in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, and can be electrical, mechanical or other forms.

[0185] In addition, the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0186] Furthermore, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0187] In this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations.

[0188] The above is only an embodiment of the present application and is not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various changes and variations. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A camera distortion calibration method, characterized in that: include: Obtaining a first coordinate of each grid point of the calibration plate image in a pixel coordinate system and a second coordinate in a world coordinate system, an initial value of a radial distortion coefficient of a camera, and an intrinsic parameter matrix and an extrinsic parameter of the camera; wherein the calibration plate image is acquired by the camera; Determining the radial distortion coefficient of the camera based on the first coordinate and the second coordinate of each grid point of the calibration plate image, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and the extrinsic parameter of the camera, and an improved distortion algorithm; wherein the improved distortion algorithm includes a penalty term, and the penalty term is used to ensure that the distortion curve corresponding to the improved distortion algorithm has no zero crossing point; The method of determining the radial distortion coefficient of the camera based on the first coordinates and the second coordinates of each grid point of the calibration plate image, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and the extrinsic parameter of the camera, and the improved distortion algorithm includes: S1: Determine an estimated projection point of each grid point based on the second coordinate of each grid point of the calibration plate image in the world coordinate system, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and extrinsic parameter of the camera, and the improved distortion algorithm; wherein the estimated projection point is a coordinate point in the pixel coordinate system; S2: Calculating an error value between the estimated projection point of each grid point and the first coordinate of each grid point in the pixel coordinate system; S3: Optimizing the error value to obtain an optimized radial distortion coefficient; S4: Substitute the optimized radial distortion coefficient into the initial value of the radial distortion coefficient in steps S1 to S3 to determine the radial distortion coefficient after further optimization; and iterate in this way until the preset iteration conditions are met, and then output the optimized radial distortion coefficient corresponding to the latest error value; wherein the optimized radial distortion coefficient corresponding to the latest error value output is the radial distortion coefficient of the camera.

2. The method according to claim 1, characterized in that Obtaining the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and the extrinsic parameter of the camera, including: Determining the extrinsic parameters of the camera based on the relationship between the world coordinate system and the camera coordinate system of the camera itself; and / or Determining an intrinsic parameter matrix of the camera based on a relationship between the camera coordinate system and the pixel coordinate system; and / or An initial value of the radial distortion coefficient of the camera is fitted based on the first coordinates and the second coordinates of each grid point of the calibration plate image, the external parameters of the camera, and the internal parameter matrix of the camera.

3. The method according to any one of claims 1-2, characterized in that The improved distortion algorithm is: Among them, x d and y d represents the distorted third coordinate of the grid point in the image coordinate system; d and y d Used to determine the estimated projection point; x u and y u represents the fourth coordinate of the grid point without distortion in the image coordinate system; u and y u Obtained by the second coordinate; k1~k6 represent radial distortion coefficients; Q(r 2 ) represents the penalty term.

4. The method according to claim 3, characterized in that The expression of the penalty term is: Q(r 2 )=Q1(r 2 )+Q2(r 2 ); Among them, Q1(r 2 ) represents F(r 2 ) at the edge of the field of view r = r max The penalty term of the function value, F(r 2 )=1+k1r 2 +k2r 4 +k3r 6 ;Q2(r 2 ) represents G(r 2 ) at the edge of the field of view r = r max The penalty term of the function value, G(r 2 )=1+k4r 2 +k5r 4 +k6r 6 ;q represents the penalty error value; Represents the third coordinate x u and y u The maximum value corresponding to the sum of squares; Or, the expression of the penalty term is: Q(r 2 )=Q1(r 2 )+Q2(r 2 )+Q3(r 2 )+Q4(r 2 ); Among them, Q1(r 2 ) represents F(r 2 ) at the edge of the field of view r = r max The penalty term of the function value; Q2(r 2 ) represents G(r 2 ) at the edge of the field of view r = r max The penalty sub-item of the function value; Q3(r 2 ) represents F(r 2 ) in the field of view range [0,r max ) Minimum of the inner function The penalty sub-item; Q4(r 2 ) represents G(r 2 ) in the field of view range [0,r max ) minimum value of the internal function G(r1 2 ) penalty sub-item; q represents the penalty error value; F(r 2 )=1+k1r 2 +k2r 4 +k3r 6 ;G(r 2 )=1+k4r 2 +k5r 4 +k6r 6 ; Represents the third coordinate x u and y u The maximum value corresponding to the sum of squares; represents F(r 2 ) minimum value; G(r1 2 ) represents G(r 2 ) is the minimum value of .

5. The method according to any one of claims 1-2, characterized in that The improved distortion algorithm is: x d =x u (1+k1r 2 +k2r 4 +k3r 6 )+Q(r 2 ); the d =y u (1+k1r 2 +k2r 4 +k3r 6 )+Q(r 2 ): Among them, x d and y d represents the distorted third coordinate of the grid point in the image coordinate system; d and y d Used to determine the estimated projection point; x u and y u represents the fourth coordinate of the grid point without distortion in the image coordinate system; u and y u Obtained by the second coordinate; k1~k3 represent radial distortion coefficients; Q(r 2 ) represents the penalty term.

6. The method according to claim 5, characterized in that The expression of the penalty term is: Among them, Q(r 2 ) represents F(r 2 ) at the edge of the field of view r = r max The penalty term of the function value, F(r 2 )=1+k1r 2 +k2r 4 +k3r 6 ;q represents the penalty error value; Represents the third coordinate x u and y u The maximum value corresponding to the sum of squares; Or, the expression of the penalty term is: Q(r 2 )=Q5(r 2 )+Q6(r 2 ); Among them, Q5(r 2 ) represents F(r 2 ) at the edge of the field of view r = r max The penalty sub-item of the function value; Q6(r 2 ) represents F(r 2 ) in the field of view range [0,r max ) Minimum of the inner function The penalty sub-item; F(r 2 )=1+k1r 2 +k2r 4 +k3r 6 ;q represents the penalty error value; Represents the third coordinate x u and y u The maximum value corresponding to the sum of squares; represents F(r 2 ) minimum value; F(r 2 )=1+k1r 2 +k2r 4 +k3r 6 .

7. The method according to claim 1, characterized in that The calibration plate image includes images of the calibration plate at at least three different angles.

8. A camera imaging method, characterized in that: include: Obtaining a first coordinate of each grid point of the calibration plate image in a pixel coordinate system and a second coordinate in a world coordinate system, an initial value of a radial distortion coefficient of a camera, and an intrinsic parameter matrix and an extrinsic parameter of the camera; wherein the calibration plate image is acquired by the camera; The radial distortion coefficient of the camera is determined based on the first coordinates and the second coordinates of each grid point of the calibration plate image, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and the extrinsic parameter of the camera, and the improved distortion algorithm; wherein the improved distortion algorithm includes a penalty term, and the penalty term is used to make the distortion curve corresponding to the improved distortion algorithm have no zero crossing point; wherein, the radial distortion coefficient of the camera is determined based on the first coordinates and the second coordinates of each grid point of the calibration plate image, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and the extrinsic parameter of the camera, and the improved distortion algorithm, including: S1: based on the second coordinates of each grid point of the calibration plate image in the world coordinate system, the initial value of the radial distortion coefficient of the camera, the intrinsic parameter matrix and External parameters, and the improved distortion algorithm determine the estimated projection point of each grid point; wherein the estimated projection point is a coordinate point in the pixel coordinate system; S2: calculating the error value between the estimated projection point of each grid point and the first coordinate of each grid point in the pixel coordinate system; S3: optimizing the error value to obtain an optimized radial distortion coefficient; S4: substituting the optimized radial distortion coefficient into the initial value of the radial distortion coefficient in steps S1 to S3 to determine the radial distortion coefficient optimized again; and so on, iterating until the preset iteration condition is met, and outputting the optimized radial distortion coefficient corresponding to the latest error value; wherein the output optimized radial distortion coefficient corresponding to the latest error value is the radial distortion coefficient of the camera; The distortion of the image captured by the camera is corrected based on the radial distortion coefficient of the camera.

9. A computer program product, characterized in that The method comprises computer program instructions, and when the computer program instructions are read and executed by a processor, the method according to any one of claims 1 to 7 or the method according to any one of claim 8 is executed.

10. An electronic device, characterized in that: include: a processor and a memory, the processor and the memory being connected; The memory is used to store programs; The processor is configured to run a program stored in the memory, and execute the method according to any one of claims 1 to 7, or execute the method according to any one of claims 8.

11. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is run by a computer, the computer program executes the method according to any one of claims 1 to 7, or executes the method according to any one of claim 8.

Images