Camera Distortion Center Calibration Method, Device, Terminal Device and Medium

Abstract

This application is applicable to the field of camera technology, and particularly relates to a method, device, terminal device and medium for correcting the center of distortion of a camera. According to the internal parameters and distortion parameters of the camera, this method undistorts the original center coordinates through an undistortion model to obtain the undistorted center coordinates, and obtains the world coordinate matrix representation of the ring in the world coordinate system based on the radius of the ring, the external parameters of the camera and the undistorted center coordinates. The world coordinate matrix representation is projected onto the image coordinate system to obtain the projected image coordinate matrix representation, thereby determining the geometric center coordinates of the image coordinate matrix representation. According to the error between the undistorted center coordinates and the geometric center coordinates, the original center coordinates are corrected, realizing the correction of the original center coordinates before calibration, improving the accuracy of subsequent calibration using the corrected original center coordinates, and thus improving the accuracy of visual measurement.

Description

Technical Field

[0001] This application belongs to the technical field of cameras, and particularly relates to a method, device, terminal device and medium for correcting the center of distortion removal of a camera. Background Technique

[0002] Currently, in a three-dimensional vision system, in order to determine the mutual relationship between a point on the surface of a spatial object in three-dimensional space and its corresponding point in the camera imaging, it is necessary to establish a geometric model of the camera imaging, and the process of solving the geometric model parameters is camera calibration. Camera calibration is the bridge connecting images and the three-dimensional world, which determines the conversion relationship between physical measurement units and image units, and its calibration accuracy directly determines the measurement accuracy. Therefore, before using the vision system for high-precision measurement, it is necessary to accurately calibrate various parameters of the camera, generally including the internal parameters, external parameters and distortion parameters of the camera. Due to the possible coaxiality deviation and lens distortion between the lenses of the camera lens module during the actual assembly and manufacturing process, the image of the calibration board with a ring collected by the camera and the center coordinates of the ring extracted from the image have deviations, resulting in a large error in the calibration result when using the center coordinates for camera calibration, and the calibration accuracy is low, affecting the accuracy of visual measurement. Summary of the Invention

[0003] In view of this, the embodiments of this application provide a method, device, terminal device and medium for correcting the center of distortion removal of a camera to solve the problem of low calibration accuracy caused by the deviation of the center coordinates extracted from the image in the prior art.

[0004] In a first aspect, the embodiments of this application provide a method for correcting the center of distortion removal of a camera, and the method for correcting the center of distortion removal includes:

[0005] According to the first internal parameter and the first distortion parameter of the camera, undistort the original center coordinates to obtain undistorted center coordinates, where the original center coordinates are the coordinates of the center of the ring in the calibration board image in the image coordinate system, and the calibration board image is the image of the calibration board with a ring collected by the camera;

[0006] According to the radius of the ring, the first internal parameter, the first external parameter of the camera and the undistorted center coordinates, obtain the world coordinate matrix representation of the ring in the world coordinate system of the calibration board;

[0007] Project the world coordinate matrix representation into the image coordinate system to obtain the projected image coordinate matrix representation, and determine the geometric center coordinates of the image coordinate matrix representation;

[0008] Correct the original center coordinates according to the error between the undistorted center coordinates and the geometric center coordinates.

[0009] In a second aspect, an undistortion center correction device for a camera provided by an embodiment of the present application includes:

[0010] An undistortion module, configured to undistort the original center coordinates according to the first internal parameter and the first distortion parameter of the camera, so as to obtain undistorted center coordinates, where the original center coordinates are the coordinates of the center of the ring in the calibration plate image in the image coordinate system, and the calibration plate image is an image of a calibration plate with a ring collected by the camera;

[0011] A matrix representation determination module, configured to obtain a world coordinate matrix representation of the ring in the world coordinate system of the calibration plate according to the radius of the ring, the first internal parameter, the first external parameter of the camera, and the undistorted center coordinates;

[0012] A center coordinate determination module, configured to project the world coordinate matrix representation onto the image coordinate system to obtain a projected image coordinate matrix representation, and determine the geometric center coordinates of the image coordinate matrix representation;

[0013] An original coordinate correction module, configured to correct the original center coordinates according to the error between the undistorted center coordinates and the geometric center coordinates.

[0014] In a third aspect, an embodiment of the present application provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the undistortion center correction method described in the first aspect is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the undistortion center correction method described in the first aspect is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device is caused to execute the undistortion center correction method described in the first aspect above.

[0017] The beneficial effects of the embodiments of the present application compared with the prior art are as follows: According to the internal parameters and distortion parameters of the camera, the present application undistorts the original center coordinates of the circle to obtain the undistorted center coordinates of the circle. Then, according to the radius of the ring, the external parameters of the camera, and the undistorted center coordinates of the circle, the world coordinate matrix representation of the ring in the world coordinate system is obtained. The world coordinate matrix representation is projected onto the image coordinate system to obtain the projected image coordinate matrix representation, so as to determine the geometric center coordinates of the image coordinate matrix representation. According to the error between the undistorted center coordinates and the geometric center coordinates, the original center coordinates are corrected, realizing the correction of the original center coordinates before calibration, improving the accuracy of subsequent calibration using the corrected original center coordinates, and thus improving the accuracy of visual measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for use in the embodiments or the description of the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 is a schematic flowchart of a method for correcting the undistorted center of a camera provided in Embodiment 1 of the present application;

[0020] Figure 2 is a schematic structural diagram of a device for correcting the undistorted center of a camera provided in Embodiment 2 of the present application;

[0021] Figure 3 is a schematic structural diagram of a terminal device provided in Embodiment 3 of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are presented in order to thoroughly understand the embodiments of the present application. However, those skilled in the art should understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0023] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0024] It should also be understood that the term "and / or" as used in the specification of this application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0025] As used in the specification of this application and the appended claims, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" depending on the context.

[0026] In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are only used for differential description and should not be construed as indicating or implying relative importance.

[0027] Reference to "one embodiment" or "some embodiments" etc. described in the specification of this application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0028] A method for correcting the distortion center of a camera provided by an embodiment of this application can be applied to devices such as a palm computer, a desktop computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud server, a personal digital assistant (PDA), etc. The embodiment of this application does not impose any restrictions on the specific type of the terminal device.

[0029] It should be understood that the magnitudes of the sequence numbers of the steps in the following embodiments do not mean the order of execution is prior or posterior. The order of execution of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiment of this application.

[0030] To illustrate the technical solution of this application, the following specific embodiments are used for illustration.

[0031] As Figure 1FIG. 1 is a flow chart of a method for correcting the center of a camera's dedistortion provided in the first embodiment of the present application. The method may include the following steps:

[0032] Step S101 : Dedistorting the original circle center coordinates according to a first intrinsic parameter and a first distortion parameter of the camera to obtain dedistorted circle center coordinates.

[0033] The original circle center coordinates are the coordinates of the circle center in the calibration plate image in the image coordinate system, and the calibration plate image is the image of the calibration plate with the circle captured by the camera.

[0034] The calibration plate is a circular grid, such as white circles on a white background, or black circles on a white background. The circular grid can be a symmetrical or asymmetric grid. In a symmetrical grid, all circles have the same radius and are evenly distributed across the calibration plate. In an asymmetric grid, the circles can have varying radii and do not need to be evenly distributed.

[0035] The camera captures the calibration plate, obtaining an image of the calibration plate. The calibration plate image is then recognized to determine the ring and its center. The coordinates of the pixel corresponding to the circle center in the calibration plate image are the original center coordinates. The original center coordinates can be the coordinates of the center of any ring in the calibration plate image.

[0036] Due to the presence of the lens in the camera lens, light will be distorted during the projection process. The process of converting the camera coordinate system to the image coordinate system will also produce distortion. The distortion is corrected by the distortion model. The first distortion parameter of the camera includes the radial distortion coefficient and the tangential distortion coefficient. Among them, the radial distortion coefficient can include three parameters: K1, K2, and K3. K1 represents the first-order radial distortion coefficient, K2 represents the second-order radial distortion coefficient, and K3 represents the third-order radial distortion coefficient. The tangential distortion coefficient can include two parameters: P1 and P2. P1 represents the first-order tangential distortion coefficient, and P2 represents the second-order tangential distortion coefficient. The number of parameters in the distortion parameter can increase with the improvement of accuracy. Radial distortion can also be called barrel distortion and pincushion distortion. It appears in the process of converting the camera coordinate system to the physical coordinate system. The reason for the tangential distortion is that the lens surface in the camera lens is not completely parallel to the image.

[0037] The camera's first intrinsic parameters include the focal length and the translation of the origin. The origin of the image coordinate system is typically the upper-left corner of the image. The camera's first intrinsic parameters and first distortion parameters are derived from factory testing or calibration. Both are built-in parameters of the camera.

[0038] Optionally, dedistorting the original circle center coordinates according to the first intrinsic parameter and the first distortion parameter of the camera, and obtaining the dedistorted circle center coordinates includes:

[0039] Input the original center coordinates, the first intrinsic parameters of the camera, and the first distortion parameters into the undistortion model, and combine with the interpolation method to obtain the undistorted center coordinates.

[0040] Among them, the undistortion process can map the undistorted image coordinates to the distorted image coordinates, or map the distorted image coordinates to the undistorted image coordinates. Since the process of mapping from the distorted image coordinates to the undistorted image coordinates is the inverse process of adding distortion and requires iteration in the algorithm, which is more cumbersome. Therefore, the undistortion in this application is from the undistorted image coordinates to the distorted image coordinates.

[0041] According to mapping the undistorted image coordinates to the distorted image coordinates, the undistortion model can be obtained as follows:

[0042] u_ud = x d *f x +c x

[0043] v_ud = y d *f y +c y

[0044] In the formula, (u_ud, v_ud) are the undistorted center coordinates;

[0045]

[0046]

[0047]

[0048] Among them, c x 、c y 、f x 、f y are all intrinsic parameters, K1 to K8, P1 to P 10 are all distortion parameters, is the high-order correction term of x d , m(i, j) is the high-order correction coefficient of the i-th power of x d corresponding to the i-th power of x and the j-th power of y, is the high-order correction term of y d , n(i, j) is the high-order correction coefficient of the i-th power of y d corresponding to the i-th power of x and the j-th power of y, (x, y) are the original center coordinates.

[0049] Input the original center coordinates, the first intrinsic parameters of the camera, and the first distortion parameters into the undistortion model, and solve the model by using the interpolation method to obtain the undistorted center coordinates.

[0050] Step S102: Obtain the matrix representation of the world coordinates of the ring in the world coordinate system of the calibration board based on the radius of the ring, the first intrinsic parameters, the first extrinsic parameters of the camera, and the coordinates of the undistorted center.

[0051] Among them, the first extrinsic parameters of the camera include the orthogonal rotation matrix R and the translation matrix T. The orthogonal rotation matrix R and the translation matrix T are the transformation matrices between the world coordinate system and the camera coordinate system, and the first extrinsic parameters of the camera can be obtained during the camera operation. The radius of the ring refers to the true radius of the ring on the calibration board.

[0052] Transform the coordinates of the undistorted center to the world coordinate system according to the first extrinsic parameters of the camera to obtain the coordinates of the center in the world coordinate system. Then, combined with the radius of the ring, the matrix representation of the world coordinates of the ring can be determined, that is, the matrix representation of the ring in the world coordinate system. The matrix representation can refer to describing the characteristic parameters of the curve to be represented in the form of a matrix, which helps to perform operations using algorithms. The structure of the matrix in the matrix representation determines the image shape corresponding to the curve to be represented. For example, the image shape corresponding to the curve represented by a 3×3 matrix is a circle or an ellipse.

[0053] Optionally, obtaining the matrix representation of the world coordinates of the ring in the world coordinate system of the calibration board based on the radius of the ring, the first intrinsic parameters, the first extrinsic parameters of the camera, and the coordinates of the undistorted center includes:

[0054] Project the coordinates of the undistorted center to the world coordinate system of the calibration board according to the first extrinsic parameters of the camera to obtain the projected center coordinates;

[0055] Obtain the matrix representation of the world coordinates of the ring in the world coordinate system based on the radius of the ring and the projected center coordinates.

[0056] Among them, the homography matrix H from image coordinates to world coordinates can be determined according to the first extrinsic parameters and the first intrinsic parameters of the camera w2c :

[0057]

[0058] In the formula, r 00 ~r 21 are the parameters in the orthogonal rotation matrix R, and t0~t2 are the parameters in the translation matrix T.

[0059] According to the homography matrix H w2c and the coordinates of the undistorted center, the projected center coordinates (cx_w, cy_w) of the coordinates of the undistorted center in the world coordinate system can be obtained.

[0060] The matrix representation Cw of the world coordinates of the ring in the world coordinate system can be obtained through the actual radius z of the ring in the world coordinate system as follows:

[0061]

[0062] Step S103: Project the world coordinate matrix representation onto the image coordinate system to obtain the projected image coordinate matrix representation, and determine the geometric center coordinates of the image coordinate matrix representation.

[0063] Among them, according to the homography matrix H from the image coordinates to the world coordinates w2c , it can be known that the homography matrix for projecting the world coordinate system onto the image coordinate system is H c2w , and this H c2w is the T w2c transpose matrix of

[0064] According to this homography matrix H c2w and the world coordinate matrix representation of the circular ring, the matrix representation after projecting the circular ring onto the image coordinate system can be determined, that is, the image coordinate matrix representation Cc, as follows: Cc = H c2w -T * Cw * H c2w -1 . According to this image coordinate matrix representation, the coordinates of its geometric center can be determined.

[0065] Optionally, determining the geometric center coordinates of the image coordinate matrix representation includes:

[0066] If the image shape corresponding to the image coordinate matrix representation is an ellipse, determine the center coordinates of the ellipse as the geometric center coordinates of the image coordinate matrix representation.

[0067] Among them, due to the existence of distortion, when converting the world coordinate matrix representation of the circular ring in the world coordinate system to the image coordinate matrix representation in the image coordinate system, the image shape corresponding to this image coordinate matrix representation may be an ellipse, no longer a perfect circle. Therefore, when the image shape corresponding to the image matrix representation is an ellipse, the center of the ellipse is the geometric center of the image coordinate matrix representation. If the image coordinate matrix representation Cc is:

[0068]

[0069] In the formula, c0 to c8 are respectively the results of the corresponding items in H c2w -T * Cw * H c2w [[ID=4,6]] -1 and H c2w -T * Cw * H c2w -1 Specifically as follows:

[0070]

[0071] According to the calculation formula of the center of the ellipse, the geometric center coordinates (cx_e, cy_e) can be obtained as follows:

[0072]

[0073]

[0074] Among them,

[0075] Step S104: Correct the original center coordinates according to the error between the undistorted center coordinates and the geometric center coordinates.

[0076] Among them, the inaccuracy of the first external parameter of the camera causes an error between the undistorted center coordinates and the geometric center coordinates. Therefore, there is also an error caused by the first external parameter of the camera in the extraction of the original center coordinates in the calibration plate image. According to this error, the original center coordinates can be corrected. For example, add the corresponding error term to the coordinate items of the original center coordinates.

[0077] The error between the undistorted center coordinates (u_ud, v_ud) and the geometric center coordinates (cx_e, cy_e) is (err_x, err_y). Therefore, correcting the original center coordinates (X, Y) is:

[0078]

[0079] Optionally, after correcting the original center coordinates, it further includes:

[0080] According to the corrected original center coordinates of all the rings in the calibration plate image, use the Zhang Zhengyou calibration method to determine the second internal parameter, the second external parameter and the second distortion parameter of the camera for the calibration plate image;

[0081] Use the gradient descent method to optimize the second internal parameter, the second external parameter and the second distortion parameter to obtain the third internal parameter, the third external parameter and the third distortion parameter.

[0082] Among them, the camera calibration process is as follows:

[0083] First, obtain N sampling images, generally calibration plate images with feature points (including the centers of the rings), where N is an integer greater than or equal to 3;

[0084] Secondly, extract the pixel coordinates of the feature points in the calibration plate image;

[0085] Then, according to the pixel coordinates, estimate the five internal parameters and six external parameters in the case of ideal undistortion;

[0086] Finally, apply the least squares method model to estimate the distortion coefficient.

[0087] After extracting and correcting the center coordinates of all the rings in the calibration plate image, according to the extracted and corrected center coordinates and the world coordinates of each center in the world coordinate system of the calibration plate, the Zhang-Zhengyou calibration method is used to determine the second internal parameters, second external parameters and second distortion parameters of the camera. Then, through the gradient descent method, the second internal parameters, second external parameters and second distortion parameters are optimized to obtain the finally optimized internal and external parameters and distortion coefficients, namely the third internal parameters, third external parameters and third distortion coefficients. Correcting the center coordinates can reduce the extraction error of feature points.

[0088] The depth accuracy of the images captured by the camera includes absolute accuracy and relative accuracy. The absolute accuracy is reflected in the measurement error between the depth value of a single point and the true distance, and the relative accuracy is reflected in the overall fluctuation on the depth plane. The more accurate the internal parameters, external parameters and distortion coefficients obtained by the above optimization are, the higher the accuracy of calculating the depth of the image.

[0089] In the calibration process of the camera, the adopted distortion fitting equation adds high-order correction terms, the order of radial distortion and the order of tangential distortion on the basis of the Brown model, thus improving the distortion fitting ability and retaining the adaptability of the Brown model.

[0090] The distortion fitting equation is as follows:

[0091]

[0092]

[0093] Among them, a total of 8 radial distortion coefficients K1 to K8 can increase the order of radial distortion, and a total of 10 tangential distortion coefficients P1 to P 10 can increase the order of tangential distortion. is the high-order correction term of x d and m(i, j) is the high-order correction coefficient of x d corresponding to the i-th power of x and the j-th power of y. is the high-order correction term of y d and n(i, j) is the high-order correction coefficient of y d corresponding to the i-th power of x and the j-th power of y.

[0094] In an embodiment of the present application, according to the internal parameters and distortion parameters of the camera, the original center coordinates are undistorted to obtain the undistorted center coordinates. Then, according to the radius of the circular ring, the external parameters of the camera, and the undistorted center coordinates, the world coordinate matrix representation of the circular ring in the world coordinate system is obtained. The world coordinate matrix representation is projected onto the image coordinate system to obtain the projected image coordinate matrix representation, thereby determining the geometric center coordinates of the image coordinate matrix representation. According to the error between the undistorted center coordinates and the geometric center coordinates, the original center coordinates are corrected, realizing the correction of the original center coordinates before calibration, improving the accuracy of subsequent calibration using the corrected original center coordinates, and thus improving the accuracy of visual measurement.

[0095] The method for correcting the undistorted center of the camera corresponding to the above embodiment is applied to a terminal device. Figure 2 FIG. shows the structural block diagram of the device for correcting the undistorted center of the camera provided in the second embodiment of the present application. For the sake of convenience of description, only the parts related to the embodiments of the present application are shown.

[0096] See Figure 2 , the device for correcting the undistorted center includes:

[0097] The undistortion module 21 is configured to undistort the original center coordinates according to the first internal parameters and the first distortion parameters of the camera to obtain the undistorted center coordinates, where the original center coordinates are the coordinates of the center of the circular ring in the calibration plate image in the image coordinate system, and the calibration plate image is an image of the calibration plate with a circular ring collected by the camera;

[0098] The matrix representation determination module 22 is configured to obtain the world coordinate matrix representation of the circular ring in the world coordinate system of the calibration plate according to the radius of the circular ring, the first internal parameters, the first external parameters of the camera, and the undistorted center coordinates;

[0099] The center coordinate determination module 23 is configured to project the world coordinate matrix representation onto the image coordinate system to obtain the projected image coordinate matrix representation and determine the geometric center coordinates of the image coordinate matrix representation;

[0100] The original coordinate correction module 24 is configured to correct the original center coordinates according to the error between the undistorted center coordinates and the geometric center coordinates.

[0101] Optionally, the center coordinate determination module 23 includes:

[0102] The ellipse center coordinate determination unit is configured to determine the center coordinates of the ellipse as the geometric center coordinates of the image coordinate matrix representation when the shape of the image corresponding to the image coordinate matrix representation is an ellipse.

[0103] Optionally, the undistortion module 21 includes:

[0104] A de-distortion unit, which is configured to input the original center coordinates, the first internal parameters of the camera, and the first distortion parameters into a de-distortion model, and combine with the interpolation method to obtain the de-distorted center coordinates.

[0105] Optionally, the above de-distortion model is:

[0106] u_ud = x d *f x +c x

[0107] v_ud = y d *f y +c y

[0108] In the formula, (u_ud, v_ud) are the de-distorted center coordinates;

[0109]

[0110]

[0111]

[0112] Among them, c x , c y , f x , f y all belong to the first internal parameters, K1 to K8, P1 to P 10 all belong to the first distortion parameters, is the high-order correction term of x d , m(i, j) is the high-order correction coefficient corresponding to the i-th power of x and the j-th power of y of x d , is the high-order correction term of y d , n(i, j) is the high-order correction coefficient corresponding to the i-th power of x and the j-th power of y of y d , (x, y) are the original center coordinates.

[0113] Optionally, the matrix representation determination module 22 includes:

[0114] A projection unit, which is configured to project the de-distorted center coordinates into the world coordinate system of the calibration board according to the first external parameters of the camera to obtain the projected center coordinates;

[0115] A matrix representation determination unit, which is configured to obtain the matrix representation of the world coordinates of the ring in the world coordinate system according to the radius of the ring and the projected center coordinates.

[0116] Optionally, the de-distorted center coordinate correction device further includes:

[0117] The calibration module is used to, after correcting the original center coordinates of the circles, determine the second internal parameters, second external parameters, and second distortion parameters of the camera with respect to the calibration plate image according to the corrected original center coordinates of all the circles in the calibration plate image by using the Zhang Zhengyou calibration method.

[0118] The first optimization module is used to optimize the second internal parameters, second external parameters, and second distortion parameters by using the gradient descent method to obtain the third internal parameters, third external parameters, and third distortion parameters.

[0119] Optionally, the above calibration module includes:

[0120] The first determination unit is used to determine the second internal parameters and second external parameters of the camera with respect to the calibration plate image.

[0121] The second determination unit is used to determine the second distortion parameters according to the second internal parameters and second external parameters in combination with the least squares method model.

[0122] It should be noted that the information interaction, execution process, etc. between the above modules, due to being based on the same concept as the method embodiment of the present application, for their specific functions and the technical effects brought, reference can be specifically made to the method embodiment part, and details will not be elaborated here.

[0123] Figure 3 This is a schematic structural diagram of a terminal device provided in Embodiment 4 of the present application. As Figure 3 shown, the terminal device 3 of this embodiment includes: at least one processor 30 ( Figure 3 only one is shown in the figure), a memory 31, and a computer program 32 stored in the memory 31 and executable on at least one processor 30. When the processor 30 executes the computer program 32, the steps in any of the above method embodiments of the distortion removal center correction method for each camera are implemented.

[0124] The terminal device may include, but is not limited to, the processor 30 and the memory 31. Those skilled in the art can understand that Figure 3 this is only an example of the terminal device 3 and does not constitute a limitation on the terminal device 3. It may include more or fewer components than shown in the figure, or combine some components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0125] The so-called processor 30 may be a CPU, and the processor 30 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0126] The memory 31 may be an internal storage unit of the terminal device 3 in some embodiments, such as the hard disk or memory of the terminal device 3. The memory 31 may also be an external storage device of the terminal device 3 in some other embodiments, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 3. Further, the memory 31 may also include both the internal storage unit and the external storage device of the terminal device 3. The memory 31 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of a computer program. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0127] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above device can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above method embodiments of this application, a computer program can be used to instruct the relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device capable of carrying the computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0128] All or part of the processes in the above method embodiments of this application can also be completed by a computer program product. When the computer program product runs on a terminal device, the terminal device can be made to execute the steps in the above method embodiments.

[0129] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0130] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0131] In the embodiments provided in this application, it should be understood that the disclosed device / terminal device and method can be implemented in other ways. For example, the device / terminal device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0132] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0133] The above embodiments are only used to illustrate the technical solutions of this application, rather than to limit them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included in the protection scope of this application.

Claims

1. A method for correcting the center of distortion removal of a camera, characterized in that, The distortion removal center correction method includes: According to the first internal parameters and the first distortion parameters of the camera, undistort the original center coordinates to obtain the undistorted center coordinates, including: input the original center coordinates, the first internal parameters and the first distortion parameters of the camera into the undistortion model, and combine the interpolation method to obtain the undistorted center coordinates. The undistortion model is: where (u_ud, v_ud) are the undistorted center coordinates; Among them, c x , c y , f x , f y all belong to the first internal reference, and K1 to K8, P1 to P 10 all belong to the first distortion parameter. is the high-order correction term of x d , m(i, j) is the high-order correction coefficient corresponding to the i-th power of x and the j-th power of y of x d . is the high-order correction term of y d , n(i, j) is the high-order correction coefficient corresponding to the i-th power of x and the j-th power of y of y d . (x, y) is the original center coordinate, where the original center coordinate is the coordinate of the center of the ring in the calibration plate image in the image coordinate system, and the calibration plate image is the image of the calibration plate with a ring collected by the camera; Obtaining the world coordinate matrix representation of the ring in the world coordinate system of the calibration board based on the radius of the ring, the first internal parameter, the first external parameter of the camera, and the distortion removal center coordinates, including: projecting the distortion removal center coordinates into the world coordinate system of the calibration board according to the first external parameter of the camera to obtain the projected center coordinates; obtaining the world coordinate matrix representation of the ring in the world coordinate system according to the radius of the ring and the projected center coordinates; Projecting the world coordinate matrix representation into the image coordinate system to obtain the projected image coordinate matrix representation, and determining the geometric center coordinates of the image coordinate matrix representation; Correcting the original center coordinates according to the error between the distortion removal center coordinates and the geometric center coordinates.

2. The de-distortion center correction method according to claim 1, characterized in that The determination of the geometric center coordinates of the image coordinate matrix representation includes: When the image shape corresponding to the image coordinate matrix representation is an ellipse, determining the center coordinates of the ellipse as the geometric center coordinates of the image coordinate matrix representation.

3. The de - distortion center correction method according to any one of claims 1 to 2, characterized in that, After correcting the original center coordinates, it further includes: According to the corrected original center coordinates of all the rings in the calibration board image, using the Zhang Zhengyou calibration method to determine the second internal parameter, the second external parameter, and the second distortion parameter of the camera for the calibration board image; Using the gradient descent method to optimize the second internal parameter, the second external parameter, and the second distortion parameter to obtain the third internal parameter, the third external parameter, and the third distortion parameter.

4. The de-distortion center correction method according to claim 3, characterized in that The determination of the second internal parameter, the second external parameter, and the second distortion parameter of the camera for the calibration board image includes: Determining the second internal parameter and the second external parameter of the camera for the calibration board image; Determining the second distortion parameter according to the second internal parameter and the second external parameter in combination with the least squares model.

5. A de-distortion center correction device for a camera, characterized in that, The distortion removal center correction device includes: The undistortion module is used to undistort the original center coordinates according to the first internal parameter and the first distortion parameter of the camera, and obtain the undistorted center coordinates, including: inputting the original center coordinates, the first internal parameter and the first distortion parameter of the camera into the undistortion model, and combining the interpolation method to obtain the undistorted center coordinates. The undistortion model is: where (u_ud, v_ud) are the undistorted center coordinates; Among them, c x and c y , f x and f y all belong to the first internal reference, K1 to K8, P1 to P 10 all belong to the first distortion parameter. is the high-order correction term of x d , m(i, j) is the high-order correction coefficient corresponding to the i-th power of x d and the j-th power of y, is the high-order correction term of y d , n(i, j) is the high-order correction coefficient corresponding to the i-th power of x d and the j-th power of y, (x, y) is the original center coordinate, where the original center coordinate is the coordinate of the center of the ring in the calibration plate image in the image coordinate system, and the calibration plate image is the image of the calibration plate with a ring collected by the camera; A matrix representation determination module for obtaining the world coordinate matrix representation of the ring in the world coordinate system of the calibration board based on the radius of the ring, the first internal parameter, the first external parameter of the camera, and the distortion removal center coordinates, including: projecting the distortion removal center coordinates into the world coordinate system of the calibration board according to the first external parameter of the camera to obtain the projected center coordinates; obtaining the world coordinate matrix representation of the ring in the world coordinate system according to the radius of the ring and the projected center coordinates; A center coordinate determination module for projecting the world coordinate matrix representation into the image coordinate system to obtain the projected image coordinate matrix representation, and determining the geometric center coordinates of the image coordinate matrix representation; An original coordinate correction module for correcting the original center coordinates according to the error between the distortion removal center coordinates and the geometric center coordinates.

6. A terminal device, characterized in that, The terminal device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the distortion removal center correction method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the de-distortion center correction method according to any one of claims 1 to 4.

Images