Distortion parameter calibration method, electronic device and computer-readable storage medium

Abstract

The embodiments of the present application relate to the field of machine vision technology and disclose a distortion parameter calibration method, electronic device, and computer-readable storage medium. The method includes: obtaining N structured light images taken by a camera at different positions of a target plane; wherein each structured light image contains a complete preset structured light pattern; calculating the matching similarity corresponding to each same-name point in the N structured light images according to a preset matching algorithm, and retaining the same-name points with a matching similarity greater than a preset matching threshold as target same-name points; recalibrating the distortion parameters of the camera according to each target same-name point and the distortion model used during factory calibration of the camera to obtain new distortion parameters of the camera. The distortion parameter calibration method provided in the embodiments of the present application can achieve efficient and high-precision distortion parameter calibration using the structured light pattern projected by the depth camera itself, thereby improving the accuracy and robustness of depth recovery of the depth camera in large distortion and wide-angle scenes.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of machine vision technology, and in particular to a distortion parameter calibration method, an electronic device, and a computer-readable storage medium. Background Art

[0002] In recent years, monocular structured light technology has demonstrated its prowess in consumer electronics, industrial measurement and other fields. Depth cameras based on monocular structured light technology have high precision, high integration, small size and low cost. They have become the preferred solution for depth recovery and 3D solution. The core of the monocular structured light depth camera includes a structured light projector and a camera module. The structured light projector is used to project structured light patterns onto the target scene and target object, and the camera module is used to capture the projection of the structured light pattern on the target scene and target object. During depth recovery, the depth camera matches the captured projection image with the pre-calibrated reference image, calculates the parallax between the same-name points in the two images, and then combines parameters such as baseline length, focal length and distortion to complete the 3D solution.

[0003] Among the evaluation indicators of depth cameras, absolute accuracy is of vital importance. It can characterize the accuracy of depth recovery of depth cameras. Absolute accuracy is closely related to the parameter calibration of cameras. When calibrating camera parameters in the industry, calibration objects are generally used for calibration when the camera leaves the factory. Due to the limitations of calibration objects and tooling fixtures, the calibration objects cannot cover the entire field of view of the depth camera, and the edge areas lack calibration information. The calculated distortion parameters are under-fitted. When the depth camera calibrated in this way is used in large distortion and wide-angle usage scenarios, the absolute accuracy will produce large errors, resulting in low depth recovery accuracy. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a distortion parameter calibration method, an electronic device, and a computer-readable storage medium, which can achieve efficient and high-precision distortion parameter calibration using the structured light pattern projected by the depth camera itself, thereby improving the accuracy and robustness of depth recovery of the depth camera in large distortion and wide-angle scenes.

[0005] To solve the above technical problems, an embodiment of the present application provides a distortion parameter calibration method, comprising the following steps: obtaining N structured light images taken by a camera at different positions of a target plane; wherein N is an integer greater than 1, and each of the structured light images contains a complete preset structured light pattern; calculating the matching similarity corresponding to each homonymous point in the N structured light images according to a preset matching algorithm, and retaining the homonymous points whose matching similarity is greater than a preset matching threshold as target homonymous points; and recalibrating the distortion parameters of the camera according to each target homonymous point and the distortion model used when the camera was calibrated at the factory to obtain new distortion parameters of the camera.

[0006] An embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned distortion parameter calibration method.

[0007] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned distortion parameter calibration method when executed by a processor.

[0008] The distortion parameter calibration method, electronic device and computer-readable storage medium provided in the embodiments of the present application are as follows: the server obtains N structured light images taken by the camera at different positions of the target plane, each structured light image contains a complete preset structured light pattern, and then calculates the matching similarity corresponding to each homonymous point in the N structured light images according to the preset matching algorithm, and retains the homonymous points whose corresponding matching similarity is greater than the preset matching threshold as the target homonymous points. Finally, according to each target homonymous point and the distortion model used when the camera is calibrated at the factory, the distortion parameters of the camera are recalibrated to obtain the new distortion parameters of the camera. Considering that when the depth camera is calibrated at the factory in the industry, due to the calibration object and the tooling card Due to the limitations of factors such as objects, certain objects cannot fill the field of view of the depth camera, so the calculated distortion parameters are under-fitted and inaccurate. The depth camera calibrated in this way is used in large distortion and wide-angle usage scenarios, and the absolute precision error is large, resulting in low depth recovery accuracy. In the embodiments of the present application, a number of structured light images are captured using the structured light pattern projected by the depth camera itself, and the camera is recalibrated based on the target same-name points in the number of structured light images and the internal parameters calibrated at the factory and the distortion model used. That is, efficient and high-precision distortion parameter calibration can be achieved only using the structured light pattern projected by the depth camera itself, thereby improving the accuracy and robustness of depth recovery of the depth camera in large distortion and wide-angle scenes.

[0009] In addition, the number of target homonymous points is B, the distortion model used when the camera is calibrated at the factory includes s distortion parameters, and s is an integer greater than 1. The distortion parameters of the camera are recalibrated according to each of the target homonymous points and the distortion model used when the camera is calibrated at the factory to obtain new distortion parameters of the camera, including: traversing each of the target homonymous points, and constructing an equation corresponding to the current target homonymous point according to the coordinates of the current target homonymous point, the internal parameters calibrated when the camera is shipped from the factory, and the distortion model used when the camera is calibrated at the factory; wherein the equation includes s unknown new distortion parameters, and B, N, and s satisfy 2*N*B=N*B+B+s; combining the equations corresponding to the homonymous points to obtain a system of equations, solving the system of equations, and obtaining the solved s new distortion parameters. The role of the distortion model In fact, it is to correct the distorted image into an undistorted image. According to the principle of similar triangles, if the image is not distorted, the vertical coordinates of the points of the same name on each image should be the same. Therefore, the vertical coordinates of the points of the same name after distortion correction are the same. When constructing the equation corresponding to the current target point of the same name, the horizontal coordinates of each point after distortion correction are unknown. Since the purpose of this application is to recalibrate the distortion parameters, the new distortion parameters in the equation are unknown. When there are N structured light images and B points of the same name, a total of 2*N*B equations can be established. The corrected horizontal coordinates of N*B points are unknown, the corrected vertical coordinates of B points of the same name are unknown, and s new distortion parameters are unknown. Therefore, in the embodiment of this application, 2*N*B=N*B+B+s is specified, so that the exact solution of the system of equations can be solved to obtain s new distortion parameters.

[0010] In addition, the number of the target homonymous points is M, and the M, the N, and the s satisfy 2*N*M>N*M+M+s. The equations corresponding to the homonymous points are combined to obtain an equation group, and the equation group is solved to obtain s new distortion parameters, including: combining the equations corresponding to the homonymous points to obtain an overdetermined equation group; establishing an objective function corresponding to the overdetermined equation group according to the coordinates of the homonymous points, the internal parameters of the camera calibrated at the factory, and the distortion model used when the camera is calibrated at the factory; and obtaining the distortion parameters according to the target. The calibration function and the preset iterative algorithm are used to iteratively solve the overdetermined equations to obtain s new distortion parameters. In order to calibrate the new distortion parameters more accurately, this application requires that M, N and s satisfy the relationship 2*N*M>N*M+M+s, that is, the number of constructed equations is greater than the number of unknowns. These equations are combined to obtain an overdetermined equation group. The overdetermined equation group constructed for the distortion scene is often nonlinear. This application uses an iterative method to solve it, thereby solving s new distortion parameters that are more accurate and more in line with the actual situation of the camera.

[0011] In addition, the matching similarity corresponding to each of the homonymous points in the N structured light images is calculated according to a preset matching algorithm, and the homonymous points whose matching similarity is greater than a preset matching threshold are retained as target homonymous points, including: randomly selecting a structured light image from the N structured light images as a reference image; traversing each of the homonymous points, taking the point of the current homonymous point on the reference image as the reference point, and taking the point of the current homonymous point in the other N-1 structured light images as the point to be matched; calculating the matching similarity between the reference point and each of the points to be matched according to a preset matching algorithm; retaining the reference point and each of the points to be matched. The homonymous points whose matching similarities between the points to be matched are greater than the preset matching threshold are target homonymous points. Considering that when the value of N is large, that is, a large number of structured light images are obtained, the computational cost of calculating the matching similarities between the homonymous points in the N structured light images is large. For example, when N is 10, each homonymous point needs to be matched and calculated 45 times. In this application, a reference image is randomly selected from the N structured light images, and the matching similarities between the other images and the reference image are calculated. When N is 10, each homonymous point only needs to be matched and calculated 9 times, which saves computing resources and improves the efficiency of recalibration of distortion parameters.

[0012] In addition, after recalibrating the distortion parameters of the camera to obtain the new distortion parameters of the camera, the process further includes: recalibrating the internal parameters of the camera according to the target homonymous points, the distortion model, and the new distortion parameters to obtain the new internal parameters of the camera; wherein the new internal parameters of the camera include the new focal length of the camera and the new principal point position coordinates of the camera. After the server recalibrates the distortion parameters of the camera, these new distortion parameters that are scientifically accurate and in line with the actual situation can be fixed, and the internal parameters of the camera can be recalibrated according to the target homonymous points to obtain the new internal parameters of the camera. The new internal parameters are more suitable for scenes with large distortion and wide angle.

[0013] In addition, the number of the target homonymous points is Q, the distortion model used when the camera is factory calibrated includes s distortion parameters, and s is an integer greater than 1. The distortion parameters of the camera are recalibrated according to each of the target homonymous points and the distortion model used when the camera is factory calibrated to obtain new distortion parameters of the camera, including: traversing each of the target homonymous points, and constructing an equation corresponding to the current target homonymous point according to the coordinates of the current target homonymous point and the distortion model used when the camera is factory calibrated; wherein the equation includes s unknown new distortion parameters and p unknown corresponding distortion parameters. The internal parameters of the camera, the Q, the N and the s satisfy 2*N*Q=N*Q+Q+s+p; the equations corresponding to the target homonymous points are combined to obtain a system of equations, and the system of equations is solved to obtain s new distortion parameters and p new internal parameters. In the embodiment of the present application, the internal parameters of the camera can be simultaneously used as unknown quantities. During recalibration, not only the distortion parameters of the camera are recalibrated, but also the internal parameters of the camera are recalibrated. There are a total of N*Q+Q+s+p unknown quantities, so the number of equations in the system of equations must be ensured to be 2*M*N=N*Q+Q+s+p.

[0014] In addition, before acquiring N structured light images taken by the camera at different positions on the target plane, it also includes: detecting whether the baseline direction of the camera is the x-axis direction; if the baseline direction of the camera is not the x-axis direction, rotating the coordinate system of the camera so that the baseline direction is the x-axis and the y-axis is perpendicular to the baseline direction. The technical solution of this application needs to ensure that the baseline direction of the camera is the x-axis direction. At the beginning of the whole process, it is first detected whether the baseline direction of the camera is the x-axis direction. If not, the coordinate system of the camera is rotated in time to ensure that the distortion parameter calibration can be carried out smoothly. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] One or more embodiments are exemplarily described by the figures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments.

[0016] Figure 1 This is the process of the distortion parameter calibration method of an embodiment of the present application Figure 1 ;

[0017] Figure 2 This is a flowchart of an embodiment of the present application, in which the matching similarity corresponding to each homonymous point in N structured light images is calculated according to a preset matching algorithm, and homonymous points with matching similarity greater than a preset matching threshold are retained as target homonymous points;

[0018] Figure 3 In one embodiment of the present application, the server recalibrates the distortion parameters of the camera based on the target homonymous points and the distortion model used when the camera is factory calibrated to obtain the new distortion parameters of the camera.

[0019] Figure 4 1 is a schematic diagram of the imaging principle of a monocular structured light camera provided in one embodiment of the present application;

[0020] Figure 5 is a schematic diagram of a distorted image provided according to an embodiment of the present application;

[0021] Figure 6 According to one embodiment of the present application, the equations corresponding to the target homonymous points are combined to obtain an equation system, and the equation system is solved to obtain s new distortion parameters.

[0022] Figure 7 This is a process of the distortion parameter calibration method according to another embodiment of the present application. Figure 2 ;

[0023] Figure 8 is a structural diagram of an electronic device according to another embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those skilled in the art that in each embodiment of the present application, many technical details are proposed to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be implemented. The division of the following embodiments is for convenience of description and should not constitute any limitation on the specific implementation of the present application. The various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0025] An embodiment of the present application relates to a distortion parameter calibration method, which is applied to an electronic device, wherein the electronic device can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described using a server as an example. The implementation details of the distortion parameter calibration method of this embodiment are described in detail below. The following content is only the implementation details provided for easy understanding and is not necessary for implementing this solution.

[0026] The specific process of the distortion parameter calibration method of this embodiment can be as follows: Figure 1 Shown, including:

[0027] Step 101: Acquire N structured light images captured by a camera at different positions on a target plane.

[0028] Specifically, the purpose of this application is to recalibrate the distortion parameters of the camera, that is, the depth camera has completed the preliminary calibration based on the preset calibration object using the calibration method commonly used in the industry when it leaves the factory, such as using a checkerboard as a calibration object and calibrating it using the Zhang Zhengyou calibration method. The camera has calibrated the internal parameters and distortion parameters when it leaves the factory. In this embodiment and the following embodiments, the internal parameters calibrated by the camera when it leaves the factory are referred to as factory internal parameters, and the distortion parameters calibrated by the camera when it leaves the factory are referred to as factory distortion parameters.

[0029] In a specific implementation, when the server recalibrates the distortion parameters of the camera, it needs to first obtain N structured light images of the target plane taken by the camera at different positions, that is, N structured light images of the target plane are taken by the camera at different positions, N is an integer greater than 1, and each structured light image obtained by the server contains a complete preset structured light pattern, that is, the target plane can envelop the full frame, wherein the target plane, different positions, and preset structured light patterns can all be set by technicians in this field according to actual needs.

[0030] In one example, the reflectivity of the target plane needs to be greater than a preset reflectivity threshold, and the target plane is a diffuse reflection plane, that is, it is necessary to ensure that the target plane has good reflectivity.

[0031] In one example, the camera is within a preset working distance, and the camera shoots the target plane at different positions by continuously changing the shooting angle of the camera and the distance between the camera and the target plane.

[0032] Step 102 : Calculate the matching similarity corresponding to each of the homonymous points in the N structured light images according to a preset matching algorithm, and retain the homonymous points with matching similarity greater than a preset matching threshold as target homonymous points.

[0033] In a specific implementation, after obtaining N structured light images of the target plane taken by the camera at different positions, the server can find all the same-name points in these N structured light images. The same-name points are the projection points corresponding to a point in the preset structured light pattern in different structured light images. After determining all the same-name points, the server can calculate the matching similarity corresponding to each of the same-name points in the N structured light images according to the preset matching algorithm, and judge whether the matching similarity corresponding to each of the same-name points is greater than the preset matching threshold. The same-name points with large corresponding matching similarity have better quality, and the corresponding data are more real and accurate. Therefore, the server retains the same-name points whose corresponding matching similarity is greater than the preset matching threshold as target same-name points, and discards the same-name points whose corresponding matching similarity is less than or equal to the preset matching threshold. The preset matching algorithm can be selected by technical personnel in this field according to actual needs.

[0034] In one example, the preset matching algorithm may be a window-based matching algorithm, such as a zero-normalized-correlation-cost (ZNCC) matching algorithm, a sum of absolute differences (SAD) matching algorithm, and the like.

[0035] In another example, the preset matching algorithm may be a feature description-based point matching algorithm, such as a speed-up robust features matching algorithm (Surf), a scale-invariant feature transformation matching algorithm (SIFT), and an oriented fast and rotated brief matching algorithm (ORB).

[0036] In an example, N=2, that is, the server obtains two structured light images. For each point with the same name, it includes a first pixel point in the first structured light image and a second pixel point in the second structured light image. The matching similarity corresponding to the point with the same name is the matching similarity between the first pixel point and the second pixel point.

[0037] Step 103 : recalibrate the distortion parameters of the camera based on the target homonymous points and the distortion model used when the camera is calibrated at the factory, and obtain new distortion parameters of the camera.

[0038] Specifically, after determining the target homonymous points, the server can recalibrate the camera's distortion parameters according to the target homonymous points and the distortion model used when the camera is factory calibrated to obtain new distortion parameters of the camera.

[0039] In an example, the server can determine the number of distortion parameters corresponding to the camera based on the distortion model used when the camera is calibrated at the factory. The server obtains the coordinates of the same-name points of each target and inputs the coordinates of the same-name points of each target into the distortion model used when the camera is calibrated at the factory. The distortion parameters in the distortion model are the factory distortion parameters. The server further trains the distortion model according to a preset cost function and continuously updates the distortion parameters of the distortion model until the distortion model converges. The server saves the distortion parameters when the distortion model converges, and thus obtains the new distortion parameters of the camera.

[0040] In this embodiment, the server obtains N structured light images taken by the camera at different positions of the target plane, each structured light image contains a complete preset structured light pattern, and then calculates the matching similarity corresponding to each homonymous point in the N structured light images according to the preset matching algorithm, and retains the homonymous points whose corresponding matching similarity is greater than the preset matching threshold as the target homonymous points. Finally, according to each target homonymous point and the distortion model used when the camera is calibrated at the factory, the distortion parameters of the camera are recalibrated to obtain the new distortion parameters of the camera. Considering that when the depth camera is calibrated at the factory in the industry, due to the limitations of factors such as calibration objects and tooling fixtures, the fixed objects cannot be laid out. The field of view of the full-depth camera, the distortion parameters calculated in this way are under-fitting and inaccurate. The depth camera calibrated in this way is used in large distortion and wide-angle usage scenarios, and the absolute precision error is large, resulting in low depth recovery accuracy. In the embodiment of the present application, a number of structured light images are captured using the structured light pattern projected by the depth camera itself, and the target same-name points in the number of structured light images are used, and the camera is recalibrated based on the internal parameters calibrated at the factory and the distortion model used. That is, only the structured light pattern projected by the depth camera itself can be used to achieve efficient and high-precision distortion parameter calibration, thereby improving the accuracy and robustness of depth recovery of the depth camera in large distortion and wide-angle scenes.

[0041] In one embodiment, the server calculates the matching similarity of each homonymous point in the N structured light images according to a preset matching algorithm, and retains the homonymous points whose matching similarity is greater than a preset matching threshold as target homonymous points. Figure 2 The steps shown in the figure are as follows:

[0042] Step 201 : randomly select a structured light image from N structured light images as a reference image.

[0043] Step 202 , traverse all points with the same name, take the point of the current point with the same name on the reference image as the reference point, and take the point of the current point with the same name in the other N-1 structured light images as the point to be matched.

[0044] Step 203 : Calculate the matching similarity between the reference point and each to-be-matched point according to a preset matching algorithm.

[0045] Step 204 : retaining the same-name points whose matching similarities between the reference point and each to-be-matched point are greater than a preset matching threshold as target same-name points.

[0046] In a specific implementation, when the value of N is large, that is, when a large number of structured light images are obtained, the amount of calculation required by the server to calculate the matching similarity between the same-name points in the N structured light images is very large. For example, when N is 10, the same-name points correspond to 10 pixels in these 10 structured light images. If the matching similarity between these 10 pixels is calculated, each same-name point needs to be matched and calculated 9+8+7+6+5+4+3+2+1=45 times, which wastes too much computing resources. Therefore, this embodiment first randomly selects one of the N structured light images as the reference image, traverses each same-name point, and compares the current same-name point in the same structured light image to the reference image. The points on the reference image are used as reference points, and the points of the current homonymous point in the other N-1 structured light images are used as points to be matched. The server only calculates the matching similarity between the reference point and each point to be matched according to the preset matching algorithm, and determines whether the matching similarity between the reference point and each point to be matched is greater than the preset matching threshold. The server retains the homonymous points whose matching similarity between the reference point and each point to be matched is greater than the preset matching threshold as target homonymous points. Based on this scheme, when N is 10, each homonymous point only needs to be matched and calculated 9 times, which saves computing resources and thus improves the efficiency of distortion parameter recalibration.

[0047] In one embodiment, before obtaining N structured light images taken by the camera at different positions of the target plane, the server can detect whether the baseline direction of the camera is the x-axis direction. If the baseline direction of the camera is the x-axis direction, the N structured light images taken by the camera at different positions of the target plane can be directly obtained; if the baseline direction of the camera is not the x-axis direction, the server first rotates the camera's coordinate system so that the baseline direction is the x-axis and the y-axis perpendicular to the baseline direction, and then obtains N structured light images taken by the camera at different positions of the target plane to ensure that the distortion parameter calibration can be carried out smoothly.

[0048] In one embodiment, the number of target homonymous points is B, and the distortion model used when the camera is calibrated at the factory includes s distortion parameters, where s is an integer greater than 1. The server recalibrates the distortion parameters of the camera based on the target homonymous points and the distortion model used when the camera is calibrated at the factory, and obtains the new distortion parameters of the camera, which can be obtained by Figure 3 The steps shown in the figure are as follows:

[0049] Step 301 traverses each target homonymous point and constructs an equation corresponding to the current target homonymous point based on the coordinates of the current target homonymous point, the internal parameters calibrated by the camera at the factory, and the distortion model used when the camera is calibrated at the factory.

[0050] Based on the imaging principle of the monocular structured light depth camera, it can be seen that the corresponding points of the target with the same name in each structured light image should be on a straight line in the camera coordinate system, and this straight line is parallel to the baseline, such as Figure 3As shown in the figure, point P and point Q are the positions of the points in the camera coordinate system corresponding to the structured light image corresponding to position 1 and the structured light image corresponding to position 2. The vertical coordinates of the two points are the same, but the horizontal coordinates are different. In large distortion and wide-angle scenes, the actual positions of point P and point Q will be distorted, such as Figure 4 As shown in the figure, the straight lines where points P and Q are located in the distorted image are not parallel to the baseline, that is, the longitudinal coordinates are different. The function of the distortion model is actually to correct the distorted image into an undistorted image. Although the longitudinal coordinates of points P and Q are different in the distorted image, the longitudinal coordinates of points P and Q must be the same in the image after distortion correction, that is, the longitudinal coordinates of the points with the same name after distortion correction are the same in each image.

[0051] In this embodiment, when the server recalibrates the camera, it can only calibrate the distortion parameters without calibrating the internal parameters. The factory internal parameters of the camera can still be used. The factory internal parameters are known quantities. The server traverses each target homonymous point and constructs an equation corresponding to the current target homonymous point based on the coordinates of the current target homonymous point, the internal parameters calibrated when the camera leaves the factory, and the distortion model used when the camera leaves the factory calibration. The unknown quantities in the equation include s unknown new distortion parameters, the horizontal coordinates of each point after distortion correction (the corresponding points of each target homonymous point in each corrected structured light image), and the vertical coordinates of B homonymous points after correction. The total number of unknown quantities is N*B+B+s. For the B target homonymous points, the server can establish 2*N*B equations. As long as B, N, and s satisfy 2*N*B=N*B+B+s, the exact solution of the equation can be calculated.

[0052] In one example, the server constructs the equation corresponding to the current target's keypoint based on the coordinates of the current target's keypoint, the internal parameters of the camera's factory calibration, and the distortion model used when the camera was calibrated at the factory. This can be achieved using the following formula:

[0053] X dij =G x [h,K|(X ij , Y j )]

[0054] Y dij =G y [h,K|(X ij , Y j )]

[0055] Where, X dij Y is the horizontal coordinate of the j-th target homonymous point in the i-th structured light image, dij is the vertical coordinate of the j-th target point of the same name in the i-th structured light image, G x (·) is the expression of the distortion model in the x-axis direction, G y(·) is the expression of the distortion model in the y-axis direction, h is used to represent the s unknown new distortion parameters, K is used to represent the internal parameters calibrated when the camera leaves the factory, X ij Y is the horizontal coordinate of the j-th target homonymous point in the i-th corrected structured light image, j is the corrected vertical coordinate of the j-th target homonymous point.

[0056] In another example, the distortion model used during factory calibration of a camera contains two distortion parameters, h1 and h2. The server constructs the equation corresponding to the current target homonymous point based on the coordinates of the current target homonymous point, the internal parameters of the camera calibration at the factory, and the distortion model used during factory calibration. This can be achieved using the following formula:

[0057]

[0058]

[0059]

[0060] Where, X dij Y is the horizontal coordinate of the j-th target homonymous point in the i-th structured light image, dij is the ordinate of the jth target homonymous point in the i-th structured light image, h1 is the first new distortion parameter, h2 is the second new distortion parameter, (C x , C y ) is the coordinate of the factory master point position calibrated when the camera leaves the factory, (F x , F y ) is the factory focal length of the camera, X ij Y is the horizontal coordinate of the j-th target homonymous point in the i-th corrected structured light image, j is the corrected vertical coordinate of the j-th target homonymous point.

[0061] In step 302, the equations corresponding to the target points of the same name are combined to obtain a system of equations, and the system of equations is solved to obtain s new distortion parameters.

[0062] In a specific implementation, the server establishes a total of 2*N*B equations for each target homonymous point. The server combines these 2*N*B equations to obtain an equation group, and solves the equation group to obtain s new distortion parameters.

[0063] In this embodiment, it is considered that the role of the distortion model is actually to correct the distorted image into an undistorted image. According to the principle of similar triangles, if the image is not distorted, the vertical coordinates of the homonymous points should be the same. Therefore, the vertical coordinates of the homonymous points on each image after distortion correction are the same. When constructing the equation corresponding to the current target homonymous point, the horizontal coordinates of each point after distortion correction are unknown. Since the purpose of this application is to recalibrate the distortion parameters, the new distortion parameters in the equation are unknown. When there are N structured light images and B homonymous points, a total of 2*N*B equations can be established. The horizontal coordinates of the N*B points after correction are unknown, the vertical coordinates of the B homonymous points after correction are unknown, and s new distortion parameters are unknown. Therefore, in the embodiment of this application, 2*N*B=N*B+B+s is specified, so that the exact solution of the equation group can be solved to obtain s new distortion parameters.

[0064] In one embodiment, the number of target homonymous points is M, and the distortion model used when the camera is factory calibrated contains s distortion parameters, where s is an integer greater than 1, and M, N, and s satisfy 2*N*M>N*M+M+s. The server combines the equations corresponding to the target homonymous points to obtain an equation system, solves the equation system, and obtains the solved s new distortion parameters, which can be obtained by Figure 6 The steps shown in the figure are as follows:

[0065] Step 401: Simultaneously solve the equations corresponding to the target points of the same name to obtain an overdetermined system of equations.

[0066] In specific implementations, the equations established by the server are often nonlinear, and the set of simultaneous equations is naturally also a nonlinear set of equations, which is more suitable for solving using least squares method, iterative method and other methods. Therefore, this embodiment requires that M, N and s satisfy the relationship 2*N*M>N*M+M+s, so that the number of equations is greater than the number of unknown quantities, and these equations can be combined to obtain an overdetermined set of equations.

[0067] Step 402 : Establish an objective function corresponding to the overdetermined set of equations based on the coordinates of the target homonymous points, the internal parameters of the camera calibrated at the factory, and the distortion model used when the camera was calibrated at the factory.

[0068] Step 403 : Iteratively solve the overdetermined equations according to the objective function and a preset iterative algorithm to obtain s new distortion parameters.

[0069] Specifically, the embodiment of the present application uses an iterative method to solve the overdetermined system of equations. The server first establishes an objective function corresponding to the overdetermined system of equations based on the coordinates of the same-name points of each target, the internal parameters calibrated when the camera leaves the factory, and the distortion model used when the camera leaves the factory for calibration. Then, based on the objective function and the preset iterative algorithm, the overdetermined system of equations is iteratively solved to obtain s new distortion parameters. The s new distortion parameters solved in this way are more accurate.

[0070] In one example, the objective function corresponding to the overdetermined equations set up by the server can be expressed by the following formula:

[0071]

[0072] Where, X dij Y is the horizontal coordinate of the j-th target homonymous point in the i-th structured light image, dij is the vertical coordinate of the j-th target point of the same name in the i-th structured light image, G x (·) is the expression of the distortion model in the x-axis direction, G y (·) is the expression of the distortion model in the y-axis direction, h is used to represent the s unknown new distortion parameters, K is used to represent the internal parameters calibrated when the camera leaves the factory, X ij Y is the horizontal coordinate of the j-th target homonymous point in the i-th corrected structured light image, j is the corrected vertical coordinate of the j-th target homonymous point.

[0073] In another example, the distortion model used during factory calibration of a camera contains two distortion parameters, h1 and h2. The objective function corresponding to the overdetermined equations set up by the server can be expressed as follows:

[0074]

[0075]

[0076] Where, X dij Y is the horizontal coordinate of the j-th target homonymous point in the i-th structured light image, dij is the ordinate of the jth target homonymous point in the i-th structured light image, h1 is the first new distortion parameter, h2 is the second new distortion parameter, (C x , C y ) is the coordinate of the factory master point position calibrated when the camera leaves the factory, (F x , F y ) is the factory focal length of the camera, X ij Y is the horizontal coordinate of the j-th target homonymous point in the i-th corrected structured light image, j is the corrected vertical coordinate of the j-th target homonymous point.

[0077] In this embodiment, in order to more accurately calibrate the new distortion parameters, it is required that M, N, and s satisfy the relationship 2*N*M>N*M+M+s. That is, the number of constructed equations is greater than the number of unknowns. By combining these equations, an overdetermined system of equations can be obtained. However, the overdetermined system of equations constructed for the distortion scene is often nonlinear. This application uses an iterative method to solve it, thereby solving s new distortion parameters that are more accurate and more consistent with the actual situation of the camera.

[0078] An embodiment of the present application relates to a method for calibrating distortion parameters. The following describes the implementation details of the method for calibrating distortion parameters of this embodiment in detail. The following content is only for the convenience of understanding the implementation details and is not necessary for implementing this solution. The specific process of the method for calibrating distortion parameters of this embodiment can be as follows: Figure 7 Shown, including:

[0079] Step 501: Acquire N structured light images captured by a camera at different positions on a target plane.

[0080] Step 502 : Calculate the matching similarity of each of the homonymous points in the N structured light images according to a preset matching algorithm, and retain the homonymous points with matching similarity greater than a preset matching threshold as target homonymous points.

[0081] Step 503 : recalibrate the distortion parameters of the camera based on the target homonymous points and the distortion model used when the camera is calibrated at the factory, and obtain new distortion parameters of the camera.

[0082] Among them, steps 501 to 503 are substantially the same as steps 101 to 103 and are not described again here.

[0083] Step 504 : recalibrate the internal parameters of the camera based on the target homonymous points, the distortion model used when the camera is factory calibrated, and the obtained new distortion parameters to obtain new internal parameters of the camera.

[0084] In a specific implementation, after recalibrating the distortion parameters of the camera and obtaining the new distortion parameters of the camera, the server can fix the new distortion parameters and recalibrate the internal parameters of the camera based on the same-name points of each target, the distortion model used during factory calibration of the camera, and the new distortion parameters obtained to obtain the new internal parameters of the camera. The new internal parameters of the camera include the new focal length of the camera and the new principal point position coordinates of the camera.

[0085] In an example, the distortion parameters have been recalibrated, and the obtained new distortion parameters are accurate and reliable. The server treats all new distortion parameters as known quantities and the internal parameters of the camera as unknown quantities. When the camera has a total of p internal parameters, the server only needs to select V target synonyms to ensure that 2*N*V≥N*V+V+p. The server reconstructs the equations corresponding to each target synonym based on the coordinates of each V target synonym, the distortion model corresponding to the camera, and all new distortion parameters. The 2*N*V equations are combined to obtain a system of equations or an overdetermined system of equations. The system of equations or the overdetermined system of equations are solved to obtain p new internal parameters.

[0086] In this embodiment, after the server recalibrates the camera's distortion parameters, it can fix these new distortion parameters that are scientifically accurate and in line with actual conditions, and recalibrate the camera's internal parameters based on the same-name points of each target to obtain new internal parameters of the camera. The new internal parameters are more suitable for scenes with large distortion and wide angles.

[0087] In one embodiment, the number of target homonymous points is Q. The distortion model used during factory calibration of the camera includes s distortion parameters, where s is an integer greater than 1. The corresponding internal parameters of the camera are p, where p is an integer greater than 1. The server can simultaneously recalibrate the internal parameters and distortion parameters of the camera, that is, both the internal parameters and the distortion parameters of the camera are considered unknown. The server traverses each target homonymous point and constructs an equation corresponding to the current target homonymous point based on the coordinates of the current target homonymous point and the distortion model used during factory calibration of the camera. The equation includes s unknown new distortion parameters and p unknown internal parameters of the camera. The relationship between Q, N, s, and p satisfies 2*N*Q=N*Q+Q+s+p, ensuring that the number of equations is equal to the number of unknowns. The server combines these 2*N*Q equations to obtain an equation system, and finds an exact solution to the equation system to obtain the s new distortion parameters and p new internal parameters.

[0088] In an example, Q, N, s, and p satisfy the relationship 2*N*Q>N*Q+Q+s+p, so the number of equations is greater than the number of unknowns. The server can combine these 2*N*Q equations to obtain an overdetermined system of equations. The server can use an iterative algorithm to iteratively solve the overdetermined system of equations, thereby obtaining s new distortion parameters and p new internal parameters.

[0089] The steps of the various methods above are divided only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0090] Another embodiment of the present application relates to an electronic device, such as Figure 8 As shown, it includes: at least one processor 601; and a memory 602 communicatively connected to the at least one processor 601; wherein the memory 602 stores instructions that can be executed by the at least one processor 601, and the instructions are executed by the at least one processor 601 to enable the at least one processor 601 to execute the distortion parameter calibration method in the above-mentioned embodiments.

[0091] The memory and processor are connected using a bus, which can include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors and memories. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and are therefore not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor.

[0092] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0093] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0094] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0095] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A method for calibrating distortion parameters, characterized in that: include: Acquire N structured light images captured by a camera at different positions of a target plane; wherein N is an integer greater than 1, and each structured light image contains a complete preset structured light pattern; Calculating the matching similarity of each of the homonymous points in the N structured light images according to a preset matching algorithm, and retaining the homonymous points whose matching similarity is greater than a preset matching threshold as target homonymous points; Recalibrating the distortion parameters of the camera according to the target homonymous points and the distortion model used when the camera is factory calibrated to obtain new distortion parameters of the camera; The number of target homonymous points is B, the distortion model used in the factory calibration of the camera includes s distortion parameters, where s is an integer greater than 1, and the distortion parameters of the camera are recalibrated according to the target homonymous points and the distortion model used in the factory calibration of the camera to obtain new distortion parameters of the camera, including: Traversing each of the target homonymous points, constructing an equation corresponding to the current target homonymous point based on the coordinates of the current target homonymous point, the internal parameters of the camera calibrated at the factory, and the distortion model used when the camera was calibrated at the factory; wherein the equation includes s unknown new distortion parameters, and B, N, and s satisfy 2*N*B=N*B+B+s; The equations corresponding to the target points of the same name are combined to obtain a system of equations, and the system of equations is solved to obtain s new distortion parameters.

2. The distortion parameter calibration method according to claim 1, characterized in that: The step of calculating the matching similarity corresponding to each of the homonymous points in the N structured light images according to a preset matching algorithm, and retaining the homonymous points having a matching similarity greater than a preset matching threshold as target homonymous points, includes: Randomly selecting a structured light image from the N structured light images as a reference image; Traversing the same-name points, taking the point of the current same-name point on the reference image as the reference point, and taking the point of the current same-name point in the other N-1 structured light images as the point to be matched; Calculating the matching similarity between the reference point and each of the points to be matched according to a preset matching algorithm; The points with the same name and whose matching similarity between the reference point and each of the points to be matched is greater than a preset matching threshold are retained as target points with the same name.

3. The distortion parameter calibration method according to claim 1, characterized in that: Before acquiring N structured light images captured by the camera at different positions of the target plane, the method further includes: Detecting whether the baseline direction of the camera is the x-axis direction; If the baseline direction of the camera is not the x-axis direction, the coordinate system of the camera is rotated so that the baseline direction is the x-axis and the y-axis is perpendicular to the baseline direction.

4. The distortion parameter calibration method according to claim 1, wherein: The equation corresponding to the current target homonymous point is constructed using the following formula based on the coordinates of the current target homonymous point, the internal parameters of the camera calibrated at the factory, and the distortion model used when the camera was calibrated at the factory: X dij =G x [h,K|(X ij ,Y j )] Y dij =G y [h,K|(X ij ,Y j )] Among them, X dij is the horizontal coordinate of the j-th target homonymous point in the i-th structured light image, Y dij is the vertical coordinate of the j-th target point of the same name in the i-th structured light image, G x (·) is the expression of the distortion model in the x-axis direction, G y (·) is the expression of the distortion model in the y-axis direction, h is used to represent the s unknown new distortion parameters, K is used to represent the internal parameters of the camera calibrated at the factory, X ij Y is the horizontal coordinate of the j-th target homonymous point in the i-th corrected structured light image, j is the corrected vertical coordinate of the j-th target homonymous point.

5. The distortion parameter calibration method according to claim 1, characterized in that: The number of the target homonymous points is M, and M, N, and s satisfy 2*N*M>N*M+M+s. The equations corresponding to the target homonymous points are combined to obtain a system of equations, and the system of equations is solved to obtain s new distortion parameters, including: The equations corresponding to the target points of the same name are combined to obtain an overdetermined system of equations; Establishing an objective function corresponding to the overdetermined set of equations according to the coordinates of the target homonymous points, the internal parameters of the camera calibrated at the factory, and the distortion model used when the camera was calibrated at the factory; According to the objective function and a preset iterative algorithm, the overdetermined equations are iteratively solved to obtain s new distortion parameters.

6. The distortion parameter calibration method according to claim 1, characterized in that: After recalibrating the distortion parameters of the camera to obtain new distortion parameters of the camera, the method further includes: Recalibrate the internal parameters of the camera according to the target homonymous points, the distortion model, and the new distortion parameters to obtain new internal parameters of the camera; wherein the new internal parameters of the camera include a new focal length of the camera and a new principal point position coordinate of the camera.

7. The distortion parameter calibration method according to any one of claims 1 to 3, characterized in that: The number of the target homonymous points is Q, the distortion model used when the camera is factory calibrated includes s distortion parameters, where s is an integer greater than 1, and the distortion parameters of the camera are recalibrated according to the target homonymous points and the distortion model used when the camera is factory calibrated to obtain new distortion parameters of the camera, including: Traversing each of the target homonymous points, and constructing an equation corresponding to the current target homonymous point based on the coordinates of the current target homonymous point and the distortion model used during factory calibration of the camera; wherein the equation includes s unknown new distortion parameters and p unknown internal parameters of the camera, and Q, N, s, and p satisfy 2*N*Q=N*Q+Q+s+p; The equations corresponding to the target points of the same name are combined to obtain a system of equations, and the system of equations is solved to obtain s new distortion parameters and p new internal parameters.

8. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the distortion parameter calibration method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the distortion parameter calibration method according to any one of claims 1 to 7 is implemented.

Images