Monocular camera calibration method based on phase target and bidirectional radial basis function neural network

Abstract

The application discloses a monocular camera calibration method based on a phase target and a double radial basis function neural network, adopts a three-step phase shift method for feature extraction, and uses a multi-frequency method to calculate absolute phase, converts absolute phase borne by feature points of the phase target to a three-dimensional space, establishes a corresponding relationship between image coordinates and world coordinates of the feature points, and finally uses a double RBF neural network to complete direct mapping of two-dimensional image coordinates to three-dimensional space coordinates, the whole RBF neural network uses a genetic algorithm to iteratively optimize width parameters and uses an improved least mean square learning rule to prevent over-computation. Meanwhile, a function model is used for constraint in the depth direction, the convenience and accuracy of the monocular camera calibration method are improved, and the demand of camera calibration can still be met under a complex distortion environment.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically to a monocular camera calibration method based on a phase target and a bidirectional path basis function neural network. Background Technology

[0002] Camera calibration has important applications in computer vision and machine vision, such as pose estimation, 3D reconstruction, and depth estimation. Currently, camera calibration methods primarily extract feature points from the corners of a checkerboard grid and the centers of circular markers, then perform calibration using the geometric mapping between the feature point coordinates and the camera's spatial coordinates. Regardless of whether a checkerboard grid or circular markers are used, the final calibration accuracy largely depends on the accuracy of the feature point extraction. These methods require predefined, precise mathematical models and pre-setting many camera parameters, and are prone to errors during the camera calibration process. Summary of the Invention

[0003] To address the aforementioned shortcomings in existing technologies, the monocular camera calibration method based on a phase target and a bidirectional path basis function neural network provided by this invention solves the problems of low accuracy and susceptibility to errors in existing monocular camera parameter calibration.

[0004] To achieve the aforementioned objectives, the present invention employs the following technical solution: a monocular camera calibration method based on a phase target and a bidirectional path basis function neural network, comprising the following steps:

[0005] S1. Obtain the absolute phase map of the phase target features of the monocular camera;

[0006] S2. Transform the absolute phase carried by the phase target feature points in the absolute phase image to three-dimensional space, and establish the correspondence between the feature point image coordinates and the world coordinates.

[0007] S3. Based on the correspondence between image coordinates and world coordinates, the two-dimensional image coordinates are input into the trained dual RBF neural network to complete the direct mapping from two-dimensional image coordinates to three-dimensional spatial coordinates, thereby realizing monocular camera calibration.

[0008] Further, step S1 specifically includes:

[0009] S11. Divide the phase target features of a monocular camera into two phase diagrams: horizontal and vertical phase-shifting fringes;

[0010] S12. Using a feature extraction method combining the three-step phase shifting method and the multi-frequency method, feature extraction is performed on the two-phase diagram to calculate the preliminary absolute phase diagram.

[0011] S13. Select samples that satisfy ψ on the preliminary absolute phase diagram. u =2πm and ψ v=2πn absolute phase map, as the absolute phase map of the phase target feature of the unit camera;

[0012] Where, ψ u , ψ v These represent the absolute phase diagrams calculated from the horizontal and vertical fringes, respectively. m and n are both integers, and π is a radian value.

[0013] Furthermore, in step S12, the expression for the three-step phase-shifting method is:

[0014] I1(u,v)=I′(u,v)+I″(u,v)cosθ

[0015] I2(u,v)=I′(u,v)+I″(u,v)cos(θ+2 / 3π)

[0016] I3(u,v)=I′(u,v)+I″(u,v)cos(θ+3 / 4π)

[0017] In the formula, I1, I2, I3 represent the gray values ​​of the phase map, (u, v) represent the coordinates of the two-dimensional image, I′ represents the background value of the stripe light intensity, I″ represents the modulation intensity of the modulated light intensity, and θ represents the principal value of the phase.

[0018]

[0019] In step S12, the multi-frequency method is used to dewrap the wrapped phase using phase diagrams of multiple frequencies in stages, obtaining continuous absolute phase, and thus obtaining a preliminary absolute phase diagram, the expression of which is:

[0020]

[0021] ψ(u,v)=θ h +2πk(u,v)

[0022] In the formula, k(u,v) represents the fringe order, ψ represents the continuous phase of the high-frequency fringe, θ1 represents the continuous phase of the low-frequency fringe, and θ h The truncated phases of the high-frequency fringes are represented by f1 and f2. h Represent θ1 and θ respectively h The frequency of the round value, where round represents the closest integer value returned;

[0023] In step S13, filtering is performed to satisfy ψ u =2πm and ψ v When the absolute phase diagram is 2πn, it satisfies Pixels are used as substitute feature points, and |ψ is selected. u -2πm|+|ψ vThe minimum value of -2πn| is used as the required feature point. Sub-pixel optimization is performed using least squares linear interpolation to obtain the absolute phase map of the monocular camera phase target feature; where ε is a manually set threshold.

[0024] Furthermore, in step S2, the correspondence between the feature point image coordinates and the world coordinates is as follows:

[0025]

[0026] In the formula, q represents the pixel pitch, p represents the number of pixels per stripe period on the LCD display, and (X,Y) represents the world coordinates in the horizontal direction.

[0027] Furthermore, in step S3, the dual RBF neural network includes a first GA-LMS-RBF neural network and a second GA-LMS-RBF neural network;

[0028] The world coordinate output of the dual RBF neural network is constrained by the depth-direction imaging function model, which divides the imaging model into two functions f. xz and f yz These correspond to the first GA-LMS-RBF neural network and the second GA-LMS-RBF neural network, respectively.

[0029] Among them, the first GA-LMS-RBF neural network uses the function f xz The input two-dimensional image coordinates are processed to obtain the output world coordinates (X). w Z w The second GA-LMS-RBF neural network uses the function f yz The input two-dimensional image coordinates are processed to obtain the output world coordinates (Y). w Z w ).

[0030] Furthermore, the expression for the depth-direction imaging function model is:

[0031]

[0032]

[0033]

[0034] P = R -1 T

[0035] Z w =Z c K(3,1)-P(3,1)

[0036]

[0037] In the formula, Z C Let X be the depth coordinates in the camera coordinate system, u and v be the x and y coordinates of the input 2D image, M1 be the intrinsic parameter matrix, R be the rotation matrix, and X be the depth coordinates. w Y w Z w Let T be the Z, Y, and Z-axis coordinates of the world coordinate system, T be the translation matrix, K be the intermediate parameter matrix, and P be a 3×1 matrix. K(3,1) and P(3,1) represent the third row and first column of matrices K and P, respectively.

[0038] Furthermore, the objective function ξ of the dual RBF neural network is:

[0039]

[0040] In the formula, N represents the N training samples of the dual RBF neural network, and e w To output the expected value, e' w This is the actual output value.

[0041] Furthermore, the dual RBF neural network incorporates a genetic algorithm to iteratively optimize the width parameter based on the RBF neural network and uses an improved least mean square learning rule to prevent overcomputation. The implementation method is as follows:

[0042] A1. Assess the fitness of the individual corresponding to each chromosome;

[0043] A2. Select two individuals from the population as the father and mother;

[0044] A3. Extract chromosomes from both parents, perform crossover, and produce offspring;

[0045] A4. Incorporate mutations into the chromosomes of offspring;

[0046] A5. Repeat steps A2 to A4 until a new population is generated, and use the individuals with the optimal width parameter as the population.

[0047] S26. After obtaining the optimized width parameter individuals, the connection weights in the dual RBF neural network are calculated using the least squares method, and the network is trained using an improved least mean square learning rule.

[0048] Furthermore, the number of hidden layer neurons in the dual RBF neural network is determined by using the K-means+ model to calculate the optimal number of clusters and the optimal cluster centers.

[0049] The beneficial effects of this invention are as follows:

[0050] (1) The camera calibration method of the present invention, which combines neural networks, omits the cumbersome calibration process of the traditional calibration method and only needs to consider data quality, network structure and parameter settings.

[0051] (2) The entire neural network calibration in this invention adopts the method of parallel calibration of dual neural networks and uses the RBF neural network algorithm based on GA-LMS (genetic-least mean square) to improve the calibration accuracy.

[0052] (3) Experimental results show that the calibration method proposed in this invention has better prediction effect than the traditional BP neural network, and has higher accuracy in the depth direction than other calibration targets. It is more suitable for application in the field of single-camera calibration and has high feasibility. Attached Figure Description

[0053] Figure 1 The flowchart of the monocular camera calibration method based on phase target and bidirectional path basis function neural network provided by the present invention is shown. Detailed Implementation

[0054] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0055] This invention provides a monocular camera calibration method based on a phase target and a bidirectional path basis function neural network, such as... Figure 1 As shown, it includes the following steps:

[0056] S1. Obtain the absolute phase map of the phase target features of the monocular camera;

[0057] S2. Transform the absolute phase carried by the phase target feature points in the absolute phase image to three-dimensional space, and establish the correspondence between the feature point image coordinates and the world coordinates.

[0058] S3. Based on the correspondence between image coordinates and world coordinates, the two-dimensional image coordinates are input into the trained dual RBF neural network to complete the direct mapping from two-dimensional image coordinates to three-dimensional spatial coordinates, thereby realizing monocular camera calibration.

[0059] Step S1 in this embodiment of the invention is specifically as follows:

[0060] S11. Divide the phase target features of a monocular camera into two phase diagrams: horizontal and vertical phase-shifting fringes;

[0061] S12. Using a feature extraction method combining the three-step phase shifting method and the multi-frequency method, feature extraction is performed on the two-phase diagram to calculate the preliminary absolute phase diagram.

[0062] S13. Select samples that satisfy ψ on the preliminary absolute phase diagram. u =2πm and ψ v =2πn absolute phase map, as the absolute phase map of the phase target feature of the unit camera;

[0063] Where, ψ u , ψ v These represent the absolute phase diagrams calculated from the horizontal and vertical fringes, respectively. m and n are both integers, and π is a radian value.

[0064] In step S12 of this embodiment, the expression for the three-step phase-shifting method is:

[0065] I1(u,v)=I′(u,v)+I″(u,v)cosθ

[0066] I2(u,v)=I′(u,v)+I″(u,v)cos(θ+2 / 3π)

[0067] I3(u,v)=I′(u,v)+I″(u,v)cos(θ+3 / 4π)

[0068] In the formula, I1, I2, I3 represent the gray values ​​of the phase map, (u, v) represent the coordinates of the two-dimensional image, I′ represents the background value of the stripe light intensity, I″ represents the modulation intensity of the modulated light intensity, and θ represents the principal value of the phase.

[0069]

[0070] In step S12 of this embodiment, the multi-frequency method is used to dewrap the wrapped phase using phase maps of multiple frequencies in stages to obtain continuous absolute phase, thereby obtaining a preliminary absolute phase map, the expression of which is:

[0071]

[0072] ψ(u,v)=θ h +2πk(u,v)

[0073] In the formula, k(u,v) represents the fringe order, ψ represents the continuous phase of the high-frequency fringe, θ1 represents the continuous phase of the low-frequency fringe, and θ h The truncated phases of the high-frequency fringes are represented by f1 and f2. h Represent θ1 and θ respectively h The frequency of the round value, where round represents the closest integer value returned;

[0074] In step S13 of this embodiment, the filter satisfies ψu =2πm and ψ v When the absolute phase diagram is 2πn, it satisfies Pixels are used as substitute feature points, and |ψ is selected. u -2πm|+|ψ v The minimum value of -2πn| is used as the required feature point. Sub-pixel optimization is performed using least squares linear interpolation to obtain the absolute phase map of the monocular camera phase target feature; where ε is a manually set threshold.

[0075] In step S2 of this embodiment of the invention, the correspondence between the feature point image coordinates and the world coordinates is as follows:

[0076]

[0077] In the formula, q represents the pixel pitch, p represents the number of pixels per stripe period on the LCD display, and (X,Y) represents the world coordinates in the horizontal direction.

[0078] Based on the above correspondence, the phase carried by each feature point can be easily converted into world coordinates.

[0079] In step S3 of this embodiment of the invention, the dual RBF neural network includes a first GA-LMS-RBF neural network and a second GA-LMS-RBF neural network;

[0080] The world coordinate output of the dual RBF neural network is constrained by the depth-direction imaging function model, which divides the imaging model into two functions f. xz and f yz These correspond to the first GA-LMS-RBF neural network and the second GA-LMS-RBF neural network, respectively.

[0081] Among them, the first GA-LMS-RBF neural network uses the function f xz The input two-dimensional image coordinates are processed to obtain the output world coordinates (X). w Z w The second GA-LMS-RBF neural network uses the function f yz The input two-dimensional image coordinates are processed to obtain the output world coordinates (Y). w Z w ).

[0082] In this embodiment, the expression for the depth-direction imaging function model is:

[0083]

[0084]

[0085]

[0086] P = R -1 T

[0087] Z w =Z c K(3,1)-P(3,1)

[0088]

[0089] In the formula, Z C Let X be the depth coordinates in the camera coordinate system, u and v be the x and y coordinates of the input 2D image, M1 be the intrinsic parameter matrix, R be the rotation matrix, and X be the depth coordinates. w Y w Z w Let T be the Z, Y, and Z-axis coordinates of the world coordinate system, T be the translation matrix, K be the intermediate parameter matrix, and P be a 3×1 matrix. K(3,1) and P(3,1) represent the third row and first column of matrices K and P, respectively.

[0090] In this embodiment, the input value of the dual RBF neural network is the phase coordinates carried by the phase target feature point. The hidden layer consists of n neurons, w is the connection weight from the hidden layer to each output layer, and the output layer is the world space coordinates (X) corresponding to the feature point. w Z w1 ), (Y w Z w2 ), input training samples into the dual RBF neural network, the error function is defined as the objective function, and its objective function ξ is:

[0091]

[0092] In the formula, N represents the N training samples of the dual RBF neural network, and e w To output the expected value, e' w This is the actual output value.

[0093] In this embodiment of the dual RBF neural network, a Gaussian activation function is used, and the gradient expression of the objective function is obtained as follows:

[0094]

[0095] Where m represents the number of iterations, and η1, η2, and η3 are different convergence factors.

[0096] In this forty-mile study, the dual RBF neural network incorporates a genetic algorithm to iteratively optimize the width parameter based on the RBF neural network and uses an improved least mean square learning rule to prevent overcomputation. The implementation method is as follows:

[0097] A1. Assess the fitness of the individual corresponding to each chromosome;

[0098] A2. Select two individuals from the population as the father and mother;

[0099] A3. Extract chromosomes from both parents, perform crossover, and produce offspring;

[0100] A4. Incorporate mutations into the chromosomes of offspring;

[0101] A5. Repeat steps A2 to A4 until a new population is generated, and use the individuals with the optimal width parameter as the population.

[0102] S26. After obtaining the optimized width parameter individuals, the connection weights in the dual RBF neural network are calculated using the least squares method, and the network is trained using an improved least mean square learning rule.

[0103] In this ongoing embodiment, the number of hidden layer neurons in the dual RBF neural network is determined by using the K-means+ model to calculate the optimal number of clusters and the optimal cluster centers.

[0104] In the description of this invention, it should be understood that the terms "center," "thickness," "upper," "lower," "horizontal," "top," "bottom," "inner," "outer," and "radial," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying the relative importance or the number of technical features implicitly specified. Therefore, a feature defined by "first," "second," and "third" may explicitly or implicitly include one or more of that feature.

[0105] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

[0106] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A monocular camera calibration method based on a phase target and a two-way path basis function neural network, characterized in that, Includes the following steps: S1. Obtain the absolute phase map of the phase target features of the monocular camera; S2. Transform the absolute phase carried by the phase target feature points in the absolute phase image to three-dimensional space, and establish the correspondence between the feature point image coordinates and the world coordinates. S3. Based on the correspondence between image coordinates and world coordinates, the two-dimensional image coordinates are input into the trained dual RBF neural network to complete the direct mapping from two-dimensional image coordinates to three-dimensional spatial coordinates, thereby realizing monocular camera calibration. In step S3, the dual RBF neural network includes a first GA-LMS-RBF neural network and a second GA-LMS-RBF neural network; The world coordinate output of the dual RBF neural network is constrained by the depth-direction imaging function model, which divides the imaging model into two functions. and These correspond to the first GA-LMS-RBF neural network and the second GA-LMS-RBF neural network, respectively. Among them, the first GA-LMS-RBF neural network uses a function The input two-dimensional image coordinates are processed to obtain the output world coordinates. The second GA-LMS-RBF neural network uses a function The input two-dimensional image coordinates are processed to obtain the output world coordinates. ; The expression for the depth-direction imaging function model is: In the formula, This represents the depth direction value in the camera coordinate system. These are the x and y coordinates of the input two-dimensional image, respectively. The intrinsic parameter matrix, Let be a rotation matrix. These are the X, Y, and Z coordinates of the world coordinate system. It is a translation matrix. This is the intermediate parameter matrix. All are 3 A matrix of 1, , express , The first column of the third row of the matrix.

2. The monocular camera calibration method based on a phase target and a bidirectional path basis function neural network according to claim 1, characterized in that, Step S1 specifically involves: S11. Divide the phase target features of a monocular camera into two phase diagrams: horizontal and vertical phase-shifting fringes; S12. Using a feature extraction method combining the three-step phase shifting method and the multi-frequency method, feature extraction is performed on the two-phase diagram to calculate the preliminary absolute phase diagram. S13. Select samples that meet the requirements on the preliminary absolute phase diagram. and The absolute phase map, serving as the absolute phase map of the phase target feature of a unit camera; in, , These represent the absolute phase diagrams calculated from the horizontal and vertical fringes, respectively. , All are integers. The value is in radians.

3. The monocular camera calibration method based on a phase target and a bidirectional path basis function neural network according to claim 2, characterized in that, In step S12, the expression for the three-step phase-shifting method is: In the formula, , , This represents the grayscale value of the phase map. Represents the coordinates of a two-dimensional image. The background value representing the intensity of the striped light. The modulation intensity represents the intensity of the modulated light. Indicates the principal value of the phase; In step S12, the multi-frequency method is used to dewrap the wrapped phase using phase diagrams of multiple frequencies in stages, obtaining continuous absolute phase, and thus obtaining a preliminary absolute phase diagram, the expression of which is: In the formula, Indicates the stripe order. This represents the continuous phase of the high-frequency stripes. This indicates the continuous phase of the low-frequency stripes. Indicates the cutoff phase of the high-frequency stripe. , They represent and frequency, Indicates the closest integer value returned; In step S13, filtering is performed to meet the following conditions. and When the absolute phase diagram is obtained, it satisfies Pixels are selected as alternative feature points. The minimum value is used as the required feature point, and sub-pixel optimization is performed using least-squares linear interpolation to obtain the absolute phase map of the monocular camera's phase target features; where, The threshold is set manually.

4. The monocular camera calibration method based on a phase target and a bidirectional path basis function neural network according to claim 2, characterized in that, In step S2, the correspondence between the feature point image coordinates and the world coordinates is as follows: In the formula, Indicates pixel spacing. This represents the number of pixels per stripe cycle on the LCD display, and (X,Y) represents the world coordinates in the horizontal direction.

5. The monocular camera calibration method based on a phase target and a bidirectional path basis function neural network according to claim 1, characterized in that, The objective function of the dual RBF neural network for: In the formula, N For dual RBF neural networks N training samples, To output the expected value, This is the actual output value.

6. The monocular camera calibration method based on a phase target and a bidirectional path basis function neural network according to claim 1, characterized in that, The dual RBF neural network incorporates a genetic algorithm to iteratively optimize the width parameter based on the RBF neural network and uses an improved least mean square learning rule to prevent overcomputation. Its implementation method is as follows: A1. Assess the fitness of the individual corresponding to each chromosome; A2. Select two individuals from the population as the father and mother; A3. Extract chromosomes from both parents, perform crossover, and produce offspring; A4. Incorporate mutations into the chromosomes of offspring; A5. Repeat steps A2 to A4 until a new population is generated, and use the individuals with the optimal width parameter as the population. S26. After obtaining the optimized width parameter individuals, the connection weights in the dual RBF neural network are calculated using the least squares method, and the network is trained using an improved least mean square learning rule.

7. The monocular camera calibration method based on a phase target and a bidirectional path basis function neural network according to claim 1, characterized in that, The number of hidden layer neurons in the dual RBF neural network is determined by using the K-means+ model to calculate the optimal number of clusters and the optimal cluster centers.

Images