Multi-camera Calibration Optimization Method Based on EOBL-SSA Algorithm

Abstract

The present invention relates to a multi-camera calibration optimization method based on the EOBL-SSA algorithm, which adopts the fusion model optimization technology to study the problem of the calibration accuracy of visual cameras in the visual positioning of robots. According to the advantages and disadvantages of current new intelligent algorithms, a robot vision camera calibration optimization algorithm based on the fusion of the elite opposition-based learning algorithm and the sparrow search algorithm is proposed, and a comparative analysis is carried out between the sparrow search algorithm and the EOBL-SSA optimization method proposed in the present invention. The present invention can improve the calibration accuracy of robot vision cameras and provide higher visual accuracy for the three-dimensional reconstruction and path planning of robot vision.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision, and particularly relates to a multi-camera calibration optimization method based on the EOBL-SSA algorithm. Background Technique

[0002] Cameras are widely used in various aspects of the industrial field, including driverless cars, mobile robots, industrial robots, etc., providing visual assistance for production and manufacturing. Compared with expensive lidar, visual cameras are having an increasing influence in the field of target recognition and detection. Accurately obtaining the pose of the target is the key to target recognition and detection, and camera parameter calibration is the basis and key technology to achieve this goal. The reliability of the calibration of the internal and external parameters of the camera directly affects the accuracy and stability of the measurement of the target pose. Therefore, improving the accuracy of camera calibration is of great significance for industrial production and manufacturing.

[0003] Reverse learning is a novel intelligent computing method proposed by Tizhoosh and is widely applied to improve the performance of various optimization algorithms. Based on the effectiveness of the reverse learning strategy, Wang et al. improved the reverse learning strategy, introduced a generalized reverse factor on the basis of reverse learning, proposed the general reverse learning strategy, and conducted a series of comparative experiments. The experimental results show that the general reverse learning strategy has superior performance compared with the reverse learning strategy. Subsequently, Wang Shenwen et al. introduced the idea of elite learning on the basis of the general reverse learning strategy, proposed the elite reverse learning strategy, and the comparative experimental results on a series of function optimization test problems show that the elite reverse learning strategy can enhance the performance of the general reverse learning strategy to a great extent. Summary of the Invention

[0004] The purpose of the present invention is to provide a multi-camera calibration optimization method based on the EOBL-SSA algorithm, which optimizes the internal parameters and distortion coefficients of camera calibration based on the traditional Zhang Zhengyou calibration method, thereby improving the accuracy of robot vision camera calibration.

[0005] The technical solution for achieving the purpose of the present invention is: a multi-camera calibration optimization method based on the EOBL-SSA algorithm, including the following steps:

[0006] Step 1, obtain calibration board images at different angles, preprocess the pictures and extract corner features;

[0007] Step 2, solve the initial values of the internal parameters and distortion coefficients;

[0008] Step 3, initialize the relevant parameters of the SSA algorithm and the parameters of the EOBL;

[0009] Step 4, update the positions of the discoverers, joiners and guardians in the sparrows;

[0010] Step 5: Use the elite reverse strategy to perform reverse solution on excellent individuals, retain the excellent individuals, compare them with the sparrow positions in the previous iteration and replace them, and update the positions of this group of sparrows.

[0011] Step 6: Determine whether the fitness value of the optimal individual meets the preset accuracy or reaches the maximum number of iterations. If not, return to Step 4. Otherwise, retain the position of the optimal individual, and the parameters corresponding to this individual are the results of camera calibration.

[0012] A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above camera calibration optimization method are implemented.

[0013] A computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, the steps of the above camera calibration optimization method are implemented.

[0014] Compared with the prior art, the present invention has the following significant advantages: 1) The reprojection error of the present invention is smaller and the calibration accuracy is higher than that of the traditional method. 2) The present invention has strong robustness, can be reused, can better improve the problem of local convergence of the sparrow algorithm, and obtains better results. 3) The present invention has good accuracy and feasibility for optimizing the internal parameters of the camera. 4) The algorithm proposed by the present invention can be combined with actual engineering cases and can be accurately and effectively used for optimizing the solution of multi-dimensional non-linear problems. Description of the Drawings

[0015] Figure 1 It is a flowchart of the present invention.

[0016] Figure 2 It is partial calibration pictures.

[0017] Figure 3 They are objective function curves of SSA iterated 500 times and 1000 times.

[0018] Figure 4 They are objective function curves of EOBL-SSA iterated 500 times and 1000 times. Detailed Embodiments

[0019] The Sparrow Search Algorithm (SSA) is a new meta-heuristic intelligent optimization algorithm proposed in 2020. The generation of SSA is mainly inspired by the foraging behavior and anti-predation behavior of sparrows, and continuously updates the discoverers, joiners, and vigilants. It has the advantages of strong optimization ability and fast convergence speed.

[0020] Based on the sparrow algorithm, the present invention combines the elite opposition-based learning strategy to optimize camera parameters, uses the elite opposition-based learning strategy to reduce the possibility of the sparrow algorithm falling into local optimum, improve the diversity of the population, and increase the convergence speed of the algorithm, thereby ensuring the global solution accuracy and efficiency of the algorithm, and reducing the reprojection error of the camera.

[0021] The present invention proposes an application of camera calibration in robot visual positioning based on an improved sparrow algorithm, adopts a fusion model optimization technology, and studies the problem of the calibration accuracy of the vision camera in robot visual positioning. According to the problems of the advantages and disadvantages of the current new intelligent algorithms, an optimized algorithm for robot vision camera calibration based on the fusion of the elite opposition-based learning algorithm (EOBL) and the sparrow search algorithm (SSA) is proposed, and a comparative analysis is carried out between the sparrow search algorithm and the EOBL-SSA optimization method proposed by the present invention.

[0022] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] A multi-camera calibration optimization method based on the EOBL-SSA algorithm includes the following steps:

[0024] Step 1: Obtain calibration board images at different angles, preprocess the images, and extract corner features.

[0025] Step 2: Solve the initial values of the internal parameters f x ,f y ,u0, v0 and distortion coefficients k1, k2, k3, p1, p2.

[0026] Step 3: Initialize the relevant parameters of the SSA algorithm and the parameters of the EOBL.

[0027] Step 4: Update the positions of the discoverers, joiners, and scouters in the sparrows.

[0028] Step 5: Use the elite opposition-based learning strategy to perform reverse solution on the top 10% of the excellent individuals, retain the excellent individuals, compare with the positions of the sparrows in the previous iteration and replace them, and update the positions of this group of sparrows.

[0029] Step 6: Determine whether the fitness value of the optimal individual meets the preset accuracy or whether the maximum number of iterations is reached. If not, return to Step 4. Otherwise, retain the position of the optimal individual, and the parameters corresponding to this individual are the results of camera calibration.

[0030] Furthermore, in Step 1, obtain images of the calibration board in different directions by the camera to be calibrated, and the number of images should be more than 6; perform grayscale processing on the obtained images, and extract the corner points of the checkerboard in the images.

[0031] Furthermore, Step 2 is specifically:

[0032] According to the camera imaging relationship and the internal relationship of the camera, f x = f c1 / d x , f y = f c2 / d y , the initial values of f x , f y , u0, and v0 are obtained; where f c1 and f c2 are the camera focal lengths; d x and d y are the physical lengths of the pixels; u0 and v0 are the intersection points of the camera optical axis and the image plane;

[0033] The camera imaging relationship formula is:

[0034]

[0035] where z c is the object distance, x d , y d is the pixel coordinate system, x w , y w , z w is the world coordinate system, and R, T are the image rotation and translation matrices;

[0036] Since these initial values are obtained under ideal conditions, distortion coefficients k1, k2, k3, p1, p2 need to be introduced for correction, where k1, k2, k3 are the radial distortion coefficients and p1, p2 are the tangential distortion coefficients;

[0037] Using the radial distortion mathematical model

[0038]

[0039] and the tangential distortion mathematical model

[0040]

[0041] and by combining them, we get

[0042]

[0043] the initial values under distortion are obtained.

[0044] The camera is calibrated using the Zhang Zhengyou calibration method to obtain the parameter values before optimization.

[0045] where [x u , y u is the coordinate of any point p on the image normalization plane, and r is the distance between point p and the origin of the coordinate system.

[0046] Furthermore, the process of initializing the relevant parameters of the SSA algorithm and the EOBL algorithm in step 3 is as follows:

[0047] It includes the number of populations n, the range of the search space, and the number of iterations iter max Randomly select the initial positions of n individuals Find the position of the optimal individual in the current population and calculate its fitness, where d is the dimension of the population individuals;

[0048] Establish the objective function of the camera calibration problem:

[0049]

[0050] where p ij is the matching point of the image, and p is the re-projected point corresponding to p ij ;

[0051] Obtain the fitness function according to the objective function to find the fitness value of the optimal individual in the iteration. Define the fitness function as:

[0052]

[0053] where (x, y) are the actual pixel coordinate points obtained by the corner extraction algorithm; (u, v) are the pixel coordinate points calculated by the camera imaging relationship; m is the total number of corners.

[0054] Furthermore, step 4 is specifically as follows:

[0055] During each iteration, the position update of the discoverer is described as follows:

[0056]

[0057] where t represents the current iteration number, j = 1, 2, 3..., d;

[0058] iter max is a constant representing the maximum number of iterations; X i,j represents the position information of the i-th sparrow in the j-th dimension; α ∈ (0, 1] is a random number; R2 and ST represent the warning value and the safety value respectively, R2 ∈ [0, 1], ST ∈ [0.5, 1]; Q is a random number subject to the normal distribution; L represents a 1×d matrix, where each element in the matrix is all 1;

[0059] The position update of the joiner is described as follows:

[0060]

[0061] where, X pis the optimal position occupied by the current discoverer, X worst represents the current globally worst position; A represents a 1×d matrix, where each element is randomly assigned a value of 1 or -1, and A + = A T (AA T ) -1 ;

[0062]

[0063] where X best is the current global optimal position; β, as a step size control parameter, is a random number that follows a normal distribution with a mean of 0 and a variance of 1; K ∈ [-1, 1] is a random number, and f i is the fitness value of the current sparrow individual; f g and f ω are the current global best and worst fitness values respectively; ε is a constant to avoid a zero denominator; when f i > f g it means that the sparrow is at the edge of the population at this time and is extremely vulnerable to predators; X best indicates that the sparrow at this position is the best in the population and is also very safe; f i = f g when this indicates that the sparrows in the population reduction are aware of the danger and need to get closer to other sparrows to minimize their risk of being preyed upon; K represents the direction of the sparrow's movement and is also a step size control parameter.

[0064] Furthermore, step 5 is specifically as follows:

[0065] Take the top 10% of the sparrow fitness rankings as the elite solutions, and at the same time obtain the dynamic boundaries of the elite sparrows. Use the reverse learning strategy to solve the reverse solutions. Compare the sparrows before and after the update. If it is better, replace the previous sparrows;

[0066]

[0067] are the individuals in the sparrow population with the top 10% fitness rankings;

[0068] Its reverse solution is expressed as:

[0069]

[0070] where K1 is a dynamic coefficient on (0, 1); β j = max(X i,j ), α j , β jis a dynamic boundary; if the dynamic boundary operation makes crossing the boundary become an infeasible solution, reset

[0071] To verify the feasibility and effectiveness of the proposed EOBL-SSA algorithm in the field of camera calibration in machine vision, as Figure 2 shown, a set of acquired calibration board images is selected as the camera calibration material, and the camera that captured this set of images is calibrated. The results of the Zhang-Zhengyou calibration method are shown in the following table:

[0072]

[0073]

[0074] The calibration results of the camera internal parameters and distortion coefficients after 500 iterations of the SSA algorithm and the EOBL-SSA algorithm are as follows:

[0075]

[0076] The calibration results of the camera internal parameters after 1000 iterations of the SSA algorithm and the EOBL-SSA algorithm are shown in the following table:

[0077]

[0078] The calculated optimized reprojection error is shown in the following table:

[0079]

[0080] Combined with Figure 3 and Figure 4 it can be seen that the SSA algorithm has a fast convergence speed, but it is easy to fall into local optimum. The EOBL-SSA can improve the convergence speed and convergence accuracy of the algorithm on the basis of the SSA algorithm, and improve the overall performance of the algorithm. It can be seen from the table that both the SSA algorithm and the EOBL-SSA algorithm can achieve good optimization of the results of the Zhang-Zhengyou calibration method. The EOBL-SSA algorithm generally has better results than the SSA algorithm. It has strong robustness, can be reused, and can better improve the problem of local convergence of the ordinary SSA algorithm and obtain better results.

Claims

1. A multi-camera calibration optimization method based on the EOBL-SSA algorithm, characterized in that, It includes the following steps: Step 1: Obtain calibration board images from different angles, preprocess the pictures, and extract corner features; Step 2: Solve the initial values of the internal parameters and distortion coefficients. Specifically: According to the camera imaging relationship and the internal relationship of the camera f x = f c1 / d x , f y = f c2 / d y , the initial values of u0 and v0 are obtained; where f c1 and f c2 are the camera focal lengths; d x and d y are the physical lengths of the pixels; u0 and v0 are the intersections of the camera optical axis and the image plane; The camera imaging relationship formula is: where z c is the object distance, x d , y d is the pixel coordinate system, x w , y w , z w is the world coordinate system, and R and T are the image rotation and translation matrices; Since this initial value is obtained under ideal conditions, distortion coefficients need to be introduced for correction. k1, k2, and k3 are radial distortion coefficients, and p1 and p2 are tangential distortion coefficients; Use the radial distortion mathematical model and the tangential distortion mathematical model as well as Combined Obtain the initial value under distortion; where, [x u , y u is the coordinate of any point p on the image normalization plane, and r is the distance between point p and the origin of the coordinate system; Calibrate the camera using Zhang Zhengyou's calibration method to obtain the parameter values before optimization; Step 3: Initialize the relevant parameters of the SSA algorithm and the parameters of EOBL. The specific process is as follows: Including the population size n, the range of the search space, and the number of iterations iter max Randomly select the initial positions of n individuals Find the position of the optimal individual in the current population and calculate its fitness, where d is the dimension of the population individuals; Establish the objective function of the camera calibration problem: where p ij is the matching point of the image, and p is the ij corresponding reprojected point; Obtain the fitness function according to the objective function to find the optimal individual fitness value of the iteration. Define the fitness function as: Where (x, y) is the actual pixel coordinate point obtained through the corner extraction algorithm, (u, v) is the pixel coordinate point calculated through the camera imaging relationship, and m is the total number of corners; Step 4: Update the positions of the discoverers, joiners, and scouts in the sparrows. Specifically: During each iteration, the position update of the discoverers is described as follows: where t represents the current iteration number, j = 1, 2, 3,..., d; X i,j represents the position information of the i-th sparrow in the j-th dimension; α ∈ (0, 1] is a random number; R2 and ST represent the early warning value and the safety value respectively, R2 ∈ [0, 1], ST ∈ [0.5, 1]; Q is a random number subject to a normal distribution; L represents a 1×d matrix, where each element in the matrix is all 1; The position update of the joiners is described as follows: Among them, X p is the optimal position occupied by the current discoverer, and X worst represents the current globally worst position; A represents a 1×d matrix, where each element is randomly assigned a value of 1 or -1, and A + = A T (AA T ) -1 ; Among which X best is the current global optimal position; β, as the step size control parameter, is a random number following a normal distribution with a mean of 0 and a variance of 1; K ∈ [-1, 1] is a random number, and f i is the fitness value of the current sparrow individual; f g and f ω are the current global best and worst fitness values respectively; ε is a constant; X best indicates that the sparrow at this position is the best in the population; K represents the moving direction of the sparrow Step 5: Use the elite reverse strategy to perform reverse solution on the excellent individuals, retain the excellent individuals, compare them with the sparrow positions in the previous iteration and replace them, and update the positions of this group of sparrows. Specifically as follows: Take 10% of the sparrow fitness rankings as the elite solutions. At the same time, obtain the dynamic boundaries of the elite sparrows, use the reverse learning strategy to solve the reverse solutions, compare the sparrows before and after the update, and replace the previous sparrows if they are better; Individuals with fitness ranking in the top 10% in the sparrow population; Its inverse solution It is expressed as: Among them, K1 is a dynamic coefficient on (0, 1); α j = min(X i,j ), β j = max(X i,j ), α j , β j are dynamic boundaries; if the dynamic boundary operation makes cross the boundary and become an infeasible solution, reset Step 6: Determine whether the value of the optimal individual fitness meets the preset accuracy or reaches the maximum number of iterations. If not, return to Step 4. Otherwise, retain the position of the optimal individual, and the parameters corresponding to this individual are the results of the camera calibration.

2. The multi-camera calibration optimization method based on the EOBL-SSA algorithm according to claim 1, characterized in that In Step 1, obtain the images of the calibration board in different directions by the camera to be calibrated. The number of images should be more than 6; perform grayscale processing on the obtained images, and extract the corners of the checkerboard in the images.

3. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it realizes the steps of the multi-camera calibration optimization method described in any one of claims 1 to 2.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it realizes the steps of the multi-camera calibration optimization method described in any one of claims 1 to 2.

Images