Optimization Method for Underwater Camera Calibration Based on Improved Slime Mould Algorithm

Abstract

The present invention discloses an optimization method for underwater camera calibration based on an improved slime mold optimization algorithm. On the basis of the slime mold algorithm, it integrates the optimal neighborhood perturbation and the reverse learning strategy to optimize the parameters, so as to improve the convergence speed and solve the problem of local convergence of the algorithm, and improve the calibration accuracy of the underwater camera. Moreover, the Zhang calibration method, the SMA algorithm, the SOA algorithm, the PSO algorithm and the ORSMA fusion algorithm proposed by the present invention are analyzed and compared, and the present invention can effectively optimize the internal parameters of the underwater camera.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision, and particularly relates to an underwater camera calibration optimization method based on an improved slime mold algorithm. Background Art

[0002] As the development of onshore resources gradually reaches saturation, the vast ocean resources have become the objects pursued by countries around the world for development. The development of the marine economy is an important part of the current world economic development. Therefore, implementing the strategy of strengthening the sea with science and technology is very important for the development of each country. In recent years, underwater robots have gradually replaced humans to explore and exploit underwater resources. Underwater radars and underwater cameras have become important ways for humans to understand the underwater world through robots. Compared with expensive underwater radars, cheap underwater cameras have become one of the main sensors of underwater robots. The calibration accuracy of the camera is particularly important for achieving accurate positioning and tracking of underwater targets. Accurate camera calibration is the premise for realizing accurate positioning of underwater targets and is also one of the key technologies to promote the implementation of the strategy of strengthening the sea with science and technology.

[0003] Different from calibration in air, the change of the underwater imaging model has an important impact on camera calibration. In recent years, foreign researchers have proposed some more suitable calibration methods. For example, in 2012, Anne Jordt-Sedlazeck et al. proposed a camera calibration method in "Refractive Calibration of Underwater Cameras (Computer Vision–ECCV2012: pp. 846-859.)" to calibrate the internal parameters, external parameters and waterproof equipment parameters of the camera. This method is applicable to mono and stereo calibration, but the process is relatively complex and time-consuming. In 2021, Lan Rongfu used the Zhang's calibration method to calibrate the internal and external parameters of the camera in "Research on Underwater Target Detection and Underwater Camera Calibration Method Based on Vision (Harbin Institute of Technology, 2021)", established an error model, and used the particle swarm optimization algorithm to optimize and solve the waterproof equipment parameters, but the algorithm is prone to falling into local optimum and the calibration accuracy is not high. Summary of the Invention

[0004] The purpose of the present invention is to provide an underwater camera calibration optimization method based on an improved slime mold algorithm to optimize the internal parameters and distortion coefficients of camera calibration based on the traditional Zhang Zhengyou calibration method, thereby improving the calibration accuracy of the underwater camera.

[0005] The technical solution for achieving the purpose of the present invention is: a method for optimizing camera calibration based on an improved slime mold algorithm, comprising the following steps:

[0006] Step 1: Obtain calibration board images at different angles, preprocess the images and extract corner features;

[0007] Step 2: Solve the initial values of the internal parameters and distortion coefficients;

[0008] Step 3: Initialize the relevant parameters of the SMA algorithm, initialize the initial positions of the population, and initialize the relevant parameters of ONP and OBL;

[0009] Step 4: Calculate the fitness of each slime mold individual, and select the current optimal fitness and its corresponding position;

[0010] Step 5: Use the optimal neighborhood perturbation strategy to update the global position, generate the reverse solution using the reverse learning strategy; and use the greedy mechanism to determine whether the generated neighborhood position is saved; retain the solution with higher fitness and update its position.

[0011] Step 6: Determine whether the value of the optimal individual fitness meets the preset accuracy or reaches the maximum number of iterations. If not, return to Step 4. Otherwise, retain the position of the optimal individual, and the parameters corresponding to this individual are the results of camera calibration.

[0012] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the above-mentioned underwater camera calibration optimization method based on the improved slime mold algorithm.

[0013] A computer-readable storage medium stores a computer program, and when the program is executed by a processor, it implements the above-mentioned underwater camera calibration optimization method based on the improved slime mold algorithm.

[0014] Compared with the prior art, the present invention has the following significant advantages: 1) The reprojection error of the present invention is smaller than that of the traditional method, and the calibration accuracy is higher; 2) The present invention has strong robustness, can be reused, can better improve the problems of slow convergence speed and local convergence of the slime mold algorithm, and obtains better results; 3) The present invention has good accuracy and feasibility for optimizing the internal parameters of underwater cameras; 4) The algorithm proposed by the present invention can be combined with actual engineering cases and can be accurately and effectively used for optimizing the solution of multi-dimensional non-linear problems. Description of the Drawings

[0015] Figure 1 It is a flowchart of the underwater camera calibration optimization method based on the improved slime mold algorithm of the present invention.

[0016] Figure 2 It is partial calibration pictures.

[0017] Figure 3 It is the objective function curve graphs of SOA, PSO, SMA, and ORSMA iterated 500 times.

[0018] Figure 4Objective function curves for 1000 iterations of SOA, PSO, SMA, and ORSMA. Detailed implementation

[0019] The slime mould algorithm (SMA) is a swarm intelligence algorithm proposed by Li Shimin, Mirjalili et al. in 2020. SMA simulates the diffusion behavior and foraging behavior of slime moulds, and the optimal food connection path formed has good exploration ability.

[0020] The optimal neighborhood perturbation (ONP) algorithm can randomly search the area near the optimal position to find a better global value, which can improve the convergence speed and avoid the "premature" phenomenon.

[0021] Opposition-Based learning (OBL) is a new intelligent optimization mechanism. The concept of opposition-based learning was proposed by Tizhoosh in 2005. The idea of opposition-based learning is to consider both the forward solution and the backward solution, and select the optimal solution as the initial population. Using opposition-based learning to initialize the population can expand the search range of the population and improve the speed and efficiency of the algorithm to find the optimal solution. Soon after opposition-based learning was proposed, Rahnamayan et al. proved by mathematical derivation that the optimization speed of the opposition-based learning strategy is faster and the learning ability is stronger, and verified this result through experiments.

[0022] Based on the slime mould algorithm, this invention optimizes the camera parameters by integrating the optimal neighborhood perturbation and the opposition-based learning strategy, uses the optimal neighborhood perturbation and the opposition-based learning strategy to reduce the possibility of the slime mould algorithm falling into local optimum, improve the diversity of the population, improve the convergence speed of the algorithm, and then ensure the global solution accuracy and efficiency of the algorithm, so as to reduce the reprojection error of the camera.

[0023] As Figure 1 shown, a method for underwater camera calibration optimization based on an improved slime mould algorithm of the present invention includes the following steps:

[0024] Step 1: Obtain images of the calibration board taken by the camera to be calibrated in different directions, and the number of images should be greater than 6. Gray-scale the obtained images and extract the corner points of the checkerboard in the images.

[0025] Step 2: According to the camera imaging relationship and the camera internal relationship f x = f c1 / d x , f y = f c2 / d y, obtain f x , f y , u 0 , v 0 The initial values. Among them, f c1 and f c2 are the camera focal lengths; d x and d y are the physical lengths of the pixels; u 0 and v 0 are the intersections of the camera optical axis and the image plane.

[0026] The camera imaging relationship formula is:

[0027]

[0028] Among them, z c is the object distance, x d , y d is the pixel coordinate system, x w , y w , z w is the world coordinate system, and R, T are the image rotation and translation matrices.

[0029] Since these initial values are obtained under ideal conditions, distortion coefficients k 1 , k 2 , k 3 , p 1 , p 2 are needed to correct them.

[0030] Using the radial distortion mathematical model

[0031]

[0032] Among them, [x u , y u are the coordinates of any point p on the image normalized plane, and s is the distance between point p and the origin of the coordinate system.

[0033] And the tangential distortion mathematical model

[0034]

[0035] And Combining them to obtain

[0036]

[0037] Obtain the initial values under distortion.

[0038] Use the Zhang Zhengyou calibration method to calibrate the camera and obtain the parameter values before optimization.

[0039] Step 3: Initialize the relevant parameters of the SMA algorithm, initialize the initial positions of the population, and initialize the relevant parameters of ONP and OBL. The process includes: the number of individuals in the population n, the search space dimension d, the maximum number of iterations max_t, the upper and lower bounds of the initial values ub and lb, and randomly select the initial positions of n individuals within the search range.

[0040] Step 4: Calculate the fitness of each slime mold individual, and select the current optimal fitness and its corresponding position as:

[0041] Establish the objective function of the camera calibration problem:

[0042]

[0043] where N is the number of corner points, p ij is the matching point of the image, and p is the reprojection point corresponding to p ij .

[0044] Obtain the fitness function according to the objective function to find the optimal individual fitness value of the iteration. Define the fitness function as:

[0045]

[0046] where (x, y) is the actual pixel coordinate point obtained by the corner extraction algorithm; (u, v) is the pixel coordinate point calculated through the camera imaging relationship; m is the total number of corner points.

[0047] Step 5: In the process of each iteration, the update formula of the slime mold position is as follows:

[0048]

[0049] where, is the position of the slime mold after update, is a parameter in the range [-a, a], is a parameter that linearly decreases from 1 to 0, t represents the current iteration number, represents the position of the individual with the highest current fitness, represents the position of the slime mold, and represent the positions of two randomly selected individuals in the slime mold, represents the weight coefficient of the slime mold, p is the control parameter, LB and UB represent the upper and lower bounds of the search range, rand and r represent random values in [0, 1]. z is a custom parameter, and let z be 0.03.

[0050] The formula for the parameter a is as follows:

[0051]

[0052] The formula for the control parameter p is as follows:

[0053] p = thnh|S(i) - DF|, i ∈ 1, 2, 3,......, n (9)

[0054] Where S(i) represents the fitness, and DF represents the best fitness obtained in all iterations.

[0055] Weight coefficient The formula is as follows:

[0056]

[0057] SmellIndex = sort(S) (11)

[0058] Where r is a random value within the interval [0, 1], bF represents the optimal fitness obtained in the current iteration process, wF represents the worst fitness value obtained in the current iteration process, condition represents the individuals in the slime mold population whose fitness values rank among, others represents the remaining individuals in the slime mold population, and SmellIndex represents the sequence sorting of fitness values (for the minimum problem, it is an increasing sequence).

[0059] Use the optimal neighborhood perturbation strategy to perform global position update. The formula for performing a random perturbation operation on the optimal position is as follows:

[0060]

[0061] In the formula, X * (t) is the optimal position obtained in formula (7), is the new position obtained by performing a random perturbation on X * (t), rand1 and rand2 are random numbers uniformly generated within the interval [0, 1]; X(t) is the newly generated position.

[0062] Use the reverse learning strategy to generate reverse solutions. If the individual X i in the population can be represented by the following formula:

[0063] X i = [X i,1 , X i,2 , … X i,j , X i,D (13)

[0064] Then its reverse solution can be represented as:

[0065] X i ' = [X i,1 ', Xi,2 ',…X i,j ',X i,D '] (14)

[0066] where \(i = 1, 2, 3, \ldots, n\), \(j = 1, 2, 3, \ldots, D\), and the reverse solution and the forward solution also need to satisfy the following relationship:

[0067] X i,j ' = k(A j +B j ) - X i,j (15)

[0068] where \(k\) is a random number uniformly distributed between \([0, 1]\), which is a general reverse factor, \(A j and B j are the lower and upper bounds of the \(j\)-th dynamic decision variable.

[0069] Use the greedy strategy to determine whether the generated neighborhood position is saved. The formula is as follows:

[0070]

[0071] where \(f(x)\) is the fitness value at position \(x\).

[0072] Step 6. Whether the value of the fitness of the optimal individual meets the preset accuracy or whether the maximum number of iterations is reached. If not, return to Step 4. Otherwise, retain the position of the optimal individual, and the parameters corresponding to this individual are the results of camera calibration.

[0073] To verify the feasibility and effectiveness of the proposed ORSMA algorithm in the field of camera calibration in machine vision, a set of acquired calibration board images is selected as the camera calibration material, and the camera that captured this set of images is calibrated. Figure 2 For some calibration picture examples, the results of Zhang Zhengyou's calibration method are shown in Table 1:

[0074] Table 1

[0075]

[0076] The calibration results of the camera internal parameters and distortion coefficients of the SOA algorithm, PSO algorithm, SMA algorithm, and ORSMA algorithm after 500 iterations are shown in Table 2:

[0077] Table 2

[0078]

[0079] The calibration results of the camera internal parameters and distortion coefficients of the SOA algorithm, PSO algorithm, SMA algorithm, and ORSMA algorithm after 1000 iterations are shown in Table 3:

[0080] Table 3

[0081]

[0082] The reprojection error after calculation optimization is shown in Table 4 as follows:

[0083] Table 4

[0084]

[0085] From Tables 1, 2, 3 and 4, it can be seen that the reprojection errors of the SOA algorithm and the PSO algorithm are both larger than those of the Zhang Zhengyou calibration method, while the reprojection errors of the SMA algorithm and the ORSMA algorithm are both smaller than those of the Zhang Zhengyou calibration method, and the reprojection error of the ORSMA algorithm is smaller and the calibration accuracy is higher.

[0086] Combined with Figure 3 , Figure 4 it can be seen that the SOA algorithm has the slowest convergence speed in the early stage and is very easy to fall into local optimum; the PSO algorithm has the fastest convergence speed and is relatively easy to fall into local optimum; the convergence curves of the SMA algorithm and the ORSMA algorithm cross frequently in the early stage, and neither of them falls into local optimum. However, with the increase of the number of iterations, the convergence speed and accuracy of the ORSMA algorithm are significantly higher than those of the SMA algorithm, and the ORSMA algorithm improves the overall performance of the algorithm. It can be seen from the table that the SOA algorithm cannot optimize the results of the Zhang calibration method, while the PSO algorithm, the SMA algorithm and the ORSMA algorithm can all achieve good optimization of the results of the Zhang calibration method, and their optimization effects increase in turn. The effect of the ORSMA algorithm is generally better than that of the PSO algorithm and the SMA algorithm. It can better improve the local convergence problem of the ordinary SMA algorithm and obtain better results.

Claims

1. An underwater camera calibration optimization method based on an improved slime mold algorithm, characterized in that, it includes the following steps: Step 1: Obtain calibration plate images at different angles, preprocess the images, and extract corner features; Step 2: Solve the initial values of the internal parameters and distortion coefficients, specifically: According to the camera imaging relationship and the internal relationship of the camera f x = f c1 / d x , f y = f c2 / d y , the initial values of f x , f y , u 0 , v 0 are obtained; where f c1 and f c2 are the camera focal lengths; d x and d y are the physical lengths of the pixels; u 0 and v 0 are the intersections of the camera optical axis and the image plane; The camera imaging relationship is: where z c is the object distance, x d , y d is the pixel coordinate system, x w , y w , z w is the world coordinate system, and R and T are the image rotation and translation matrices; Since this initial value is obtained under ideal conditions, it is necessary to introduce a distortion coefficient k 1 , k 2 , k 3 , p 1 , p 2 for correction; using the radial distortion mathematical model Among them, [x u , y u are the coordinates of any point p on the image normalization plane, and s is the distance between point p and the origin of the coordinate system; and the tangential distortion mathematical model as well as Combined obtain the initial values under distortion; Use the Zhang Zhengyou calibration method to calibrate the camera and obtain the parameter values before optimization; Step 3: Initialize the relevant parameters of the SMA algorithm, initialize the initial positions of the population, and initialize the relevant parameters of ONP and OBL; Step 4: Calculate the fitness of each slime mold individual, and select the current optimal fitness and its corresponding position; Step 5: Use the optimal neighborhood perturbation strategy to update the global position, use the reverse learning strategy to generate reverse solutions; and adopt a greedy mechanism to judge whether the generated neighborhood positions are saved; retain the solutions with higher fitness and update their positions; specifically: In the process of each iteration, the update formula for the slime mold position is as follows: Among them, is the updated position of the slime mold, is a parameter in the range [-a, a], is a parameter that linearly decreases from 1 to 0, where t represents the current iteration number, represents the position where the individual with the highest current fitness is located, represents the position of the slime mold, and represent the positions of two randomly selected individuals in the slime mold, represents the weight coefficient of the slime mold, p is a control parameter, LB and UB represent the upper and lower bounds of the search range, rand and r represent random values in [0, 1]; z is a user-defined parameter; The formula for parameter a is as follows: max_t represents the maximum number of iterations; The formula for the control parameter p is as follows: p = thnh|S(i)-DF|, i∈1,2,3,......,n (9) where S(i) represents the fitness, and DF represents the best fitness obtained in all iterations; Weight coefficient The formula is as follows: SmellIndex = sort(S) (11) where r is a random value within the interval [0,1], bF represents the optimal fitness obtained in the current iteration process, wF represents the worst fitness value obtained in the current iteration process, condition represents the individuals in the slime mold population with fitness values ranked in the top 1 / 2, other represents the remaining individuals in the slime mold population, and SmellIndex represents the sequence sorting of the fitness values; Use the optimal neighborhood perturbation strategy to update the global position, where the formula for randomly perturbing the optimal position is as follows: where X * (t) is the optimal position obtained from Equation (7), is the new position obtained by randomly perturbing X * (t), rand1 and rand2 are random numbers uniformly generated in the interval [0, 1]; X(t) is the newly generated position; Generate the reverse solution using the reverse learning strategy. If the individual X in the population i can be expressed by the following formula: X i = [X i,1 , X i,2 , … X i,j , X i,D (13) Then its reverse solution can be expressed as: X i ' = [X i,1 ', X i,2 ', …X i,j ', X i,D '] (14) In the formula, i = 1,2,3,…,n, j = 1,2,3,…,D, and the reverse solution and the forward solution also need to satisfy the following relationship: X i,j ' = k(A j + B j ) - X i,j (15) where k is a random number uniformly distributed between [0, 1], A j and B j are the lower and upper bounds of the j-th dynamic decision variable; Adopt a greedy strategy to judge whether the generated neighborhood positions are saved, and the formula is as follows: In the formula, f(x) is the fitness value at position x; Step 6: Judge whether the value of the optimal individual fitness meets the preset accuracy or whether the maximum number of iterations is reached. If not, return to Step 4. Otherwise, retain the position of the optimal individual, and the parameters corresponding to this individual are the results of the camera calibration.

2. The underwater camera calibration optimization method based on the improved slime mold algorithm according to claim 1, characterized in that, in Step 1, obtain the images of the calibration plate in different directions by the camera to be calibrated, and the number of images should be greater than 6; perform grayscale processing on the obtained images, and extract the corner points of the checkerboard in the images.

3. The underwater camera calibration optimization method based on the improved slime mold algorithm according to claim 1, characterized in that, in Step 3, the process of initializing the relevant parameters of the SMA algorithm, initializing the initial positions of the population, and initializing the relevant parameters of ONP and OBL is: It includes the population size n, the search space dimension d, the maximum number of iterations max_t, the upper and lower bounds of the initial values ub and lb, and randomly selects the initial positions of n individuals within the search range.

4. The underwater camera calibration optimization method based on the improved slime mold algorithm according to claim 1, characterized in that in step 4, the fitness of each slime mold individual is calculated, and the current optimal fitness and its corresponding position are selected specifically as follows: Establish the objective function of the camera calibration problem: where N is the number of corner points, p ij is the matching point of the image, and p is p ij corresponding reprojection point; Obtain the fitness function according to the objective function to obtain the fitness value of the optimal individual in the iteration, and define the fitness function as: where (x, y) is the actual pixel coordinate point obtained by the corner extraction algorithm; (u, v) is the pixel coordinate point calculated by the camera imaging relationship; m is the total number of corners.

5. The underwater camera calibration optimization method based on the improved slime mold algorithm according to claim 1, characterized in that z takes the value of 0.

03.

6. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that when the processor executes the program, it implements the underwater camera calibration optimization method based on the improved slime mold algorithm as described in any one of claims 1-5.

7. A computer-readable storage medium, on which a computer program is stored, characterized in that when the program is executed by the processor, it implements the underwater camera calibration optimization method based on the improved slime mold algorithm as described in any one of claims 1-5.

Images