Dynamic calibration method for underwater binocular vision system coordinated with vortex light ranging

Abstract

This invention relates to the field of underwater environmental perception technology and discloses a dynamic calibration method for an underwater binocular vision system using vortex laser ranging. The method comprises two steps: first, in clear, static water, the initial calibration parameters of the binocular vision system are solved using an underwater light refraction model; second, when applied in actual water, vortex laser ranging technology is used to acquire accurate distance information of the target area in real time, assisting the underwater binocular vision system in detecting changes in the water's refractive index and optimizing the calibration parameters, thereby achieving accurate positioning of three-dimensional spatial points in complex aquatic environments. The beneficial effects of this invention are: it overcomes the influence of changes in the aquatic environment on the accuracy of underwater three-dimensional information perception, achieves accurate calculation of the calibration parameters of the underwater binocular camera, and improves the positioning accuracy of three-dimensional spatial points in dynamically changing aquatic environments.

Description

Technical Field

[0001] This invention relates to the field of underwater environmental perception technology, and specifically to a dynamic calibration method for an underwater binocular vision system using vortex optical ranging. Background Technology

[0002] Underwater binocular vision sensing technology utilizes two optical cameras to acquire two images of the same underwater scene from different perspectives, reconstructing the scene's true three-dimensional spatial information and enabling the measurement of key data such as target distance and size. It is a crucial means of acquiring underwater three-dimensional information and achieving refined target detection. This technology boasts advantages such as high imaging resolution and rich detail information, and is frequently used in shallow water and close-range operational scenarios for functions such as terrain reconnaissance, engineering construction and maintenance, and resource exploration, possessing significant academic research and engineering application value.

[0003] Calibration of binocular vision system parameters is a crucial step in achieving underwater 3D information perception. Because underwater binocular cameras typically require a waterproof layer, light reflected from the underwater target surface is refracted through three different media—water, waterproofing material, and air—before being captured by the camera's sensor. Therefore, calibration of an underwater binocular vision system not only requires obtaining the camera's intrinsic and extrinsic parameters but also calculating refractive parameters such as the thickness of the waterproofing material and the plane normal vector based on the refractive indices of these three media. However, current underwater binocular vision system parameter calibration methods based on refraction models treat the water refractive index as a known constant, which is unsuitable for real-world applications where the refractive index of water bodies such as rivers and lakes dynamically changes, leading to a significant decrease in the accuracy of underwater 3D information perception. Therefore, it is urgent to research new parameter estimation methods for underwater binocular vision systems to improve the accuracy of underwater binocular vision 3D information perception, providing theoretical basis and technical support for underwater engineering inspection, underwater resource exploration, and other fields. This is of great significance for promoting the development of underwater exploration technology in my country and improving the country's technological level and competitiveness in the marine field. Summary of the Invention

[0004] To address the problem that existing underwater binocular camera calibration parameters cannot adapt to changes in refractive index in water, leading to a decrease in the accuracy of 3D information perception, this invention provides a dynamic calibration method for an underwater binocular vision system using vortex optical ranging. This method can overcome the impact of changes in the water environment on the accuracy of underwater 3D information perception, achieve accurate calculation of underwater binocular camera calibration parameters, and improve the positioning accuracy of 3D spatial points in dynamically changing water environments.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A dynamic calibration method for an underwater binocular vision system with vortex optical ranging coordination includes the following steps:

[0007] Step S1: Solve for the basic parameters of the binocular camera itself in air;

[0008] Step S2: In clear, static water, solve for the initial calibration parameters of the binocular vision system using an underwater light refraction model;

[0009] Step S3: In actual water bodies, vortex laser ranging is used to obtain the distance information of the target area in real time, which assists the underwater binocular vision system in detecting changes in the water refractive index of the detection area and optimizing the calibration parameters of the binocular vision system, thereby obtaining accurate calibration parameters in real time in a dynamically changing water environment.

[0010] Furthermore, prior to step S1, an underwater vortex laser ranging and binocular camera synchronous information acquisition device is constructed. This acquisition device includes a vortex laser ranging module, a binocular camera, a processor, and a waterproof casing. The vortex laser ranging module includes a vortex laser modulation system and a receiving detection system. The vortex laser modulation system includes a laser and an electro-optic modulator. The electro-optic modulator includes a first shaping optical system and a first spiral phase plate arranged sequentially. The laser generates Gaussian light and emits it to the first shaping optical system, where it is then modulated into a vortex laser by the first spiral phase plate and emitted towards the underwater target. The vortex laser is generated by water... The reflected beam is sent to a receiving and detection system, which includes a second shaping optical system, a second spiral phase plate, a light-shielding plate, a demodulation system, a noise-filtering aperture, a photoelectric detection system, and a phase difference ranging system. The shaping optical system shapes the reflected beam, and after passing through the second spiral phase plate, the signal on the vortex ring is demodulated by the demodulation system. The noise-filtering aperture filters out background noise from the signal. The filtered signal finally reaches the photoelectric detection system and is identified and detected by the system, then transmitted to the phase difference ranging system. The phase difference ranging system is used to analyze the phase difference between the echo signal and the transmitted signal to obtain the distance to the underwater target.

[0011] Further, the specific method of step S1 is as follows: The underwater vortex laser ranging and binocular camera synchronous information acquisition device is placed in the air, and a checkerboard calibration board is placed in front of it. Images of N are taken from different angles and distances. a Using the binocular images from the calibration board, and employing Zhang's calibration method, the basic parameters of the binocular camera itself are determined. These basic parameters include the intrinsic parameter A of the left eye camera. l Right eye camera internal reference A r The rotation matrix R between the left and right cameras rl With translation vector T rl .

[0012] Further, the specific method of step S2 is as follows: The underwater vortex laser ranging and binocular camera synchronous information acquisition device is placed in clear, static water; a checkerboard calibration board is placed in front of the device; and N images are taken from different angles and distances. w Using the binocular images from the calibration board, the initial refractive parameters of the underwater binocular vision system were solved using Zhang's calibration method and a ray refraction model. These initial refractive parameters include the vertical distance d between the optical center of the left eye camera and the upper interface of the waterproof glass. l The vertical distance d between the optical center of the right eye camera and the upper interface of the waterproof glass. r The thickness h of the glass and the normal vector n of the glass plane.

[0013] Further, step S2 includes the following steps:

[0014] Step S21: Establish an underwater binocular vision light refraction model;

[0015] Step S22: Establish a refraction parameter optimization model;

[0016] Step S23: Based on steps S21 and S22, a method for solving the refraction parameters of an underwater binocular camera is proposed, the nonlinear refraction function of the underwater binocular vision system is derived, and the initial refraction parameters are solved.

[0017] Step S24: Use the basic parameters obtained in step S1 and the initial refraction parameters obtained in step S23 as the initial calibration parameters of the binocular vision system.

[0018] Furthermore, the method for establishing the light refraction model in step S21 is as follows:

[0019] Assuming the left-eye camera coordinate system coincides with the world coordinate system, π1 and π2 are the waterproof glass planes, i.e., the refraction planes, n(n x ,n y ,n z ) is its normal vector, h is the thickness of the waterproof glass, and d is the normal vector. l n is the perpendicular distance between the optical center of the left eye camera and π1; a n g n w Let be the relative refractive indices of air, glass, and water, respectively; for an underwater target P, the reflected light from the object's surface, after two refractions, will be imaged at points p1, p2, and p3, respectively. l (x l ,y l ), p r (x l ,y l ); P il P ol P represents the intersection points of the left refracted light path with the π1 and π2 planes, respectively. ir Por These are the intersection points of the right refracted ray path with π1 and π2, respectively; the left refracted ray path originates from... The composition, and their corresponding unit vectors are respectively These represent the incident light rays from air to the waterproof glass on the left and their angles of incidence, respectively. These represent the refracted light rays produced when the left-path light enters the waterproof glass and their refraction angles, respectively. Let represent the refracted ray from the waterproof glass into the water and its refraction angle, respectively; according to the coplanar constraint condition of the refracted rays, the rays passing through the refracted surface satisfy the following formula.

[0020]

[0021] Based on the above formula, the intersection point P of the ray with π1 and π2 is obtained. il P ol The coordinates;

[0022]

[0023] As above, the refracted light path of underwater target P in the right eye camera. The composition, and their corresponding unit vectors are respectively The intersection points of the right-refracted ray path with the π1 plane and the π2 plane are P and P, respectively. ir P or ;

[0024] Using the rotation matrix R between the binocular cameras rl With translation vector T rl , will P ir P or Transform to the world coordinate system to obtain a new

[0025]

[0026] in, These are the unit vectors of the right refracted ray corresponding to the underwater target P in the right eye camera in the world coordinate system; Let P be the intersection point of the right-refracted ray and π1 and π2. ir P or The coordinates in the world coordinate system; then the refracted light rays from the left and right cameras in the water are...

[0027] Since the vectors of the two refracted rays passing through the optical center of the camera are coplanar with the intersection point, there exists a point in space that is simultaneously coplanar with the intersection point. Vertical vector And vector The intersection points with L1 and L2 are M1 and M2 respectively. The coordinates of M1 and M2 are calculated using the following formula;

[0028]

[0029] The three-dimensional coordinates of the underwater target P can be determined by minimizing the distance between M1 and M2, that is:

[0030]

[0031] Furthermore, the method for establishing the refraction parameter optimization model in step S22 is as follows:

[0032] Assuming the size and relative positions of the chessboard squares can be determined in advance, and that the chessboard has H×W' interior corner points, with each square having a size of w', C i,j Let represent the three-dimensional coordinates of the interior corner point of the i-th row and j-th column of the chessboard grid, i = 1, 2, 3, ..., W', j = 1, 2, 3, ..., H; if the calibration parameters are accurate, the three-dimensional spatial coordinates determined from the two-dimensional image coordinates of the chessboard grid corner points should satisfy the following conditions:

[0033] Condition 1: The distance between adjacent corner points is the same, that is, the distance between adjacent corner points is w';

[0034] Condition 2: Corner points in the same direction are collinear; therefore, a certain corner point C... i,j Its adjacent corner point C i+1,j C i,j+1 The vector formed They are perpendicular to each other, and their vectors are perpendicular. The angle between them should be close to

[0035] Condition 3: The lines where the corner points are located in the same direction are parallel to each other, that is, the vector formed by any corner point to the adjacent corner point to the right should be parallel to the horizontal direction of the chessboard; at the same time, the vector formed by the corner point to the adjacent corner point below should be parallel to the vertical direction of the chessboard.

[0036] Using the prior information that adjacent corner points are equidistant, corner points in the same direction are collinear, and the lines containing corner points in the same direction are parallel, the following three directional functions are established respectively:

[0037]

[0038] Wherein, F1 is the distance difference between adjacent corner points, used to measure the difference between the average distance between the reconstructed adjacent interior corner points and the true distance w; F2 is the position difference between adjacent corner points, used to represent the error between the angle between the reconstructed interior corner point forming vector and the true angle; F3 is the angle difference in the parallel direction, used to calculate the difference in angle between the straight lines formed by the corner points in the horizontal or vertical directions.

[0039] Establish the overall objective function F GF G =F1+F2+F3, by minimizing F G Able to obtain refraction parameters (n,d) l ,d r ,h), where n=(n x ,n y ,n z Therefore, there are 6 refraction parameters. To improve computational efficiency, the objective function is further simplified by the constraint relationships between the parameters.

[0040] The constraint relationship is as follows: and

[0041] The simplified objective function has four parameters to be determined, including (n x ,n y ,d l h); Set the solution range n x ∈[-0.3,0.3], n y ∈[-0.3,0.3], and assume d l The manual measurement value of h is d′ l If h′, then set d l The solution range for h is d l ∈[0.5·d′ l ,1.5·d′ l ], h∈[0.5·h′,1.5·h′ l ].

[0042] Furthermore, the method for solving the refraction parameters of the underwater binocular camera in step S23 is as follows:

[0043] Step S231: Determine the race size as N nsga The t-th individual in the population is initialized with X using a continuous real number encoding. t ={x t1 ,x t2 ,x t3 ,x t4},t∈[1,N nsga ], generating the first generation population P0, (x t1 ,x t2 ,x t3 ,x t4 ) respectively correspond to (n x ,n y ,d l ,h);

[0044] Step S232: Perform crossover and mutation operations on the individuals in the initial population to generate a population of size N. nsgaGiven the offspring population Q0, calculate the objective function value of the corresponding parameters of the current population, and use a non-dominated sorting method to divide the individuals in population P0∪Q0 into at least two non-dominated levels (G1, G2, ..., G...) based on the dominance relationship according to the cost value. l ), and G1, G2, ..., G l Priority decreases sequentially;

[0045] Step S232: Starting from level G1, select non-dominated sets according to priority and retain them in the next generation until the number of the next generation is equal to N. nsga Or the first time exceeding N nsga And assume that the current non-dominated level is G. l Then G l+1 All individuals in the layer will be eliminated, l = 1, 2, ..., N so N so Let G1 be the total number of the next generation; assuming G1 ∪ ... G l-1 The number of N l-1 And N l-1 <N nsga Then, the reference point method is further used in G. l Further select N in the hierarchy nsga -N l-1 Individuals, forming a new scale of N. nsga The evolutionary population;

[0046] Step S234: Iterate through steps S232 and S233 until the maximum number of iterations is reached, and obtain the optimal solution set of refraction parameters. Select the individual with the smallest variance of the cost values ​​of the three corresponding objectives as the final refraction parameters of the underwater camera.

[0047] Furthermore, the specific steps of step S3 are as follows:

[0048] Step S31: Use the vortex laser ranging module to emit a vortex laser, and obtain a high-precision target distance measurement result D based on the phase difference between the emitted and returned vortex laser signals;

[0049] Step S32: Extract the center matching point pairs of the vortex laser spot in the binocular image and analyze the left eye image Im. l With right eye image Im r Based on the grayscale fluctuation and the imaging color characteristics of the beam, the bright vortex laser spot region in the binocular image is detected and extracted, and the center point of the laser spot in the left and right viewpoints is located as the matching point pair.

[0050] Step S33: Import the matching point pairs of the vortex laser in the binocular image, the initial refraction parameters of the binocular camera obtained in the static water body, and the target distance information obtained by the vortex laser in step S31 into step S21 to establish an underwater binocular vision light refraction model, re-derive the refraction parameters of the current water body, dynamically invert the water refractive index of the current measurement target area, and realize the dynamic optimization of the calibration parameters of the underwater binocular vision system.

[0051] Furthermore, the method for locating the matching point pairs in step S32 is as follows:

[0052] Left eye image Im l With right eye image Im r Convert each image to a weighted grayscale image Im lg Im rg The calculation method is as follows, where Im l (r), Im l (g), Im l (b) Im respectively l The red image channel, blue-green image channel, and green channel, where Im r (r), Im r (g), Im r (b) Im respectively r The red image channel, blue-green image channel, and green channel;

[0053]

[0054] Set threshold α p α p ∈(0,255), using this threshold for Im lg Im rg Binarization is performed to obtain the binarized image ROI. l With ROI r The images are the initial regions of the vortex laser spots in the left and right eye images, respectively. Then, through morphological processing, a new binarized image ROI containing the laser spot regions is obtained. l ′ and ROI r ′;

[0055] Utilizing a new binarized image ROI l ′ and ROI r ′, segmenting the left eye image Im l With right eye image Im r Laser spot image Im l ′ and Im r ′;Extract left eye image Im l The center point P of the laser spot lcl respectively for Im l ′ and Imr ′Perform the Census transform and obtain C sl With C sr , and calculate the Hamming distance between the Census transform bit strings of each point in the segmented light spot area, Im r ′The point in P lcl The point P with the closest distance to the corresponding Census transform bit string lcr , which is P​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​This is a flowchart of the dynamic calibration method in an embodiment of the present invention;

[0062] Figure 2 This is a schematic diagram of the underwater vortex laser ranging and binocular camera synchronous information acquisition device in an embodiment of the present invention;

[0063] Figure 3 This is a schematic diagram of the vortex laser ranging module in an embodiment of the present invention;

[0064] Figure 4 This is a schematic diagram of underwater spatial point positioning based on a light refraction model in an embodiment of the present invention;

[0065] Figure 5 This is a schematic diagram of the method for solving the refractive parameters of an underwater binocular camera in an embodiment of the present invention;

[0066] Figure 6 This is a schematic diagram of the dynamic optimization method for water refractive index parameters in conjunction with vortex laser ranging in an embodiment of the present invention;

[0067] Figure 7 This is a schematic diagram of the method for matching the center point of a vortex laser spot in the left and right eye images in an embodiment of the present invention.

[0068] Meaning of the reference numerals in the diagram:

[0069] 1-Left eye camera; 2-Right eye camera; 3-Vortex laser emitter; 4-Vortex laser ranging module; 5-Waterproof housing. Detailed Implementation

[0070] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] like Figure 2 As shown, this invention proposes a set of underwater vortex laser ranging and binocular camera synchronous information acquisition devices. The device includes a vortex laser ranging module, a binocular camera, a processor, and a waterproof housing. The structure of the vortex laser ranging module is as follows: Figure 3 As shown, it includes a vortex laser modulation system and a receiving and detection system.

[0072] (1) The vortex laser modulation system includes a laser and an electro-optic modulator. The electro-optic modulator includes a first shaping optical system and a first spiral phase plate arranged in sequence. The laser generates blue-green Gaussian light with a wavelength of 532nm that is visible to the human eye and emits the Gaussian light to the first shaping optical system. The first spiral phase plate then modulates the light into a vortex laser and emits it to an underwater target.

[0073] (2) The vortex beam is reflected from the underwater target to the receiving and detection system. The receiving and detection system includes a second shaping optical system, a second spiral phase plate, a light shield, a demodulation system, a noise filter, a photoelectric detection system, and a phase difference ranging system. First, the shaping optical system shapes the reflected beam. After passing through the second spiral phase plate of the same specifications as the vortex laser modulation system, the black light shield blocks part of the center of the vortex to reduce the reception of scattered light. The demodulation system demodulates only the signal on the vortex ring. Finally, the noise filter filters the background noise in the signal. The noise-filtered signal finally reaches the photoelectric detection system and is identified and detected by the system. It is then transmitted to the phase difference ranging system to analyze the phase difference between the echo signal and the transmitted signal and accurately calculate the distance to the underwater target.

[0074] Figure 3 In the diagram, P represents the underwater target, T1 represents the vortex laser emitted by the transmitter, T2 represents the reflected light of the vortex laser reflected by the underwater target, and D represents the distance between the underwater target and the underwater vortex laser ranging and binocular camera synchronous information acquisition device.

[0075] like Figure 1 As shown, this invention proposes a dynamic calibration method for an underwater binocular vision system based on vortex optical ranging, comprising the following steps:

[0076] Step 1: Determine the basic parameters of the binocular camera itself in air. Place the underwater vortex laser ranging and binocular camera synchronization information acquisition device in air, and place a checkerboard calibration board in front of it. Take pictures of N from different angles and distances. a (N a ≥15) sets of calibration board binocular images, using Zhang's calibration method, to solve for the basic parameters of the binocular camera itself, including the intrinsic parameter A of the left eye camera. l Right eye camera internal reference A r The rotation matrix R between the left and right cameras rl With translation vector T rl .

[0077] Step 2: Solve for the initial refraction parameters of the underwater binocular vision system in water. Place the underwater vortex laser ranging and binocular camera synchronous information acquisition device in clear, static water. Place a checkerboard calibration board in front of the device and take pictures of N from different angles and distances. w (Nw ≥15) sets of calibration plate binocular images were used. Zhang's calibration method was employed, and the initial refractive parameters of the underwater binocular vision system were solved using a ray refraction model. These parameters included the vertical distance d between the optical center of the left eye camera and the upper interface of the waterproof outer shell (i.e., waterproof glass). l The vertical distance d between the optical center of the right eye camera and the upper interface of the waterproof housing (i.e., waterproof glass). r The thickness h of the glass and the normal vector n of the glass plane.

[0078] Step 2.1: Establish an underwater binocular vision light refraction model (i.e., an underwater binocular vision three-dimensional spatial point localization model). For example... Figure 4 As shown, O cl X cl Y cl These represent the origin, x-coordinate, and y-coordinate of the left-eye camera in the coordinate system of the light refraction model, respectively. W X W Y W Let O be the origin, x-coordinate, and y-coordinate of the left-eye camera in the world coordinate system, respectively. Assume the left-eye camera coordinate system coincides with the world coordinate system. cr X cr Y cr Here, π1 and π2 are the origin, x-coordinate, and y-coordinate of the right eye camera in the coordinate system of the light refraction model, respectively; n(n) represents the waterproof glass plane (refractive plane). x ,n y ,n z ) is its normal vector, h is the thickness of the waterproof glass, and d is the normal vector. l n is the perpendicular distance between the optical center of the left eye camera and π1. a n g n w Let be the relative refractive indices of air, glass, and water, respectively. For an underwater target P, the reflected light from the object's surface undergoes two refractions, resulting in image points p1, p2, and p3 respectively. l (x l ,y l ), p r (x l ,y l P il P ol P represents the intersection points of the left refracted light path with the π1 and π2 planes, respectively. ir P or These are the intersection points of the right refracted ray path with π1 and π2, respectively. The left refracted ray path originates from... The composition, and their corresponding unit vectors are respectively These represent the incident light rays from air to the waterproof glass on the left and their angles of incidence, respectively. These represent the refracted light rays produced when the left-path light enters the waterproof glass and their refraction angles (i.e., the first refraction angle of the left-path light ray); Let represent the refracted ray of the left-path ray as it exits the waterproof glass and enters the water, and its refraction angle (i.e., the second refraction angle of the left-path ray). According to the coplanar constraint condition of the refracted rays, the rays passing through the refracting surface satisfy the following equation.

[0079] in,

[0080] Based on the above formula, the intersection point P of the light ray with π1 and π2 can be derived. il P ol The coordinates.

[0081]

[0082] As above, the refracted light path of underwater target P in the right eye camera. The composition, and their corresponding unit vectors are respectively The intersection points of the right-refracted ray path with the π1 plane and the π2 plane are P and P, respectively. ir P or .

[0083] Using the rotation matrix R between the binocular cameras rl With translation vector T rl P can be ir P or Transform to the world coordinate system to obtain a new

[0084]

[0085] in, These are the unit vectors of the right refracted ray corresponding to the underwater target P in the right eye camera in the world coordinate system; Let P be the intersection point of the right-refracted ray and π1 and π2. ir P or The coordinates in the world coordinate system. Then the refracted light rays from the left and right cameras in the water are L. l (P ol ,r l 2 ),

[0086] Since the vectors of the two refracted rays passing through the optical center of the camera are coplanar with the intersection point, there exists a point in space that is simultaneously coplanar with the intersection point. Vertical vector And vector The intersection points with L1 and L2 are M1 and M2, respectively. The coordinates of M1 and M2 can be calculated using the following formula.

[0087]

[0088] The three-dimensional coordinates of the underwater target P can be determined by minimizing the distance between M1 and M2, that is:

[0089]

[0090] Step 2.2: Establish a refraction parameter optimization model. As the above analysis shows, two sets of parameters need to be determined for binocular cameras to perform 3D information perception underwater. One set consists of basic parameters, including the intrinsic parameter A of the left eye camera. l Right eye camera internal reference A r The rotation matrix R between the left and right cameras rl With translation vector T rl The other set consists of refraction parameters, including the vertical distance d between the optical center of the left eye camera and the upper interface of the waterproof glass. l The vertical distance d between the optical center of the right eye camera and the upper interface of the waterproof glass. r The parameters include the glass thickness h and the normal vector n of the glass plane. There exists a nonlinear relationship F between these parameters. NL Therefore, for an underwater target P, if its imaging points in the left and right cameras are known to be p1 and p2 respectively... l (x l ,y l ), p r (x l ,y l If the coordinates of its object point in three-dimensional space are P = F, then the coordinates of its object point in three-dimensional space can be expressed as P = F. NL (p l ,p r ,n,d l ,d r ,h,n g ,n w ).

[0091] like Figure 4 As shown, assuming the size and relative positions of the chessboard squares can be determined in advance, and that the chessboard has H×W' interior corner points, with each square having a size of w', C i,j Let W' represent the three-dimensional coordinates of the inner corner point of the i-th row and j-th column of the chessboard grid, where i = 1, 2, 3, ..., W', j = 1, 2, 3, ..., H. If the calibration parameters are accurate, the three-dimensional spatial coordinates determined from the two-dimensional image coordinates of the chessboard grid corner points should satisfy the following conditions: (1) The distance between adjacent corner points is consistent, that is, the distance between adjacent corner points is w'. (2) Corner points in the same direction are collinear, therefore, a certain corner point C i,j Its adjacent corner point C i+1,j C i,j+1 The vector formed They are perpendicular to each other, and their vectors are perpendicular. The angle between them should be close to (3) The lines where corner points are located in the same direction are parallel to each other, that is, the vector formed by any corner point to the adjacent corner point on the right should be parallel to the horizontal direction of the chessboard. At the same time, the vector formed by the corner point to the adjacent corner point below should be parallel to the vertical direction of the chessboard.

[0092] Using the prior information that adjacent corner points are equidistant, corner points in the same direction are collinear, and the lines on which corner points in the same direction are located are parallel, the following three directional functions can be established respectively.

[0093]

[0094] Wherein, F1 is the distance difference between adjacent corner points, used to measure the difference between the average distance between the reconstructed adjacent inner corner points and the true distance w; F2 is the position difference between adjacent corner points, used to represent the error between the angle formed by the reconstructed inner corner point vectors and the true angle; F3 is the angle difference in the parallel direction, used to calculate the difference in angle between the straight lines formed by the corner points in the horizontal or vertical directions.

[0095] It should be noted that: nonlinear relationship F NL This refers to a non-linear relationship between the three-dimensional coordinates and various parameters, but this non-linear function is difficult to solve, so it is converted into the F below. G To estimate, F G Let F1, F2, and F3 be the overall objective function in the parameter calibration process.

[0096] Establish the overall objective function F G F G =F1+F2+F3, thus transforming the binocular camera refraction parameter calibration into a multi-objective optimization problem, by minimizing F G The refraction parameters (n, d) can then be obtained. l ,d r ,h), where n=(n x ,n y ,n z From this, we can see that there are 6 refraction parameters. To improve computational efficiency, the objective function can be further simplified through the constraints between the parameters. ① Since the normal vector is a unit vector, there are constraints: ②Based on the pose geometry of the binocular cameras, we can see that: The simplified objective function has fewer than four parameters to be determined, containing only (n) x ,n y ,d l In practice, the camera's waterproof glass and the camera's mounting angle will be as parallel as possible; therefore, the solution range n is set. x∈[-0.3,0.3], n y ∈[-0.3,0.3], and assume d l The manual measurement value of h is d′ l If h′, then set d l The solution range for h is d l ∈[0.5·d′ l ,1.5·d′ l ], h∈[0.5·h′,1.5·h′ l ].

[0097] Step 2.3: A method for solving the refraction parameters of an underwater binocular camera is proposed, the nonlinear refraction function of the underwater binocular vision system is derived, and the initial calibration parameters are solved. NSGA-III (a non-dominated sorting method based on reference points) exhibits good performance in multi-objective optimization problems and is one of the most popular multi-objective genetic optimization algorithms. Therefore, as... Figure 5 As shown, this invention utilizes the NSGA-III optimization algorithm to minimize multiple constraints to obtain refraction parameters. Finally, the obtained basic parameters and refraction parameters are used as the initial calibration parameters for the binocular vision system.

[0098] (1) First, determine the race size as N. nsga The t-th individual in the population is initialized with X using a continuous real number encoding. t ={x t1 ,x t2 ,x t3 ,x t4},t∈[1,N nsga ], generating the first generation population P0, (x t1 ,x t2 ,x t3 ,x t4 ) respectively correspond to (n x ,n y ,d l ,h).

[0099] (2) Perform crossover, mutation, and other operations on the individuals in the initial population to generate a population of size N. nsga Given the offspring population Q0, calculate the objective function value of the corresponding parameters of the current population, and use a non-dominated ranking method to divide the individuals in the population P0∪Q0 into multiple non-dominated levels (G1, G2, ..., G...) according to the dominance relationship based on the cost value. l ), and G1, G2, ..., G l The priority decreases sequentially.

[0100] (3) Starting from level G1, select non-dominated sets according to priority and retain them in the next generation until the number of the next generation is equal to N. nsgaOr the first time exceeding N nsga And assume that the current non-dominated level is G. l Then G l+1 All individuals in the layer will be eliminated, l = 1, 2, ..., N so N so This represents the total number of the next generation. In most cases, G... l Only a portion of the individuals in the group can be preserved in the next generation, assuming G1∪...G l-1 The number of N l-1 And N l-1 <N nsga Then, the reference point method is further used in G. l Further select N in the hierarchy nsga -N l-1 Individuals, forming a new scale of N. nsga The evolutionary population.

[0101] (4) Iterate through (2)-(3) steps until the maximum number of iterations is reached, and obtain the optimal solution set of the refraction parameters. Select the individual with the smallest variance of the cost values ​​of the three corresponding targets as the final refraction parameters of the underwater camera.

[0102] Step 2.4: Finally, the obtained basic parameters and refraction parameters are used as the initial calibration parameters of the binocular vision system.

[0103] Step 3: Propose a dynamic optimization method for water refractive index parameters in conjunction with vortex laser ranging. In the underwater binocular vision 3D spatial point positioning model established above based on light refraction and propagation path analysis, the intrinsic parameters of the binocular camera, the normal vector of the glass plane, and the refractive index of the waterproof material are all known parameters. Due to the continuous movement of the non-uniformly distributed medium in the actual water environment, the water refractive index changes continuously, leading to inaccurate pre-acquired calibration parameters and a decrease in the accuracy of underwater 3D spatial point positioning. In order to obtain accurate calibration parameters in real time in a dynamically changing water environment and achieve precise perception of 3D information, such as... Figure 6 As shown, this invention proposes to dynamically adjust and optimize the refractive index of water in real time by using precise distance measurements obtained from a single-point vortex laser, based on initial parameters.

[0104] Step 3.1: Use the vortex laser ranging module to emit a vortex laser, and obtain a high-precision target distance measurement result D based on the phase difference between the emitted and returned vortex laser signals.

[0105] Step 3.2: Extract the center matching point pairs of the vortex laser spot in the binocular image. Analyze the left eye image Im. l With right eye image Im rBased on the grayscale fluctuations and the imaging color characteristics of the beam, the bright vortex laser spot regions in the binocular images are detected and extracted, and the center points of the laser spots in the left and right viewpoints are located as matching point pairs.

[0106] (1) Because the vortex laser spot has high brightness, therefore, Figure 7 As shown, first, the left eye image Im... l With right eye image Im r Convert each image to a weighted grayscale image Im lg Im rg The calculation method is as follows, where Im l (r), Im l (g), Im l (b) Im respectively l The red image channel, blue-green image channel, and green channel, where Im r (r), Im r (g), Im r (b) Im respectively r The red image channel, blue-green image channel, and green channel.

[0107]

[0108] Weighted grayscale image Im lg Im rg The fluctuation chart is as follows Figure 7 As shown in the second row, the fluctuation in the area where the laser spot is located is significantly higher than that in other areas, therefore a threshold α is set. p (α p ∈(0,255)), using this threshold for Im lg Im rg Binarization is performed to obtain the binarized image ROI. l With ROI r The images show the initial regions of the vortex laser spot in the left and right eye images, respectively. Morphological processing is then used to remove narrow discontinuities and elongated gaps in the binarized images, and small voids are filled in, resulting in a new binarized image ROI containing the approximate region of the laser spot. l ′ and ROI r ′.

[0109] (2) Utilizing the new binarized image ROI l ′ and ROI r ′, segmenting the left eye image Im l With right eye image Im r Laser spot image Im l ′ and Im r Extract the left eye image Im. l The center point P of the laser spotlcl , perform Census transform on Im l ' and Im r ' respectively and obtain C sl and C sr , and calculate the Hamming distance between the Census transform bit strings of each point in the segmented light spot area. The point P r ' that is closest to the Census transform bit string corresponding to the P lcl point in Im lcr [[ID=!5]] is the matching point of P lcl .

[0110] Step 3.3: Import the matching point pairs of the vortex laser in the binocular images, the initial refraction parameters of the binocular cameras pre-acquired in the static water body, and the target distance information obtained by the vortex laser in Step 3.1 into the underwater binocular vision ray refraction model established in Step 2.1, re-derive the refraction parameters of the current water body, dynamically invert the refractive index of the current measured target area of the water body, and realize the dynamic optimization of the calibration parameters of the underwater binocular vision system.

[0111] This method substitutes the high-precision ranging data D obtained by the vortex laser as a known parameter into the underwater binocular vision three-dimensional space point positioning model, enabling the above-mentioned underwater three-dimensional space point positioning method based on the refraction model to adapt to the refractive index parameter fluctuations in the flowing water body and improving the underwater three-dimensional information perception accuracy.

[0112] (1) Set the parameter n w (i.e., the refractive index of the water body) in Step 2.1 as the variable to be optimized. Assume that are the minimum and maximum values of n w [[ID=!3]] respectively. Within the range of , take values from low to high with a step size of st n . is the q-th value, and the int(·) symbol represents rounding down.

[0113] (2) Substitute the matching point pairs of the laser spot center points (P lcl , P lcr ) and into Step 2.1 in sequence to recalculate the three-dimensional coordinates P cq of the center point of the water vortex laser spot, and calculate the depth error value cq between the coordinate of P on the Z-axis and the ranging data D. After sequentially traversing and calculating the depth error values, take the value of the variable to be optimized corresponding to the case with the minimum depth error value as the refractive index of the water body in the current measured target area to complete the update of the refractive index of the water body in different underwater environments.

[0114] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0115] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A dynamic calibration method for an underwater binocular vision system with vortex optical ranging coordination, characterized in that, Includes the following steps: Step S1: Solve for the basic parameters of the binocular camera itself in air; Step S2: In clear, static water, solve for the initial calibration parameters of the binocular vision system using an underwater light refraction model; Step S3: In actual water bodies, vortex laser ranging is used to obtain the distance information of the target area in real time, which assists the underwater binocular vision system in detecting changes in the refractive index of the water in the detection area and optimizing the calibration parameters of the binocular vision system, so as to obtain accurate calibration parameters in real time in the dynamically changing water environment. Step S2 includes the following steps: Step S21: Establish an underwater binocular vision light refraction model; Step S22: Establish a refraction parameter optimization model; Step S23: Based on steps S21 and S22, a method for solving the refraction parameters of an underwater binocular camera is proposed, the nonlinear refraction function of the underwater binocular vision system is derived, and the initial refraction parameters are solved. Step S24: Use the basic parameters obtained in step S1 and the initial refraction parameters obtained in step S23 as the initial calibration parameters of the binocular vision system; The specific steps of step S3 are as follows: Step S31: Employ the vortex laser ranging module to emit a vortex laser, and obtain a high-precision target distance measurement result based on the phase difference between the emitted and returned vortex laser signals. ; Step S32: Extract the center matching point pairs of the vortex laser spot in the binocular image and analyze the left eye image. With right eye image Based on the grayscale fluctuation and the imaging color characteristics of the beam, the bright vortex laser spot region in the binocular image is detected and extracted, and the center point of the laser spot in the left and right viewpoints is located as the matching point pair. Step S33: Import the matching point pairs of the vortex laser in the binocular image, the initial refraction parameters of the binocular camera obtained in the static water body in advance, and the target distance information obtained by the vortex laser in step S31 into step S21 to establish an underwater binocular vision light refraction model, re-derive the refraction parameters of the current water body, dynamically invert the water refractive index of the current measurement target area, and realize the dynamic optimization of the calibration parameters of the underwater binocular vision system. The method for locating the matching point pairs in step S32 is as follows: Left eye image With right eye image Convert to weighted grayscale images respectively , The calculation method is as follows, where , , They are respectively The red image channel, blue-green image channel, and green channel, among which , , They are respectively The red image channel, blue-green image channel, and green channel; Set threshold , Using this threshold , Binarization is performed to obtain a binarized image. and The images show the initial regions of the vortex laser spots in the left and right eye images, respectively. Morphological processing is then used to obtain new binarized images containing the laser spot regions. and ; Using a new binarized image and Segment the left eye image With right eye image Laser spot image and Extract the left eye image. Center point of laser spot , respectively and Perform Census transformation and obtain and And calculate the Hamming distance between the Census transform bit strings between each point in the segmented spot region. Zhongyu The point corresponds to the closest point in the Census transform bit string. That is matching points; The method for calculating the water refractive index of the current target area in step S33 is as follows: The refractive index of the water body in step S21 Let this be the variable to be optimized, and assume... , They are respectively The minimum and maximum values, in Within the range, from low to high with step size as To retrieve values, For the first One value, , The symbol indicates rounding down; Match the center point of the laser spot with the point pair ( , )and Substitute these values ​​sequentially into step S21 to recalculate the three-dimensional coordinates of the center point of the water vortex laser spot. ,calculate Coordinates and distance measurement data on the Z-axis Depth error value between After sequentially calculating the depth error values, the value of the variable to be optimized corresponding to the minimum depth error value is taken as the water refractive index of the current measurement target area. This allows for the updating of the refractive index of water in different underwater environments.

2. The dynamic calibration method for an underwater binocular vision system with vortex optical ranging coordination according to claim 1, characterized in that, The specific method of step S1 is as follows: place the underwater vortex laser ranging and binocular camera synchronous information acquisition device in the air, place a checkerboard calibration board in front of it, and take pictures from different angles and distances. Using the binocular images from the calibration board, and employing Zhang's calibration method, the basic parameters of the binocular camera itself are determined. These basic parameters include the intrinsic parameters of the left eye camera. Right eye camera internal reference Rotation matrix between left and right cameras With translation vector .

3. The dynamic calibration method for an underwater binocular vision system with vortex optical ranging coordination according to claim 1, characterized in that, The specific method of step S2 is as follows: place the underwater vortex laser ranging and binocular camera synchronous information acquisition device in clear, static water, place a checkerboard calibration board in front of the device, and take pictures from different angles and distances. Using the binocular images from the calibration board, the initial refractive parameters of the underwater binocular vision system were solved using Zhang's calibration method and a ray refraction model. These initial refractive parameters include the vertical distance between the optical center of the left eye camera and the upper interface of the waterproof glass. The vertical distance between the optical center of the right eye camera and the upper interface of the waterproof glass. Glass thickness The normal vector of the glass plane .

4. The dynamic calibration method for an underwater binocular vision system with vortex optical ranging coordination according to claim 3, characterized in that, The method for establishing the light refraction model in step S21 is as follows: Assuming the left-eye camera coordinate system coincides with the world coordinate system, , The surface of the waterproof glass is the refractive plane. Its normal vector, For the thickness of waterproof glass, For the left eye camera optical center and The vertical distance; , , These are the relative refractive indices of air, glass, and water, respectively; for underwater targets... The reflected light from the object's surface, after being refracted twice, forms images at the following points: , ; , The left refraction path and flat, Intersection of planes , The right refraction path and , The intersection; the refracted light rays on the left are from , , Composed of, and their corresponding unit vectors are respectively , , ; , These represent the incident light rays from air to the waterproof glass on the left and their angles of incidence, respectively. , These represent the refracted light rays produced when the left-path light enters the waterproof glass and their refraction angles, respectively. , Let represent the refracted ray of the left-path ray as it exits the waterproof glass and enters the water, and its refraction angle, respectively; according to the coplanar constraint condition of the refracted rays, the rays passing through the refracted surface satisfy the following equation; ,in, Based on the above formula, we can deduce the relationship between light and... , intersection , The coordinates; ， As above, underwater target in the right eye camera. Refracted light path , , Composed of, and their corresponding unit vectors are respectively , , Right refracted light path and flat, The intersection points of the planes are respectively , ; Using the rotation matrix between binocular cameras With translation vector ,Will , Transform to the world coordinate system to obtain a new ; in, These are the corresponding underwater targets in the right eye camera in the world coordinate system. The unit vector of the right-reflected ray; For the right refracted ray and , intersection , The coordinates in the world coordinate system; then the refracted light rays from the left and right cameras in the water are... , ; Since the vectors of the two refracted rays passing through the optical center of the camera are coplanar with the intersection point, there exists a point in space that is simultaneously coplanar with the intersection point. , Vertical vector And vector and , The intersection points are respectively and ,but and The coordinates are calculated using the following formula; Corresponding underwater target The three-dimensional coordinates can be minimized and The distance is determined, that is: 。 5. The dynamic calibration method for an underwater binocular vision system with vortex optical ranging coordination according to claim 3, characterized in that, The method for establishing the refraction parameter optimization model in step S22 is as follows: Assuming the size and relative positions of the chessboard squares can be determined in advance, and assuming there are a total of chessboard squares... There are 1 interior corner point, and the size of each chessboard square is 1. , Indicates the first square of the chessboard Line number The three-dimensional coordinates of the inner corner points of the column. , If the calibration parameters are accurate, the three-dimensional spatial coordinates determined from the two-dimensional image coordinates of the chessboard corner points should meet the following conditions: Condition 1: The distance between adjacent corner points is the same, that is, the distance between adjacent corner points is equal to the distance between adjacent corner points. ; Condition 2: Corner points in the same direction are collinear; therefore, a certain corner point Its adjacent corner points , The vector formed , They are perpendicular to each other, and their vectors are perpendicular. , The angle between them should be close to ; Condition 3: The lines where the corner points are located in the same direction are parallel to each other, that is, the vector formed by any corner point to the adjacent corner point to the right should be parallel to the horizontal direction of the chessboard; at the same time, the vector formed by the corner point to the adjacent corner point below should be parallel to the vertical direction of the chessboard. Using the prior information that adjacent corner points are equidistant, corner points in the same direction are collinear, and the lines containing corner points in the same direction are parallel, the following three directional functions are established respectively: in, The distance difference between adjacent corner points is used to measure the average distance between reconstructed adjacent interior corner points and the true distance. The gap between them; The difference between adjacent corner points represents the error between the angle formed by the reconstructed inner corner point vector and the true angle. This is the angle difference in parallel directions, used to calculate the difference in angle between the straight lines formed by the corner points of two horizontal or vertical directions. Establish the overall objective function , By minimizing Able to obtain refraction parameters ,in, Therefore, there are 6 refraction parameters. To improve computational efficiency, the objective function is further simplified by the constraint relationship between the parameters. The constraint relationship is as follows: and ; in, This represents the translation vector between the left and right cameras; The simplified objective function has four parameters to be determined, including... Set the solution range , And assume and The manual measurement value is , Then set and The solution range is , .

6. The dynamic calibration method for an underwater binocular vision system with vortex optical ranging coordination according to claim 3, characterized in that, The method for solving the refractive parameters of the underwater binocular camera in step S23 is as follows: Step S231: Determine the race size as The first in the population Each individual is initialized using continuous real number encoding. The first generation population was generated. , Corresponding to ; Step S232: Perform crossover and mutation operations on individuals in the initial population to generate a population of size [missing information]. offspring population Calculate the objective function value for the parameters corresponding to the current population, and use a non-dominated sorting method to sort the population based on the cost value. Individuals in the hierarchy are divided into at least two non-dominant levels based on their dominance relationships. ),and Priority decreases sequentially; Step S233: From The hierarchy begins by selecting non-dominated sets based on priority and retaining them in the next generation, until the number of sets in the next generation equals the number of sets in the next generation. Or the first time exceeding And assume that the current non-dominated level is ,but All individuals in the layer will be eliminated. , The total number of the next generation; assuming The number of ,and Then, the reference point method is further used in Further selection within the hierarchy Individuals, forming a new scale The evolutionary population; Step S234: Iterate through steps S232 and S233 until the maximum number of iterations is reached, and obtain the optimal solution set of refraction parameters. Select the individual with the smallest variance of the cost values ​​of the three corresponding objectives as the final refraction parameters of the underwater camera.

Images