A method for coordinated control of underwater light source delivery direction based on acoustic information guidance

Abstract

The present invention provides a method for coordinated control of underwater light source delivery direction based on acoustic information guidance, and belongs to the field of underwater acoustic and optical imaging technology. The present invention converts the coordinates of a target object detected by sonar (i.e., the coordinates in a world coordinate system with sonar as the origin) into coordinates in a world coordinate system with a point on the pan-tilt axis as the origin. After the method detects the target object using sonar, the coordinates of the target object detected by sonar (i.e., the coordinates in a world coordinate system with sonar as the origin) are converted into coordinates in a world coordinate system with a point on the pan-tilt axis as the origin, and the pan-tilt angle is adjusted according to the target object coordinates to make it face the target object, thereby changing the underwater lighting conditions and improving the underwater optical imaging quality. The present invention intervenes in the light source during the imaging process to obtain a high-contrast image, making full use of the respective technical advantages of light and sound in underwater imaging, and can be used to complete underwater target detection tasks where search and identification coexist.

Description

Technical Field

[0001] The present invention belongs to the field of underwater acoustic and optical imaging technology, and in particular relates to a method for collaboratively controlling the delivery direction of an underwater light source based on acoustic information guidance. Background Art

[0002] The underwater target detection task is characterized by the coexistence of search and identification, which requires the detection system to have the characteristics of long detection distance, wide detection range and high image quality. Acoustic imaging, due to its long imaging distance and low resolution, can very well adapt to the contour imaging needs of large-scale underwater targets, and has occupied an absolute dominant position in earlier underwater visual applications such as marine exploration, seafloor topography mapping, underwater defense, and underwater search. As human beings continue to deepen their understanding and development of the ocean, facing the high-resolution imaging needs of underwater fine-grained targets such as underwater welding, submarine optical cable laying, deep-sea biological survey, and aquaculture disease diagnosis, optical imaging can bring high target detail resolution compared to acoustic imaging, but its range of action is greatly affected by water quality, so it is only suitable for local small-scale spatial imaging scenes. In other words: the acoustic method is relatively less affected by water quality. Although the resolution is low, the long range is suitable for target search. The optical method is greatly affected by water quality. Although the range is short, the resolution is high and suitable for target identification. Therefore, it is of great significance to develop light and sound joint detection technology.

[0003] At present, most underwater acoustic and optical fusion methods are image-level fusion, that is, directly using the acoustic and optical image spatial registration method to achieve acoustic and optical information fusion, which is achieved by preprocessing the acquired acoustic and optical images or improving the image-level registration method. It is an image post-processing technology. This technology only focuses on the role of sonar and camera in the underwater imaging system, and does not consider the impact of the light source on the quality of the acquired image. It is impossible to control the quality of the acquired image, resulting in poor quality of the acquired unprocessed image, which affects the effect of the acoustic and optical fusion technology in the application. Summary of the invention

[0004] In view of the above problems, the present invention converts the coordinates of the target object detected by the sonar (i.e., the coordinates in the world coordinate system with the sonar as the origin) into coordinates in the world coordinate system with a point on the pan-tilt axis as the origin, so that the pan-tilt angle for placing the light source can be calculated according to the position of the target object identified by the sonar, thereby realizing the change of the energy delivery direction of the light source. The method proposed in the present invention mainly intervenes in the light source during the imaging process to obtain a high-contrast image, which also has a certain positive effect on the acoustic-optical fusion technology at the image level, thereby achieving the purpose of improving the quality of underwater imaging.

[0005] The present invention proposes a method for collaboratively controlling the delivery direction of an underwater light source based on acoustic information guidance, comprising the following steps:

[0006] Step 1: The light source is set on a pan-tilt assembly with adjustable angles, and two three-dimensional rectangular coordinate systems are set, including a world coordinate system with the sonar as the origin. and the world coordinate system with a point on the gimbal axis as the origin ;

[0007] Step 2: According to the camera calibration principle, the camera internal parameter matrix and distortion matrix are obtained; the checkerboard is used as a standard reference to obtain the coordinates of the corner points in the two world coordinate systems; the checkerboard image is captured by a camera to obtain the pixel coordinates of the checkerboard corner points to solve the rotation matrix between the world coordinate system with the sonar as the origin and the camera coordinate system. and translation matrix And the rotation matrix between the world coordinate system with a point on the gimbal axis as the origin and the camera coordinate system and translation matrix ;

[0008] Step 3, according to , , , And the plane pixel coordinates of the target object in the image, solve the conversion relationship between the coordinates of the target object in the two world coordinate systems;

[0009] Step 4, obtaining the coordinates of the target object in the world coordinate system with the sonar as the origin, and calculating the coordinates of the target object in the world coordinate system with a point on the gimbal axis as the origin according to the conversion relationship in step 3;

[0010] Step 5, based on the coordinates of the target object in the world coordinate system with a point on the rotation axis of the gimbal as the origin, calculate the angle of the gimbal where the light source is placed, and control the direction of light field energy delivery so that it is delivered to the target object.

[0011] Preferably, the two three-dimensional rectangular coordinate systems set in step 1 are used to set the world coordinate system , with the sonar as the coordinate origin Establish the world coordinate system. axis, Axis and The axis satisfies the right-hand screw rule; set the world coordinate system , with a point on the gimbal axis as the coordinate origin Establish the world coordinate system. axis, Axis and The shaft satisfies the right-hand screw rule.

[0012] Preferably, in step 2, according to the camera calibration principle, the camera internal parameter matrix and the distortion matrix are obtained, and the specific process is:

[0013] Use a checkerboard calibration plate with known dimensions and grid size as a standard reference, and collect at least 10 images of the checkerboard calibration plate at different positions and angles;

[0014] Use the built-in calibrateCamera function in the openCV library for calibration. The calibration program reads the pixel coordinates and world coordinates of the corner points, and calculates the camera's intrinsic parameter matrix and distortion matrix based on maximum likelihood estimation.

[0015] Run the calibration program to obtain the calibration results and save the calibration results in a file.

[0016] Preferably, in step 2, the rotation matrix between the world coordinate system with the sonar as the origin and the camera coordinate system is solved and translation matrix And the rotation matrix between the world coordinate system with a point on the gimbal axis as the origin and the camera coordinate system and translation matrix , the specific process is:

[0017] S11, place the chessboard in front of the camera, ensure that the camera can capture all the corners of the chessboard, and use the built-in findChessboardCorners in the openCV library to find the pixel coordinates of the corners of the chessboard;

[0018] S12, using the sonar to obtain the coordinates of the corner points of the chessboard in a world coordinate system with the sonar as the origin and the coordinates in a world coordinate system with a point on the rotation axis of the gimbal as the origin;

[0019] S13, according to the camera calibration principle, solve the camera's external parameter matrix, the formula is as follows:

[0020]

[0021] During calibration, the world coordinate system The plane is defined on the calibration plate plane. represents the world coordinates of the corner points of the chessboard in this world coordinate system, are the pixel coordinates of the corner points of the chessboard, represents the extrinsic parameter matrix of the camera, and They represent the rotation matrix and translation matrix of the world coordinate system defined during calibration relative to the camera coordinate system. is the intrinsic parameter matrix of the camera;

[0022] S14, the coordinates of the corner points of the chessboard in the world coordinate system with the sonar as the origin, the pixel coordinates of the corner points of the chessboard in the image and the internal parameter matrix Substitute into the built-in solvePnP function in openCV to find the rotation vector and translation matrix , and use the Rodrigues function to convert the rotation vector into a rotation matrix ;

[0023] S15, convert the world coordinates into coordinates in the world coordinate system with a point on the pan / tilt axis as the origin, repeat the processes of S13 and S14, and obtain the rotation matrix between the world coordinate system with a point on the pan / tilt axis as the origin and the camera coordinate system and translation matrix .

[0024] Preferably, the specific steps of step 4 are:

[0025] The conversion relationship between the homogeneous coordinates of a vertex of the target object in the world coordinate system with the sonar as the origin and the homogeneous coordinates of the image of the point in the pixel coordinate system is:

[0026]

[0027] is the intrinsic parameter matrix of the camera, 、 Represents the rotation matrix and translation matrix of the world coordinate system with the sonar as the origin relative to the camera coordinate system;

[0028] Express the above equation in Cartesian coordinates:

[0029]

[0030] 、 They are the rotation matrix and translation matrix of the world coordinate system with the sonar as the origin relative to the camera coordinate system;

[0031] The conversion relationship between the homogeneous coordinates of a vertex of the target object in the world coordinate system with a point on the gimbal axis as the origin and the homogeneous coordinates of the image of the point in the pixel coordinate system is:

[0032]

[0033] is the intrinsic parameter matrix of the camera, 、 It represents the rotation matrix and translation matrix of the world coordinate system with a point on the gimbal axis as the origin relative to the camera coordinate system; the above formula is expressed in Cartesian coordinates:

[0034]

[0035] because , In the pixel coordinate system, it is the same point, so we have ;

[0036] The coordinates of the known target in the world coordinate system with the sonar as the origin , get the coordinates of the point in the world coordinate system with a point on the gimbal axis as the origin, the formula is:

[0037] .

[0038] Preferably, the gimbal assembly includes a first gimbal and a second gimbal hinged on the first gimbal; the first gimbal can rotate left and right 360 degrees, and the second gimbal can rotate up and down 180 degrees based on the hinge point with the first gimbal, thereby controlling the angle of the light source.

[0039] Preferably, the control angle calculation process of the first pan-tilt and the second pan-tilt is:

[0040] Take a point on the first gimbal axis As the origin of coordinates, establish a spherical coordinate system and a spatial rectangular coordinate system; the spatial rectangular coordinate system Axis and the world coordinate system with the sonar as the origin Parallel; measure the length of the shaft ,set up It is the intersection with the second cloud platform. is the coordinate of a vertex of the target object in the world coordinate system with the gimbal axis as the origin, denoted as ;

[0041] The coordinates of the point in the world coordinate system with the first gimbal optical axis as the origin are:

[0042]

[0043] Since the two axes of rotation are perpendicular to each other, we have:

[0044]

[0045] Will The coordinates of the points and Substituting the coordinates of into the above formula, we get:

[0046]

[0047] In the above formula, is a constant, measured Value According to the coordinate transformation, solving the above formula gives ;

[0048] According to the rotation range of the first pan / tilt, the calculated The value is used for angle value conversion and error compensation;

[0049] The angle of the second gimbal Calculated by the following formula:

[0050]

[0051] The mapping relationship between the light field energy delivery direction and the coordinates of the target position identified by the sonar is obtained.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] The experimental results show that the world coordinates of different checkerboard corner points are input into the light source energy coordinated control system to control the light source energy delivery direction. After the light source energy delivery direction is coordinated and adjusted, images are collected to verify whether the pan-tilt angle calculation is accurate. It can be seen that the light source energy direction can be delivered according to the location of the target object. The present invention converts the coordinates of the target object detected by the sonar into coordinates in a world coordinate system with a point on the pan-tilt axis as the origin, thereby calculating the pan-tilt angle for placing the light source according to the position of the target object identified by the sonar, realizing the coordinated control of the light source energy delivery direction, making full use of the respective technical advantages of light and sound in underwater imaging, and can be used to complete underwater target detection tasks where search and identification coexist. The present invention is used in the detection system to enable it to have the characteristics of long detection distance, wide detection range, and high image quality.

[0054] The method of using acoustic information to guide the coordinated regulation of light sources proposed in the present invention is essentially different from the existing image-level acoustic-optical fusion technology. The present invention intervenes in the light source during the imaging process to obtain a high-contrast image. Since the image preprocessing method and the image-level registration method in the image-level acoustic-optical fusion technology have certain requirements on image contrast, the present invention also has a certain positive effect on the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a schematic diagram of the overall process flow of the present invention.

[0056] Figure 2 Schematic diagram of a chessboard calibration plate in an embodiment.

[0057] Figure 3 Schematic diagram of the gimbal angle calculation model.

[0058] Figure 4 Schematic diagram of the upper right corner of the chessboard grid when the pan / tilt adjusts the angle in the embodiment.

[0059] Figure 5It is the lower right corner point of the lighting chessboard when the pan / tilt adjusts the angle in the embodiment. DETAILED DESCRIPTION

[0060] This patent proposes a method for coordinated control of the underwater light source delivery direction based on acoustic information guidance, which converts the coordinates of the target object detected by the sonar (i.e., the coordinates in the world coordinate system with the sonar as the origin) into coordinates in the world coordinate system with a point on the pan-tilt axis as the origin. This allows the pan-tilt angle for placing the light source to be calculated based on the position of the target object identified by the sonar, thereby changing the direction of the light source energy delivery. The method proposed in this patent mainly intervenes in the light source during the imaging process to obtain a high-contrast image, which also has a certain positive effect on the acoustic-optical fusion technology at the image level, thereby achieving the purpose of improving the underwater imaging quality. The overall process is as follows: Figure 1 shown.

[0061] The invention will be further described below in conjunction with specific embodiments.

[0062] 1. Set two world coordinate systems and

[0063] Set the world coordinate system , with the sonar as the coordinate origin Establish the world coordinate system. Axis and The plane formed by the axes is parallel to the ground. Axis perpendicular to Axis and The plane formed by the axes establishes the world coordinate system. , and The shaft satisfies the right-hand screw rule.

[0064] Set the world coordinate system , with a point on the gimbal axis as the origin , the world coordinate system Axis and the world coordinate system with the sonar as the origin parallel.

[0065] 2. Camera internal parameter matrix and distortion matrix

[0066] In this embodiment, a corner point number is used. And an alumina checkerboard calibration plate with a grid size of 20 mm each as a standard reference, such as Figure 2 As shown in Figure 1, the calibration plate image is captured using a Deepsea IPMSC-3105 camera with a resolution of 1080p. In order to ensure the accuracy of the calibration, 14 images of the checkerboard calibration plate at different positions and angles are collected.

[0067] Use the built-in calibrateCamera function in the openCV library for calibration. The calibration program reads the pixel coordinates and world coordinates of the corner points, and calculates the intrinsic parameters and distortion matrix of the camera based on the maximum likelihood estimation.

[0068] Run the calibration program to obtain the calibration results and save the calibration results in a file.

[0069] 3. Obtaining the rotation matrix and translation matrix between the two world coordinate systems and the camera coordinate system

[0070] S1, place the chessboard in front of the camera, make sure the camera can capture all the corners of the chessboard, and use the built-in findChessboardCorners in the openCV library to find the pixel coordinates of the corners of the chessboard.

[0071] S2, obtain the coordinates of the corner points of the chessboard in the world coordinate system with the sonar as the origin and the coordinates in the world coordinate system with a point on the gimbal axis as the origin.

[0072] S3, according to the camera calibration principle, solve the camera's external parameter matrix, the formula is as follows:

[0073]

[0074] In order to facilitate calculation during calibration, the world coordinate system is The plane is defined on the calibration plate plane. represents the world coordinates of the corner points of the chessboard in this world coordinate system, are the pixel coordinates of the corner points of the chessboard. Represents the extrinsic parameter matrix of the camera. and They represent the rotation matrix and translation matrix of the world coordinate system defined during calibration relative to the camera coordinate system. is the intrinsic parameter matrix of the camera. The intrinsic parameter matrix of the camera is the intrinsic parameter of the camera and is therefore a constant.

[0075] The coordinates of the corner points of the chessboard in the world coordinate system with the sonar as the origin, the pixel coordinates of the corner points of the chessboard in the image and the intrinsic parameter matrix Substitute into the built-in solvePnP function in openCV to find the rotation vector and translation matrix , and use the Rodrigues function to convert the rotation vector into a rotation matrix .

[0076] S4, convert the world coordinates into coordinates in the world coordinate system with a point on the gimbal axis as the origin, repeat S3, and you can get the rotation matrix and translation matrix .

[0077] 4. Based on the known coordinates of the target in the world coordinate system with the sonar as the origin, calculate the coordinates of the target in the world coordinate system with a point on the gimbal axis as the origin.

[0078] The conversion relationship between the homogeneous coordinates of a vertex of the target object in the world coordinate system with the sonar as the origin and the homogeneous coordinates of the image of the point in the pixel coordinate system is:

[0079]

[0080] is the intrinsic parameter matrix of the camera, , Represents the rotation matrix and translation matrix of the world coordinate system with the sonar as the origin relative to the camera coordinate system.

[0081] Expressing the above equation in Cartesian coordinates, we have:

[0082]

[0083] , They are the rotation matrix and translation matrix of the world coordinate system with the sonar as the origin relative to the camera coordinate system.

[0084] The conversion relationship between the homogeneous coordinates of a vertex of the target object in the world coordinate system with a point on the gimbal axis as the origin and the homogeneous coordinates of the image of the point in the pixel coordinate system is:

[0085]

[0086] is the intrinsic parameter matrix of the camera, , It represents the rotation matrix and translation matrix of the world coordinate system with a point on the pan / tilt axis as the origin relative to the camera coordinate system. The above formula is expressed in Cartesian coordinates:

[0087]

[0088] because , In the pixel coordinate system, it is the same point, so we have Therefore, the coordinates of the target in the world coordinate system with the sonar as the origin are known. , we can get the coordinates of the point in the world coordinate system with a point on the gimbal axis as the origin

[0089]

[0090] 5. Calculate the angle of the gimbal assembly where the light source is placed

[0091] In this embodiment, the gimbal assembly includes a first gimbal and a second gimbal hinged on the first gimbal; the first gimbal can rotate 360 ​​degrees left and right, and the second gimbal can rotate 180 degrees up and down based on the hinge point with the first gimbal, so as to control the angle of the light source. In this embodiment, the control angle calculation process of the first gimbal and the second gimbal is:

[0092] S1, the direction of light field energy delivery is controlled by two pan-tilts. In order to calculate the two pan-tilt angles conveniently, a pan-tilt angle calculation model is established, such as Figure 3 . Take a point on the first gimbal axis As the origin of coordinates, establish the spherical coordinate system and the spatial rectangular coordinate system. Axis and the world coordinate system with the sonar as the origin In the figure, is the length of the shaft, It is the intersection with the second cloud platform. is the coordinate of a vertex of the target object in the world coordinate system with the gimbal axis as the origin, denoted as . Given by measuring In this embodiment, .

[0093] S2, according to the established model, The coordinates of the point in the world coordinate system with the first gimbal optical axis as the origin are:

[0094]

[0095] S3, since the two axes are perpendicular to each other, we can get:

[0096]

[0097] Will The coordinates of the points and Substituting the coordinates of into the above formula, we get:

[0098]

[0099] In the above formula, the inclination angle of the first gimbal is fixed, so is a constant, measured The value of , It can be obtained by coordinate transformation. Solving the above formula can be obtained ; express and The angle of the axis; express exist , Axis Projection on a plane The angle between

[0100] S4, according to the rotation range of the first pan / tilt head, The first pan / tilt rotation range is , so the angle of the gimbal rotation is .

[0101] S5, angle of the second gimbal Calculated by the following formula:

[0102]

[0103] In summary, the mapping relationship between the light field energy delivery direction and the coordinates of the target position identified by the sonar can be obtained.

[0104] Input the coordinates of the upper right corner of the chessboard, and you will get the effect diagram of the gimbal adjusting angle as shown below Figure 4 , input the coordinates of the lower right corner of the chessboard, and get the gimbal adjustment effect as follows Figure 5 The experimental results show that the world coordinates of different checkerboard corner points are input into the light source energy coordinated control system to control the light source energy delivery direction. After the light source energy delivery direction is coordinated and adjusted, images are collected to verify whether the gimbal angle calculation is accurate. It can be seen that the light source energy direction can be delivered according to the location of the target object.

[0105] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0106] Although the above describes the specific implementation methods of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for collaboratively controlling the delivery direction of underwater light sources based on acoustic information guidance, characterized in that: The following steps are involved: Step 1: The light source is set on a pan-tilt assembly with adjustable angles, and two three-dimensional rectangular coordinate systems are set, including a world coordinate system with the sonar as the origin. and the world coordinate system with a point on the gimbal axis as the origin ; Step 2: According to the camera calibration principle, the camera internal parameter matrix and distortion matrix are obtained; the checkerboard is used as a standard reference to obtain the coordinates of the corner points in the two world coordinate systems; the checkerboard image is captured by a camera to obtain the pixel coordinates of the checkerboard corner points to solve the rotation matrix between the world coordinate system with the sonar as the origin and the camera coordinate system. and translation matrix And the rotation matrix between the world coordinate system with a point on the gimbal axis as the origin and the camera coordinate system and translation matrix ; Step 3, according to , , , And the plane pixel coordinates of the target object in the image, solve the conversion relationship between the coordinates of the target object in the two world coordinate systems; Step 4, obtaining the coordinates of the target object in the world coordinate system with the sonar as the origin, and calculating the coordinates of the target object in the world coordinate system with a point on the gimbal axis as the origin according to the conversion relationship in step 3; Step 5, based on the coordinates of the target object in the world coordinate system with a point on the rotation axis of the gimbal as the origin, calculate the angle of the gimbal where the light source is placed, and control the direction of light field energy delivery so that it is delivered to the target object; The pan-tilt assembly includes a first pan-tilt and a second pan-tilt hingedly arranged on the first pan-tilt; the first pan-tilt can rotate 360 ​​degrees left and right, and the second pan-tilt can rotate 180 degrees up and down based on the hinge point with the first pan-tilt, so as to control the angle of the light source; The calculation process of the control angle of the first and second gimbals is as follows: Take a point on the first gimbal axis As the coordinate origin, establish a spherical coordinate system and a spatial rectangular coordinate system; the spatial rectangular coordinate system Axis and the world coordinate system with the sonar as the origin Parallel; measure the length of the shaft ,set up It is the intersection with the second cloud platform. is the coordinate of a vertex of the target object in the world coordinate system with the gimbal axis as the origin, denoted as ; The coordinates of the point in the world coordinate system with the first gimbal optical axis as the origin are: Since the two axes of rotation are perpendicular to each other, we have: Will The coordinates of the points and Substituting the coordinates of into the above formula, we get: In the above formula, is a constant, measured The value of According to the coordinate transformation, solving the above formula gives ; express and The angle of the axis; express exist , Axis Projection on a plane The angle between According to the rotation range of the first pan / tilt, the calculated The value is used for angle value conversion and error compensation; Get the rotation angle of the first gimbal; The angle of the second gimbal Calculated by the following formula: The mapping relationship between the light field energy delivery direction and the coordinates of the target position identified by the sonar is obtained.

2. The method for coordinated control of underwater light source delivery direction based on acoustic information guidance according to claim 1, characterized in that: The two three-dimensional rectangular coordinate systems set in step 1 set the world coordinate system , with the sonar as the coordinate origin Establish the world coordinate system. axis, Axis and The axis satisfies the right-hand screw rule; set the world coordinate system , with a point on the gimbal axis as the coordinate origin Establish the world coordinate system. axis, Axis and The shaft satisfies the right-hand screw rule.

3. The method for coordinated control of underwater light source delivery direction based on acoustic information guidance according to claim 1, characterized in that: In step 2, according to the camera calibration principle, the camera internal parameter matrix and distortion matrix are obtained. The specific process is as follows: Use a checkerboard calibration plate with known dimensions and grid size as a standard reference, and collect at least 10 images of the checkerboard calibration plate at different positions and angles; Use the built-in calibrateCamera function in the openCV library for calibration. The calibration program reads the pixel coordinates and world coordinates of the corner points, and calculates the camera's intrinsic parameter matrix and distortion matrix based on maximum likelihood estimation. Run the calibration program to obtain the calibration results and save the calibration results in a file.

4. The method for coordinated control of underwater light source delivery direction based on acoustic information guidance according to claim 1, characterized in that: In step 2, the rotation matrix between the world coordinate system with the sonar as the origin and the camera coordinate system is solved. and translation matrix And the rotation matrix between the world coordinate system with a point on the gimbal axis as the origin and the camera coordinate system and translation matrix , the specific process is: S11, place the chessboard in front of the camera, ensure that the camera can capture all the corners of the chessboard, and use the built-in findChessboardCorners in the openCV library to find the pixel coordinates of the corners of the chessboard; S12, using the sonar to obtain the coordinates of the corner points of the chessboard in a world coordinate system with the sonar as the origin and the coordinates in a world coordinate system with a point on the rotation axis of the gimbal as the origin; S13, according to the camera calibration principle, solve the camera's external parameter matrix, the formula is as follows: During calibration, the world coordinate system The plane is defined on the calibration plate plane. represents the world coordinates of the corner points of the chessboard in this world coordinate system, are the pixel coordinates of the corner points of the chessboard, represents the extrinsic parameter matrix of the camera, and They represent the rotation matrix and translation matrix of the world coordinate system defined during calibration relative to the camera coordinate system. is the intrinsic parameter matrix of the camera; S14, the coordinates of the corner points of the chessboard in the world coordinate system with the sonar as the origin, the pixel coordinates of the corner points of the chessboard in the image and the internal parameter matrix Substitute into the built-in solvePnP function in openCV to find the rotation vector and translation matrix , and use the Rodrigues function to convert the rotation vector into a rotation matrix ; S15, convert the world coordinates into coordinates in the world coordinate system with a point on the pan / tilt axis as the origin, repeat the processes of S13 and S14, and obtain the rotation matrix between the world coordinate system with a point on the pan / tilt axis as the origin and the camera coordinate system and translation matrix .

5. The method for coordinated control of underwater light source delivery direction based on acoustic information guidance according to claim 1, characterized in that: The specific steps of step 4 are as follows: The conversion relationship between the homogeneous coordinates of a vertex of the target object in the world coordinate system with the sonar as the origin and the homogeneous coordinates of the image of the point in the pixel coordinate system is: is the intrinsic parameter matrix of the camera, 、 Represents the rotation matrix and translation matrix of the world coordinate system with the sonar as the origin relative to the camera coordinate system; Express the above equation in Cartesian coordinates: 、 They are the rotation matrix and translation matrix of the world coordinate system with the sonar as the origin relative to the camera coordinate system; The conversion relationship between the homogeneous coordinates of a vertex of the target object in the world coordinate system with a point on the gimbal axis as the origin and the homogeneous coordinates of the image of the point in the pixel coordinate system is: is the intrinsic parameter matrix of the camera, 、 Represents the rotation matrix and translation matrix of the world coordinate system with a point on the gimbal axis as the origin relative to the camera coordinate system; The above formula is expressed in Cartesian coordinates: because , In the pixel coordinate system, it is the same point, so we have ; The coordinates of the known target in the world coordinate system with the sonar as the origin , get the coordinates of the point in the world coordinate system with a point on the gimbal axis as the origin, the formula is: 。

Images