3D reconstruction method for underwater damage of marine equipment based on vision and IMU fusion

Abstract

The present invention provides a 3D reconstruction method for underwater damage of marine equipment based on vision and IMU fusion. The method includes the steps of performing, in an underwater environment, internal parameter calibration of the left and right cameras of a stereo camera and extrinsic parameter calibration between the left and right cameras of the stereo camera and a platform IMU; calibrating the extrinsic parameter matrices of the drive IMU coordinate system and the laser sensor coordinate system on each axis of the drive system; identifying and roughly locating the damaged area based on the 3D reconstruction of the underwater vision; planning an optimal path for the underwater mobile platform, performing local obstacle avoidance based on the 3D reconstructed point cloud, and controlling the underwater mobile platform to move near the damaged area; planning a trajectory for a drainage system and draining water using the drainage system; and determining the laser position of a laser sensor using drive IMU data to realize a precise 3D reconstruction of the damaged area based on the laser sensor data. This method provides a highly accurate 3D reconstruction of the damaged area, assisting other equipment in autonomous repair and improving the work efficiency of the marine equipment.

Description

[Technical Field]

[0001] The present invention relates to the technical field of underwater 3D reconstruction, and in particular to a method for 3D reconstruction of underwater damage to marine equipment based on the fusion of vision and IMU. [Background technology]

[0002] Marine equipment, including ships, offshore oil and gas platforms, and offshore wind power plants, are susceptible to long-term structural damage due to adverse factors such as large waves, humid environments, seawater corrosion, and collisions. Traditional methods require returning to port for repairs or manual underwater inspections, which require significant time and cost, and pose numerous safety risks. Using an underwater mobile platform to locate the damage effectively solves this problem. The underwater mobile platform's autonomous positioning and 3D reconstruction technology allows for the establishment of a clear and accurate underwater damage model, which can then be used in conjunction with an autonomous repair system to complete the repair work.

[0003] Currently, commonly used underwater positioning systems include underwater acoustic positioning systems and underwater SLAM (Simultaneous Landing and Landing Amplification) techniques. Underwater acoustic positioning systems include ultra-short baseline positioning, short baseline positioning, and long baseline positioning, and require expensive equipment and are difficult to install. Common SLAM techniques include sonar and camera-based techniques. Sonar equipment is expensive and, because it is an acoustic technique, has low resolution, making it suitable for deep-sea positioning. However, camera-based techniques must overcome light refraction in the water and are sensitive to light rays, making them prone to positioning failure when feature points are unclear.

[0004] Conventional camera-based 3D reconstruction systems have two procedures: camera distortion correction and 3D reconstruction. Camera distortion correction includes methods based on a single-viewpoint model and methods based on a calibration board or auxiliary hardware. Single-viewpoint model methods only consider the perspective model and do not consider the underwater refraction model, resulting in low accuracy. Calibration board and auxiliary hardware methods consider the underwater refraction model, resulting in high accuracy. 3D reconstruction primarily involves directly or indirectly acquiring a 3D point cloud based on camera parameters and then superimposing the 3D point cloud using positioning data. However, as mentioned above, the accuracy of underwater positioning using only a camera is low, which also results in low accuracy of 3D reconstruction. Summary of the Invention [Problem to be solved by the invention]

[0005] To overcome the drawbacks and deficiencies of the prior art, the present invention provides a method for 3D reconstruction of underwater damage to marine equipment based on the fusion of vision and IMU, which can provide highly accurate 3D reconstruction results of the damaged area, assist other equipment in autonomous repair, and improve the work efficiency of marine equipment. [Means for solving the problem]

[0006] To achieve the above object, the present invention is realized by the following technical means: A method for 3D reconstruction of underwater damage of marine equipment based on the fusion of vision and IMU is realized by a 3D underwater damage reconstruction system, the 3D underwater damage reconstruction system comprising an underwater mobile platform and a host computer, the underwater mobile platform comprising an underwater mobile platform body, a stereo camera mounted on the underwater mobile platform body, a platform IMU, a laser sensor, a laser driving system, a communication system, and a drainage system, each axis of the laser driving system is provided with a driving IMU, and the communication system is used for communication between the underwater mobile platform and the host computer, 3D reconstruction method for underwater damage of marine equipment S1: Fixing the stereo camera and the platform IMU to the underwater moving platform, respectively, and performing internal parameter calibration of the left and right cameras of the stereo camera and external parameter calibration between the left and right cameras of the stereo camera and the platform IMU in an underwater environment; S2, fixing a laser sensor to the tip of the drive system, and calibrating the external parameter matrix of the drive IMU coordinate system and the laser sensor coordinate system on each axis of the drive system; S3: collecting stereo camera image data and platform IMU data, integrating acceleration data and angular velocity data in the platform IMU data to obtain position and attitude observation in the platform IMU coordinate system, performing position and attitude observation, damage detection and 3D point cloud generation in the stereo camera coordinate system for frames corresponding to the stereo camera image data, fusing the position and attitude observation results and superimposing the continuous 3D point clouds to obtain an underwater 3D reconstructed point cloud for use in verifying the damage detection results, planning an optimal path for the underwater mobile platform based on the position and attitude observation and damage detection results, performing local obstacle avoidance based on the 3D reconstructed point cloud, and controlling the underwater mobile platform to move near the damaged area; S4: Planning the trajectory of the drainage system, draining the damaged area with the drainage system, and determining the laser position of the laser sensor using the driving IMU data based on the external parameter matrix obtained in S2, thereby realizing a three-dimensional reconstruction of the detailed laser sensor data of the damaged area.

[0007] Preferably, S1 is S11: fixing a stereo camera to the front end of the underwater mobile platform body, with the visual direction tilted downward between 10° and 30°, and fixing a platform IMU to the center of the underwater mobile platform body corresponding to the center of mass of the underwater mobile platform; S12. The calibration board and the underwater moving platform are simultaneously placed in the water, so that the calibration board appears in the field of view of the left and right cameras of the stereo camera at the same time; The method includes the steps of moving the underwater mobile platform so that the calibration boards are distributed at each position in the field of view of the left and right cameras of the stereo camera, recording multiple sets of stereo camera image data, transmitting the multiple sets of stereo camera image data to a host computer via a communication system, and using the host computer to perform related calibration calculations, internal parameter calibration of the left and right cameras of the stereo camera, and external parameter calibration between the left and right cameras of the stereo camera and the platform IMU.

[0008] Preferably, in S12, The internal parameter calibration of the left and right cameras of a stereo camera is as follows:

number

number

number

number

number

[0009] Preferably, in step S2, aligning the drive IMU coordinate system and the laser sensor coordinate system on each axis of the drive system includes: Based on the positional relationship between the stereo camera, the drainage system, and the laser drive system, a transformation relationship from the underwater mobile platform mass center coordinate system to the drainage system coordinate system and a transformation relationship between the drainage system coordinate system and the laser drive system coordinate system are obtained; The laser point of the laser sensor is controlled to move on a calibration board with known parameters, the laser sensor and the driving IMU are connected via a communication system to acquire data, which is then transmitted to a host computer, where calibration calculations are performed by the host computer to acquire the transformation relationship between the laser driving system coordinate system and the laser sensor coordinate system, Align the four coordinate systems of the laser sensor, the laser drive system, the drainage system, and the center of mass of the underwater mobile platform.

[0010] Preferably, the method for aligning the four coordinate systems of the laser sensor, the laser drive system, the drainage system, and the center of mass of the underwater mobile platform comprises: Calibrate the extrinsic parameter matrix, including the rotation matrix and translation vector of any two coordinate systems of the laser sensor, the laser drive system, the drainage system, and the center of mass of the underwater mobile platform.

number

[0011] Preferably, in step S3, the attitude observation in the platform IMU coordinate system is The velocity V, translation vector T, and rotation matrix R obtained by integrating the platform IMU data from time k to time k+1 are respectively

number

number

number

number

[0012] Preferably, in step S3, the posture observation in the stereo camera coordinate system is Feature points are extracted from the stereo camera image data, and a circular region is constructed with the feature points as the center.

number

number

[0013] Preferably, in S3, the 3D point cloud generation in the stereo camera coordinate system is performed by: The above feature points are extracted and matched for the left and right camera images of the same frame of the stereo camera, and the disparity is calculated based on the grayscale squared error algorithm.

number

number

[0014] Preferably, S4 plans a movement trajectory of the underwater mobile platform based on the location of the damaged area so that the drainage system covers and drains the damaged area to form a dry space, uses a laser driving system to control the laser sensor to perform three-dimensional scanning in the dry space, and transmits the laser sensor data and driving IMU data to a host computer via a communication system, and the host computer obtains the attitude of the laser driving system using the driving IMU data based on the external parameter matrix obtained in S2, converts it to obtain the position of the laser sensor, obtains a fine three-dimensional reconstruction of the laser sensor data based on the laser sensor position and point cloud data, and detects the damage location based on the three-dimensional reconstruction result.

[0015] Preferably, in step S4, the three-dimensional reconstruction of the laser sensor data is performed by: The laser sensor emits laser pulses at a fixed frequency and receives the reflected light from the receiver to determine the distance, and roughly distinguishes the target material based on the reflection intensity. The distance measurement formula is: L=tc / 2 where L is the target distance, t is the return time, and c is the speed of light. The position and orientation of the laser sensor are predicted using the driving IMU, and a 3D reconstruction result of the laser sensor is obtained based on the rotation matrix R and translation vector T. [Effects of the Invention]

[0016] Compared with the prior art, the present invention has the following advantages and beneficial effects. 1. The present invention can autonomously detect underwater damage and perform 3D reconstruction of marine equipment, thereby reducing labor and economic costs and improving safety. 2. By combining vision and IMU, the present invention improves positioning accuracy and underwater 3D reconstruction accuracy, and by using a damage detection method based on image and point cloud fusion verification, the location of damage in underwater marine equipment can be more accurately determined. 3. The present invention can accurately drain water near the damaged area, thereby realizing high-precision laser-based 3D reconstruction and damage identification, and providing convenience to other autonomous repair equipment. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a flowchart of a method for 3D reconstruction of underwater damage to marine equipment based on vision and IMU fusion according to the present invention. [Figure 2] 1 is a structural schematic diagram of a three-dimensional reconstruction system for underwater damage according to the present invention; [Figure 3] FIG. 1 is a diagram illustrating the communication of a 3D reconstruction method for underwater damage to marine equipment based on vision and IMU fusion according to the present invention. [Figure 4] FIG. 1 is a diagram illustrating coordinate system transformation in the 3D reconstruction method for underwater damage to marine equipment based on the fusion of vision and IMU according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0018] The present invention will be described in detail below by way of specific embodiments with reference to the drawings.

[0019] (Example) The specific process of the 3D reconstruction method for underwater damage of marine equipment based on the fusion of vision and IMU according to this embodiment is shown in FIG. 1 and is realized by a 3D reconstruction system for underwater damage.

[0020] The 3D reconstruction system for underwater damage comprises an underwater mobile platform and a host computer. As shown in FIG. 2 , the underwater mobile platform comprises an underwater mobile platform main body 1, a stereo camera 4 mounted on the underwater mobile platform main body 1, a platform IMU 6, a laser sensor 5, a laser driving system 3, a communication system, and a drainage system 2.

[0021] A driving IMU is provided for each axis of the laser driving system 3, and a communication system is used for communication between the underwater mobile platform and the host computer. Specifically, the communication system is fixed to the center of the underwater mobile platform body and is used to collect and transmit stereo camera image data and IMU data to the host computer, as well as receive related control commands from the host computer to drive the underwater mobile platform.

[0022] The method for three-dimensionally reconstructing underwater damage to marine equipment includes the following steps S1 to S4.

[0023] S1: The stereo camera and the platform IMU are fixed to the underwater mobile platform, and in the underwater environment, internal parameter calibration of the left and right cameras of the stereo camera and external parameter calibration between the left and right cameras of the stereo camera and the platform IMU are performed.

[0024] S1 is S11: fixing a stereo camera to the front end of the underwater mobile platform body, with the visual direction tilted downward between 10° and 30°, and fixing a platform IMU to the center of the underwater mobile platform body corresponding to the center of mass of the underwater mobile platform; S12. Place the calibration board and the underwater moving platform in the water at the same time, so that the calibration board appears in the field of view of the left and right cameras of the stereo camera at the same time, and rotate it as much as possible in each direction to ensure calibration with the three axes of the driving IMU while ensuring that the calibration board is completely included in the field of view of the stereo camera at the same time. The data recording time in this step does not need to be long, and the stereo camera should be 15 frames / second or more, and the driving IMU should be 100 frames / second or more. The method includes the steps of moving the underwater mobile platform so that the calibration boards are distributed at each position in the field of view of the left and right cameras of the stereo camera, recording multiple sets of stereo camera image data, transmitting the multiple sets of stereo camera image data to a host computer via a communication system, and using the host computer to perform related calibration calculations, internal parameter calibration of the left and right cameras of the stereo camera, and external parameter calibration between the left and right cameras of the stereo camera and the platform IMU.

[0025] In S12, the internal parameter calibration of the left and right cameras of the stereo camera is

number

number

number

number

number

[0026] S2: Fix the laser sensor to the tip of the drive system and calibrate the external parameter matrix between the drive IMU coordinate system and the laser sensor coordinate system on each axis of the drive system.

[0027] Specifically, as shown in Figure 4, based on the positional relationship between the stereo camera, the drainage system, and the laser drive system, the transformation relationship from the underwater mobile platform mass center coordinate system to the drainage system coordinate system, and the transformation relationship between the drainage system coordinate system and the laser drive system coordinate system are obtained; The laser point of the laser sensor is controlled to move on a calibration board with known parameters, the laser sensor and the driving IMU are connected via a communication system to acquire data, which is then transmitted to a host computer, where calibration calculations are performed by the host computer to acquire the transformation relationship between the laser driving system coordinate system and the laser sensor coordinate system, Align the four coordinate systems of the laser sensor, the laser drive system, the drainage system, and the center of mass of the underwater mobile platform. After the offline calibration is completed, the transformation relationships of all the coordinate systems in Figure 4 are known.

[0028] The method for aligning the four coordinate systems of the laser sensor, the laser driving system, the drainage system, and the center of mass of the underwater mobile platform is as follows: Calibrate the extrinsic parameter matrix, including the rotation matrix and translation vector of any two coordinate systems of the laser sensor, the laser drive system, the drainage system, and the center of mass of the underwater mobile platform.

number

[0029] S3: The stereo camera image data and platform IMU data are collected by the host computer, and the previous internal parameter calibration results and external parameter calibration results are read. Then, the platform IMU data and stereo camera image data are merged to obtain positioning results in the left camera coordinate system. Based on the detection principle of the stereo camera, damage information is detected in the left camera coordinate system to generate a 3D point cloud for the current frame image. The positioning results and the 3D point cloud information are merged, and the point clouds for each frame are filtered and superimposed to generate continuous 3D reconstruction results. The damage location detected by the stereo camera is verified based on the 3D reconstruction point cloud. Next, based on the positioning results and the location of the damaged area in the left camera coordinate system, the overall movement path of the underwater mobile platform is planned and transformed into the underwater mobile platform's center of mass coordinate system. A control signal is sent to the communication system of the underwater mobile platform via the communication bus. As the underwater mobile platform moves, local obstacle avoidance is performed based on the 3D information stored in the real-time 3D reconstruction results until the underwater mobile platform moves near the damaged area.

[0030] Attitude observation in the platform IMU coordinate system is The velocity V, translation vector T, and rotation matrix R obtained by integrating the platform IMU data from time k to time k+1 are respectively

number

number

number

number

[0031] The pose observation in the stereo camera coordinate system is Feature points are extracted from the stereo camera image data, and a circular region is constructed with the feature points as the center.

number

number

[0032] 3D point cloud generation in the stereo camera coordinate system is as follows: The above feature points are extracted and matched for the left and right camera images of the same frame of the stereo camera, and the disparity is calculated based on the grayscale squared error algorithm.

number

number

[0033] S4: Plan the trajectory of the drainage system, control the drainage system to drain, and determine the laser position of the laser sensor using the driving IMU data based on the external parameter matrix obtained in S2, thereby realizing 3D reconstruction of the detailed laser sensor data of the damaged area.

[0034] Specifically, based on the location of the damaged area, the movement trajectory of the underwater mobile platform is planned so that the drainage system covers and drains the damaged area to form a dry space, and the laser driving system is used to control the laser sensor to perform three-dimensional scanning in the dry space, and the laser sensor data and driving IMU data are sent to the host computer via the communication system.The host computer obtains the attitude of the laser driving system using the driving IMU data based on the external parameter matrix obtained in S2, converts it to obtain the position of the laser sensor, obtains a fine three-dimensional reconstruction of the laser sensor data based on the laser sensor position and point cloud data, and detects the damage location based on the three-dimensional reconstruction results.

[0035] 3D reconstruction of laser sensor data The laser sensor emits laser pulses at a fixed frequency and receives the reflected light from the receiver to determine the distance, and roughly distinguishes the target material based on the reflection intensity. The distance measurement formula is: L=tc / 2 where L is the target distance, t is the return time, and c is the speed of light. The position and orientation of the laser sensor are predicted using the driving IMU, and a 3D reconstruction result of the laser sensor is obtained based on the rotation matrix R and translation vector T. This enables precise detection of the damage position, with an error of less than 0.2 mm, and can provide highly accurate positioning results to other autonomous repair equipment.

[0036] The above examples are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above examples. Any other changes, modifications, substitutions, combinations and simplifications made without departing from the spirit and principles of the present invention should be considered as equivalent substitutions, and all are intended to fall within the protection scope of the present invention.

[0037] (Addendum) (Appendix 1) A method for 3D reconstruction of underwater damage to marine equipment based on vision and IMU fusion, comprising: The present invention is realized by a 3D reconstruction system for underwater damage, the 3D reconstruction system for underwater damage comprising an underwater mobile platform and a host computer, the underwater mobile platform comprising an underwater mobile platform body, a stereo camera mounted on the underwater mobile platform body, a platform IMU, a laser sensor, a laser driving system, a communication system, and a drainage system, each axis of the laser driving system is provided with a driving IMU, and the communication system is used for communication between the underwater mobile platform and the host computer, 3D reconstruction method for underwater damage of marine equipment S1: Fixing the stereo camera and the platform IMU to the underwater moving platform, respectively, and performing internal parameter calibration of the left and right cameras of the stereo camera and external parameter calibration between the left and right cameras of the stereo camera and the platform IMU in an underwater environment; S2, fixing a laser sensor to the tip of the drive system, and calibrating the external parameter matrix of the drive IMU coordinate system and the laser sensor coordinate system on each axis of the drive system; S3: collecting stereo camera image data and platform IMU data, integrating acceleration data and angular velocity data in the platform IMU data to obtain position and attitude observation in the platform IMU coordinate system, performing position and attitude observation, damage detection and 3D point cloud generation in the stereo camera coordinate system for frames corresponding to the stereo camera image data, fusing the position and attitude observation results and superimposing the continuous 3D point clouds to obtain an underwater 3D reconstructed point cloud for use in verifying the damage detection results, planning an optimal path for the underwater mobile platform based on the position and attitude observation and damage detection results, performing local obstacle avoidance based on the 3D reconstructed point cloud, and controlling the underwater mobile platform to move near the damaged area; S4: Planning the trajectory of a drainage system, draining the damaged area with the drainage system, and determining the laser position of the laser sensor using the driving IMU data based on the external parameter matrix obtained in S2, thereby realizing a precise 3D reconstruction of the laser sensor data of the damaged area.

[0038] (Appendix 2) The S1 is S11: fixing a stereo camera to the front end of the underwater mobile platform body, with the visual direction tilted downward between 10° and 30°, and fixing a platform IMU to the center of the underwater mobile platform body corresponding to the center of mass of the underwater mobile platform; S12. The calibration board and the underwater moving platform are simultaneously placed in the water, so that the calibration board appears in the field of view of the left and right cameras of the stereo camera at the same time; a step of moving an underwater mobile platform so that calibration boards are distributed at each position of the field of view of the left and right cameras of the stereo camera, recording multiple sets of stereo camera image data, transmitting the multiple sets of stereo camera image data to a host computer via a communication system, and using the host computer to perform related calibration calculations, internal parameter calibration of the left and right cameras of the stereo camera, and external parameter calibration between the left and right cameras of the stereo camera and the platform IMU.

[0039] (Appendix 3) In S12, The internal parameter calibration of the left and right cameras of a stereo camera is as follows:

number

number

number

number

number

[0040] (Appendix 4) In step S2, aligning the drive IMU coordinate system and the laser sensor coordinate system on each axis of the drive system is performed by: Based on the positional relationship between the stereo camera, the drainage system, and the laser drive system, a transformation relationship from the underwater mobile platform mass center coordinate system to the drainage system coordinate system and a transformation relationship between the drainage system coordinate system and the laser drive system coordinate system are obtained; The laser point of the laser sensor is controlled to move on a calibration board with known parameters, the laser sensor and the driving IMU are connected via a communication system to acquire data, which is then transmitted to a host computer, where calibration calculations are performed by the host computer to acquire the transformation relationship between the laser driving system coordinate system and the laser sensor coordinate system, 3. A 3D reconstruction method for underwater damage to marine equipment based on vision and IMU fusion as described in Appendix 1, characterized in that four coordinate systems of the laser sensor, the laser drive system, the drainage system, and the center of mass of the underwater moving platform are aligned.

[0041] (Appendix 5) The method for aligning the four coordinate systems of the laser sensor, the laser driving system, the drainage system, and the center of mass of the underwater mobile platform is as follows: 5. The 3D reconstruction method for underwater damage to marine equipment based on vision and IMU fusion described in Appendix 4, characterized in that an external parameter matrix including a rotation matrix and a translation vector of any two coordinate systems of the laser sensor, the laser drive system, the drainage system, and the center of mass of the underwater mobile platform is calibrated.

number

[0042] (Appendix 6) In S3, the attitude observation in the platform IMU coordinate system is The velocity V, translation vector T, and rotation matrix R obtained by integrating the platform IMU data from time k to time k+1 are respectively

number

number

number

number

[0043] (Appendix 7) In S3, the posture observation in the stereo camera coordinate system is Feature points are extracted from the stereo camera image data, and a circular region is constructed with the feature points as the center.

number

number

[0044] (Appendix 8) In S3, the 3D point cloud generation in the stereo camera coordinate system is performed as follows: The above feature points are extracted and matched for the left and right camera images of the same frame of the stereo camera, and the disparity is calculated based on the grayscale squared error algorithm.

number

number

[0045] (Appendix 9) and (c) detecting the damage location based on the 3D reconstruction result of the laser sensor data. The method for 3D reconstruction of underwater damage to marine equipment based on vision and IMU fusion, as described in Appendix 1, further comprises: (i) planning a movement trajectory of the underwater mobile platform based on the location of the damaged area, so that a drainage system can cover and drain the damaged area to form a dry space; (ii) using a laser driving system to control the laser sensor to perform 3D scanning in the dry space; (iii) transmitting the laser sensor data and driving IMU data to a host computer via a communication system; (iv) the host computer obtaining the attitude of the laser driving system using the driving IMU data based on the external parameter matrix obtained in (i) and converting it to obtain the position of the laser sensor; (v) obtaining a fine 3D reconstruction of the laser sensor data based on the position of the laser sensor and the point cloud data; and (vi) detecting the damage location based on the 3D reconstruction result.

[0046] (Appendix 10) In S4, the three-dimensional reconstruction of the laser sensor data is performed by: The laser sensor emits laser pulses at a fixed frequency and receives the reflected light from the receiver to determine the distance, and roughly distinguishes the target material based on the reflection intensity. The distance measurement formula is: L=tc / 2 where L is the target distance, t is the return time, and c is the speed of light. 10. The 3D reconstruction method for underwater damage to marine equipment based on fusion of vision and IMU described in Appendix 9, characterized in that: a driving IMU is used to predict the position and orientation of the laser sensor; and a 3D reconstruction result of the laser sensor is obtained based on a rotation matrix R and a translation vector T.

Claims

1. A method for three-dimensional reconstruction of underwater damage to marine equipment based on vision and IMU fusion, comprising: The present invention is realized by a three-dimensional reconstruction system for underwater damage, the three-dimensional reconstruction system for underwater damage comprising an underwater mobile platform and a host computer, the underwater mobile platform comprising an underwater mobile platform body, a stereo camera mounted on the underwater mobile platform body, a platform IMU, a laser sensor, a laser driving system, a communication system, and a drainage system, a driving IMU is provided on each axis of the laser driving system, and the communication system is used for communication between the underwater mobile platform and the host computer, A three-dimensional reconstruction method for underwater damage of marine equipment is S1: Fixing the stereo camera and the platform IMU to the underwater moving platform, respectively, and performing internal parameter calibration of the left and right cameras of the stereo camera and external parameter calibration between the left and right cameras of the stereo camera and the platform IMU in an underwater environment; S2: Fixing a laser sensor to the tip of the drive system, and calibrating the external parameter matrix of the drive IMU coordinate system and the laser sensor coordinate system on each axis of the drive system; S3: collecting stereo camera image data and platform IMU data, integrating acceleration data and angular velocity data in the platform IMU data to obtain position and attitude observation in the platform IMU coordinate system, performing position and attitude observation, damage detection, and 3D point cloud generation in the stereo camera coordinate system for frames corresponding to the stereo camera image data, respectively, fusing the position and attitude observation results, and superimposing the continuous 3D point clouds to obtain an underwater 3D reconstructed point cloud, which is used to verify the damage detection results, planning an optimal path for the underwater mobile platform based on the position and attitude observation and damage detection results, performing local obstacle avoidance based on the 3D reconstructed point cloud, and controlling the underwater mobile platform to move near the damaged area; S4: Planning the trajectory of a drainage system, draining the damaged area with the drainage system, and determining the laser position of the laser sensor using the driving IMU data based on the external parameter matrix obtained in S2, thereby realizing a 3D reconstruction of the fine laser sensor data of the damaged area.

2. The S1 is S11: fixing a stereo camera to the front end of the underwater mobile platform body, with the visual direction tilted downward between 10° and 30°, and fixing a platform IMU to the center of the underwater mobile platform body corresponding to the center of mass of the underwater mobile platform; S12. The calibration board and the underwater moving platform are simultaneously placed in the water, so that the calibration board appears in the field of view of the left and right cameras of the stereo camera at the same time; 2. The method for 3D reconstruction of underwater damage to marine equipment based on fusion of vision and IMU according to claim 1, further comprising the steps of: moving the underwater mobile platform so that calibration boards are distributed at each position of the field of view of the left and right cameras of the stereo camera; recording multiple sets of stereo camera image data; transmitting the multiple sets of stereo camera image data to a host computer through a communication system; and using the host computer to perform related calibration calculations, internal parameter calibration of the left and right cameras of the stereo camera, and external parameter calibration of the left and right cameras of the stereo camera and the platform IMU.

3. In S12, The internal parameter calibration of the left and right cameras of a stereo camera is as follows: [Equation 1] (where l represents the left camera, r represents the right camera, and K ｌ , K ｒ represent the intrinsic parameter matrices of the left and right cameras, respectively, and f ｘｌ , f ｙｌ , f ｘｒ , f ｙｒ represent the focal lengths of the left and right cameras in the x-axis and y-axis directions, respectively, in pixels, and (u ０ｌ , v ｏｌ ), (u ０ｒ , v ０ｒ ) represent the actual pixel coordinates of the principal point in the image plane coordinate systems of the left and right cameras, respectively. and The external parameter calibration between the left and right cameras of the stereo camera and the platform IMU is as follows: If the platform IMU coordinate system is the world coordinate system, the transformation relationship from the image points of the left and right cameras of the stereo camera to the platform IMU coordinate system is as follows: [Equation 2] [Equation 3] (In the formula, [Equation 4] are two-dimensional coordinates in the left and right camera coordinate systems, respectively, [Equation 5] is the three-dimensional coordinate in the platform IMU coordinate system, and R ｌｒ , R ｒｉ are 3*3 rotation matrices from the right camera to the left camera and from the left camera to the platform IMU coordinate system, respectively, and T ｌｒ , T ｒｉ are 1*3 translation vectors from the right camera to the left camera and from the left camera to the platform IMU coordinate system, respectively.

4. In step S2, the alignment of the driving IMU coordinate system and the laser sensor coordinate system on each axis of the drive system is performed by: Based on the positional relationship between the stereo camera, the drainage system, and the laser drive system, a transformation relationship from the underwater mobile platform mass center coordinate system to the drainage system coordinate system and a transformation relationship between the drainage system coordinate system and the laser drive system coordinate system are obtained; The laser point of the laser sensor is controlled to move on a calibration board with known parameters, the laser sensor and the driving IMU are connected via a communication system to acquire data, which is then transmitted to a host computer, and the host computer performs calibration calculations to acquire the transformation relationship between the laser driving system coordinate system and the laser sensor coordinate system. The method for 3D reconstruction of underwater damage to marine equipment based on fusion of vision and IMU according to claim 1, characterized in that four coordinate systems of the laser sensor, the laser drive system, the displacement system, and the center of mass of the underwater moving platform are aligned.

5. The method for aligning the four coordinate systems of the laser sensor, the laser driving system, the drainage system, and the center of mass of the underwater mobile platform is as follows: The 3D reconstruction method for underwater damage to marine equipment based on fusion of vision and IMU according to claim 4, characterized in that an external parameter matrix including a rotation matrix and a translation vector of any two coordinate systems of the laser sensor, the laser drive system, the displacement system, and the center of mass of the underwater mobile platform is calibrated. [Equation 6] (where A and B represent two coordinate systems, respectively, X represents a 4*4 extrinsic parameter matrix, R represents a 3*3 rotation matrix, and T represents a 1*3 translation vector)

6. In the step S3, the attitude observation in the platform IMU coordinate system is The velocity V, translation vector T, and rotation matrix R obtained by integrating the platform IMU data from time k to time k+1 are respectively [Equation 7] [Equation 8] [Equation 9] (In the formula, V ｋ , V ｋ＋１ are the velocities at time k and time k+1, respectively, a is the acceleration, Δt is the time interval, and T ｋ , T ｋ＋１ are translation vectors at time k and time k+1, respectively, and R ｋ , R ｋ＋１ are rotation matrices at time k and time k+1, respectively, ω is the angular velocity, [Equation 10] 2. The method for 3D reconstruction of underwater damage to marine equipment based on fusion of vision and IMU according to claim 1, wherein:

7. In S3, the posture observation in the stereo camera coordinate system is Feature points are extracted from the stereo camera image data, and a circular region is constructed with the feature points as the center. [0011] θ=arctan(m ０１ / m １０ ) [where C represents the center of mass of the circular region, θ represents the direction vector of the feature point, and m ｐｑ represents the moment of the circular region, [0012] (where R represents the radius of the circular region, x and y represent the x-axis and y-axis coordinates, and I(x, y) represents the grayscale equation.) The 3D reconstruction method for underwater damage to marine equipment based on fusion of vision and IMU according to claim 6, characterized in that feature points from consecutive frames of stereo camera image data are extracted and matched, and a PnP problem is set using the matched pixel points to obtain a rotation matrix R and a translation vector T of the stereo camera.

8. In S3, the three-dimensional point cloud in the stereo camera coordinate system is generated as follows: The above feature points are extracted and matched for the left and right camera images of the same frame of the stereo camera, and the disparity is calculated based on the grayscale squared error algorithm. [0013] (where x, y, d are the x-axis coordinate, y-axis coordinate, and parallax, respectively; i and j are the change values ​​in the x-axis and y-axis directions, respectively; m and n are the maximum values ​​in the x-axis and y-axis directions, respectively; and I １ (x, y), I ２ (x, y) represents the grayscale equation) Three-dimensional point cloud data is generated based on the parallax and the original coordinates, and the three-dimensional coordinates are [0014] [In the formula, x ｌ , x ｒ are the abscissa values ​​corresponding to the left and right cameras, respectively, and y ｌ , y ｒ are the ordinate values ​​of the left and right cameras, respectively, and f ｘ , f ｙ are the corresponding focal lengths of the internal parameters of the left and right cameras, respectively, X, Y, and Z are the three-dimensional coordinates, and D is the depth value, which is expressed by the following formula: D = Bf / d (where B is the baseline length, f is the focal length of the camera, and d is the disparity between the left and right images) is calculated.

9. 2. The method for 3D reconstruction of underwater damage to marine equipment based on fusion of vision and IMU according to claim 1, wherein S4 plans a movement trajectory of the underwater mobile platform based on the location of the damaged area so that a drainage system can cover and drain the damaged area to form a dry space, controls the laser sensor using a laser driving system to perform 3D scanning in the dry space, and transmits the laser sensor data and driving IMU data to a host computer via a communication system. The host computer obtains the attitude of the laser driving system using the driving IMU data based on the external parameter matrix obtained in S2, and converts it to obtain the position of the laser sensor, obtains a fine 3D reconstruction of the laser sensor data based on the position of the laser sensor and the point cloud data, and detects the damage location based on the 3D reconstruction result.

10. In step S4, the three-dimensional reconstruction of the laser sensor data is performed as follows: The laser sensor emits laser pulses at a fixed frequency and receives the reflected light from the receiver to determine the distance, and roughly distinguishes the target material based on the reflection intensity. The distance measurement formula is: L = tc / 2 where L is the target distance, t is the return time, and c is the speed of light. The 3D reconstruction method for underwater damage to marine equipment based on fusion of vision and IMU according to claim 9, characterized in that the position and orientation of the laser sensor are predicted using a driving IMU, and a 3D reconstruction result of the laser sensor is obtained based on a rotation matrix R and a translation vector T.

Images