An underwater RGB-D three-dimensional reconstruction system and method

Abstract

The application belongs to the field of machine vision, and particularly relates to an underwater RGB-D three-dimensional reconstruction system and method, which comprises an underwater target fine three-dimensional perception system, a three-dimensional perception data processing system and a data acquisition unit; the underwater target fine three-dimensional perception system is used for receiving underwater target scene data collected by the data acquisition unit and generating underwater target three-dimensional perception data packets; the three-dimensional perception data processing system is used for analyzing the underwater target three-dimensional perception data packets sent by the underwater target fine three-dimensional perception system and generating RGB three-dimensional point cloud of the underwater target scene; and the data acquisition unit is used for collecting underwater target scene data and sending the data to the underwater target fine three-dimensional perception system. The application provides color information for three-dimensional point cloud, and can obtain more real and more accurate underwater three-dimensional point cloud data.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of machine vision, in particular to an underwater RGB-D three-dimensional reconstruction system and method. BACKGROUND

[0002] With the increasing exploration, development and related environmental protection of deep-sea seabed resources, it is urgent to improve the fine perception measurement capability of seabed topography; the fine perception measurement technology of seabed topography can also be used for the detection of seabed targets such as sunken ships, seabed pipelines and cables.

[0003] Underwater three-dimensional reconstruction technology provides a more accurate and fine means for human exploration of the ocean and more intuitive analysis of the ocean world. However, underwater three-dimensional reconstruction is often accompanied by the problem of missing color information, and color information is the most important information for human perception of the environment. How to assign color information to underwater three-dimensional point cloud has become the key to the application and promotion of underwater three-dimensional reconstruction technology. The existing underwater three-dimensional reconstruction method cannot obtain more intuitive color information, and can only compare with two-dimensional RGB pictures to have a certain perception of the actual situation of three-dimensional point cloud, which causes the loss of underwater three-dimensional scanning information, and thus seriously reduces the resolution capability of underwater three-dimensional reconstruction technology. SUMMARY

[0004] The application aims to provide an underwater RGB-D three-dimensional reconstruction system and three-dimensional reconstruction method capable of obtaining more realistic and accurate underwater three-dimensional point cloud data to obtain three-dimensional point cloud data, so as to overcome the defects of the existing underwater three-dimensional reconstruction method.

[0005] The technical scheme adopted by the application to achieve the above-mentioned purpose is: an underwater RGB three-dimensional reconstruction system, comprising: an underwater target fine three-dimensional perception system, a three-dimensional perception data processing system and a data acquisition unit;

[0006] The underwater target fine three-dimensional perception system is used for receiving underwater target scene data collected by the data acquisition unit and generating underwater target three-dimensional perception data packets;

[0007] The three-dimensional perception data processing system is used for analyzing the underwater target three-dimensional perception data packets sent by the underwater target fine three-dimensional perception system and generating RGB three-dimensional point clouds of the underwater target scene;

[0008] The data acquisition unit is used for collecting underwater target scene data and sending it to the underwater target fine three-dimensional perception system.

[0009] The data acquisition unit comprises: a point cloud reconstruction camera, a picture acquisition camera and an underwater turntable connected with the underwater target fine three-dimensional perception system;

[0010] The point cloud reconstruction camera is a black-and-white camera, which is used to acquire original point cloud data of the underwater target scene and store the original point cloud data in the form of single-frame data, and calibrate the point cloud reconstruction camera to acquire calibration data of the point cloud reconstruction camera; and transmit the original point cloud data and the calibration data of the point cloud reconstruction camera to the underwater target fine three-dimensional perception system.

[0011] The picture acquisition camera is a color camera, which is used to acquire underwater RGB images of the underwater target scene, and calibrate the picture acquisition camera to acquire calibration data of the picture acquisition camera; and transmit the underwater RGB images and the calibration data of the point cloud reconstruction camera to the underwater target fine three-dimensional perception system.

[0012] The underwater turntable is used to acquire turntable position parameters of positions corresponding to the output original point cloud data and the underwater RGB images, and transmit the turntable position parameters to the underwater target fine three-dimensional perception system.

[0013] The point cloud reconstruction camera and the picture acquisition camera acquire the underwater target scene according to a set frame rate, and the point cloud reconstruction camera and the picture acquisition camera are calibrated before acquiring the underwater target scene.

[0014] The underwater target scene data includes underwater point cloud, underwater scene RGB image, turntable position parameter, and calibration file including calibration data of the point cloud reconstruction camera and the picture acquisition camera.

[0015] An underwater RGB-D three-dimensional reconstruction method, comprising the following steps:

[0016] 1) calibrate the data acquisition unit, and transmit the calibration data;

[0017] 2) the data acquisition unit acquires underwater target scene data of the underwater target scene, and transmits the underwater target scene data to the data underwater target fine three-dimensional perception system; the underwater target fine three-dimensional perception system generates underwater target three-dimensional perception data packets from the underwater target scene data, and transmits the underwater target three-dimensional perception data packets to the three-dimensional perception data processing system;

[0018] 3) the three-dimensional perception data processing system analyzes and processes the data packets offline to generate RGB three-dimensional point cloud data.

[0019] The step 1) is specifically:

[0020] The calibration of the data acquisition unit includes internal participant distortion calibration of the picture acquisition camera and external parameter calibration of the picture acquisition camera and the point cloud reconstruction camera.

[0021] 1-1) The internal participant distortion calibration of the picture acquisition camera is specifically:

[0022] The underwater RGB-D three-dimensional reconstruction system is submerged in water, and the calibration board images in different postures are continuously collected by the picture collection camera;

[0023] The corner points of the calibration board in the collected picture are extracted, and the Zhang Zhengyou calibration method is used to calibrate the internal parameters M of the picture collection camera c and distortion D c .

[0024] 1-2) The picture collection camera and the picture collection camera external parameter calibration, specifically:

[0025] The underwater RGB-D three-dimensional reconstruction system is placed in the air and fixed; the front hatch of the camera sealing cabin is removed;

[0026] The same calibration board images are collected by the point cloud reconstruction camera and the picture collection camera respectively, and the positions of the calibration board images collected by the two cameras are ensured to be unchanged; the calibration board is moved, and the point cloud reconstruction camera and the picture collection camera are used to collect multiple groups of pictures respectively;

[0027] According to the size and the number of corner points of the calibration board, the double-camera external parameter calibration is completed by the Zhang Zhengyou calibration method, that is, the camera external parameters R o and T o are obtained.

[0028] The step 3) comprises the following steps:

[0029] S1: The three-dimensional perception data processing system reads the calibration file, and the internal parameters of the picture collection camera are recorded as M c , the double-camera external parameters are recorded as M o , and the external parameters between the point cloud reconstruction camera and the underwater turntable are recorded as M p .

[0030] S2: The three-dimensional perception data processing system reads the data packet, extracts the turntable position parameters and the data time stamp corresponding to the set frame rate ratio, and constructs the original point cloud data queue A and the RGB image data queue B;

[0031] S3: The original point cloud data queue A and the RGB image data queue B are arranged in ascending order according to the data time stamp;

[0032] S4: The rotation matrix M α corresponding to the point cloud PI in a point cloud data pair in the original point cloud data queue A is obtained, and the rotation matrix M β corresponding to the RGB image S adjacent in position in the RGB image data queue B is obtained.

[0033] S5: read a point P[X, Y, Z, 1] from the point cloud Pl, transform the coordinate point from the point cloud reconstruction camera coordinate system to the picture acquisition camera coordinate system through the coordinate transformation formula to obtain a coordinate point [X', Y', Z', 1], and convert the coordinate point [X', Y', Z', 1] from the alpha position to the beta position;

[0034] S6: map the three-dimensional point to the RGB image S through the picture acquisition camera internal parameter to obtain a two-dimensional point p[u, v] corresponding to the three-dimensional point in S;

[0035] S7: generate a point cloud with color information according to the two-dimensional point p[u, v] corresponding to the three-dimensional point in S;

[0036] S8: traverse the original data queue A, and repeat the steps of S3-S5 to generate an RGB point cloud.

[0037] The step S4 comprises the following steps:

[0038] (1) read a point cloud data pair from the original point cloud data queue A, read the point cloud according to the data full path, and record the point cloud as Pl, the underwater turntable angle corresponding to the point cloud Pl is recorded as alpha, and the rotation matrix M corresponding to the point cloud Pl is recorded as M α :

[0039]

[0040] (2) according to the underwater turntable angle alpha of the point cloud and the time stamp, select the RGB image adjacent to the point cloud Pl in the RGB image data queue B, record the RGB image as S, the corresponding turntable angle is recorded as beta, and the rotation matrix M corresponding to the RGB image S is recorded as M β :

[0041]

[0042] The step S5 is specifically:

[0043] read a point P[X, Y, Z, 1] from the point cloud Pl, transform the coordinate point from the point cloud reconstruction camera coordinate system to the picture acquisition camera coordinate system through the coordinate transformation formula to obtain a coordinate point [X', Y', Z', 1], and convert the coordinate point [X', Y', Z', 1] from the alpha position to the beta position, i.e.

[0044] [X′,Y′,Z′,1] T =M0·[X,Y,Z,1] T .

[0045] Use the external parameter between the turntable and the camera, and the angle position of the point cloud and the RGB image to convert [X', Y', Z', 1] from the alpha position to the beta position, i.e. obtain the point coordinates [X'', Y'', Z'', 1] of the point cloud in the beta position picture acquisition camera coordinate system:

[0046]

[0047] The three-dimensional point corresponds to a two-dimensional point p[u,v] in S, that is:

[0048]

[0049]

[0050] Wherein, X'', Y'', Z'' are point coordinates of the point cloud in the picture acquisition camera coordinate system at the beta position, (u0, v0) is the pixel coordinate of the picture acquisition camera lens optical center.

[0051] The step S9 is specifically:

[0052] The RGB color of the corresponding two-dimensional point p[u,v] in the RGB image S is given to the point P[X,Y,Z,1], and the point P is mapped to the turntable coordinate system through M p And M α Mapping to the turntable coordinate system, that is:

[0053] [X''', Y''', Z''', 1] T =·M α ·M P ·[X,Y,Z,1] T

[0054] The point cloud with color information is: [X''', Y''', Z''', R, G, B].

[0055] The present application has the following beneficial effects and advantages:

[0056] 1. The present application can obtain underwater three-dimensional point cloud data in real time.

[0057] 2. The present application provides a stable and reliable calibration method for an underwater optical system.

[0058] 3. The present application provides color information for three-dimensional point cloud, and can obtain more real and more accurate underwater three-dimensional point cloud data. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 System architecture diagram of the present application;

[0060] Figure 2 System operation flowchart of the present application;

[0061] Figure 3 System calibration flowchart of the present application;

[0062] Figure 4 Underwater target scene data acquisition process schematic diagram of the present application;

[0063] Figure 5 The structural schematic diagram of the data acquisition unit of the application;

[0064] Wherein, 1 is a picture acquisition camera, 2 is a point cloud reconstruction camera, 3 is a floodlight, 4 is an embedded processor, 5 is a line laser, 6 is a rigid horizontal rod, and 7 is an underwater turntable. DETAILED DESCRIPTION

[0065] The application will be further described in detail below in combination with the drawings and embodiments.

[0066] The application comprises an underwater target fine three-dimensional perception system, a three-dimensional perception data processing system and a data acquisition unit, and the basic architecture is as shown in the figure. Figure 1

[0067] The underwater target fine three-dimensional perception system is used for receiving the underwater target scene data collected by the data acquisition unit and generating an underwater target three-dimensional perception data packet.

[0068] The three-dimensional perception data processing system is used for analyzing the underwater target three-dimensional perception data packet sent by the underwater target fine three-dimensional perception system and generating an RGB three-dimensional point cloud of the underwater target scene.

[0069] The data acquisition unit is used for collecting underwater target scene data and sending the data to the underwater target fine three-dimensional perception system. The underwater target scene data comprises underwater point cloud, underwater scene RGB image, turntable position parameter and calibration file containing calibration data of the point cloud reconstruction camera and the picture acquisition camera.

[0070] It comprises a point cloud reconstruction camera, a picture acquisition camera and an underwater turntable connected with the underwater target fine three-dimensional perception system.

[0071] The point cloud reconstruction camera is a black-and-white camera, which is used for acquiring original point cloud data of the underwater target scene and storing the data in the form of single frame data. Meanwhile, the point cloud reconstruction camera is calibrated to acquire calibration data of the point cloud reconstruction camera. The original point cloud data and the calibration data of the point cloud reconstruction camera are sent to the underwater target fine three-dimensional perception system.

[0072] The picture acquisition camera is a color camera, which is used for acquiring underwater RGB image of the underwater target scene. Meanwhile, the picture acquisition camera is calibrated to acquire calibration data of the picture acquisition camera. The underwater RGB image and the calibration data of the point cloud reconstruction camera are sent to the underwater target fine three-dimensional perception system.

[0073] The underwater turntable is used for acquiring turntable position parameter containing the position corresponding to the output original point cloud data and the underwater RGB image and sending the parameter to the underwater target fine three-dimensional perception system.

[0074] ​The point cloud reconstruction camera and the picture acquisition camera are used to collect the underwater target scene according to a set frame rate, and before the underwater target scene is collected, the point cloud reconstruction camera and the picture acquisition camera are calibrated.

[0075] The underwater target fine three-dimensional perception system calibrates the point cloud reconstruction camera and the picture acquisition camera, the principle steps of calibration and the hardware structure of the data acquisition unit are known technologies, which are disclosed in the invention authorized patent with the application number 202110162626.2, the applicant is Shenyang Institute of Automation, Chinese Academy of Sciences, and the name is a kind of underwater target fine three-dimensional perception method, the specific principle steps of calibration of the underwater target fine three-dimensional perception system and the structure of the data acquisition unit of the present application are disclosed in detail;

[0076] The specific structure is as follows:

[0077] The hardware system of the present application is as shown in Figure 5 The picture acquisition camera 1, the point cloud reconstruction camera 2, the floodlight 3, the embedded processor 4 and the line laser 5 are fixed on the rigid horizontal rod 6 in sequence; the rigid horizontal rod is fixed on the underwater turntable 7; the relative position of the point cloud reconstruction camera 2 and the line laser 5 is fixed.

[0078] The picture acquisition camera 1 is a color camera, which is placed in the underwater sealed cabin and fixed on the rigid horizontal rod 6 through the point cloud reconstruction camera 2. The point cloud reconstruction camera 2 is a black and white camera, which is placed in the underwater sealed cabin and fixed on the rigid horizontal rod 6. If it is placed in the outdoor shallow water or other environment with bright light, a band-pass interference filter can be added in front of the camera. Considering the blue shift effect of the band-pass interference filter, the short-wave cutoff frequency of the passband should be determined according to the field of view of the point cloud reconstruction camera. The picture acquisition camera 1 and the point cloud reconstruction camera 2 are connected with the embedded processor 4 through a group of gigabit network lines, trigger lines and power lines, the embedded processor 4 controls the picture acquisition camera 1 and the point cloud reconstruction camera 2 through the gigabit network line, and transmits the collected picture data, the trigger line transmits the trigger signal synchronously emitted by the picture acquisition camera 1 and the point cloud reconstruction camera 2 to the circuit board where the embedded processor 4 is located, so as to trigger the floodlight 3 and the line laser 5 respectively.

[0079] The floodlight 3 is a white LED array capable of emitting high-brightness flash, and the line laser 5 is a green or blue high-power line laser emitter, both of which are underwater sealed and fixed on the rigid crossbar 6. The floodlight 3 and the line laser 5 are connected with the embedded processor 4 through a group of serial lines and power lines respectively, the embedded processor 4 configures the brightness parameters of the floodlight 3 and the line laser 5 through the serial lines, and the circuit board where the embedded processor 4 is located connects the trigger lines generated by the picture acquisition camera 1 and the point cloud reconstruction camera 2 to the trigger input ports of the floodlight 3 and the line laser 5 respectively. The floodlight 3 uniformly illuminates the front area covering the field of view when the picture acquisition camera 1 is exposed; and the line laser 5 projects a vertical line laser in the field of view when the point cloud reconstruction camera 2 is exposed.

[0080] The underwater turntable 7 is fixed at the bottom of the rigid crossbar 6 and has been underwater sealed. The underwater turntable 7 is connected with the embedded processor 4 through a group of serial lines and power lines, the embedded processor 4 controls the rotation of the underwater turntable 7 through the serial lines, and the underwater turntable 7 drives the whole device to rotate through the rigid crossbar.

[0081] The embedded processor 4 is placed in the underwater sealed cabin, fixed on the rigid crossbar 6, and connected to the water surface through the transmission cable for power supply and offline data transmission. The embedded processor 4 is used to control other underwater devices, and to process and store data in real time.

[0082] As shown in Figure 2 , the present application provides an underwater RGB-D three-dimensional reconstruction method, mainly including three parts:

[0083] Part1: Calibration of underwater target fine three-dimensional perception system, that is, calibration of point cloud reconstruction camera and picture acquisition camera in data acquisition unit;

[0084] Part2: Collecting three-dimensional perception data packets by using the underwater target fine three-dimensional perception system;

[0085] Part3: Offline analysis and processing of data packets and generation of RGB three-dimensional point cloud data.

[0086] The calibration of the underwater target fine three-dimensional perception system (Part1) in the present application includes two aspects of internal distortion calibration of the picture acquisition camera (S11) and external parameter calibration of the picture acquisition camera and the point cloud reconstruction camera (S12), as shown in Figure 3 , and the specific calibration process is as follows:

[0087] S111: Submerge the system in water and fix it;

[0088] S112: Use the picture acquisition camera to continuously collect images of the calibration board under different postures;

[0089] S113: Extract the corner points of the calibration board in the collected picture, and use Zhang Zhengyou's calibration method to calibrate the camera internal parameter Mc and the distortion Dc of the picture collection camera.

[0090] S121: Place the system in the air and fix it.

[0091] S122: Remove the front hatch of the camera sealed cabin.

[0092] S123: Collect the same calibration board image using the point cloud reconstruction camera and the picture collection camera respectively, and ensure that the positions of the calibration board images collected by the two cameras are unchanged.

[0093] S124: Move the calibration board and repeat S3 to collect multiple groups of pictures.

[0094] S125: According to the size and corner point number of the calibration board, complete the double-camera external parameter calibration using Zhang Zhengyou's calibration method, and obtain the camera external parameters Ro and To.

[0095] After the system calibrates the data collection unit, the underwater target fine three-dimensional perception data is collected using the data collection unit, and the data packet mainly includes underwater point cloud, underwater RGB image and turntable position parameter data. The underwater point cloud is the original point cloud data output by the underwater target fine three-dimensional perception system; the underwater RGB image is the scene RGB image collected by the picture collection camera; and the turntable position parameter is the position corresponding to the point cloud and image output by the underwater turntable.

[0096] The original point cloud data is generated by the point cloud reconstruction camera and stored in the form of single-frame data. When the underwater target fine three-dimensional perception system collects the data packet, the original point cloud data and the scene RGB image in the data packet are collected according to the preset frame rate ratio, and the preset frame rate ratio is generated by software configuration and data collection timing, and the point cloud reconstruction camera and the picture collection camera are controlled by the hardware trigger module according to the data collection timing.

[0097] After the data collection unit completes the data collection, the host computer end (three-dimensional perception data processing system) downloads the original data from the device end (underwater target fine three-dimensional perception system) through the data communication port, and obtains the data packet after data decompression and data decoding.

[0098] As shown in Figure 4 , the specific steps of the host computer end (three-dimensional perception data processing system) offline processing data packet and integrating to generate RGB three-dimensional point cloud data (Part 3) are as follows:

[0099] S1: Read the calibration file, and record the camera internal parameter as M c , the double-camera external parameter as M o , and the external parameter between the point cloud reconstruction camera and the turntable as Mp wherein:

[0100]

[0101]

[0102]

[0103] S2: read the data packet, extract the full path of the data, the turntable position corresponding to the data and the data timestamp, and construct the point cloud data queue A and the RGB image data queue B in the system memory;

[0104] S3: arrange in ascending order according to the data timestamp;

[0105] S4: read and extract a point cloud data pair from the point cloud data queue A, and read the point cloud according to the data full path, denoted as point cloud Pl, the corresponding turntable angle denoted as a, and the corresponding rotation matrix M α :

[0106]

[0107] S5: select the RGB image adjacent in position from the RGB image data queue B according to the turntable angle a and the timestamp of the point cloud, denoted as S, the corresponding turntable angle denoted as β, and the corresponding rotation matrix M β :

[0108]

[0109] S6: read a point P[X, Y, Z, 1] from the point cloud Pl, and transform the coordinate point from the point cloud reconstruction camera coordinate system to the picture acquisition camera coordinate system by the following coordinate transformation formula:

[0110] [X', Y', Z', 1] T = M0·[X, Y, Z, 1] T

[0111] S7: convert [X', Y', Z', 1] from a position to β position by using the external parameters between the turntable and the camera, and the angle position of the point cloud and the RGB image:

[0112]

[0113] S8: map the three-dimensional point to the RGB image S by using the picture acquisition camera intrinsic parameters to obtain the two-dimensional point p[u, v] corresponding to the three-dimensional point in S:

[0114]

[0115]

[0116] S9: RGB color of the corresponding two-dimensional point p[u,v] in the RGB image S is assigned to the point P[X,Y,Z,1], and the point P is mapped to the turntable coordinate system through M p and M α Mapping to the turntable coordinate system:

[0117] [X″′,Y″′,Z″′,1] T =·M α ·M P ·[X,Y,Z,1] T

[0118] Finally, a point cloud with color information [X″′,Y″′,Z″′,R,G,B] is formed.

[0119] S10: Traverse the data queue A, repeat the steps of S3-S7 to generate a complete RGB point cloud.

Claims

1. An underwater RGB three-dimensional reconstruction system, characterized in that, include: Underwater target fine three-dimensional perception system, three-dimensional perception data processing system and data acquisition unit; The underwater target fine three-dimensional perception system is used to receive underwater target scene data collected by the data acquisition unit and generate underwater target three-dimensional perception data packets. The three-dimensional perception data processing system is used to parse the underwater target three-dimensional perception data packets sent by the underwater target fine three-dimensional perception system and generate RGB three-dimensional point clouds of the underwater target scene. S1: The 3D perception data processing system reads the calibration file and records the intrinsic parameters of the image acquisition camera as M. c The dual-camera extrinsic parameters are denoted as M. o The extrinsic parameters between the point cloud reconstruction camera and the underwater turntable are denoted as M. p ; S2: The 3D perception data processing system reads data packets, extracts turntable position parameters and sets the data timestamps corresponding to the frame rate ratio, and constructs the original point cloud data queue A and the RGB image data queue B. S3: Sort the original point cloud data queue A and RGB image data queue B in ascending order according to the data timestamp; S4: Obtain the rotation matrix M corresponding to the point cloud PI in a point cloud data pair from the raw point cloud data queue A. α Simultaneously, obtain the rotation matrix M corresponding to the RGB image S located adjacent to it in the RGB image data queue B. β ; (1) Read and extract a point cloud data pair from the original point cloud data queue A, and read the point cloud according to the full data path, denoted as point cloud Pl, the corresponding underwater turntable angle is denoted as α, and the rotation matrix M corresponding to point cloud Pl is... α : (2) Based on the underwater turntable angle α and the timestamp of the point cloud, select the RGB image of the point cloud Pl that is adjacent to it in the RGB image data queue B, denoted as S, and the corresponding turntable angle is denoted as β. The rotation matrix M corresponding to the RGB image S is... β : S5: Read a point P[X,Y,Z,1] from the point cloud Pl, and transform the coordinate point from the point cloud reconstruction camera coordinate system to the image acquisition camera coordinate system [X',Y',Z',1] using the coordinate transformation formula. Transform the coordinate point [X',Y',Z',1] from position α to position β. Read a point P[X,Y,Z,1] from the point cloud Pl, and transform the coordinates of the point from the point cloud reconstruction camera coordinate system to the image acquisition camera coordinate system using the coordinate transformation formula, that is: [X′,Y′,Z′,1] T =M0·[X,Y,Z,1] T Using the extrinsic parameters between the turntable and the camera, as well as the angular positions of the point cloud and the RGB image, [X',Y',Z',1] is transformed from position α to position β, thus obtaining the point coordinates [X″,Y″,Z″,1] of the point cloud in the coordinate system of the image acquisition camera at position β: S6: Map the 3D points to the RGB image S by acquiring the camera intrinsic parameters of the image, and obtain the 2D point p[u,v] corresponding to the 3D point in S; S7: Generate a point cloud with color information based on the two-dimensional point p[u,v] corresponding to the three-dimensional point in S; The two-dimensional point p[u,v] corresponding to the three-dimensional point in S is: Where X″, Y″, Z″ are the point coordinates of the point cloud in the image acquisition camera coordinate system when it is at position β, and (u0, v0) are the pixel coordinates of the optical center of the image acquisition camera lens. S8: Traverse the original data queue A, repeat steps S3 to S5, and generate RGB point clouds; Assign the RGB color of the corresponding two-dimensional point p[u,v] in the RGB image S to point P[X,Y,Z,1]. At the same time, pass point P through M. p With M α Mapping to the turntable coordinate system, i.e.: [X″′,Y″′,Z″′,1] T =·M α ·M P ·[X,Y,Z,1] T The point cloud with color information is generated as: [X″′,Y″′,Z″′,R,G,B]; The data acquisition unit is used to collect underwater target scene data and send it to the underwater target fine three-dimensional perception system.

2. The underwater RGB three-dimensional reconstruction system according to claim 1, characterized in that, The data acquisition unit includes: a point cloud reconstruction camera, an image acquisition camera, and an underwater turntable connected to the underwater target fine three-dimensional perception system; The point cloud reconstruction camera is a monochrome camera used to acquire raw point cloud data of underwater target scenes and store it in the form of single-frame data. At the same time, the point cloud reconstruction camera is calibrated to acquire calibration data. The raw point cloud data and the calibration data of the point cloud reconstruction camera are sent to the underwater target fine 3D perception system. The image acquisition camera is a color camera used to acquire underwater RGB images of underwater target scenes. At the same time, the image acquisition camera is calibrated to obtain calibration data. The underwater RGB images and the calibration data of the point cloud reconstruction camera are sent to the underwater target fine 3D perception system. The underwater turntable is used to acquire the turntable position parameters, which include the output raw point cloud data and the position corresponding to the underwater RGB image, and send them to the underwater target fine 3D perception system. The point cloud reconstruction camera and the image acquisition camera collect underwater target scenes at a set frame rate ratio, and the point cloud reconstruction camera and the image acquisition camera are calibrated before collecting underwater target scenes.

3. The underwater RGB three-dimensional reconstruction system according to claim 1, characterized in that, The underwater target scene data includes: underwater point clouds, underwater scene RGB images, turntable position parameters, and calibration files containing calibration data for point cloud reconstruction cameras and image acquisition cameras.

4. The three-dimensional reconstruction method of the underwater RGB-D three-dimensional reconstruction system according to claim 1, characterized in that, Includes the following steps: 1) Calibrate the data acquisition unit and record the calibration data; 2) The data acquisition unit collects underwater target scene data and sends it to the data underwater target fine three-dimensional perception system; The underwater target fine 3D perception system generates underwater target 3D perception data packets from underwater target scene data and sends them to the 3D perception data processing system. 3) The 3D perception data processing system analyzes and processes data packets offline and generates RGB 3D point cloud data.

5. The three-dimensional reconstruction method of an underwater RGB-D three-dimensional reconstruction system according to claim 4, characterized in that, Step 1) specifically includes: The calibration of the data acquisition unit includes: internal parameter distortion calibration of the image acquisition camera, and external parameter calibration of the image acquisition camera and the point cloud reconstruction camera. 1-1) The distortion calibration process within the image acquisition camera specifically involves: The underwater RGB-D 3D reconstruction system is submerged in water, and the calibration board images under different postures are continuously acquired by the image acquisition camera. Extract the calibration plate corner points from the acquired images and use Zhang Zhengyou's calibration method to calibrate the intrinsic parameters M of the image acquisition camera. c With distortion D c ; The calibration of the image acquisition camera and its extrinsic parameters as described in 1-2) is as follows: Place and fix the underwater RGB-D 3D reconstruction system in the air; remove the front cover of the camera's sealed compartment; Use both the point cloud reconstruction camera and the image acquisition camera to capture images of the same calibration board, ensuring that the position of the calibration board images captured by the two cameras remains unchanged; move the calibration board and repeatedly use both the point cloud reconstruction camera and the image acquisition camera to capture multiple sets of images. Based on the size and number of corner points of the calibration plate, the dual-camera extrinsic parameters were calibrated using the Zhang Zhengyou calibration method, thus obtaining the camera extrinsic parameters R. o With T o .

Images