A Construction Method of a Dataset for the External Oil Transfer Operation Scenario of a Shuttle Tanker Based on Virtual Simulation

Abstract

A method for constructing a dataset of the external oil transfer operation scenario of a shuttle tanker based on virtual simulation. Simulation of real physical binocular cameras and depth cameras; making a calibration board virtual model, generating corresponding binocular images for virtual camera calibration, and obtaining the internal and external parameter matrices and distortion coefficients of the virtual binocular cameras; constructing a corresponding 3D digital model based on the parameters of the real shuttle tanker and performing texture rendering; obtaining the three-dimensional position, attitude and corresponding virtual binocular images of the shuttle tanker model in real time; constructing a parameterized virtual natural environment system and ocean scene, controlling the movement of the shuttle tanker model in the ocean scene; testing the effectiveness, authenticity and accuracy of the virtual data, and conducting a feasibility evaluation using deep learning algorithms; integrating various environmental conditions, integrating the pose information, binocular camera parameters and virtual binocular images to construct a dataset for the external oil transfer operation scenario monitoring task, providing data support for realizing real-time monitoring and early warning of offshore shuttle tankers.

Description

Technical Field

[0001] The present invention belongs to the field of virtual synthesis simulation, and particularly relates to a method for constructing a dataset of the external oil transfer operation scenario of a shuttle tanker based on virtual simulation. Background Art

[0002] Offshore oil and gas production platforms are important equipment facilities for offshore oil development, capable of preliminarily processing, storing, and transporting energy such as oil and natural gas collected offshore. The production platform transports substances such as oil and natural gas stored inside to a shuttle tanker through an external transfer hose, and the shuttle tanker shuttles between the ocean and the land to transport oil energy. When the production platform and the shuttle tanker perform external transfer operations, the working conditions are complex and the weather environment is harsh. Moreover, the production platform - shuttle tanker is loaded with a large amount of flammable and explosive items, weighing up to tens of thousands of tons, and the distance between them is relatively small compared to the length of the hull itself, making it extremely easy to have dangerous collision accidents. Therefore, during external transfer operations, it is necessary to manually monitor the relative distance and angle information between the production platform and the shuttle tanker in real time outside, so as to timely feedback to the operation end of the shuttle tanker to make corresponding position adjustments. However, each external transfer operation lasts continuously for up to 24 hours without interruption, and it is extremely easy for manual workers to get tired at night, which leads to untimely information feedback and major safety hazards in the external transfer operation process.

[0003] An urgent problem is to use machinery to replace manual labor to monitor the pose information such as the relative distance and relative angle of the production platform - shuttle tanker, and once the shuttle tanker exceeds the set safety range, the system immediately issues a safety alarm of the corresponding level, and transmits the monitoring information and early warning information to the operation end of the shuttle tanker in real time. The operation end timely adjusts the relative posture of the ship's body according to this information to keep it always within a safe range and reduce the risk of collision.

[0004] In recent years, with the improvement of computing resources, deep learning has ushered in rapid development. Powerful neural networks have extremely strong representational power and perform well in image-based target three-dimensional detection tasks. For this task, it is planned to use binocular cameras as sensors to provide data input, and use deep learning algorithms as the core to calculate and estimate binocular image data to obtain the three-dimensional pose information of the shuttle tanker. The effectiveness of the algorithm depends to a great extent on the data set. A high-quality and large data set plays an irreplaceable role in completing this task. The production of data sets for this task is extremely difficult, so there is a lack of three-dimensional detection data sets for this working condition today. The difficulties are reflected in several aspects: First, it is difficult to collect data under extreme working conditions. As mentioned above, the marine environment is complex and changeable, and a lot of manpower, material resources and time are required. Secondly, it is extremely difficult to accurately obtain the supervised true value of the data itself during collection. In the field of autonomous driving, the solution is mostly to rely on laser radar to measure the distance, but the laser radar itself is expensive and the information provided is very sparse. More importantly, the calibration between the laser radar and the camera coordinate system will introduce additional errors. The longer the distance, the greater the error.

[0005] Currently, industries such as animation and games have also achieved unprecedented development, and the pictures can be rendered exactly like real scenes. This method combines virtual simulation technology, deep learning technology and the mining platform-shuttle tanker export operation conditions, and establishes a virtual simulation scene that is twin to the real world for this task condition. At the same time, accurate three-dimensional pose information is output to produce multi-condition, accurate and large-scale data sets. This method can not only break through the various difficulties in making data sets mentioned above, but also simulate complex and diverse weather environments such as rain, snow, and fog. At the same time, the label customization of the entire data set is realized, and the true value information with millimeter-level accuracy is obtained, which has the advantages of convenient and fast scene derivation and unlimited data stream output. Summary of the invention

[0006] The present invention provides a method for constructing a dataset of shuttle tanker offloading operation scenarios based on virtual simulation. Aiming at the problems of complex real data collection and difficult to guarantee the collection accuracy in this environmental condition, a twin virtual body of the real offloading operation environment is virtual simulated, including the construction of main bodies such as virtual binocular cameras, ocean scenes, and shuttle tanker models, and a three-dimensional object detection dataset with multiple environmental conditions, accuracy, and a large amount of data is produced. The idea of this method is as follows: construct a virtual binocular camera model and a depth camera model; calibrate the internal and external parameter matrices and distortion coefficients of the virtual binocular camera; construct and render the three-dimensional model of the shuttle tanker, and obtain its theoretical size in the virtual development engine; obtain information such as the real-time three-dimensional position, attitude, and virtual binocular images of the target; construct a parametric environment system and an ocean scene, and evaluate the effectiveness of the virtual binocular images; integrate the internal and external parameter matrices and distortion coefficients of the binocular camera, the three-dimensional position information of the shuttle tanker model, and the virtual binocular image data to make a high-quality simulation dataset for three-dimensional object detection.

[0007] The technical solution of the present invention is as follows:

[0008] A method for constructing a dataset of shuttle tanker offloading operation scenarios based on virtual simulation is as follows:

[0009] Step 1, construct a binocular camera model and a depth camera model in the virtual development engine according to a real physical camera;

[0010] (1.1) Set the coordinates (x, y, z) and (x’, y’, z’) of the left and right virtual monocular cameras in the world coordinate system respectively. The left and right monocular cameras are set on the same horizontal line in space, and are consistent in the height and depth directions. The horizontal distance is set as b, which is the baseline distance of the binocular camera;

[0011] (1.2) Set the field of view angle F v 、focal length F, length L, and width W of the photosensitive chip of each virtual camera in the binocular camera model;

[0012] (1.3) Copy the left monocular camera model in step (1.1) as the depth camera model, set the depth rendering range of the depth camera model, and represent the depth value between each point in the rendering range of the depth camera coordinate system and the camera plane with pixel values from 0 to 255. The closer the point is to the camera plane, the smaller the pixel value, and a depth map representing the depth by pixel values is obtained;

[0013] Step 2, test the accuracy of the binocular camera simulation, and at the same time obtain accurate internal and external parameters of the virtual camera;

[0014] (2.1) Make a checkerboard calibration plate model in the virtual development engine;

[0015] (2.2) Obtaining images of the checkerboard calibration plate model from the perspective of a binocular camera, and collecting multiple sets of left and right image data pairs;

[0016] (2.3) Calibrate the image data pair obtained in step (2.2) to obtain the sensor internal and external parameter matrix and distortion coefficient of the binocular camera;

[0017] (2.4) Determine the three-dimensional space point corresponding to the checkerboard feature corner point in the image obtained in step (2.2) by using the geometric relationship between the left and right images, and reproject the three-dimensional space point to the image through the internal and external parameter matrix and distortion coefficient of the virtual binocular camera in step (2.3), thereby obtaining a virtual pixel point, and calculate the error between the virtual pixel point and the checkerboard feature corner point. If the error value is within an acceptable range, it indicates the effectiveness of the binocular camera simulation;

[0018] Step 3, constructing and coloring the three-dimensional digital model of the shuttle tanker, importing the three-dimensional digital model of the shuttle tanker into the virtual development engine, and calculating the theoretical size of the three-dimensional digital model of the shuttle tanker in the virtual development engine;

[0019] (3.1) Proportional modeling of the shuttle tanker is performed, and the three-dimensional digital model of the shuttle tanker is mapped and rendered according to the surface color and texture of the real shuttle tanker;

[0020] (3.2) Import the three-dimensional digital model of the shuttle tanker into the virtual development engine, and note that the scale transformation factor between the shuttle tanker modeling software and the virtual development engine is k. The theoretical size of the three-dimensional digital model of the shuttle tanker in the virtual development engine is calculated by the transformation factor k;

[0021] Step 4, obtaining the real-time three-dimensional position and attitude information of the ship model;

[0022] (4.1) acquiring a virtual binocular image pair including a shuttle tanker model in real time;

[0023] (4.2) The center coordinates (x0, y0, z0) and the deflection angle (A) of the shuttle tanker model contained in each frame of the image are x , A y , A z ) outputs and records, performs difference processing on the coordinates of the center point of the shuttle tanker model and the camera position coordinates, and obtains the three-dimensional position information of the shuttle tanker model in the camera coordinate system;

[0024] Step 5, constructing the environmental system and ocean scene of rain, snow, fog, dusk, night, normal illumination and backlighting, and controlling the movement of the shuttle tanker model in step 3 in the ocean scene;

[0025] (5.1) Create the shape appearances of raindrops, mist, and snowflakes, design to control the density, movement speed, and movement direction of raindrops, water mist, and snowflakes in the scene, and construct an adjustable environmental system for rain, snow, and fog;

[0026] (5.2) Construct a parameterized system for dusk, night, normal lighting, and backlighting, and set the direction, color, and intensity of the light according to the lighting changes caused by the time change in a day to achieve the adjustment of the light over time;

[0027] (5.3) Construct an ocean scene, simulate the sea surface waves, water splashes, and colors, and achieve the adjustment of the wave height and wave shape;

[0028] (5.4) Control the floating mode of the shuttle tanker model in the ocean scene to be sinusoidal and cosinusoidal fluctuations, and the movement direction and speed change with the direction and size of the waves;

[0029] Step 6, cross-validate the virtual simulation data obtained from the operation sequence of steps 1 - 5 with the real data, use the deep learning object recognition algorithm to identify and detect the shuttle tanker in the virtual image, and verify the authenticity of the virtual simulation data;

[0030] (6.1) According to the virtual binocular images obtained from the operation sequence of steps 1 - 5 and the internal and external parameter matrices and distortion coefficients of the virtual binocular camera obtained in step (2.3), perform stereo matching on the virtual binocular images to obtain disparity data and back-project to obtain the three-dimensional point cloud of the shuttle tanker model. Compare the size of the three-dimensional point cloud of the shuttle tanker model with the theoretical size of the three-dimensional digital model of the shuttle tanker when constructing the virtual scene. If the two can correspond to each other, it indicates the validity of the virtual data;

[0031] (6.2) Convert the depth map obtained in step (1.3) through the internal and external parameter matrices of the virtual binocular camera in step (2.3) to obtain the three-dimensional point cloud of the ship model. Compare the size of the three-dimensional point cloud of the ship model with the theoretical size of the three-dimensional digital model of the shuttle tanker when constructing the virtual scene. If the two can correspond to each other, it indicates the validity of the virtual data;

[0032] (6.3) Perform recognition training on the shuttle tanker in the virtual image through the object recognition algorithm of deep learning, and use the real physical image for testing; if the shuttle tanker in the real image can be accurately recognized, it indicates the authenticity of the virtual data;

[0033] Step 7, effectively match various environmental conditions, integrate the internal and external parameter matrices and distortion coefficients of the virtual binocular camera in step (2.3), the three-dimensional position information of the shuttle tanker model in step (4.2), and the virtual binocular image pairs and depth maps obtained from the operation sequence of steps 1 - 5 to construct a high-fidelity, high-precision, and multi-condition data set;

[0034] All the internal and external sensor parameters matrices and distortion coefficients of the virtual binocular camera obtained in step (2.3), the three-dimensional position information of the shuttle tanker model in step (4.2), and the effective combination with the marine scene through various environmental systems in the operation sequence of steps 1-5, and including the virtual binocular image pairs and depth maps of the ship model; the internal and external sensor parameter matrices and distortion coefficients of the virtual binocular camera constitute the sensor parameter module in the dataset; the three-dimensional position information of the shuttle tanker model and the depth map constitute the supervised data module in the dataset; the virtual binocular images constitute the image data module in the dataset; Summarize the data of the above three modules to form a virtual simulation dataset for three-dimensional detection of marine ships.

[0035] Beneficial effects of the present invention: In the external transportation operation scenario of the production platform - shuttle tanker, when using a deep learning algorithm driven by big data, there is a lack of an adaptation dataset with comprehensive, accurate annotation information and including various extreme environmental conditions. Through the simulation of the actual physical scene, the present invention virtually simulates a binocular camera sensor, a three-dimensional model of a shuttle tanker, a parametric environmental system, and a marine scene, calibrates the internal and external parameters of the binocular camera and far meets the industrial use accuracy, creates a virtual twin of the external transportation operation scenario, effectively simulates the real working conditions of the scenario, and integrates all data to form a standardized dataset for use in deep learning algorithm training, meeting the requirements of the algorithm for data, and providing strong data support for realizing real-time monitoring and early warning of marine shuttle tankers. Brief Description of the Drawings

[0036] Figure 1 It is a flowchart of the present invention.

[0037] Figure 2 It is a schematic diagram of the space of the virtual binocular camera.

[0038] Figure 3 It is a model diagram of the virtual calibration board.

[0039] Figure 4 It is a schematic diagram of the detection of the characteristic corner points of the calibration board.

[0040] Figure 5 It is a schematic diagram of the calibration secondary projection error.

[0041] Figure 6 It is a schematic diagram of the shuttle tanker model.

[0042] Figure 7 It is a schematic diagram of the virtual binocular image including the shuttle tanker model, (a) virtual left image, (b) virtual right image.

[0043] Figure 8 It is a schematic diagram of the dusk lighting environment.

[0044] Figure 9 It is a schematic diagram of the disparity map.

[0045] Figure 10 A schematic diagram of a depth map.

[0046] Figure 11 Schematic diagram of the shuttle tanker point cloud. DETAILED DESCRIPTION

[0047] In order to explain the steps of the present invention in more detail, the specific implementation process of the present invention is introduced by way of drawings and cases. The cases described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative labor conditions belong to the protection scope of the present invention.

[0048] See also Figure 1 This method provides a method for constructing a shuttle tanker export operation scene dataset based on virtual simulation. For the export operation scene, a twin system is virtualized, including a binocular camera model, a shuttle tanker 3D model, a parametric environment system and an ocean scene, and the 3D position information of the ship model, the virtual binocular image and the virtual binocular camera internal and external parameter matrix and distortion coefficient are obtained, which together constitute a real, standardized and high-precision 3D detection dataset.

[0049] The method for constructing a shuttle tanker export operation scene data set based on virtual simulation mainly includes the following steps:

[0050] Step 1 specifically includes the following sub-steps:

[0051] (1.1) Please refer to Figure 2 , respectively set the coordinates of the left and right virtual monocular cameras in the world coordinate system (x, y, z) and (x', y', z'). The left and right monocular cameras are set on the same horizontal line in space, and the height H and depth D directions are kept consistent. The horizontal spacing is set to b, which is the baseline distance of the binocular camera.

[0052] (1.2) Set the field of view angle Fv, focal length F, photosensitive chip length L, width W and other parameter information of each virtual camera in the binocular camera. The above internal parameters of the left and right cameras are shared, that is, they are set exactly the same;

[0053] (1.3) Copy the left monocular camera model in step (1.1) as the depth camera model, set the depth rendering range of the depth camera model, and represent the depth value between each point in the rendering range of the depth camera coordinate system and the camera plane with a pixel value of 0-255. The closer the point is to the camera plane, the smaller the pixel value is, and a depth map in which the depth is represented by pixel values is obtained;

[0054] Step 2 specifically includes the following sub-steps:

[0055] (2.1) Refer to Figure 3 , create a checkerboard calibration board model in the virtual development engine. In the middle are small black and white square blocks, and the side length of the small square block is denoted as B. The checkerboard model is rectangular as a whole, that is, the number of small square blocks in the horizontal and vertical directions is inconsistent, and the checkerboard calibration board model has no border.

[0056] (2.2) Obtain the images of the checkerboard calibration board model from the perspective of the virtual binocular camera. Transform the position of the calibration board model in the field of view of the virtual binocular camera so that the calibration board model is in as many positions as possible in the upper, lower, left, and right of the field of view. Repeat this process to collect multiple groups of virtual binocular images containing the calibration board model;

[0057] (2.3) Calibrate the virtual binocular images obtained in step (2.2). Refer to Figure 4 , detect and extract the feature corner points in the virtual image, that is, the corner points of the small square blocks; use the coordinates of the detected feature corner points in the image and the side length B of the small square to estimate and optimize parameters such as the internal and external parameter matrices and distortion coefficients of the virtual binocular camera;

[0058] (2.4) Determine the three-dimensional space points corresponding to the feature corner points of the checkerboard in the images obtained in step (2.2) using the geometric relationship between the left and right images. Project the three-dimensional space points onto the image again through the internal and external parameter matrices and distortion coefficients of the virtual binocular camera in step (2.3) to obtain virtual pixel points, and calculate the error between the virtual pixel points and the feature corner points of the checkerboard. If the error value is within the acceptable range, it indicates the accuracy of the binocular camera simulation. For example, refer to Figure 5 , in a specific example, a total of 20 groups of virtual binocular images were collected in step (2.2), and the average secondary projection error after calibration was 0.05 pixels. In the industry, the calibration error is mostly required to be less than 0.1 pixel. In the specific example, the calibration accuracy is much higher than the required standard, strongly verifying the accuracy of the binocular camera simulation;

[0059] Step 3 specifically includes the following sub-steps:

[0060] (3.1) Refer to Figure 6 , use 3D modeling software to perform equal-scale modeling with reference to the real shuttle tanker to create a shuttle tanker model; perform corresponding texture mapping and rendering operations on the shuttle tanker model with reference to the surface color texture and other information of the real shuttle tanker, so that the shuttle tanker model is consistent with the real shuttle tanker in terms of shape and appearance;

[0061] (3.2) Import files such as textures, material properties, and naming together with the shuttle tanker model into the virtual development engine. Denote the scale transformation factor between the 3D modeling software and the virtual development engine as k, and calculate the theoretical size of the shuttle tanker model in the virtual development engine through the transformation factor k. For example, if the model in the 3D modeling software is proportional to the real physical model, there is a scaling coefficient m when exporting the ship model, and there is a scaling scale n when importing the ship model into the virtual development engine again. The scale transformation factor k = m * n, and use the scale transformation factor to calculate the size of the ship model;

[0062] Step 4 specifically includes the following sub-steps:

[0063] (4.1) Please refer to Figure 7 , and at the same time intercept the virtual binocular image pair containing the shuttle tanker model;

[0064] (4.2) Output and record the center point coordinates (x0, y0, z0) of the shuttle tanker model contained in each frame of the picture and the deflection angles (A x , A y , A z ) around the space coordinate axes. Subtract the center point coordinates of the shuttle tanker model from the camera position coordinates in step (1.1), and transform the shuttle tanker model from the virtual development engine coordinate system to the virtual camera coordinate system to obtain the position coordinates of the shuttle tanker model in the camera coordinate system;

[0065] Step 5 specifically includes the following sub-steps:

[0066] (5.1) Construct the shape and appearance of individual raindrops, fog, and snowflakes, design to control the density, movement speed, and movement direction of raindrops, water mist, and snowflakes in the scene, and construct an environmental system of rain, snow, and fog with adjustable parameters;

[0067] (5.2) Please refer to Figure 8 , construct a parameterized system related to environmental light such as dusk, night, normal light, and backlight. Set parameters such as the direction, color, and intensity of the light according to the light changes caused by the change of time of day, so that adjusting the time can adjust the light.

[0068] (5.3) Construct the ocean scene, simulate information such as sea surface waves, water splashes, and colors, and realize the adjustment of parameters such as wave height and wave shape;

[0069] (5.4) Control the floating mode of the shuttle tanker model in the ocean scene to be sinusoidal fluctuation, and the movement direction and speed change with the direction and size of the waves;

[0070] Step 6 specifically includes the following sub-steps:

[0071] (6.1) Refer to Figure 9 , according to the virtual binocular images obtained from the operation sequence of steps 1 to 5, the internal and external parameter matrices and distortion coefficients of the virtual binocular camera obtained in step (2.3), the virtual binocular images are corrected and stereo-matched to obtain disparity data; the disparity data is back-projected through the internal and external parameter matrices of the camera to obtain the three-dimensional point cloud of the shuttle tanker model; compare the length, width, and height dimensions of the three-dimensional point cloud of the shuttle tanker model with the theoretical dimensions of the three-dimensional digital model of the shuttle tanker when constructing the virtual scene. If the two can correspond to each other, it indicates the validity of the virtual data;

[0072] (6.2) Refer to Figure 10 and Figure 11 , convert the depth map obtained in step (1.3) through the internal and external parameter matrix of the virtual binocular camera in step (2.3) to obtain the three-dimensional point cloud of the ship model. Compare the dimensions of the three-dimensional point cloud of the ship model with the theoretical dimensions of the three-dimensional digital model of the shuttle tanker when constructing the virtual scene. If the two can correspond to each other, it indicates the validity of the virtual depth map;

[0073] (6.3) Use the object recognition algorithm of deep learning to identify and train the shuttle tanker in the virtual image, and use the physical image containing the real shuttle tanker for testing; if the shuttle tanker in the real image can be accurately identified, it indicates the authenticity of the virtual data;

[0074] Step 7 specifically includes the following sub-steps:

[0075] All the internal and external parameter matrices and distortion coefficients of the sensors of the virtual binocular camera obtained in step (2.3), the three-dimensional position information of the shuttle tanker model in step (4.2), and the operation sequence of steps 1 to 5 are effectively matched with the ocean scene through various environmental systems, and include the virtual binocular image pairs and depth maps containing the ship model; the internal and external parameter matrices and distortion coefficients of the virtual binocular camera constitute the sensor parameter module in the dataset; the three-dimensional position information and depth map of the shuttle tanker model constitute the supervised data module in the dataset; the virtual binocular images constitute the image data module in the dataset; summarize the data of the above three modules to form a virtual simulation dataset for three-dimensional detection of marine ships.

Claims

1. A method for constructing a dataset of the external oil transfer operation scenario of a shuttle tanker based on virtual simulation, characterized in that, Here are the steps: Step 1: Build a binocular camera model and a depth camera model in the virtual development engine according to the real physical camera; (1.1) Set the coordinates of the left and right virtual monocular cameras in the world coordinate system (x, y, z) and (x', y', z') respectively. The left and right monocular cameras are set on the same horizontal line in space, and the height and depth directions are consistent. The horizontal spacing is set to b, which is the baseline distance of the binocular cameras. (1.2) Set the field of view angle F, focal length F, length L, and width W of the photosensitive chip of each virtual camera in the binocular camera model. v , focal length F, length L of the photosensitive chip, and width W; (1.3) Copy the left monocular camera model in step (1.1) as the depth camera model, set the depth rendering range of the depth camera model, and represent the depth value between each point in the rendering range of the depth camera coordinate system and the camera plane with a pixel value of 0-255. The closer the point is to the camera plane, the smaller the pixel value is, and a depth map in which the depth is represented by pixel values is obtained; Step 2: Test the accuracy of the binocular camera simulation and obtain accurate virtual camera internal and external parameters; (2.1) Create a checkerboard calibration board model in the virtual development engine; (2.2) Obtaining images of the checkerboard calibration plate model from the perspective of a binocular camera, and collecting multiple sets of left and right image data pairs; (2.3) Calibrate the image data pair obtained in step (2.2) to obtain the sensor internal and external parameter matrix and distortion coefficient of the binocular camera; (2.4) Determine the three-dimensional space point corresponding to the checkerboard feature corner point in the image obtained in step (2.2) by using the geometric relationship between the left and right images, and reproject the three-dimensional space point to the image through the internal and external parameter matrix and distortion coefficient of the virtual binocular camera in step (2.3), thereby obtaining a virtual pixel point, and calculate the error between the virtual pixel point and the checkerboard feature corner point. If the error value is within an acceptable range, it indicates the accuracy of the binocular camera simulation; Step 3, constructing and coloring the shuttle tanker three-dimensional digital model, importing the shuttle tanker three-dimensional digital model into the virtual development engine, and calculating the theoretical size of the shuttle tanker three-dimensional digital model in the virtual development engine; (3.1) Proportional modeling of the shuttle tanker is performed, and the three-dimensional digital model of the shuttle tanker is mapped and rendered according to the surface color and texture of the real shuttle tanker; (3.2) Import the three-dimensional digital model of the shuttle tanker into the virtual development engine, and note that the scale transformation factor between the shuttle tanker modeling software and the virtual development engine is k. The theoretical size of the three-dimensional digital model of the shuttle tanker in the virtual development engine is calculated by the transformation factor k; Step 4, obtaining the real-time three-dimensional position and attitude information of the ship model; (4.1) acquiring a virtual binocular image pair including a shuttle tanker model in real time; (4.2) Output and record the center point coordinates (x0, y0, z0) of the shuttle tanker model contained in each frame of the image and the deflection angles (A x , A y , A z ) around the space coordinate axes. Subtract the center point coordinates of the shuttle tanker model from the camera position coordinates to obtain the three-dimensional position information of the shuttle tanker model in the camera coordinate system; Step 5, constructing the environmental system and ocean scene of rain, snow, fog, dusk, night, normal illumination and backlighting, and controlling the movement of the shuttle tanker model in step 3 in the ocean scene; (5.1) Create the shape and appearance of raindrops, fog and snowflakes, design and control the density, moving speed and moving direction of raindrops, mist and snowflakes in the scene, and build an environmental system of rain, snow and fog with adjustable parameters; (5.2) Construct a parametric system for dusk, night, normal light, and backlight, and set the direction, color, and intensity of the light according to the light changes caused by the time change during a day, so as to adjust the light with the change of time; (5.3) Construct an ocean scene, simulate the sea surface waves, water splashes, and colors, and realize the adjustment of the wave height and wave shape; (5.4) Control the floating mode of the shuttle tanker model in the ocean scene to be sinusoidal and cosinusoidal fluctuations, and the movement direction and speed change with the direction and size of the waves; Step 6: Cross-validate the virtual simulation data obtained from the operation sequence in Steps 1-5 with the real data, and use the deep learning object recognition algorithm to identify and detect the shuttle tanker in the virtual image to verify the authenticity of the virtual simulation data; (6.1) According to the virtual binocular images obtained from the operation sequence in Steps 1-5 and the internal and external parameter matrices and distortion coefficients of the virtual binocular camera obtained in Step (2.3), perform stereo matching on the virtual binocular images to obtain disparity data and back-project to obtain the three-dimensional point cloud of the shuttle tanker model. Compare the size of the three-dimensional point cloud of the shuttle tanker model with the theoretical size of the three-dimensional digital model of the shuttle tanker when constructing the virtual scene. If the two can correspond to each other, it indicates the effectiveness of the virtual data; (6.2) Convert the depth map obtained in Step (1.3) through the internal and external parameter matrices of the virtual binocular camera in Step (2.3) to obtain the three-dimensional point cloud of the ship model. Compare the size of the three-dimensional point cloud of the ship model with the theoretical size of the three-dimensional digital model of the shuttle tanker when constructing the virtual scene. If the two can correspond to each other, it indicates the effectiveness of the virtual data; (6.3) Perform recognition training on the shuttle tanker in the virtual image through the object recognition algorithm of deep learning, and use the real physical image for testing; if the shuttle tanker in the real image can be accurately recognized, it indicates the authenticity of the virtual data; Step 7: Effectively match various environmental conditions, integrate the internal and external parameter matrices and distortion coefficients of the virtual binocular camera in Step (2.3), the three-dimensional position information of the shuttle tanker model in Step (4.2), and the virtual binocular image pairs and depth maps obtained from the operation sequence in Steps 1-5 to construct a high-fidelity, high-precision, multi-condition dataset; All the internal and external parameter matrices and distortion coefficients of the sensors of the virtual binocular camera obtained in Step (2.3), the three-dimensional position information of the shuttle tanker model in Step (4.2), and the operation sequence in Steps 1-5 are effectively matched with various environmental systems and the ocean scene, and include the virtual binocular image pairs and depth maps of the ship model; the internal and external parameter matrices and distortion coefficients of the virtual binocular camera constitute the sensor parameter module in the dataset; the three-dimensional position information and depth map of the shuttle tanker model constitute the supervised data module in the dataset; the virtual binocular images constitute the image data module in the dataset; summarize the data of the above three modules to form a virtual simulation dataset for three-dimensional detection of marine ships.

Images