A multi-sequence submarine intelligent water sampling system and method based on deep learning

Abstract

The present invention provides a multi-sequence seabed intelligent water sampling system and method based on deep learning. The system includes a multi-sequence water sampler, an underwater camera, a sonar, an environmental parameter sensor unit and a power unit arranged on the multi-sequence water sampler; and a central processing unit; the central processing unit is used to receive data collected by the underwater camera, the environmental parameter sensor unit and the sonar in real time, and use a pre-trained deep learning model to perform image recognition on the data collected by the underwater camera, and control the multi-sequence water sampler to perform seawater sampling according to the recognition result and the data collected by the environmental parameter sensor unit and the sonar, and control the power unit to drive the multi-sequence water sampler to achieve adaptive movement; the present invention can realize tasks such as accurate selection of seabed sampling points, construction of three-dimensional seabed terrain, adaptive selection of water sampler sampling timing, and adaptive sample detection around seabed sampling points.

Description

Technical Field

[0001] The present invention relates to the field of marine science and marine engineering technology, and more specifically, to a multi-sequence seabed intelligent water sampling system and method based on deep learning. Background Art

[0002] High pressure and low temperature make up the extreme environment of the deep sea, which gives rise to a variety of special habitats such as cold springs, hydrothermal fluids, mud volcanoes, and brine pools. The collection and research of deep-sea water has always been a hot and key topic in the field of marine research. Collecting seawater samples that can truly reflect the composition of the in-situ water body through a water sampler is an important technical means in the field of deep-sea exploration and is crucial to the development of seabed resources. At the same time, the gas composition information of seawater samples is one of the research focuses. The gas information contained in seawater samples, such as CH 4 , H 2 S, CO 2 The content of gases such as these is an important indicator for tracking marine mineral resources such as seafloor natural gas hydrates, hydrothermal fluids, and cold springs. It is also an important development direction in the current field of deep-sea resource exploration.

[0003] At present, the research on marine water bodies usually uses a temperature-salinity-depth instrument (CTD) equipped with a water sampling system, using various sensors on the CTD water sampling device to obtain basic ocean physical and chemical parameters, and collect water samples at the same time. With the continuous deepening of marine research, higher requirements are also placed on water sampling devices. During the operation of the water sampler, the process from the seabed surface of the sampling point to the sea surface will be affected by seawater flow field factors such as ocean currents. During the vertical sampling process, the content of related gas components in seawater will change with the change of seawater flow field, with high randomness and high heterogeneity. Characteristics, such as high randomness, make it difficult to construct a three-dimensional gradient change model of various substances to be tested in seawater samples from the seabed to the sea surface. At the same time, traditional water samplers can be carried on various deep-sea operating equipment. Although high-fidelity in-situ sampling under different temperature and pressure conditions in the deep-sea environment can be achieved, the following problems still need to be solved:

[0004] 1) Most of them are single-sequence operations, and only samples at the same location at the same time can be taken in one operation;

[0005] 2) When the water sampler is sampling on the seabed, the selection of the corresponding sampling area is often carried out through real-time image transmission and manual judgment. The three-dimensional geographical environment of the seabed is difficult to determine, the water sampler landing point is inconvenient, and the judgment logic is highly subjective;

[0006] 3) In order to build a three-dimensional gradient change model of various substances to be tested from the seabed to the sea surface, the water sampler needs to conduct preliminary detection on the concentration range of a certain substance to be tested during the sampling process, and sampling in each water layer must be carried out within the concentration range area; previous water samplers only detected at a single sampling point on the seabed, resulting in a single concentration range for indicator detection, and the corresponding information could not be effectively obtained, resulting in large sampling errors, and it was difficult to accurately control the sampling time.

[0007] The existing patent document discloses a method and device for ocean observation control of adaptive sampling of multiple underwater robots, the method comprising: initializing state and network parameters; when a certain robot emerges from the water, obtaining the position of the robot and the characteristic vector of the sampled area; estimating the current position of other robots and obtaining the actual value of the distance of the robot emerging from the water; using the Kriging method to estimate the ocean characteristic field; forming a state group based on the position of each robot and the ocean characteristic field estimation; storing the state group, position memory, and characteristic vector in the memory bank; inputting the reassigned state group into the strategy neural network, outputting the robot action instruction, judging whether the neural network parameters are updated, updating each neural network parameter or further judging whether the sampling task is completed, and looping or ending; although the scheme of the prior art can enable the sampling control of multiple underwater robots to have autonomous learning and decision-making capabilities, and reduce the sampling uncertainty to a certain extent, the robot of the scheme is still a single sequence operation, and the position estimation accuracy is not high, and the sampling error is large. Summary of the invention

[0008] In order to overcome the defects of the above-mentioned existing technologies, most of which are single-sequence operations, low accuracy in judging the environmental factors of each water layer during the sampling process, and large sampling errors and randomness, the present invention provides a multi-sequence submarine intelligent water sampling system and method based on deep learning, which can realize tasks such as accurate selection of submarine sampling points, construction of three-dimensional submarine terrain, adaptive selection of water sampler sampling timing, and adaptive sample detection around submarine sampling points.

[0009] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0010] A multi-sequence seabed intelligent water sampling system based on deep learning, comprising: a multi-sequence water sampler, an underwater camera, a sonar, an environmental parameter sensor unit and a power unit arranged on the multi-sequence water sampler; and a central processing unit;

[0011] The central processing unit is respectively connected with the multi-sequence water sampler, the underwater camera, the environmental parameter sensor unit, the power unit and the sonar;

[0012] The central processing unit is used to receive data collected by the underwater camera, environmental parameter sensor unit and sonar in real time, and use a pre-trained deep learning model to perform image recognition on the data collected by the underwater camera to obtain recognition results; according to the recognition results and the data collected by the environmental parameter sensor unit and sonar, the multi-sequence water sampler is controlled to sample seawater, and the power unit is controlled to drive the multi-sequence water sampler to achieve adaptive movement.

[0013] Preferably, the multi-sequence water sampler comprises: a water sampler housing, a water sampler foot and a plurality of water sampler bottles;

[0014] The underwater camera, sonar, environmental parameter sensor unit and power unit are respectively arranged on the outside of the water sampler housing; the water sampler feet are arranged on the bottom outside the water sampler housing; each of the water sampling bottles is arranged inside the water sampler housing;

[0015] Each of the water collection bottles is provided with an electric control valve, and the electric control valve is connected to a central processing unit, and the central processing unit controls the opening and closing of each of the electric control valves.

[0016] Preferably, the environmental parameter sensor unit is provided with several different sensors, and the sensors are used to detect marine environmental parameters;

[0017] The marine environmental parameters include: the concentrations of several substances to be detected, and the water flow rate.

[0018] Preferably, the central processing unit is connected to the multi-sequence water sampler, the underwater camera, the environmental parameter sensor unit, the power unit and the sonar through optical cables respectively.

[0019] Preferably, the deep learning model in the central processing unit is specifically: an improved Deeplab semantic segmentation model;

[0020] The improved Deeplab semantic segmentation model includes an encoder and a decoder connected in sequence;

[0021] The encoder is specifically an Xception network, and the structure includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, an image pooling layer and a fifth convolutional layer connected in sequence;

[0022] The convolution kernel size of the first convolution layer and the fifth convolution layer is 1×1; the second convolution layer, the third convolution layer and the fourth convolution layer are all dilated convolution layers with a convolution kernel size of 3×3, and the dilation factors are 6, 12 and 18 respectively;

[0023] The structure of the decoder includes: a first upsampling layer, a concatenation layer, a sixth convolutional layer, and a second upsampling layer connected in sequence;

[0024] The output of the fifth convolutional layer is also connected to the concatenation layer;

[0025] The convolution kernel size of the sixth convolutional layer is 3×3.

[0026] The present invention also provides a multi-sequence submarine intelligent water sampling method based on deep learning, based on the above-mentioned multi-sequence submarine intelligent water sampling system based on deep learning, comprising the following steps:

[0027] S1: The multi-sequence water sampler dives to the seabed, uses the underwater camera to take real-time seabed images and transmits them to the central processing unit;

[0028] S2: The central processing unit uses a pre-trained deep learning model to identify the seabed image and obtain a seabed matrix segmentation result; and obtains a sampling position base point of a multi-sequence water sampler according to the seabed matrix distribution segmentation result;

[0029] S3: Scanning the sampling position base point using the sonar, and transmitting the scanning data to the central processing unit in real time;

[0030] The central processing unit constructs a three-dimensional seafloor topography distribution within a preset range around the sampling location base point based on the scan data;

[0031] Setting a sampling trajectory of a multi-sequence water sampler according to the three-dimensional seafloor topography distribution;

[0032] S4: the central processing unit controls the power unit to drive the multi-sequence water sampler to achieve adaptive movement according to the sampling trajectory;

[0033] During the adaptive movement of the multi-sequence water sampler, the environmental parameter sensor unit is used to detect the concentration of the preset type of substance to be detected in real time and transmit it to the central processing unit;

[0034] The central processing unit selects a number of target sampling points in the sampling trajectory according to the concentration of the substance to be detected;

[0035] S5: The central processing unit controls the power unit to drive the multi-sequence water sampler to move to each target sampling point respectively;

[0036] At each target sampling point, the central processing unit determines the sampling time according to the data collected in real time by the underwater camera and the environmental parameter sensor unit, and controls the multi-sequence water sampler to perform seawater sampling at the sampling time;

[0037] After completing the seawater sampling at all target sampling points, the multi-sequence water sampler rises.

[0038] Preferably, in step S4, the preset type of substance to be detected is specifically methane;

[0039] The environmental parameter sensor unit is provided with a methane concentration sensor, and the methane concentration is detected by using the methane concentration sensor;

[0040] The several target sampling points are several methane leakage nozzles with different methane concentrations.

[0041] Preferably, in step S5, at each target sampling point, the central processing unit determines the sampling time according to the data collected in real time by the underwater camera and the environmental parameter sensor unit, including:

[0042] S5.1: Using the underwater camera to take images of the target sampling points in real time and transmit the images to the central processing unit;

[0043] The methane concentration sensor is used to detect the methane concentration at the target sampling point in real time and transmit the methane concentration to the central processing unit;

[0044] S5.2: bubbles and sediment emitted from the methane seepage vent are selected as characteristic targets, and the central processing unit uses a pre-trained deep learning model to recognize the image of the target sampling point to obtain recognition results of the sediment and bubbles;

[0045] S5.3: The central processing unit determines in real time whether the identification results of the sediment and bubbles at the target sampling point and the change in methane concentration meet the preset eruption conditions at the same time. If so, this time is used as the sampling time of the target sampling point; otherwise, repeat step S5.3.

[0046] Preferably, in step S5.2, the central processing unit uses a pre-trained deep learning model to identify the positions of background content, sediment and bubble particles in the image of the target sampling point in real time;

[0047] Classifying the sediment according to the seabed matrix segmentation result, and comparing the sediment categories at different times to obtain the change in the distribution ratio of the sediment;

[0048] Calculate the bubble rising rate based on the position of the bubble particles at different times;

[0049] The change in the distribution ratio of the sediment and the rising rate of the bubbles are used as the identification results of the sediment and bubbles.

[0050] Preferably, in step S5, a water flow velocity sensor is further provided in the environmental parameter sensor unit; the water flow velocity sensor detects the water flow velocity in real time and transmits it to the central processing unit;

[0051] The central processing unit controls the power unit according to the water flow rate to drive the multi-sequence water sampler to move to each target sampling point respectively.

[0052] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0053] The present invention provides a multi-sequence seabed intelligent water sampling system and method based on deep learning, wherein the system comprises a multi-sequence water sampler, an underwater camera, a sonar, an environmental parameter sensor unit and a power unit arranged on the multi-sequence water sampler; and a central processing unit; the resolution of the underwater camera is greater than or equal to a first threshold, and the resolution of the sonar is greater than or equal to a second threshold; the central processing unit is used to receive data collected by the underwater camera, the environmental parameter sensor unit and the sonar in real time, and perform image recognition on the data collected by the underwater camera using a pre-trained deep learning model to obtain a recognition result; according to the recognition result and the data collected by the environmental parameter sensor unit and the sonar, the multi-sequence water sampler is controlled to perform seawater sampling, and the power unit is controlled to drive the multi-sequence water sampler to achieve adaptive movement;

[0054] Compared with the prior art, the present invention:

[0055] 1) Based on the deep learning model, the seabed is semantically segmented to divide the coverage of different seabed substrates, the ideal sampling location is determined according to the task requirements, and the multi-beam information fed back by the sonar is used to construct a high-precision three-dimensional seabed topography with visualization of the target sampling point. This is used as the basis for selecting the sampling location of the water sampler, avoiding the subjective uncertainty of human judgment, improving the accuracy of sampling site selection, and being able to collect more ideal seawater samples;

[0056] 2) The deep learning model in the present invention has the advantages of small size, high computing speed and high accuracy, which perfectly meets the task characteristics of the seabed matrix segmentation task, that is, the distribution of various matrixes is highly coherent and the characteristics are obviously different, and effectively improves the sampling accuracy;

[0057] 3) The present invention realizes the adaptive movement function of the water sampler by setting a power unit. The water sampler can adaptively move within a certain range around the sampling site while determining the sampling site to detect multiple groups of water samples, obtain a more reasonable concentration range of the indicator substance, and provide an effective reference for selecting appropriate sampling positions for each water layer in the process of collecting water samples from bottom to top;

[0058] 4) After selecting a suitable sampling location, the present invention also determines the sampling time and performs sampling only when the sampling conditions are met. The sampling time can be accurately controlled to obtain the best possible seabed bottom water samples, which greatly reduces the randomness of traditional water sampling systems when facing such tasks and reduces sampling errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is an architecture diagram of a multi-sequence submarine intelligent water sampling system based on deep learning provided in Example 1.

[0060] Figure 2 This is a structural diagram of a multi-sequence submarine intelligent water sampling system based on deep learning provided in Example 2.

[0061] Figure 3 This is a structural diagram of the deep learning model in the central processing unit provided in Example 2.

[0062] Figure 4 This is a flow chart of a multi-sequence seabed intelligent water sampling method based on deep learning provided in Example 3. DETAILED DESCRIPTION

[0063] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0064] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0065] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0066] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0067] Example 1

[0068] like Figure 1 As shown, this embodiment provides a multi-sequence submarine intelligent water sampling system based on deep learning, including: a multi-sequence water sampler, an underwater camera, a sonar, an environmental parameter sensor unit and a power unit arranged on the multi-sequence water sampler; and a central processing unit;

[0069] The central processing unit is respectively connected with the multi-sequence water sampler, the underwater camera, the environmental parameter sensor unit, the power unit and the sonar;

[0070] The central processing unit is used to receive data collected by the underwater camera, environmental parameter sensor unit and sonar in real time, and use a pre-trained deep learning model to perform image recognition on the data collected by the underwater camera to obtain recognition results; according to the recognition results and the data collected by the environmental parameter sensor unit and sonar, the multi-sequence water sampler is controlled to sample seawater, and the power unit is controlled to drive the multi-sequence water sampler to achieve adaptive movement.

[0071] In the specific implementation process, the purpose of this system is to improve the accuracy, objectivity and timeliness of the water sampler operation process on the basis of high-fidelity in-situ sampling of multi-sequence water samplers, to reduce the interference of seawater flow field on the concentration diffusion of indicator substances as much as possible, and to improve the applicability of sampling point selection;

[0072] This system uses a deep learning module pre-trained according to the sampling task requirements, and based on the image transmission and multi-beam data fed back by the underwater camera and sonar, it can perform semantic segmentation on the seabed to divide the coverage of different seabed substrates. At the same time, when benthic organisms exist, it can divide the living areas of different types of organisms to determine the ideal sampling location according to the task requirements; the multi-beam information fed back by the sonar can construct a visualized, high-precision three-dimensional seabed topography of the seabed sampling location, which serves as the basis for selecting the sampling location of the water sampler, increasing the objectivity of the sampling point selection and the convenience of the water sampler landing point selection;

[0073] The deep learning model involved in this system uses a series of convolution, pooling and other operations to process the input image, which has the advantages of small model size, high calculation rate and high accuracy. At the same time, combined with the three-dimensional seabed topography constructed by sonar multi-beam data, the logic of water sampler selection site is further improved;

[0074] Traditional water samplers are often carried on underwater remote control units such as underwater robots (ROVs). The multi-sequence in-situ pressure-maintaining water samplers involved in this system are heavy overall and cannot be carried conventionally. Therefore, a multi-directional power unit is installed on the multi-sequence water sampler to move horizontally within a short distance. Based on the above-mentioned ability to construct real-time three-dimensional seabed terrain, this system can control the water sampler to achieve adaptive horizontal movement. In the process of collecting water samples in the deep sea, in order to realize the construction of a three-dimensional gradient change model of various substances to be tested from the seabed to the sea surface, it is necessary to conduct preliminary detection of the concentration range of a certain indicator substance during the sampling process. Due to the relatively limited movement of traditional water samplers, they are often only tested at a single site for a long period of time. However, due to the influence of the seabed flow field, the measurement accuracy of the indicator substance concentration at a single site is limited. Therefore, this system can adaptively move within a certain range around the sampling site to detect multiple groups of water samples while determining the sampling site through the adaptive horizontal movement function based on deep learning, so as to obtain a more reasonable indicator substance concentration range, and provide an effective reference for selecting suitable sampling positions for each water layer in the process of collecting water samples from bottom to top.

[0075] Finally, when this system is sampling on the seabed, it is sometimes necessary to collect water samples carrying pore water just after the seabed gas erupts. The water samples have characteristics such as high concentration of erupted gas and rich basic microbial biomass, and are considered ideal water samples; however, due to the uncertainty of the eruption cycle, the randomness of collecting such water samples is extremely high. Using this system, deep learning can be used to train target recognition of bubbles at the vent position or surging mud and sand during gas eruption. After selecting a suitable sampling position, the camera also transmits real-time images of the vent position. When the vent erupts at any time, targets such as bubbles or mud and sand are recognized by the target detection model. At the same time, when the concentration sensor of the relevant indicator substance also detects concentration fluctuations, the multi-sequence water sampler valve is opened for sampling to obtain the best possible bottom water samples, which greatly reduces the randomness and sampling errors of traditional water sampling systems when facing such tasks.

[0076] This system mainly solves the problems of randomness caused by manual judgment of various water layers and seabed habitats at sampling sites during the sampling process of existing water samplers, heterogeneity caused by changes in seawater flow fields when various substances to be tested in seawater migrate to the sea surface, subjectivity in accurately determining the collection position in the subsequent water sample collection process under the premise of single-sequence operation, and timeliness of sampling of pore water carried by the vent. It realizes tasks such as accurate selection of seabed sampling sites, construction of three-dimensional seabed terrain, adaptive selection of water sampler sampling timing, and adaptive sample detection around seabed sampling points.

[0077] Example 2

[0078] like Figure 2 As shown, this embodiment provides a multi-sequence submarine intelligent water sampling system based on deep learning, including: a multi-sequence water sampler 1, a high-definition underwater camera 2, an environmental parameter sensor unit 3, a power unit 4 and a high-definition sonar 5 arranged on the multi-sequence water sampler 1; and a central processing unit 6;

[0079] The central processing unit 6 in this embodiment is specifically a shipborne integrated central processing unit;

[0080] The resolution of the high-definition underwater camera 2 is greater than or equal to the first threshold, and the resolution of the high-definition sonar 5 is greater than or equal to the second threshold; the high-resolution sensing equipment can effectively improve the detection accuracy, reduce the sampling error, and collect more ideal water samples that meet the requirements;

[0081] The central processor unit 6 is respectively connected to the multi-sequence water sampler 1, the high-definition underwater camera 2, the environmental parameter sensor unit 3, the power unit 4 and the high-definition sonar 5;

[0082] The central processing unit 6 is used to receive the data collected by the high-definition underwater camera 2, the environmental parameter sensor unit 3 and the high-definition sonar 5 in real time, and use the pre-trained deep learning model to perform image recognition on the data collected by the high-definition underwater camera 2 to obtain the recognition result; according to the recognition result and the data collected by the environmental parameter sensor unit 3 and the high-definition sonar 5, the multi-sequence water sampler 1 is controlled to perform seawater sampling, and the power unit 4 is controlled to drive the multi-sequence water sampler 1 to achieve adaptive movement;

[0083] The multi-sequence water sampler 1 comprises: a water sampler housing 11, a water sampler foot 12 and a plurality of water sampler bottles 13;

[0084] The high-definition underwater camera 2, the environmental parameter sensor unit 3, the power unit 4 and the high-definition sonar 5 are respectively arranged on the outside of the water sampler housing 11; the water sampler machine foot 12 is arranged on the outside bottom of the water sampler housing 11; each of the water sampling bottles 13 is arranged inside the water sampler housing 11;

[0085] Each of the water collection bottles 13 is provided with an electric control valve 131, and the electric control valve 131 is connected to the central processing unit 6, and the central processing unit 6 controls the opening and closing of each of the electric control valves 131;

[0086] The environmental parameter sensor unit 3 is provided with several different sensors, which are used to detect marine environmental parameters;

[0087] The marine environmental parameters include: the concentration of several substances to be detected, and the water flow rate;

[0088] The central processing unit 6 is connected to the multi-sequence water sampler 1, the high-definition underwater camera 2, the environmental parameter sensor unit 3, the power unit 4 and the high-definition sonar 5 through optical cables 7 respectively;

[0089] The deep learning model in the central processing unit 6 is specifically: an improved Deeplab semantic segmentation model;

[0090] The improved Deeplab semantic segmentation model includes an encoder and a decoder connected in sequence;

[0091] The encoder is specifically an Xception network, and the structure includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, an image pooling layer and a fifth convolutional layer connected in sequence;

[0092] The convolution kernel size of the first convolution layer and the fifth convolution layer is 1×1; the second convolution layer, the third convolution layer and the fourth convolution layer are all dilated convolution layers with a convolution kernel size of 3×3, and the dilation factors are 6, 12 and 18 respectively;

[0093] The structure of the decoder includes: a first upsampling layer, a concatenation layer, a sixth convolutional layer, and a second upsampling layer connected in sequence;

[0094] The output of the fifth convolutional layer is also connected to the concatenation layer;

[0095] The convolution kernel size of the sixth convolutional layer is 3×3.

[0096] In the specific implementation process, the working scene of the system is set to the cold spring area, the base sampling position is the methane seepage vent position covered with a large number of mussels, and the relevant indication parameter is set to the methane concentration in the seawater; the multi-sequence water sampler 1 involved in this embodiment is equipped with multiple multi-angle power units 4, which can achieve horizontal movement at the bottom of the offshore, and the information transmission method is to use the optical cable 7 for transmission. The integrated central processing unit 6 is carried on the mother ship. The deep learning model involved in this system has also been pre-trained, and can transmit relevant seabed information in real time and identify and process it in real time;

[0097] In this embodiment, the deep learning model function is briefly described as follows: Figure 3 As shown in the figure, the network structure includes two modules, Encoder and Decoder; the front-end network feature extraction layer of Encoder uses the Xception architecture, and uses a series of convolution, pooling and other operations to process the input image, and gradually abstracts the input image into a high-level feature representation; Xception includes the convolution layer conv1, which uses a 1×1 convolution kernel; the convolution layer conv2, which uses a 3×3 convolution kernel with a dilation factor of 6; the convolution layer conv3, which uses a 3×3 convolution kernel with a dilation factor of 12; the convolution layer conv4, which uses a 3×3 convolution kernel with a dilation factor of 18; the image pooling layer can increase the receptive field of the convolution kernel by increasing the dilation factor of the dilated convolution; finally, a 1×1 convolution layer is added, and the output of this convolution layer is called multi-scale contextual The function of this layer is to perform weighted combination of the output of the previous convolutional layer to obtain a more global image representation. The decoder backend network layer uses upsampling operation to restore the feature map extracted by the encoder module to the size of the input image and perform pixel-level prediction at the same time. The result of the encoder is first upsampled by 4 times, and then the feature map of DCNN is fused with the upsampled result to synthesize a new feature map, and finally the result is input into the next upsampling architecture. After going through the entire model network structure process, the objects of different categories in the image are segmented and merged for visualization processing.

[0098] The deep learning model in this system improves the Deeplab semantic segmentation model with a wide range of applications, uses the Xception architecture in the front-end network feature extraction layer, and uses a series of convolution, pooling and other operations to process the input image. The improved model has the advantages of small model size, high calculation rate and high accuracy; combined with the characteristics of strong coherence and obvious feature differences in the distribution of various substrates in the seabed substrate segmentation task, the advantages of the real-time detection and recognition feedback results of this model are more suitable for time-sensitive working conditions. At the same time, combined with the three-dimensional seabed terrain constructed by sonar multi-beam data, the logic of water sampler selection site can be further improved;

[0099] This system is based on a multi-sequence high-fidelity in-situ pressure-maintaining deep-sea water sampler. By installing a high-definition underwater camera 2, an environmental parameter sensor unit 3 and a power unit 4 on the multi-sequence water sampler 1, as well as setting up a high-definition sonar 5 and a central processing unit 6, it can realize tasks such as accurate selection of seabed sampling sites, construction of three-dimensional seabed terrain, adaptive selection of water sampler sampling timing, and adaptive sample detection around seabed sampling points.

[0100] Example 3

[0101] like Figure 4 As shown, this embodiment provides a multi-sequence submarine intelligent water sampling method based on deep learning, based on the multi-sequence submarine intelligent water sampling system based on deep learning described in Embodiment 1 or 2, comprising the following steps:

[0102] S1: The multi-sequence water sampler dives to the seabed, uses the underwater camera to take real-time seabed images and transmits them to the central processing unit;

[0103] S2: The central processing unit uses a pre-trained deep learning model to identify the seabed image and obtain a seabed matrix segmentation result; and obtains a sampling position base point of a multi-sequence water sampler according to the seabed matrix distribution segmentation result;

[0104] S3: Scanning the sampling position base point using the sonar, and transmitting the scanning data to the central processing unit in real time;

[0105] The central processing unit constructs a three-dimensional seafloor topography distribution within a preset range around the sampling location base point based on the scan data;

[0106] Setting a sampling trajectory of a multi-sequence water sampler according to the three-dimensional seafloor topography distribution;

[0107] S4: the central processing unit controls the power unit to drive the multi-sequence water sampler to achieve adaptive movement according to the sampling trajectory;

[0108] During the adaptive movement of the multi-sequence water sampler, the environmental parameter sensor unit is used to detect the concentration of the preset type of substance to be detected in real time and transmit it to the central processing unit;

[0109] The central processing unit selects a number of target sampling points in the sampling trajectory according to the concentration of the substance to be detected;

[0110] S5: the central processing unit controls the power unit to drive the multi-sequence water sampler to move to each target sampling point respectively;

[0111] At each target sampling point, the central processing unit determines the sampling time according to the data collected in real time by the underwater camera and the environmental parameter sensor unit, and controls the multi-sequence water sampler to perform seawater sampling at the sampling time;

[0112] After completing the seawater sampling at all target sampling points, the multi-sequence water sampler rises;

[0113] In step S4, the preset type of substance to be detected is specifically methane;

[0114] The environmental parameter sensor unit is provided with a methane concentration sensor, and the methane concentration is detected by using the methane concentration sensor;

[0115] The several target sampling points are several methane leakage nozzles with different methane concentrations;

[0116] In step S5, at each target sampling point, the central processing unit determines the sampling time according to the data collected in real time by the underwater camera and the environmental parameter sensor unit, including:

[0117] S5.1: Using the underwater camera to take images of the target sampling points in real time and transmit the images to the central processing unit;

[0118] The methane concentration sensor is used to detect the methane concentration at the target sampling point in real time and transmit the methane concentration to the central processing unit;

[0119] S5.2: bubbles and sediment emitted from the methane seepage vent are selected as characteristic targets, and the central processing unit uses a pre-trained deep learning model to identify the background content, sediment and bubble particle positions in the image of the target sampling point;

[0120] Classifying the sediment according to the seabed matrix segmentation result, and comparing the sediment categories at different times to obtain the change in the distribution ratio of the sediment;

[0121] Calculate the bubble rising rate based on the position of the bubble particles at different times;

[0122] The distribution ratio change of the sediment and the rising rate of the bubbles are used as the identification results of the sediment and the bubbles;

[0123] S5.3: The central processing unit determines in real time whether the identification results of the sediment and bubbles at the target sampling point and the change in the methane concentration meet the preset eruption conditions at the same time. If so, this time is used as the sampling time of the target sampling point; otherwise, repeat step S5.3;

[0124] In step S5, a water flow velocity sensor is also provided in the environmental parameter sensor unit; the water flow velocity sensor detects the water flow velocity in real time and transmits it to the central processing unit;

[0125] The central processing unit controls the power unit according to the water flow rate to drive the multi-sequence water sampler to move to each target sampling point respectively.

[0126] In the specific implementation process, the camera and sonar used in this embodiment are respectively a high-definition underwater camera and a high-definition sonar;

[0127] First, identify and select the seabed matrix and locate the target sampling points. The specific steps are as follows:

[0128] 1) Based on the deep learning method, this system recognizes the seabed matrix segmentation. First, when the water sampling system is moving on the seabed, a high-precision underwater camera is used to shoot the seabed video in real time and transmit it to the shipboard integrated central processing unit. Then, the pre-trained deep learning model is used to identify the seabed matrix distribution segmentation result, which is used as the judgment basis for sampling point selection.

[0129] The above-mentioned model for segmenting and identifying the seabed matrix is ​​implemented based on deep learning, and for the same recognition task, factors such as recognition rate and high recognition accuracy need to be considered; after experimental comparison of models adapted to semantic segmentation tasks, the present invention selects and improves the Deeplab semantic segmentation model proposed by Google, and uses the Xception architecture in the front-end network feature extraction layer of the model, and uses a series of convolution, pooling and other operations to process the input image, and gradually abstracts the input image into a high-level feature representation; the improved model has the advantages of small model size, high calculation rate and high accuracy;

[0130] The depth model in this embodiment does not need to process every pixel in the seabed image fed back. Since the distribution of the seabed matrix is ​​highly coherent and the characteristics are obviously different, the use of this model can realize real-time processing of image information to the greatest extent and provide matrix segmentation results with timeliness, which is conducive to the water sampler to select and judge the sampling points in real time during the travel process;

[0131] After the improved semantic segmentation model is trained for classification and segmentation of different types of seabed matrices according to task requirements, it is loaded into the ship's integrated central processing unit. After the water sampler passes over the seabed, the real-time transmitted images are processed by the model to construct a visualization result of the classification and segmentation of the seabed matrix as the basis for selection;

[0132] 2) After the sampling point selection logic judgment is performed, the sampling position base point of the water sampler can be determined. At the same time, the high-definition sonar scans the sampling position base point and transmits the scanned data back to the shipboard integrated central processing unit. Through the multi-beam data after data conversion, a high-precision three-dimensional terrain distribution of the sampling position is constructed;

[0133] 3) After constructing the three-dimensional seabed terrain distribution near the sampling location base point, combined with the segmentation results of the seabed matrix in the above steps, it is judged whether the terrain is flat, the type of seabed matrix where the water sampler can choose to land, whether it can support the stable landing of the water sampler, etc., so as to select the most suitable landing point for the water sampler's machine feet, so that the water sampler can land and sample at a location that is as flat and hard as possible without destroying the seabed habitat;

[0134] Then the water sampler is used to adaptively detect environmental factors. The specific steps are as follows:

[0135] 4) After the intelligent water sampling system involved in this embodiment completes the segmentation and identification of the seabed matrix, the sampling site and landing point of the water sampler are determined. In order to achieve high-precision fixed-point sampling during the sampling process of the water sampler from the seabed to the sea surface, it is necessary to determine whether the water sample comes from the upward diffusion of methane after the methane eruption from the seabed vent during the sampling process of each water layer. Therefore, it is necessary to establish a three-dimensional gradient change model of the indicator substance methane at the depth scale. Therefore, the detection of the reasonable change range of the indicator substance methane concentration at the sampling point position is particularly critical;

[0136] The water sampler can determine the changes in seabed topography within a certain range through the above-mentioned function of constructing seabed topography based on deep learning. After determining the surrounding seabed topography, the water sampler can preset the translational motion trajectory around the sampling site base point, while avoiding areas with large terrain changes detected by sonar;

[0137] 5) The water sampler always keeps a certain distance from the seabed along the preset motion trajectory, and detects seawater samples multiple times at different relative distances around the sampling site base point. The system is equipped with a methane concentration sensor, and the data fed back by the sensor in real time and the data after multiple detections jointly construct the concentration range of the indicator substance methane, which is C1-C2;

[0138] Finally, timely water sampling of the nozzle position is carried out. The specific steps are as follows:

[0139] 6) After the water sampler has completed the sampling position base point, the landing point, and the concentration range detection of the indicator substance methane in the surrounding environmental factors, the water sampler returns to the preset landing point to collect the water sample carrying pore water ejected from the methane leakage vent. Due to the characteristics of high methane concentration and rich microbial biomass in the water sample, it is regarded as an ideal water sample; however, the water sample is extremely time-sensitive and random, so the method involved in this embodiment is implemented by coupling the target detection model based on deep learning with the detection data of the methane concentration sensor;

[0140] 7) In order to detect the eruption moment, this method selects characteristic targets, namely, the bubbles and mud that are erupted, and uses the high-precision underwater camera carried by the system to shoot the picture, and completes the identification and locking after the target appears; in this model, since the types of identified targets are relatively single and the differences are obvious, it is more appropriate to select a small-volume model with high recognition efficiency and small carrying capacity. This embodiment still uses the improved Deeplab semantic segmentation model constructed above;

[0141] 8) The construction of this model is mainly based on the semantic segmentation model involved in the above scheme. The video information at the nozzle is extracted into image information, and the background content and the detailed information of the sediment or bubbles are semantically segmented. The sediment details are classified using the soft sediments in the seabed matrix trained in the semantic segmentation model. At the same time, the image containing bubbles is expanded and eroded to obtain the clear outline of the bubble and locate the position of the bubble particle;

[0142] 9) When calculating the bubble rising rates at the two time nodes of time t1 and time t2, the relative two-dimensional coordinates of the bubbles at the two time nodes can be detected and marked in the above manner, which are (X1, Y1) and (X2, Y2), respectively. The relative displacement of the bubble movement is divided by the time difference between t1 and t2 to obtain the bubble rising rate during the movement of the methane bubble. By comparing whether the bubble rising rate is higher than the bubble rising rate when there is no eruption, it can be determined whether the sampling time has been reached;

[0143] 10) In the image containing sediment and bubble information, determine the change in the distribution ratio of sediment in the image and the movement of the bubble particle position; at the same time, combined with the information on the change in the concentration of methane gas at the location transmitted by the methane concentration sensor, jointly correct and determine the eruption status of the vent;

[0144] 11) After confirming the eruption status of the methane seepage vent, control the valve of the water sampling bottle to open and immediately collect the bottom water sample with pore water that has just erupted;

[0145] 12) In this embodiment, the water sampler is also equipped with a water flow velocity sensor, which can be used to detect the speed and relative direction of the seawater flow field. When the water sampler leaves the seabed and rises to enter the next sampling water layer, the methane concentration Cn in the water is detected to determine whether it is within the interval range of C1 to C2. If it is not within the interval range, the water sampler moves horizontally in the direction according to the water flow velocity, looking for a point where the methane concentration is within the interval range for sampling, and repeating this step until the sea surface; this step is to select the best sampling point for each water layer, and according to the data of the substance to be tested measured by the water sampler in each target water layer during the ascending sampling process, it is possible to construct a three-dimensional gradient change model of the substance to be tested from the seabed to the sea surface, which is used for the subsequent three-dimensional depth gradient change analysis of the concentration offset of the sampled sample;

[0146] This method can improve the accuracy, objectivity and timeliness of the water sampler operation process on the basis of high-fidelity in-situ sampling of multi-sequence water samplers, reduce the interference of seawater flow field on the concentration diffusion of indicator substances as much as possible, and improve the applicability of sampling point selection;

[0147] This method mainly solves the problems of randomness that may be caused by manual judgment of the seabed habitats of various water layers and sampling sites during the sampling process of existing water samplers, heterogeneity caused by changes in seawater flow fields in the process of various substances to be tested in seawater migrating to the sea surface, subjectivity in the difficulty in accurately determining the collection position in the subsequent water sample collection process under the premise of single-sequence operation, and timeliness of sampling of pore water samples carried by the nozzle; it realizes the tasks of accurate selection of seabed sampling sites, construction of three-dimensional seabed terrain, adaptive selection of water sampler sampling timing, and adaptive sample detection around seabed sampling points.

[0148] The same or similar reference numerals correspond to the same or similar components;

[0149] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0150] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A multi-sequence submarine intelligent water sampling system based on deep learning, characterized in that: include: A multi-sequence water sampler, an underwater camera, a sonar, an environmental parameter sensor unit and a power unit arranged on the multi-sequence water sampler; and a central processing unit; The central processing unit is respectively connected with the multi-sequence water sampler, the underwater camera, the environmental parameter sensor unit, the power unit and the sonar; The central processing unit is used to receive data collected by the underwater camera, the environmental parameter sensor unit and the sonar in real time, and use the pre-trained deep learning model to perform image recognition on the data collected by the underwater camera to obtain recognition results; according to the recognition results and the data collected by the environmental parameter sensor unit and the sonar, the multi-sequence water sampler is controlled to perform seawater sampling, and the power unit is controlled to drive the multi-sequence water sampler to achieve adaptive movement; The deep learning model in the central processing unit is specifically: an improved Deeplab semantic segmentation model; The improved Deeplab semantic segmentation model includes an encoder and a decoder connected in sequence; The encoder is specifically an Xception network, and the structure includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, an image pooling layer and a fifth convolutional layer connected in sequence; The convolution kernel size of the first convolution layer and the fifth convolution layer is 1×1; the second convolution layer, the third convolution layer and the fourth convolution layer are all dilated convolution layers with a convolution kernel size of 3×3, and the dilation factors are 6, 12 and 18 respectively; The structure of the decoder includes: a first upsampling layer, a concatenation layer, a sixth convolutional layer, and a second upsampling layer connected in sequence; The output of the fifth convolutional layer is also connected to the concatenation layer; The convolution kernel size of the sixth convolutional layer is 3×3.

2. According to the multi-sequence submarine intelligent water sampling system based on deep learning in claim 1, it is characterized in that: The multi-sequence water sampler comprises: a water sampler housing, a water sampler foot and a plurality of water sampler bottles; The underwater camera, sonar, environmental parameter sensor unit and power unit are respectively arranged on the outside of the water sampler housing; the water sampler feet are arranged on the bottom outside the water sampler housing; each of the water sampling bottles is arranged inside the water sampler housing; Each of the water collection bottles is provided with an electric control valve, and the electric control valve is connected to a central processing unit, and the central processing unit controls the opening and closing of each of the electric control valves.

3. The multi-sequence submarine intelligent water sampling system based on deep learning according to claim 1 is characterized in that: The environmental parameter sensor unit is provided with several different sensors, and the sensors are used to detect marine environmental parameters; The marine environmental parameters include: the concentrations of several substances to be detected, and the water flow rate.

4. The multi-sequence submarine intelligent water sampling system based on deep learning according to claim 1 is characterized in that: The central processing unit is connected with the multi-sequence water sampler, the underwater camera, the environmental parameter sensor unit, the power unit and the sonar through optical cables respectively.

5. A multi-sequence submarine intelligent water sampling method based on deep learning, based on the multi-sequence submarine intelligent water sampling system based on deep learning as described in any one of claims 1 to 4, characterized in that: The following steps are involved: S1: The multi-sequence water sampler dives to the seabed, uses the underwater camera to take real-time seabed images and transmits them to the central processing unit; S2: The central processing unit uses a pre-trained deep learning model to identify the seabed image and obtain a seabed matrix segmentation result; and obtains a sampling position base point of a multi-sequence water sampler according to the seabed matrix distribution segmentation result; S3: Scanning the sampling position base point using the sonar, and transmitting the scanning data to the central processing unit in real time; The central processing unit constructs a three-dimensional seafloor topography distribution within a preset range around the sampling location base point based on the scan data; Setting a sampling trajectory of a multi-sequence water sampler according to the three-dimensional seafloor topography distribution; S4: the central processing unit controls the power unit to drive the multi-sequence water sampler to achieve adaptive movement according to the sampling trajectory; During the adaptive movement of the multi-sequence water sampler, the environmental parameter sensor unit is used to detect the concentration of the preset type of substance to be detected in real time and transmit it to the central processing unit; The central processing unit selects a number of target sampling points in the sampling trajectory according to the concentration of the substance to be detected; S5: The central processing unit controls the power unit to drive the multi-sequence water sampler to move to each target sampling point respectively; At each target sampling point, the central processing unit determines the sampling time according to the data collected in real time by the underwater camera and the environmental parameter sensor unit, and controls the multi-sequence water sampler to perform seawater sampling at the sampling time; After completing the seawater sampling at all target sampling points, the multi-sequence water sampler rises.

6. The multi-sequence submarine intelligent water sampling method based on deep learning according to claim 5 is characterized in that: In the step S4, the preset type of substance to be detected is specifically methane; The environmental parameter sensor unit is provided with a methane concentration sensor, and the methane concentration is detected by using the methane concentration sensor; The several target sampling points are several methane leakage nozzles with different methane concentrations.

7. The multi-sequence submarine intelligent water sampling method based on deep learning according to claim 6 is characterized in that: In step S5, at each target sampling point, the central processing unit determines the sampling time according to the data collected in real time by the underwater camera and the environmental parameter sensor unit, including: S5.1: Using the underwater camera to take images of the target sampling points in real time and transmit the images to the central processing unit; The methane concentration sensor is used to detect the methane concentration at the target sampling point in real time and transmit the methane concentration to the central processing unit; S5.2: bubbles and sediment emitted from the methane seepage vent are selected as characteristic targets, and the central processing unit uses a pre-trained deep learning model to recognize the image of the target sampling point to obtain recognition results of the sediment and bubbles; S5.3: The central processing unit determines in real time whether the identification results of the sediment and bubbles at the target sampling point and the change in methane concentration meet the preset eruption conditions at the same time. If so, this time is used as the sampling time of the target sampling point; otherwise, repeat step S5.

3.

8. The multi-sequence submarine intelligent water sampling method based on deep learning according to claim 7 is characterized in that: In step S5.2, the central processing unit uses a pre-trained deep learning model to identify the background content, the positions of the sediment and the bubble particles in the image of the target sampling point in real time; Classifying the sediment according to the seabed matrix segmentation result, and comparing the sediment categories at different times to obtain the change in the distribution ratio of the sediment; Calculate the bubble rising rate based on the position of the bubble particles at different times; The change in the distribution ratio of the sediment and the rising rate of the bubbles are used as the identification results of the sediment and bubbles.

9. A multi-sequence seabed intelligent water sampling method based on deep learning according to any one of claims 5 to 8, characterized in that: In step S5, a water flow velocity sensor is also provided in the environmental parameter sensor unit; the water flow velocity sensor detects the water flow velocity in real time and transmits it to the central processing unit; The central processing unit controls the power unit according to the water flow rate to drive the multi-sequence water sampler to move to each target sampling point respectively.

Images