Submarine hydrothermal plume autonomous sampling method based on machine vision and real-time temperature detection

Abstract

The invention discloses a submarine hydrothermal plume autonomous sampling method based on machine vision and real-time temperature detection, and the method comprises the steps: obtaining the pose parameters of a current AUV and a current visual image, and inputting the current visual image to a pre-trained YOLO model, so as to carry out the detection frame labeling of the submarine hydrothermal plume in the visual image; and performing pose parameter coarse adjustment, far-end forward thrust adjustment and near-end thrust adjustment according to the input data, so that the AUV arrives at the position corresponding to the maximum temperature of the submarine hydrothermal plume to perform hovering sampling. According to the method provided by the invention, the power distribution mode of the underwater robot can be effectively optimized, so that the sampling efficiency is effectively improved.

Description

Technical Field

[0001] This invention belongs to the field of deep-sea exploration, and in particular relates to an autonomous sampling method for seafloor hydrothermal plumes based on machine vision and real-time temperature detection. Background Technology

[0002] Targeted sampling of deep-sea hydrothermal plumes enables in-situ, precise capture of bio-mineral interfaces under extreme environments by focusing on the core region of the vent (e.g., the interface between high-temperature fluids at 300-400°C and cold seawater). This technology can obtain the actual survival substrates of microbial communities such as thermophilic archaea and chemosynthetic bacteria in the vent microregion (e.g., biofilms on the surface of nanoscale sulfide particles), directly elucidating the specific expression mechanisms of energy metabolism pathways under high temperature and pressure. Metagenomic data obtained from targeted sampling can trace the horizontal transfer pathways of key genes in the carbon-sulfur cycle, while the isotopic composition of the in-situ captured metal sulfide particles can provide a direct chain of evidence for the hypothesis that "hydrothermal systems are the initial sites of the chemical evolution of life," helping to clarify the key links in the synthesis of organic molecules catalyzed by inorganic minerals, and providing Earth-model samples for the development of extraterrestrial extreme environment life detection technologies.

[0003] Currently, deep-sea hydrothermal plume sampling is mainly conducted using manned submersibles (HOVs) and remotely operated underwater vehicles (ROVs) equipped with samplers. HOVs rely on human operation and are limited by life support systems. In the complex environment of deep-sea hydrothermal vent areas, characterized by high temperatures and strong turbulence, human operation poses high risks; furthermore, the operation is costly and time-consuming. While ROVs can be remotely controlled, the umbilical cable severely restricts their mobility and operational range.

[0004] Patent document CN116681935A discloses an autonomous identification and positioning method and system for deep-sea hydrothermal vents, relating to the field of intelligent data processing technology. This invention performs image enhancement processing on acquired images of seafloor hydrothermal vent areas, then uses the ORB feature extraction algorithm to extract features from the enhanced images. Next, after filtering the enhanced images to obtain candidate regions, feature vectors are generated based on the ORB features and the candidate regions. Then, the feature vectors and candidate regions are input into a classifier to obtain classification results. A non-maximum suppression method is used to filter the classification results to accurately obtain the identification results of the seafloor hydrothermal vents. Finally, the accurate positioning of the seafloor hydrothermal vents is achieved based on the identification results.

[0005] Patent document CN107871115A discloses an image-based method for identifying hydrothermal vents on the seabed, including the following steps: using sparse representation to remove the background and diffused hydrothermal plume, which can accurately segment the textured rocks and the main body of the hydrothermal plume; using optical flow to determine whether the image contains a hydrothermal plume, and segmenting the main body of the hydrothermal plume; and using the grayscale characteristics around the hydrothermal plume vent to effectively determine the specific location of the hydrothermal plume vent. Summary of the Invention

[0006] The purpose of this invention is to provide an autonomous sampling method for seafloor hydrothermal plumes based on machine vision and real-time temperature detection. This method can effectively optimize the power distribution of underwater robots, thereby effectively improving sampling efficiency.

[0007] To achieve the objectives of this invention, the following technical solution is provided: an autonomous sampling method for seafloor hydrothermal plumes based on robot vision and real-time temperature detection, comprising the following steps: Obtain the pose parameters of the current AUV and the current visual image, and input the current visual image into the pre-trained YOLO model to annotate the detection boxes of the hydrothermal plume in the visual image; The deviation between the center point of the visual image and the detection box corresponding to the identified hydrothermal plume is used as input to coarsely adjust the pose parameters of the current AUV so as to adjust the detection box corresponding to the hydrothermal plume to the center point of the visual image. During attitude adjustment, a motion control strategy of first aligning and then moving forward is adopted to adjust the far-end forward thrust until the ambient temperature of the AUV gradually increases. Then, the far-end forward thrust adjustment is switched to the near-end forward thrust adjustment based on the autonomous hovering sampling strategy. The near-end thrust adjustment continuously collects the current ambient temperature to update the maximum temperature. When the maximum temperature is no longer updated, the forward thrust of the current AUV is fine-tuned using the visual image corresponding to the maximum temperature, so that the AUV can hover and sample at the position corresponding to the maximum temperature of the hydrothermal plume on the seabed.

[0008] This invention controls the AUV to align with the hydrothermal plume using a hydrothermal plume bounding box for visual servoing. Once the AUV reaches the vicinity of the plume, it accurately locates the plume based on temperature information and finally hovers near the highest temperature point to complete the hydrothermal plume sampling task.

[0009] Specifically, the pose parameters include the AUV's depth and yaw angle.

[0010] Specifically, the YOLO model also includes a Kalman filter, which is used to predict the presence of hydrothermal vents in the visual image when no hydrothermal vents are detected in the visual image within a certain time period, by using the detection box results of historical visual images containing hydrothermal vents until the hydrothermal vents in the visual image are obtained again. Specifically, ; ; in, Indicates the current depth. This represents the proportional gain of the PID controller in the y-axis direction of the visual image. This represents the integral coefficient of the PID controller in the y-axis direction of the visual image. F represents the derivative coefficient of the PID controller in the y-axis direction of the visual image. OV This indicates the horizontal field of view of the camera mounted on the AUV. cam_width This indicates the horizontal resolution of the camera mounted on the AUV. Deep reference input, This represents the error along the X-axis between the center point of the current detection box and the center point of the visual image. This represents the error along the y-axis between the center point of the current detection box and the center point of the visual image. This represents the error along the y-axis between the center point of the detection box and the center point of the visual image at the previous moment. Indicates the current yaw angle. This indicates the current yaw angle reference input.

[0011] Specifically, the center point of the detection frame is calculated by selecting the coordinate values ​​of two vertices on the diagonal of the detection frame.

[0012] Specifically, the expression for the adjustment of the far-end forward thrust is as follows: ; in, This indicates the maximum forward thrust preset by the AUV. This represents the error along the X-axis between the center point of the detection box and the center point of the visual image. This represents the error along the y-axis between the center point of the detection box and the center point of the visual image.

[0013] Specifically, the autonomous hovering sampling strategy calculates the relative distance between the seafloor hydrothermal plume and the monocular camera mounted on the AUV based on the camera imaging principle and the geometric features of the detection frame, and uses the relative distance as the feedback input for the AUV's advance and retreat control.

[0014] Specifically, the expression for the proximal forward thrust adjustment is as follows: ; ; in, This indicates the relative distance between the bullet screen camera mounted on the AUV and the hydrothermal vent plume. objectwidth This represents the width of the detection box in the visual image. realwidth This indicates the actual width of the vent corresponding to the hydrothermal plume on the seabed. sensorwidth This represents the actual width of a single pixel in a visual image. This indicates the input hover control distance. and This represents the weighting parameter.

[0015] Specifically, the temperature sensor of the AUV is integrated at the end of the sampling tube.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: Image recognition technology is used to identify dynamic hydrothermal plumes, providing corresponding reference data for multi-segment motion adjustments. At the same time, an economical AUV hovering control method is proposed by collecting temperature changes at the hydrothermal vent location, thereby ensuring that the AUV can more accurately approach the high-temperature core region of the hydrothermal plume and improve sampling accuracy. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the autonomous sampling method for submarine hydrothermal plumes provided in this embodiment; Figure 2 This is a schematic diagram of AUV control provided in this embodiment; Figure 3 This is a schematic diagram of the coarse adjustment of the attitude parameters provided in this embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0019] like Figure 1 As shown in this embodiment, an autonomous sampling method for seafloor hydrothermal plumes based on machine vision and real-time temperature detection is provided, including the following steps: Obtain the current AUV's attitude parameters and the current visual image, and input the current visual image into a pre-trained YOLO model to annotate detection boxes for hydrothermal plumes in the visual image; The deviation between the center point of the visual image and the detection box corresponding to the identified hydrothermal plume is used as input to coarsely adjust the pose parameters of the current AUV so as to adjust the detection box corresponding to the hydrothermal plume to the center point of the visual image. During the pose adjustment process, a motion control strategy of first aligning and then moving forward is adopted to adjust the far-end forward thrust until the ambient temperature of the AUV gradually increases. Then, the far-end forward thrust adjustment is switched to the near-end forward thrust adjustment based on the autonomous hovering sampling strategy. The near-end thrust adjustment continuously collects the current ambient temperature to update the maximum temperature. When the maximum temperature is no longer updated, the forward thrust of the current AUV is fine-tuned using the visual image corresponding to the maximum temperature, so that the AUV can hover and sample at the position corresponding to the maximum temperature.

[0020] More specifically, in this embodiment, non-maximum suppression (NMS) based on class is applied to the detection results of the YOLO model. The specific expression is as follows: ; The above formula means that among all bounding boxes belonging to category c, the bounding box that maximizes the confidence score is retained. After NMS, at most two detection boxes are output for each image: the hydrothermal plume detection box and the vent detection box.

[0021] like Figure 2 The diagram shown is a control block diagram of the AUV provided in this embodiment. The visual servo controller acts as the upper-level controller, and its output is the input setpoint of the lower-level AUV controller. Its feedback input is the coordinates of the feature points of the hydrothermal plume target box detected by the YOLOv10 model. The lower-level controller of the AUV is an independent PID controller in the degrees of freedom such as depth, yaw angle, and pitch angle, and a thrust controller in the degrees of freedom of advance and retreat.

[0022] In this embodiment, the adjustment process is divided into three parts. After the plume appears in the camera's field of view, the target detection part identifies and locates the position of the hydrothermal plume and outputs the corresponding detection box. The visual servo controller part calculates the depth, yaw angle, and forward thrust value based on the detection box information. These control parameters serve as input settings for the AUV's underlying controller. When the target detection model cannot identify the plume's position in the current image, the missed detection processing part uses Kalman filtering to update the feature point positions based on the last detection result to help the AUV redetect the hydrothermal plume.

[0023] like Figure 3 As shown, the center of the bounding box output of the target detection is selected as the visual servo feature point, and the coordinate error between the center point of the bounding box and the center point of the image is used as the controller input to ensure that the target remains in the center of its field of view as the AUV approaches the hydrothermal plume.

[0024] A and B are the two corner points of the plume target bounding box, with coordinates of respectively... and Point C is the target feature point calculated from the plume target bounding box, with coordinates as follows: Point D is the center point of the image plane, i.e., the desired feature point, with coordinates of... , and These represent the height and width of the image, respectively. and These represent the errors between the target feature point and the desired feature point along the x-axis and y-axis of the image coordinate system, respectively. ,

[0025] The feature points required for visual servoing are calculated based on the detection box. Then, Kalman filtering is used to smooth the trajectory of the feature points to reduce the influence of system noise and other unstable factors. Finally, the error value between the feature points of the detection box and the center point of the image plane is calculated and output to the visual servo controller.

[0026] In the visual servoing strategy, the control process of an AUV approaching a hydrothermal plume is decoupled into two parts: alignment control and forward control. For alignment control, it is further decoupled into independent controls in the horizontal and vertical directions, with two independent controllers designed to control the AUV's depth and yaw angle, ensuring the target remains centered in the field of view. The vertical PID controller utilizes feature point errors along the y-axis of the image plane. As a feedback input, the output is the depth change.

[0027] AUV current depth Adding the depth change output from the controller as the reference input to the AUV's underlying depth closed-loop controller, then the depth reference input... It can be expressed by the following formula; the horizontal PID controller utilizes the feature point error in the x-axis direction of the image plane. As a feedback input, the output is the change in yaw angle. The current yaw angle of the AUV. Adding the angle change output by the controller as the reference input to the AUV's underlying yaw angle closed-loop controller, then the yaw angle reference input... It can be expressed by the following formula: ; ; in, Indicates the current depth. This represents the proportional gain of the PID controller in the y-axis direction of the visual image. This represents the integral coefficient of the PID controller in the y-axis direction of the visual image. F represents the derivative coefficient of the PID controller in the y-axis direction of the visual image. OV This indicates the horizontal field of view of the camera mounted on the AUV. cam_widthThis indicates the horizontal resolution of the camera mounted on the AUV. Deep reference input, This represents the error along the X-axis between the center point of the current detection box and the center point of the visual image. This represents the error along the y-axis between the center point of the current detection box and the center point of the visual image. This represents the error along the y-axis between the center point of the detection box and the center point of the visual image at the previous moment. Indicates the current yaw angle. This indicates the current yaw angle reference input.

[0028] For the forward and backward control of AUV during visual servoing, this embodiment uses the current target detection box as feedback input and outputs the control force of the current forward and backward direction.

[0029] To minimize the risk of target loss, a "align first, then advance" motion control strategy is adopted. Specifically, when the plume target is far from the center of the field of view in the visual image plane, its forward speed is kept very low, and visual servo alignment control is mainly performed at this time. After aligning with the hydrothermal plume target, its forward speed is increased to enable the AUV to quickly approach the target.

[0030] The expression for adjusting the far-end forward thrust is as follows: ; in, This indicates the maximum forward thrust preset by the AUV. This represents the error along the X-axis between the center point of the detection box and the center point of the visual image. This represents the error along the y-axis between the center point of the detection box and the center point of the visual image.

[0031] Furthermore, for the YOLO model, this embodiment introduces Kalman filtering to handle missed detections of hydrothermal plume locations during the AUV's approach to the plume. When a hydrothermal plume is not detected for a period of time, the Kalman filter can continue to predict the plume's location based on previous state estimates, thereby ensuring the continuity and robustness of visual servoing.

[0032] The subsequent near-end thrust adjustment involves calculating the relative distance between the plume and the monocular camera based on the camera imaging principle and the geometric characteristics of the detection frame, and using this distance as feedback input for AUV advance / retreat control. Simultaneously, the AUV maintains alignment control with the hydrothermal plume based on the target detection results.

[0033] For any point P(X,Y,Z) in space, its image point in the imaging plane is P'(X',Y'). Based on the similarity relation of triangles, we can obtain: ; The distance between the camera and the hydrothermal vent can be further obtained: ; in, This indicates the relative distance between the bullet screen camera mounted on the AUV and the hydrothermal vent plume. objectwidth This represents the width of the detection box in the visual image. realwidth This indicates the actual width of the vent corresponding to the hydrothermal plume on the seabed. sensor width This represents the actual width of a single pixel in a visual image. This represents the input hover control distance. Additionally, Kz is the scaling factor, which is a constant calculated from the measurable data mentioned above.

[0034] The calculated relative distance Z a As feedback input for AUV advance / retreat control, the thrust value, Z, can be obtained. d The hover control distance is the input: ; in, and This represents the weighting parameter.

[0035] To reduce the hovering range of the AUV, this embodiment performs median filtering on the acquired nozzle detection frame width value. Simultaneously, considering that AUV depth control requires a certain convergence time, when the AUV reaches the proximal point of the hydrothermal plume and its depth direction is aligned with the plume, the AUV will maintain a fixed depth control based on the current depth. It should be noted that the hovering process primarily focuses on relative depth values; therefore, the error between the depth calculated by the monocular ranging algorithm and the actual depth will not affect the stability of the hovering control.

[0036] When performing autonomous sampling of hydrothermal plumes, the AUV needs to hover near the plume and extend the sampling tube into it. To accurately locate the plume, this embodiment proposes a hydrothermal vent sampling strategy based on temperature and visual information, taking into account the plume's temperature characteristics. A temperature probe is integrated into the sampling tube. As the AUV passes through the plume, the relative distance between the AUV and the vent at the maximum temperature is recorded, and this distance is used as the control input for AUV hovering control. Throughout the autonomous sampling process, the AUV maintains alignment control with the hydrothermal plume.

[0037] To remove spurious peaks caused by noise, this embodiment employs a moving average window to perform low-pass filtering on the original temperature measurements. During the AUV's passage through the hydrothermal plume, the temperature is continuously monitored. If the current temperature is greater than the previously recorded maximum temperature, the maximum temperature is updated, and the relative distance between the AUV and the hydrothermal vent is recorded. When the temperature gradient stabilizes and the current temperature is significantly lower than the peak temperature, the AUV is considered to have passed through the hydrothermal plume, and the relative distance at the recorded temperature peak is used as the hovering control input.

[0038] Temperature monitoring continues even after the AUV has traversed the hydrothermal plume and entered the hovering control phase. During hovering, if the current temperature value, after low-pass filtering, is greater than the previously recorded maximum temperature value, the maximum temperature record is updated, and this relative distance is simultaneously set as the new hovering control target distance. Subsequent hovering control will be dynamically adjusted based on this updated target distance to ensure the AUV can more accurately approach the high-temperature core region of the hydrothermal plume, improving sampling accuracy.

[0039] Furthermore, the terms "upper," "lower," "inner," "outer," "front," and "rear" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Unless otherwise specifically stated, the relative steps, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0040] Of course, the above description is only a specific embodiment of the present invention and is not intended to limit the scope of the present invention. All equivalent changes or modifications made to the structure, features and principles described in the claims of the present invention should be included in the scope of the claims of the present invention.

[0041] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An autonomous sampling method for submarine hydrothermal plumes based on machine vision and real-time temperature detection, characterized in that, Includes the following steps: Obtain the pose parameters of the current AUV and the current visual image, and input the current visual image into the pre-trained YOLO model to annotate the detection boxes of the hydrothermal plume in the visual image; The deviation between the center point of the visual image and the detection box corresponding to the identified hydrothermal plume is used as input to coarsely adjust the pose parameters of the current AUV so as to adjust the detection box corresponding to the hydrothermal plume to the center point of the visual image. During the pose adjustment process, a motion control strategy of first aligning and then moving forward is adopted to adjust the far-end forward thrust until the ambient temperature of the AUV gradually increases. Then, the far-end forward thrust adjustment is switched to the near-end forward thrust adjustment based on the autonomous hovering sampling strategy. The near-end thrust adjustment continuously collects the current ambient temperature to update the maximum temperature. When the maximum temperature is no longer updated, the forward thrust of the current AUV is fine-tuned using the visual image corresponding to the maximum temperature, so that the AUV can hover and sample at the position corresponding to the maximum temperature of the hydrothermal plume on the seabed.

2. The autonomous sampling method for submarine hydrothermal plumes based on machine vision and real-time temperature detection according to claim 1, characterized in that, The pose parameters include the AUV's depth and yaw angle.

3. The autonomous sampling method for submarine hydrothermal plumes based on machine vision and real-time temperature detection according to claim 1, characterized in that, The YOLO model also includes a Kalman filter, which is used to predict the presence of hydrothermal vents in the visual image when no hydrothermal vents are detected in the visual image within a certain time period, by using the detection box results of historical visual images containing hydrothermal vents until the hydrothermal vents are re-obtained in the visual image.

4. The autonomous sampling method for submarine hydrothermal plumes based on machine vision and real-time temperature detection according to claim 1, characterized in that, The coarse adjustment of the pose parameters of the current AUV is expressed as follows: ； ； in, For image error state update time, Indicates the current depth. This represents the proportional gain of the PID controller in the y-axis direction of the visual image. This represents the integral coefficient of the PID controller in the y-axis direction of the visual image. F represents the derivative coefficient of the PID controller in the y-axis direction of the visual image. OV This indicates the horizontal field of view of the camera mounted on the AUV. cam_width This indicates the horizontal resolution of the camera mounted on the AUV. Deep reference input, This represents the error along the X-axis between the center point of the current detection box and the center point of the visual image. This represents the error along the y-axis between the center point of the current detection box and the center point of the visual image. This represents the error along the y-axis between the center point of the detection box and the center point of the visual image at the previous moment. Indicates the current yaw angle. This indicates the current yaw angle reference input.

5. The autonomous sampling method for submarine hydrothermal plumes based on machine vision and real-time temperature detection according to claim 4, characterized in that, The center point of the detection frame is calculated by selecting the coordinate values ​​of two vertices on the diagonal of the detection frame.

6. The autonomous sampling method for submarine hydrothermal plumes based on machine vision and real-time temperature detection according to claim 1, characterized in that, The expression for adjusting the far-end forward thrust is as follows: ； in, This indicates the maximum forward thrust preset by the AUV. This represents the error along the X-axis between the center point of the detection box and the center point of the visual image. This represents the error along the y-axis between the center point of the detection box and the center point of the visual image.

7. The autonomous sampling method for submarine hydrothermal plumes based on machine vision and real-time temperature detection according to claim 1, characterized in that, The autonomous hovering sampling strategy calculates the relative distance between the seafloor hydrothermal plume and the monocular camera mounted on the AUV based on the camera imaging principle and the geometric features of the detection frame, and uses the relative distance as the feedback input for the AUV's advance and retreat control.

8. The autonomous sampling method for submarine hydrothermal plumes based on machine vision and real-time temperature detection according to claim 1, characterized in that, The expression for the proximal forward thrust adjustment is as follows: ； in, This indicates the relative distance between the bullet screen camera mounted on the AUV and the hydrothermal vent plume. objectwidth This represents the width of the detection box in the visual image. realwidth This indicates the actual width of the vent corresponding to the hydrothermal plume on the seabed. sensor width This represents the actual width of a single pixel in a visual image. This indicates the input hover control distance. and This represents the weighting parameter.

9. The autonomous sampling method for submarine hydrothermal plumes based on machine vision and real-time temperature detection according to claim 1, characterized in that, The temperature sensor of the AUV is integrated at the end of the sampling tube.

Images