Method and system for positioning a seabed mining vehicle

Abstract

The application relates to the field of deep-sea exploration, and discloses a positioning method and system of a seabed mining vehicle, so as to improve positioning accuracy. The method comprises the following steps: extracting key frames in the process of exploration of an AUV on a target region to be operated to splice a global map; after the mining vehicle reaches the target region, calibrating the relative position relationship between a patrol robot and the mining vehicle and the height between the patrol robot and the seabed; in the operation process of the mining vehicle, the patrol robot collects images of the mining vehicle through a first camera, obtains the distance between the patrol robot and the mining vehicle and posture adjustment information for maintaining the relative position relationship according to the collected images of the mining vehicle; meanwhile, the patrol robot collects seabed images directly below in real time through a second camera according to the calibrated height of the patrol robot and the seabed, performs image matching in the global map according to the current seabed images, and then obtains positioning information of the mining vehicle according to the longitude and latitude of the matched images in the global map, the distance between the patrol robot and the mining vehicle and the relative position relationship.

Description

Technical Field

[0001] The present invention relates to the field of deep sea exploration technology, and in particular to a positioning method and system for a seabed mining vehicle. Background Art

[0002] During deep-sea mining operations, precise positioning is crucial to ensuring the safe operation of deep-sea mining vehicles. Existing deep-sea mining vehicles use acoustic positioning accuracy at a depth of 6,000 meters. However, acoustic signals transmitted in the deep sea are subject to changes in propagation speed due to changes in seawater density and temperature. Furthermore, the signal attenuates significantly with increasing transmission distance, resulting in a certain delay. This results in significant positioning errors when using acoustic signals to locate deep-sea mining vehicles, with a positioning accuracy of approximately 25 meters. Deep-sea mining vehicles require high-precision positioning to ensure accurate movement and operation. However, due to the significant inaccuracies of acoustic positioning systems, deviations may occur during operation, impacting mining efficiency and safety. Currently, no effective solution exists. Summary of the Invention

[0003] The present invention aims to disclose a positioning method and system for a seabed mining vehicle to improve positioning accuracy.

[0004] To achieve the above-mentioned purpose, the positioning method of the seabed mining vehicle disclosed in the present invention includes:

[0005] Step S1: Exploring the target area using an AUV (Autonomous Underwater Vehicle). During the exploration process, key frames are extracted from the exploration video data based on the AUV's speed and navigation trajectory. The longitude and latitude of each key frame based on the Earth coordinate system are calculated, and the fusion bands between adjacent key frames are spliced ​​and fused to obtain a global map of the target area.

[0006] Step S2: After the mining vehicle reaches the target area, controlling the inspection robot electrically connected to the mining vehicle in a wired manner to move to a position obliquely above the mining vehicle, and then calibrating the relative position relationship between the inspection robot and the mining vehicle and the height above the seabed;

[0007] Step S3: During the operation of the mining vehicle, the inspection robot uses a first camera to capture images of the mining vehicle, and obtains the distance between the inspection robot and the mining vehicle, as well as posture adjustment information for maintaining a relative orientation with the mining vehicle based on the captured images of the mining vehicle. Simultaneously, the inspection robot uses a second camera to capture images of the seabed directly below in real time based on a calibrated height relative to the seabed, performs image matching on a global map based on the current seabed image, and then obtains positioning information of the mining vehicle based on the latitude and longitude of the matched image on the global map, in combination with the distance and relative orientation between the inspection robot and the mining vehicle, and the calibrated height relative to the seabed.

[0008] The relative orientation relationship between the inspection robot and the mining vehicle is used to ensure that the first camera includes the mining vehicle within the target center area of ​​the captured image with a unified observation angle.

[0009] Preferably, the method of the present invention further comprises:

[0010] Step S4: Calculate the dynamic displacement information of the mining vehicle according to the dynamic changes of the positioning information of the mining vehicle.

[0011] Preferably, the inspection robot uses a sonar altimeter to track its own height information relative to the seabed, and ensures that the real-time height and the calibrated height relative to the seabed are within the required fluctuation range during the posture adjustment process, so that the distance between the second camera and the seabed directly below is maintained consistent during the collection process of the series of seabed images.

[0012] Preferably, during the exploration process, the AUV maintains the height information of the seabed within a required fluctuation range (ie, satisfies the shooting distance consistency).

[0013] Optionally, the output of the present invention after inputting the mining vehicle image captured by the first camera into the trained yolov8 network model includes the distance between the inspection robot and the mining vehicle, and posture adjustment information for maintaining the relative orientation relationship with the mining vehicle.

[0014] Alternatively, the present invention can calculate the distance between the inspection robot and the mining vehicle, and posture adjustment information for maintaining the relative orientation relationship with the mining vehicle based on the pixel distance of the feature points on the mining vehicle and the actual distance calibrated in advance in combination with the information of the field of view of the first camera.

[0015] To achieve the above-mentioned object, the present invention further discloses a positioning system for a seabed mining vehicle, comprising:

[0016] AUV, used to explore the target area for operation;

[0017] The inspection robot is used to collect images of the mining vehicle using a first camera and to collect images of the seabed directly below using a second camera in real time;

[0018] The central controller is used to perform the following steps:

[0019] Step S1: During the exploration process of the AUV, key frames are extracted from the exploration video data according to the travel speed and navigation trajectory of the AUV, the longitude and latitude of each key frame based on the earth coordinate system are calculated, and the fusion bands between adjacent key frames are spliced ​​and fused to obtain a global map of the target area;

[0020] Step S2: After the mining vehicle reaches the target area, controlling the inspection robot electrically connected to the mining vehicle in a wired manner to move to an area directly in front of the mining vehicle at a set height from the seabed;

[0021] Step S3: During the operation of the mining vehicle, the distance between the inspection robot and the mining vehicle and posture adjustment information for maintaining a relative orientation relationship with the mining vehicle are obtained based on the mining vehicle image captured by the first camera. Simultaneously, image matching is performed on a global map based on the current seabed image captured by the second camera. Then, positioning information of the mining vehicle is obtained based on the latitude and longitude of the matched image on the global map, in combination with the distance and relative orientation relationship between the inspection robot and the mining vehicle, and the calibrated height above the seabed.

[0022] The relative orientation relationship between the inspection robot and the mining vehicle is used to ensure that the first camera includes the mining vehicle within the target center area of ​​the captured image with a unified observation angle.

[0023] The present invention has the following beneficial effects:

[0024] 1. The AUV first surveys the target area and constructs a global map of the target area based on key frames extracted from the video data collected by the AUV, providing safe navigation for the mining vehicle's subsequent operations.

[0025] 2. During the operation of the mining vehicle, the inspection robot adjusts its posture according to the displacement of the mining vehicle, thereby ensuring that the first camera includes the mining vehicle in the target center area of ​​the captured image with a unified observation angle (usually also the optimal observation angle). This not only facilitates manual precise control of the mining vehicle operation process based on the image of the first camera, but also facilitates automatic collaborative operation between various devices based on global information.

[0026] 3. During the positioning process of the mining vehicle, image matching is first performed on the global map based on the current image collected in real time by the inspection robot. Then, the positioning information of the mining vehicle is obtained based on the latitude and longitude of the matched image on the global map, combined with the distance and azimuth between the inspection robot and the mining vehicle, and the calibrated height above the seabed. The positioning accuracy can be improved to within 1 meter; compared with the traditional sonar positioning method with an error of up to 25 meters, the positioning accuracy is greatly improved.

[0027] The present invention will be described in further detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0029] Figure 1 It is a schematic flow chart of a method for positioning a seabed mining vehicle disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways as defined and covered by the claims.

[0031] Example 1

[0032] This embodiment discloses a method for positioning a submarine mining vehicle. Figure 1 As shown, the following steps are included:

[0033] Step S1: Use the AUV to explore the target area to be operated. During the exploration process, key frames are extracted from the exploration video data according to the AUV's travel speed and navigation trajectory. The longitude and latitude of each key frame based on the earth coordinate system are calculated, and the fusion bands between adjacent key frames are spliced ​​and fused to obtain a global map of the target area.

[0034] In this step, preferably, the AUV maintains the required fluctuation range of the height information of the seabed (i.e., satisfies the shooting distance consistency) during the exploration process (excluding the diving and surfacing process), so as to reduce the complexity of pre-processing such as scaling and correction of the image in the process of determining the key frame to the specific longitude and latitude of the earth coordinate system.

[0035] Typically, during exploration, an AUV captures images of seafloor features at a constant speed along a planned route. This allows for a rough selection of keyframes based on the vehicle's speed. These pre-selected keyframes are then reviewed using the AUV's navigation log (which can be calculated from sensor data such as the Doppler log, depth gauge, accelerometer, and inertial measurement unit) to ensure that the overlap with the previous keyframe meets a specified requirement; for example, the overlap can be set to greater than 10% and less than 30%. If necessary, the keyframe information in the video data can be manually reviewed and added or deleted accordingly.

[0036] In summary, the essence of this step is to determine the longitude and latitude of each key frame using the same earth coordinate system; among them, the process of mapping the pixel information of the captured image to the earth coordinate system is an existing technology and will not be described in detail.

[0037] In each key frame of this step, the distribution characteristics of landforms such as manganese nodules in the seabed mining area are often unique, thus laying the foundation for subsequent image matching.

[0038] Step S2: After the mining vehicle arrives at the target area, the inspection robot electrically connected to the mining vehicle in a wired manner is controlled to move to an oblique position above the mining vehicle, and then the relative position relationship between the inspection robot and the mining vehicle and the height above the seabed are calibrated.

[0039] In this embodiment, the mining vehicle can accommodate an inspection robot, which has significantly lower purchase and operating costs than an AUV, during its descent and ascent. The relative orientation between the inspection robot and the mining vehicle ensures that the first camera consistently observes the mining vehicle within the target center area of ​​the captured image. Alternatively, during calibration, the optimal relative orientation between the inspection robot and the mining vehicle can be manually determined.

[0040] Step S3: During the operation of the mining vehicle, the inspection robot uses the first camera to collect images of the mining vehicle, and obtains the distance between the inspection robot and the mining vehicle, as well as posture adjustment information for maintaining the relative orientation relationship with the mining vehicle based on the collected images of the mining vehicle; at the same time, the inspection robot uses the second camera to collect images of the seabed directly below in real time based on the calibrated height relative to the seabed, performs image matching on the global map based on the current seabed image, and then obtains the positioning information of the mining vehicle based on the latitude and longitude of the matched image on the global map, combined with the distance and relative orientation relationship between the inspection robot and the mining vehicle, and the calibrated height relative to the seabed.

[0041] In this step, the positioning information of the mining vehicle can be accurately solved through the geometric position relationship between several key points (mainly the triangular relationship between the seabed coordinates of the center of the mining vehicle, the coordinates of the center of the inspection robot, and the coordinates of the seabed directly below the inspection robot; the main purpose of the inspection robot's posture adjustment is to maintain the consistency of this triangular relationship during the dynamic displacement of the mining vehicle, so that the seabed coordinates of the center of the mining vehicle can be quickly and accurately located based on the seabed coordinates directly below the inspection robot). The inspection robot can use a sonar altimeter (whose short-range ranging accuracy is generally relatively high) to track its own height information relative to the seabed; and during the posture adjustment process, ensure that the real-time height and the calibrated height relative to the seabed are within the required fluctuation range, so that the distance between the second camera and the seabed directly below is consistent during the acquisition of the series of seabed images; if the heights are inconsistent, the geometric position relationship between the aforementioned key points will have more variables, thereby doubling the calculation complexity, ultimately affecting real-time performance and affecting the accuracy of the results due to the introduction of errors in the calculation process. Optionally, the distance between the inspection robot and the mining vehicle, as well as its posture to maintain its relative orientation relative to the mining vehicle (posture estimation can be performed based on Lyapunov stability theory, and the inspection robot's posture can be adjusted based on whether the mining vehicle is centered in the field of view of the captured image). This information can be calculated using a trained network model such as Yolov8, or traditional methods based on the pixel distance (i.e., the number of pixels between) of feature points on the mining vehicle and a pre-calibrated actual distance combined with information about the first camera's field of view. This is also prior art and will not be described in detail here. If necessary, the distance between the inspection robot and the mining vehicle can also be rapidly calculated using a neural network that combines the pre-calibrated distance with distance constraints between corresponding pixels in the actual captured image.

[0042] Furthermore, the method of this embodiment further includes: step S4, calculating the dynamic displacement information of the mining vehicle according to the dynamic changes of the positioning information of the mining vehicle; thereby obtaining the dynamic running trajectory of the mining vehicle.

[0043] Furthermore, if necessary, this embodiment can also utilize existing closed-loop visual perception control algorithms such as Gaussian filtering, Kalman filtering, and / or contrast enhancement to enhance the texture detail of images captured by the AUV and inspection robot based on the first and second cameras. This is not detailed here. Furthermore, during mining vehicle operations, this embodiment can significantly reduce equipment acquisition and operating costs by replacing the AUV with an inspection robot.

[0044] Example 2

[0045] This embodiment discloses a positioning system for a seabed mining vehicle, comprising:

[0046] AUV is used to explore the target area for operation.

[0047] The inspection robot is used to collect images of the mining vehicle with a first camera and to collect images of the seabed directly below with a second camera in real time.

[0048] The central controller is used to perform the following steps:

[0049] Step S1: During the AUV exploration process, key frames are extracted from the exploration video data according to the AUV's travel speed and navigation trajectory. The longitude and latitude of each key frame based on the earth coordinate system are calculated, and the fusion bands between adjacent key frames are spliced ​​and fused to obtain a global map of the target area.

[0050] Step S2: After the mining vehicle reaches the target area, the inspection robot electrically connected to the mining vehicle in a wired manner is controlled to move to an area in front of the mining vehicle at a set height from the seabed.

[0051] Step S3: During the operation of the mining vehicle, the distance between the inspection robot and the mining vehicle and the posture adjustment information for maintaining the relative orientation relationship with the mining vehicle are obtained based on the mining vehicle image captured by the first camera; at the same time, image matching is performed in the global map based on the current seabed image of the second camera, and then the positioning information of the mining vehicle is obtained based on the latitude and longitude of the matched image in the global map combined with the distance and relative orientation relationship between the inspection robot and the mining vehicle, and the calibrated height with the seabed.

[0052] The relative orientation relationship between the inspection robot and the mining vehicle is used to ensure that the first camera includes the mining vehicle within the target center area of ​​the captured image with a unified observation angle.

[0053] Similarly, some detailed processing logic of the system in this embodiment is the same as that in embodiment 1 and will not be described in detail.

[0054] In summary, the methods and systems disclosed in the above embodiments of the present invention respectively have at least the following beneficial effects:

[0055] 1. The AUV first surveys the target area and constructs a global map of the target area based on key frames extracted from the video data collected by the AUV, providing safe navigation for the mining vehicle's subsequent operations.

[0056] 2. During the operation of the mining vehicle, the inspection robot adjusts its posture according to the displacement of the mining vehicle, thereby ensuring that the first camera includes the mining vehicle in the target center area of ​​the captured image with a unified observation angle (usually also the optimal observation angle). This not only facilitates manual precise control of the mining vehicle operation process based on the image of the first camera, but also facilitates automatic collaborative operation between various devices based on global information.

[0057] 3. During the positioning process of the mining vehicle, image matching is first performed on the global map based on the current image collected in real time by the inspection robot. Then, the positioning information of the mining vehicle is obtained based on the latitude and longitude of the matched image on the global map, combined with the distance and azimuth between the inspection robot and the mining vehicle, and the calibrated height above the seabed. The positioning accuracy can be improved to within 1 meter; compared with the traditional sonar positioning method with an error of up to 25 meters, the positioning accuracy is greatly improved.

[0058] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for positioning a seabed mining vehicle, characterized in that: include: Step S1: Exploring the target area for operation by using an AUV. During the exploration process, key frames are extracted from the exploration video data according to the travel speed and navigation trajectory of the AUV, the longitude and latitude of each key frame based on the earth coordinate system are calculated, and the fusion bands between adjacent key frames are spliced ​​and fused to obtain a global map of the target area; Step S2: After the mining vehicle reaches the target area, controlling the inspection robot electrically connected to the mining vehicle in a wired manner to move to a position obliquely above the mining vehicle, and then calibrating the relative position relationship between the inspection robot and the mining vehicle and the height above the seabed; Step S3: During the operation of the mining vehicle, the inspection robot uses a first camera to capture images of the mining vehicle, and obtains the distance between the inspection robot and the mining vehicle, as well as posture adjustment information for maintaining a relative orientation with the mining vehicle based on the captured images of the mining vehicle. Simultaneously, the inspection robot uses a second camera to capture images of the seabed directly below in real time based on a calibrated height relative to the seabed, performs image matching on a global map based on the current seabed image, and then obtains positioning information of the mining vehicle based on the latitude and longitude of the matched image on the global map, in combination with the distance and relative orientation between the inspection robot and the mining vehicle, and the calibrated height relative to the seabed. The relative orientation relationship between the inspection robot and the mining vehicle is used to ensure that the first camera includes the mining vehicle within the target center area of ​​the captured image with a unified observation angle.

2. The method for positioning a seabed mining vehicle according to claim 1, wherein: Also includes: Step S4: Calculate the dynamic displacement information of the mining vehicle according to the dynamic changes of the positioning information of the mining vehicle.

3. The method for positioning a seabed mining vehicle according to claim 1 or 2, wherein: The inspection robot uses a sonar altimeter to track its own height information relative to the seabed, and ensures that the real-time height and the calibrated height relative to the seabed are within the required fluctuation range during the posture adjustment process, so that the distance between the second camera and the seabed directly below is maintained consistent during the collection process of a series of seabed images.

4. The method for positioning a seabed mining vehicle according to claim 3, wherein: During the exploration process, the AUV maintains altitude information relative to the seabed within a required fluctuation range.

5. The method for positioning a seabed mining vehicle according to claim 1, wherein: The distance between the inspection robot and the mining vehicle, and the posture adjustment information for maintaining the relative orientation relationship with the mining vehicle are obtained by inputting the mining vehicle image captured by the first camera into the output of the trained yolov8 network model.

6. The method for positioning a seabed mining vehicle according to claim 1, wherein: The distance between the inspection robot and the mining vehicle and the posture adjustment information for maintaining the relative orientation relationship with the mining vehicle are calculated based on the pixel distance of the feature points on the mining vehicle and the actual distance calibrated in advance in combination with the information of the field of view of the first camera.

7. A positioning system for a seabed mining vehicle, characterized in that: include: AUV, used to explore the target area for operation; The inspection robot is used to collect images of the mining vehicle using a first camera and to collect images of the seabed directly below using a second camera in real time; The central controller is used to perform the following steps: Step S1: During the exploration process of the AUV, key frames are extracted from the exploration video data according to the travel speed and navigation trajectory of the AUV, the longitude and latitude of each key frame based on the earth coordinate system are calculated, and the fusion bands between adjacent key frames are spliced ​​and fused to obtain a global map of the target area; Step S2: After the mining vehicle reaches the target area, controlling the inspection robot electrically connected to the mining vehicle in a wired manner to move to an area directly in front of the mining vehicle at a set height from the seabed; Step S3: During the operation of the mining vehicle, the distance between the inspection robot and the mining vehicle and posture adjustment information for maintaining a relative orientation relationship with the mining vehicle are obtained based on the mining vehicle image captured by the first camera. Simultaneously, image matching is performed on a global map based on the current seabed image captured by the second camera. Then, positioning information of the mining vehicle is obtained based on the latitude and longitude of the matched image on the global map, in combination with the distance and relative orientation relationship between the inspection robot and the mining vehicle, and the calibrated height above the seabed. The relative orientation relationship between the inspection robot and the mining vehicle is used to ensure that the first camera includes the mining vehicle within the target center area of ​​the captured image with a unified observation angle.

8. The positioning system for a seabed mining vehicle according to claim 7, wherein: The central controller inputs the mining vehicle image captured by the first camera into the trained yolov8 network model, and the output includes the distance between the inspection robot and the mining vehicle, and the posture adjustment information for maintaining the relative orientation relationship with the mining vehicle.

9. The positioning system for a seabed mining vehicle according to claim 7, wherein: The central controller calculates the distance between the inspection robot and the mining vehicle, and posture adjustment information for maintaining a relative orientation relationship with the mining vehicle based on the pixel distance of the feature points on the mining vehicle and the actual distance calibrated in advance, combined with information on the field of view of the first camera.

Images