Positioning method and system of submarine mining vehicle

Abstract

The invention relates to the field of deep sea exploration, and discloses a positioning method and system for a submarine mining vehicle, so as to improve the positioning accuracy. The method comprises the following steps: extracting a global map spliced by key frames in the exploration process of a to-be-operated target area by the AUV; after the mining vehicle arrives at the target area, the relative orientation relation between the inspection robot and the mining vehicle and the height between the inspection robot and the seabed are calibrated; in the working process of the mining vehicle, the inspection robot collects images of the mining vehicle through a first camera, the distance between the inspection robot and the mining vehicle is obtained according to the collected images of the mining vehicle, and posture adjustment information for maintaining the relative orientation relation between the inspection robot and the mining vehicle is obtained; meanwhile, the inspection robot collects a seabed image under the inspection robot in real time through a second camera according to the calibrated height of the inspection robot and the seabed, and image matching is carried out in a global map according to the current seabed image; and obtaining the positioning information of the mining vehicle according to the longitude and latitude of the matched image in the global map in combination with the distance and the relative orientation relationship between the inspection robot and the mining vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of deep - sea exploration, and particularly to a positioning method and system for a subsea mining vehicle. Background Art

[0002] During the deep - sea mining operation, accurate positioning is crucial to ensure the safe operation of the deep - sea mining vehicle. The existing deep - sea mining vehicle uses acoustic positioning accuracy at 6000 meters. Since the acoustic signal not only changes in propagation speed due to the density and temperature of seawater during transmission in the deep sea, but also the signal attenuation is obvious with the increase of the transmission distance, and there is a certain delay, which leads to a large positioning error when using the acoustic signal to position the deep - sea mining vehicle. The positioning accuracy is about 25m. However, the deep - sea mining vehicle requires high - precision positioning to ensure the accurate movement and operation of the mining vehicle. Due to the obvious lack of accuracy of the acoustic positioning system, deviations may occur during the operation of the mining vehicle, affecting the mining efficiency and safety, and there is currently no good solution. Summary of the Invention

[0003] The purpose of the present invention is to disclose a positioning method and system for a subsea mining vehicle to improve the positioning accuracy.

[0004] To achieve the above - mentioned purpose, the positioning method for the subsea mining vehicle disclosed by the present invention includes: Step S1: Use an AUV (Autonomous Underwater Vehicle) to explore the target operation area. During the exploration process, extract key frames from the exploration video data according to the traveling speed and navigation trajectory of the AUV, calculate the longitude and latitude of each key frame based on the earth coordinate system, and perform splicing and fusion processing on the fusion zones between adjacent key frames to obtain the global map of the target area; Step S2: After the mining vehicle reaches the target area, control the inspection robot electrically connected to the mining vehicle in a wired manner to run above the mining vehicle diagonally, and then calibrate the relative azimuth relationship between the inspection robot and the mining vehicle and the height from the seabed; Step S3: During the operation of the mining vehicle, the inspection robot uses a first camera to collect images of the mining vehicle, and obtains the distance from the mining vehicle and the attitude adjustment information for maintaining the relative azimuth relationship with the mining vehicle according to the collected images of the mining vehicle. At the same time, the inspection robot uses a second camera to collect real - time images of the seabed directly below according to the calibrated height from the seabed, performs image matching in the global map according to the current seabed image, and then obtains the positioning information of the mining vehicle according to the longitude and latitude of the matching image in the global map, combined with the distance and relative azimuth relationship between the inspection robot and the mining vehicle, and the calibrated height from the seabed; Among them, the relative azimuth relationship between the inspection robot and the mining vehicle is used to ensure that the first camera includes the mining vehicle in the target central area of the collected image with a unified observation perspective.

[0005] Preferably, the method of the present invention further includes: Step S4: Calculate the dynamic displacement information of the mining vehicle according to the dynamic change of the positioning information of the mining vehicle.

[0006] Preferably, the inspection robot uses a sonar altimeter to track the height information between itself and the seabed, and ensures that the real-time height from the seabed during the attitude adjustment process is within a required fluctuation range from the calibrated height, so that the distance between the second camera and the seabed directly below remains consistent during the acquisition of a series of seabed images.

[0007] Preferably, during the exploration process of the AUV, the height information from the seabed is maintained within a required fluctuation range (i.e., the shooting distance consistency is satisfied).

[0008] Optionally, the output after inputting the mining vehicle image collected by the first camera into the trained yolov8 network model of the present invention includes the distance between the inspection robot and the mining vehicle, and the attitude adjustment information for maintaining the relative azimuth relationship with the mining vehicle.

[0009] Alternatively, the present invention can calculate the distance between the inspection robot and the mining vehicle, and the attitude adjustment information for maintaining the relative azimuth relationship with the mining vehicle according to the pixel distance of the feature points on the mining vehicle and the pre-calibrated actual distance combined with the information of the field of view angle of the first camera.

[0010] To achieve the above object, the present invention also discloses a positioning system for a seabed mining vehicle, including: An AUV for exploring the target operation area; An inspection robot for collecting images of the mining vehicle with a first camera and collecting images of the seabed directly below in real time with a second camera; A central controller for performing the following steps: Step S1: During the exploration process of the AUV, extract key frames from the exploration video data according to the traveling speed and navigation trajectory of the AUV, calculate the longitude and latitude of each key frame based on the earth coordinate system, and perform splicing and fusion processing on the fusion zones between adjacent key frames to obtain a global map of the target area; Step S2: After the mining vehicle reaches the target area, control the inspection robot electrically connected to the mining vehicle in a wired manner to run to the area at a set height from the seabed directly in front of the mining vehicle; Step S3: During the operation of the mining vehicle, obtain the distance from the mining vehicle based on the image of the mining vehicle captured by the first camera, and the attitude adjustment information for maintaining the relative orientation relationship with the mining vehicle; meanwhile, perform image matching of the current seabed image captured by the second camera in the global map, and then obtain the positioning information of the mining vehicle based on the longitude and latitude of the matching image in the global map, combined with the distance and relative orientation relationship between the inspection robot and the mining vehicle, and the calibrated height from the seabed. Among them, the relative orientation relationship between the inspection robot and the mining vehicle is used to ensure that the first camera includes the mining vehicle in the target central area of the captured image with a unified observation perspective.

[0011] The present invention has the following beneficial effects: 1. First, an AUV explores the operation target area, and constructs a global map of the target area based on the key frames extracted from the video data collected by the AUV, providing safe navigation for the subsequent operation process of the mining vehicle.

[0012] 2. During the operation of the mining vehicle, adjust the attitude of the inspection robot according to the displacement of the mining vehicle, so as to ensure that the first camera includes the mining vehicle in the target central area of the captured image with a unified observation perspective (usually also the best observation perspective). This is not only convenient for manual precise control of the operation process of the mining vehicle based on the image of the first camera, but also convenient for automatic cooperation among various devices based on global information.

[0013] 3. During the positioning of the mining vehicle, first perform image matching of the current image captured by the inspection robot in the global map, and then obtain the positioning information of the mining vehicle based on the longitude and latitude of the matching image in the global map, combined with the distance and azimuth angle between the inspection robot and the mining vehicle, and the calibrated height from the seabed. The positioning accuracy can be improved to within 1 meter; compared with the traditional sonar positioning method with an error of up to 25 meters, the positioning accuracy is greatly improved.

[0014] Next, the present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings forming a part of this application are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings: Figure 1 is a schematic flow chart of the positioning method of the seabed mining vehicle disclosed in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways defined and covered by the claims.

[0017] Embodiment 1 This embodiment discloses a positioning method for a subsea mining vehicle, as Figure 1 shown, including the following steps: Step S1: Use an AUV to explore the target operation area. During the exploration process, extract key frames from the exploration video data according to the traveling speed and navigation trajectory of the AUV, calculate the longitude and latitude of each key frame based on the earth coordinate system, and perform stitching and fusion processing on the fusion zones between adjacent key frames to obtain a global map of the target area.

[0018] In this step, preferably, during the exploration process (excluding the diving and surfacing processes) of the AUV, keep the height information from the seabed within a required fluctuation range (i.e., meet the consistency of the shooting distance), so as to reduce the complexity of preprocessing such as image scaling and correction during the process of determining the specific longitude and latitude of the key frame in the earth coordinate system.

[0019] Generally, during the exploration process, the AUV can uniformly collect images of the seabed geomorphic features according to the planned route, so as to roughly screen out key frames according to the traveling speed. Then, the initially screened key frames can be rechecked in combination with the navigation log of the AUV recording its own navigation trajectory (the relevant data can be calculated from the sensing data of a Doppler log, depth gauge, accelerometer, inertial measurement unit, etc.) so that the overlapping part with the previous key frame meets the set requirements; for example: the range of this overlapping part can be set to be greater than 10% and less than 30%. If necessary, the key frame information of the video data can be manually rechecked and corresponding addition and deletion processing can be performed.

[0020] In summary, the essence of this step is to determine the longitude and latitude of each key frame in the same earth coordinate system; among them, the process of mapping the pixel information of the collected image to the earth coordinate system is prior art and will not be elaborated.

[0021] Among the key frames of this step, the distribution characteristics of geomorphic features such as manganese nodules in the seabed mining area are often unique, thus laying a foundation for subsequent image matching.

[0022] Step S2: After the mining vehicle reaches the target area, control the inspection robot electrically connected to the mining vehicle by wire to run to the upper oblique position of the mining vehicle, and then calibrate the relative azimuth relationship between the inspection robot and the mining vehicle and the height from the seabed.

[0023] In this embodiment, during the diving and surfacing processes of the mining vehicle, an inspection robot with much lower purchase and operation costs than an AUV can be stowed; wherein, the relative azimuth relationship between the inspection robot and the mining vehicle is used to ensure that the first camera includes the mining vehicle in the target central area of the collected images from a unified observation perspective. Optionally, during the calibration process, the optimal relative azimuth relationship between the inspection robot and the mining vehicle can be determined manually.

[0024] Step S3: During the operation of the mining vehicle, the inspection robot uses the first camera to collect images of the mining vehicle, and obtains the distance from the mining vehicle and the attitude adjustment information for maintaining the relative azimuth relationship with the mining vehicle based on the collected images of the mining vehicle; meanwhile, the inspection robot uses the second camera to collect real-time images of the seabed directly below according to the calibrated height from the seabed, performs image matching in the global map based on the current seabed images, and then obtains the positioning information of the mining vehicle based on the longitude and latitude of the matching images in the global map, the distance and relative azimuth relationship between the inspection robot and the mining vehicle, and the calibrated height from the seabed.

[0025] In this step, the positioning information of the mining vehicle can be accurately solved through the geometric position relationship between several key points (mainly the triangular relationship among the seabed coordinates of the center of the mining vehicle, the center coordinates of the inspection robot, and the seabed coordinates directly below the inspection robot; wherein, the main purpose of the attitude adjustment of the inspection robot is to maintain the consistency of this triangular relationship during the dynamic displacement of the mining vehicle, so that the seabed coordinates of the center of the mining vehicle can be accurately located based on the seabed coordinates directly below the inspection robot). Among them, the inspection robot can use a sonar altimeter (whose ranging accuracy at short distances is usually relatively high) to track the height information between itself and the seabed; and ensure that the real-time height from the seabed during the attitude adjustment process is within the required fluctuation range of the calibrated height, so that the distance between the second camera and the seabed directly below remains consistent during the acquisition of a series of seabed images; if the heights are inconsistent, there will be more variables in the geometric position relationship among the aforementioned key points, resulting in a doubling of the computational complexity, ultimately affecting the real-time performance and the accuracy of the result due to the introduction of errors during the calculation process. Optionally, the distance between the inspection robot and the mining vehicle and the attitude adjustment information for maintaining the relative azimuth relationship with the mining vehicle (the attitude can be estimated based on the Lyapunov stability theory, and the attitude of the inspection robot can be adjusted by whether the mining vehicle is in the center of the photo field of view in the captured image) can be obtained through trained network models such as yolov8 or traditional calculation methods based on the pixel distance of the feature points on the mining vehicle (i.e., the number of pixels separated) and the information of the pre-calibrated actual distance combined with the field of view angle of the first camera, which is also prior art and will not be elaborated. When necessary, the distance between the inspection robot and the mining vehicle can also be quickly calculated by combining the distance pre-calibrated by the neural network with the distance constraint information between the corresponding pixel points in the actually captured photo.

[0026] Furthermore, the method of this embodiment further includes: Step S4, calculating the dynamic displacement information of the mining vehicle according to the dynamic change of the positioning information of the mining vehicle; thereby, the dynamic operation trajectory of the mining vehicle can be obtained.

[0027] In addition, when necessary, this embodiment can also increase the texture details of the images collected by the AUV and the inspection robot based on the first and second cameras through existing visual perception closed-loop control algorithms such as Gaussian filtering, Kalman filtering, and / or contrast enhancement, which will not be elaborated here. Meanwhile, during the operation of the mining vehicle, this embodiment replaces the AUV with an inspection robot, which can significantly reduce the equipment purchase cost and operation cost.

[0028] Embodiment 2 This embodiment discloses a positioning system for a subsea mining vehicle, including: An AUV for prospecting the target operation area.

[0029] An inspection robot for collecting images of the mining vehicle with a first camera and collecting real-time images of the seabed directly below with a second camera.

[0030] A central controller for performing the following steps: Step S1, during the prospecting process of the AUV, extracting key frames from the prospecting video data according to the traveling speed and navigation trajectory of the AUV, calculating the longitude and latitude of each key frame based on the earth coordinate system, and performing splicing and fusion processing on the fusion zone between adjacent key frames to obtain a global map of the target area.

[0031] Step S2, after the mining vehicle reaches the target area, controlling the inspection robot electrically connected to the mining vehicle in a wired manner to run to an area at a set height from the seabed directly in front of the mining vehicle.

[0032] Step S3, during the operation of the mining vehicle, obtaining the distance from the mining vehicle according to the image of the mining vehicle collected by the first camera and the attitude adjustment information for maintaining the relative orientation relationship with the mining vehicle; meanwhile, performing image matching on the current seabed image of the second camera in the global map, and then obtaining the positioning information of the mining vehicle according to the longitude and latitude of the matching image in the global map, the distance and relative orientation relationship between the inspection robot and the mining vehicle, and the calibrated height from the seabed.

[0033] Wherein, the relative orientation relationship between the inspection robot and the mining vehicle is used to ensure that the first camera includes the mining vehicle in the target central area of the collected image from a unified observation perspective.

[0034] Similarly, the logic of some detail processing of the system in this embodiment is the same as that in Embodiment 1 and will not be elaborated here.

[0035] In summary, the methods and systems separately disclosed in the above embodiments of the present invention have at least the following beneficial effects: 1. The AUV first explores the operation target area and constructs a global map of the target area based on the key frames extracted from the video data collected by the AUV, providing safe navigation for the subsequent operation process of the mining vehicle.

[0036] 2. During the operation process of the mining vehicle, the attitude of the inspection robot is adjusted according to the displacement of the mining vehicle, so as to ensure that the first camera includes the mining vehicle in the target center area of the collected image with a unified observation perspective (usually also the best observation perspective). This is not only convenient for manual precise control of the operation process of the mining vehicle according to the image of the first camera, but also convenient for automatic collaborative operation among various devices based on global information.

[0037] 3. During the positioning process of the mining vehicle, first perform image matching in the global map based on the current image collected by the inspection robot in real time, and then obtain the positioning information of the mining vehicle according to the latitude and longitude of the matching image in the global map, combined with the distance and azimuth angle between the inspection robot and the mining vehicle, and the calibrated height from the seabed. The positioning accuracy can be improved to within 1 meter. Compared with the traditional sonar positioning method with an error of up to 25 meters, the positioning accuracy is greatly improved.

[0038] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A positioning method for a subsea mining vehicle, characterized in that, Including: Step S1: The AUV is used to explore the target area to be operated. During the exploration process, key frames are extracted from the exploration video data according to the traveling speed and navigation trajectory of the AUV, the longitude and latitude of each key frame based on the earth coordinate system are calculated, and the splicing and fusion processing is performed on the fusion zone between adjacent key frames to obtain the global map of the target area; Step S2: After the mining vehicle arrives at the target area, the inspection robot electrically connected to the mining vehicle in a wired manner is controlled to run above the mining vehicle obliquely, and then the relative azimuth relationship between the inspection robot and the mining vehicle and the height from the seabed are calibrated; Step S3: During the operation of the mining vehicle, the inspection robot uses the first camera to collect images of the mining vehicle, and obtains the distance from the mining vehicle and the attitude adjustment information for maintaining the relative azimuth relationship with the mining vehicle according to the collected images of the mining vehicle; at the same time, the inspection robot uses the second camera to collect the seabed images directly below in real time according to the calibrated height from the seabed, performs image matching in the global map according to the current seabed images, and then obtains the positioning information of the mining vehicle according to the longitude and latitude of the matching images in the global map, the distance and relative azimuth relationship between the inspection robot and the mining vehicle, and the calibrated height from the seabed; Among them, the relative azimuth relationship between the inspection robot and the mining vehicle is used to ensure that the first camera includes the mining vehicle in the target center area of the collected images from a unified observation perspective.

2. The positioning method of the subsea mining vehicle according to claim 1, characterized in that Also including: Step S4: Calculate the dynamic displacement information of the mining vehicle according to the dynamic change of the positioning information of the mining vehicle.

3. The positioning method of the subsea mining vehicle according to claim 1 or 2, characterized in that, The inspection robot uses a sonar altimeter to track the height information between itself and the seabed, and ensures that the real-time height from the seabed during the attitude adjustment process is within the required fluctuation range from the calibrated height, so that the distance between the second camera and the seabed directly below remains consistent during the acquisition of a series of seabed images.

4. The positioning method of the subsea mining vehicle according to claim 3, characterized in that During the exploration process, the AUV maintains the height information from the seabed within the required fluctuation range.

5. The positioning method of the subsea mining vehicle according to claim 1, wherein The distance between the inspection robot and the mining vehicle and the attitude adjustment information for maintaining the relative azimuth relationship with the mining vehicle are obtained through the output after inputting the images of the mining vehicle collected by the first camera into the trained yolov8 network model.

6. The positioning method of the seabed mining vehicle according to claim 1, characterized in that, The distance between the inspection robot and the mining vehicle and the attitude adjustment information for maintaining the relative azimuth relationship with the mining vehicle are calculated according to the pixel distance of the feature points on the mining vehicle, the pre-calibrated actual distance, and the information of the field of view angle of the first camera.

7. A positioning system for a subsea mining vehicle, characterized in that, Including: An AUV, used to explore the target area to be operated; An inspection robot, used to collect images of the mining vehicle with the first camera and collect the seabed images directly below in real time with the second camera; A central controller, used to execute the following steps: Step S1. During the exploration process of the AUV, key frames are extracted from the exploration video data according to the traveling speed and navigation trajectory of the AUV, the longitude and latitude of each key frame based on the earth coordinate system are calculated, and after splicing and fusion processing of the fusion zones between adjacent key frames, a global map of the target area is obtained; Step S2. After the mining vehicle reaches the target area, the inspection robot electrically connected to the mining vehicle in a wired manner is controlled to run to the area at a set height from the seabed directly in front of the mining vehicle; Step S3. During the operation of the mining vehicle, the distance from the inspection robot to the mining vehicle and the attitude adjustment information for maintaining the relative orientation relationship with the mining vehicle are obtained according to the image of the mining vehicle collected by the first camera; at the same time, image matching is performed on the current seabed image of the second camera in the global map, and then the positioning information of the mining vehicle is obtained according to the longitude and latitude of the matching image in the global map, combined with the distance and relative orientation relationship between the inspection robot and the mining vehicle, and the calibrated height from the seabed; Among them, the relative orientation relationship between the inspection robot and the mining vehicle is used to ensure that the first camera includes the mining vehicle in the target central area of the collected image with a unified observation perspective.

8. The positioning system of the subsea mining vehicle according to claim 7, characterized in that, The output after the central controller inputs the image of the mining vehicle collected by the first camera into the trained yolov8 network model includes the distance between the inspection robot and the mining vehicle and the attitude adjustment information for maintaining the relative orientation relationship with the mining vehicle.

9. The positioning system of the seabed mining vehicle according to claim 7, characterized in that, The central controller calculates the distance between the inspection robot and the mining vehicle and the attitude adjustment information for maintaining the relative orientation relationship with the mining vehicle according to the pixel distance of the feature points on the mining vehicle and the pre-calibrated actual distance, combined with the information of the field of view angle of the first camera.

Images