Navigation control method and device of autonomous underwater vehicle, electronic equipment and medium

Abstract

The invention provides a navigation control method and device of an underwater autonomous vehicle, electronic equipment and a medium, and the method comprises the following steps: inputting position information, a camera image and a sonar image of the underwater autonomous vehicle into a navigation control model to obtain a yaw angle, a pitch angle and a speed of the underwater autonomous vehicle; the navigation control model is obtained by training based on a sample camera image, a sample sonar image and sample position information as well as a label yaw angle, a label pitch angle and a label speed of the autonomous underwater vehicle. According to the method, by fusing position information, a camera image and a sonar image, a navigation control model can obtain more comprehensive environment perception capability than any single sensor, so that the navigation control model can still make a comprehensive decision according to local fine visual information and position information provided by the camera image in a complex dynamic underwater environment; the defects of a single sensor are effectively compensated, and the navigation reliability of the autonomous underwater vehicle in a complex dynamic environment is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of underwater robot navigation and control technology, and in particular to a navigation and control method, device, electronic equipment and medium for an underwater autonomous vehicle. Background Technology

[0002] The underwater environment presents unique challenges to robust underwater navigation due to limited visibility and unreliable GPS signals, with sensor performance limitations being a common issue. Traditional visual simultaneous localization and mapping (SLAM) methods, such as SLAM and end-to-end neural network control, are often insufficient for reliable underwater positioning and navigation because they rely too heavily on single sensor inputs. This limitation is particularly pronounced considering the complexity and dynamics of the marine environment. Cameras have extremely low visibility in turbid or deep waters, sonar systems can only provide partial spatial coverage, and underwater position information can only be acquired using the underwater robot's own sensors. These challenges undoubtedly highlight the importance of combining multiple sensor modalities for complementary purposes to improve the reliability and accuracy of underwater navigation.

[0003] In recent years, driven by the rapid development of deep learning, end-to-end control methods have become a promising paradigm in control theory. This method directly maps sensory input to control output, mimicking human decision-making processes while avoiding the complexity of traditional control algorithms. Furthermore, by enabling deep neural networks to learn from driving data from human operators, it provides an effective alternative to traditional labor-intensive algorithmic control methods. Moreover, most underwater navigation systems are primarily associated with limited sensory input, mainly cameras and sonar. However, vision-based control systems rely heavily on the quality of input information. For example, visual information about the environment surrounding a mobile robot can be processed using a CNN to directly generate linear and angular velocities. The XX team developed an end-to-end control system for autonomous vehicle navigation that combines LiDAR and vision, imbuing visual images with environmental depth information. Undoubtedly, vision-based navigation control is a good choice under favorable environmental conditions. However, underwater, vision systems face challenges from suspended particles and reflections, resulting in significantly reduced visual information compared to land and air. Therefore, acoustic-based navigation is a central focus of underwater navigation engineering research. XX employs mechanically scanned sonar data and Dijkstra's algorithm to facilitate path planning for underwater vehicles; XX also utilizes two-dimensional forward-looking sonar for underwater navigation. However, when faced with complex and dynamic underwater environments, the limited range of sonar undoubtedly renders navigation unreliable. The limitations of these single-sensor methods highlight the need for more comprehensive environmental awareness in underwater navigation. Summary of the Invention

[0004] This invention provides a navigation control method, device, electronic equipment, and medium for an underwater autonomous vehicle, which addresses the shortcomings of existing underwater navigation methods that use a single sensor, which struggle to ensure the reliability of navigation information in complex and dynamic underwater environments, and whose limited detection range restricts the improvement of navigation accuracy of underwater autonomous vehicles.

[0005] This invention provides a navigation and control method for an underwater autonomous vehicle, comprising the following steps: Acquire camera and sonar images captured by an underwater autonomous vehicle; The position information of the underwater autonomous vehicle, the camera image, and the sonar image are input into the navigation control model to obtain the yaw angle, pitch angle, and velocity of the underwater autonomous vehicle output by the navigation control model. The navigation control model is trained based on sample camera images, sample sonar images, and sample position information of the underwater autonomous vehicle, as well as the tag yaw angle, tag pitch angle, and tag velocity of the underwater autonomous vehicle.

[0006] According to the navigation control method for an underwater autonomous vehicle provided by the present invention, the training steps of the navigation control model include: Obtain the initial navigation control model; The sample camera image, the sample sonar image, and the sample position information are input into the initial navigation control model to obtain the predicted yaw angle, predicted pitch angle, and predicted velocity output by the initial navigation control model. The yaw loss is determined based on the difference between the predicted yaw angle and the tag yaw angle; The pitch angle loss is determined based on the difference between the predicted pitch angle and the tag pitch angle; Based on the difference between the predicted speed and the label speed, determine the speed loss; Based on the yaw loss, pitch angle loss, and velocity loss, the target loss is determined, and based on the target loss, the parameters of the initial navigation control model are iterated to obtain the navigation control model.

[0007] According to the present invention, a navigation control method for an underwater autonomous vehicle is provided, wherein the navigation control model includes a feature extraction layer, a feature fusion layer, and a prediction layer; The feature extraction layer includes a location information extraction layer, a camera feature extraction layer, and a sonar feature extraction layer; the location information extraction layer is used to extract location features from the location information, the camera feature extraction layer is used to extract camera features from the camera image, and the sonar feature extraction layer is used to extract sonar features from the sonar image. The feature fusion layer is used to perform self-attention calculation on the location features, camera features, and sonar features, determine the correlation between the location features, camera features, and sonar features, and perform feature fusion on the location features, camera features, and sonar features based on the correlation to obtain fused features; The prediction layer is used to perform navigation control prediction based on the fused features to obtain the yaw angle, the pitch angle, and the speed.

[0008] According to the navigation and control method for an underwater autonomous vehicle provided by the present invention, the position features are determined based on the starting position information, current position information and ending position information of the underwater autonomous vehicle.

[0009] According to the navigation and control method of an underwater autonomous vehicle provided by the present invention, the sample camera image, the sample sonar image and the sample position information are obtained by controlling the underwater autonomous vehicle to navigate in an underwater simulator and periodically collecting the camera image, sonar image and position information captured by the underwater autonomous vehicle according to a preset acquisition rule; The underwater simulator is equipped with preset seabed topography and multiple obstacles, and visual occlusion is provided to limit the visible distance of the camera images to a preset range.

[0010] According to the navigation and control method of an underwater autonomous vehicle provided by the present invention, the preset acquisition rules include a first rule, a second rule, and a third rule; The first rule is to avoid collisions between the autonomous underwater vehicle and the obstacle or the preset seabed topography; The second rule is to maintain the navigation distance between the underwater autonomous vehicle and the preset seabed topography within a preset safe distance range; The third rule is to prioritize avoiding obstacles by using a smaller yaw angle, provided that the first rule is met.

[0011] The present invention also provides a navigation and control device for an underwater autonomous vehicle, comprising the following units: The acquisition unit is used to acquire camera and sonar images captured by the underwater autonomous vehicle; The input unit is used to input the position information of the underwater autonomous vehicle, the camera image and the sonar image into the navigation control model to obtain the yaw angle, pitch angle and velocity of the underwater autonomous vehicle output by the navigation control model; The navigation control model is trained based on sample camera images, sample sonar images, and sample position information of the underwater autonomous vehicle, as well as the tag yaw angle, tag pitch angle, and tag velocity of the underwater autonomous vehicle.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the navigation control method of any of the above-described underwater autonomous vehicles.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the navigation control method of any of the above-described underwater autonomous vehicles.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the navigation control method of any of the above-described underwater autonomous vehicles.

[0015] The navigation and control method, apparatus, electronic equipment, and medium for underwater autonomous vehicles (AUVs) provided by this invention input the AUV's position information, camera images, and sonar images into a navigation and control model to obtain the AUV's yaw angle, pitch angle, and velocity output by the navigation and control model. The navigation and control model is trained based on sample camera images, sample sonar images, and sample position information of the AUV, as well as the AUV's labeled yaw angle, labeled pitch angle, and labeled velocity. By fusing heterogeneous sensor information from different physical principles—position information, camera images, and sonar images—the navigation and control model can obtain a more comprehensive environmental perception capability than any single sensor. Thus, when facing complex and dynamic underwater environments, the navigation and control model can still make comprehensive decisions based on the local fine visual information and position information provided by camera images, effectively compensating for the deficiencies of a single sensor, and ultimately outputting stable yaw angle, pitch angle, and velocity control commands, significantly improving the navigation reliability and robustness of the AUV in complex and dynamic environments. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the navigation and control method for an underwater autonomous vehicle provided by the present invention.

[0018] Figure 2 This is a schematic diagram of the navigation control model provided by the present invention.

[0019] Figure 3 The location provided by this invention Figure 3 A schematic diagram of the D model.

[0020] Figure 4 This is a schematic diagram illustrating the setup of the simulation environment and the visualization effect of the sensors provided by the present invention.

[0021] Figure 5 This is a schematic diagram of a portion of the experimental trajectory provided by the present invention.

[0022] Figure 6 This is a schematic diagram of the navigation and control device for the underwater autonomous vehicle provided by the present invention.

[0023] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] This invention provides a navigation and control method for an underwater autonomous vehicle. Figure 1 This is a flowchart illustrating the navigation and control method for an underwater autonomous vehicle provided by the present invention, as shown below. Figure 1 As shown, the method includes steps 110 and 120.

[0026] Step 110: Acquire camera and sonar images taken by the underwater autonomous vehicle.

[0027] Specifically, camera and sonar images captured by an autonomous underwater vehicle (AUV) can be acquired. When navigating in an underwater environment, the AUV perceives its surroundings through its onboard sensor system. This sensor system includes at least a camera and sonar, an attitude sensor, and a depth sensor, among others; however, this embodiment of the invention does not specifically limit the specific components. Here, the attitude sensor incorporates an IMU (Inertial Measurement Unit).

[0028] Here, camera images are visual image data of the environment in front of or around the underwater autonomous vehicle, captured by cameras mounted on the vehicle. The camera can be a monocular camera, a binocular stereo camera, or an infrared camera, etc. In some implementations, to more realistically simulate underwater visually limited environments, such as turbid water, the images acquired by the camera may be processed, or visual obstructions may be placed in the simulated environment to limit the effective viewing distance of the camera to a preset range, such as 5 to 8 meters, beyond which objects become blurred or invisible.

[0029] Here, sonar images are acoustic image data about underwater terrain, obstacles, or other objects detected by sonar equipment aboard an autonomous underwater vehicle. The sonar can be forward-looking sonar, side-scan sonar, or multibeam sonar, etc. For example, a multibeam sonar such as the BlueView P900 can be used, whose raw acoustic echo data is processed and visualized to form two-dimensional or three-dimensional sonar images. These sonar images can provide crucial information about the outline and distance of obstacles in visually limited conditions, such as murky water or low light.

[0030] Step 120: Input the position information of the underwater autonomous vehicle, the camera image and the sonar image into the navigation control model to obtain the yaw angle, pitch angle and velocity of the underwater autonomous vehicle output by the navigation control model; The navigation control model is trained based on sample camera images, sample sonar images, and sample position information of the underwater autonomous vehicle, as well as the tag yaw angle, tag pitch angle, and tag velocity of the underwater autonomous vehicle.

[0031] Specifically, the position information, camera images, and sonar images of the underwater autonomous vehicle can be input into the navigation control model to obtain the yaw angle, pitch angle, and speed of the underwater autonomous vehicle output by the navigation control model.

[0032] Here, the position information of the underwater autonomous vehicle is used to characterize the spatial position of the underwater autonomous vehicle in a certain coordinate system. This position information can take many forms, such as the three-dimensional Cartesian coordinates (X, Y, Z coordinates) of the underwater autonomous vehicle in a global or local coordinate system, displacement information relative to the starting point, or real-time position data obtained through an inertial navigation system (INS), a Doppler Velocity Log (DVL), or an underwater acoustic positioning system. This embodiment of the invention does not specifically limit this.

[0033] The navigation control model is a trained computational model capable of mapping input environmental perception information and position information into output navigation control commands. In implementations, this navigation control model can be a deep neural network model, a support vector machine, a decision tree, or other machine learning model capable of handling complex nonlinear mapping relationships; this embodiment of the invention does not specifically limit its application. The core function of this navigation control model is to integrate information from multiple sources and make intelligent navigation decisions accordingly.

[0034] Here, the yaw angle, pitch angle, and velocity output by the navigation control model are key parameters used to directly or indirectly control the motion state of the underwater autonomous vehicle.

[0035] The yaw angle is used to control the horizontal turning of the autonomous underwater vehicle. For example, the output yaw angle can be a continuous angle value or a series of discrete yaw angles, such as seven yaw angles: [-3θ, -2θ, -θ, 0, θ, 2θ, 3θ], where θ is a basic unit of yaw angle.

[0036] Pitch angle is used to control the vertical pitch of an autonomous underwater vehicle. Similarly, the output pitch angle can be a continuous value or discrete levels, such as five levels: [-2φ, -φ, 0, φ, 2φ], where φ is a basic pitch angle unit.

[0037] Speed ​​is used to control the forward speed of the autonomous underwater vehicle (AUV). The output speed can be a continuous value or discrete levels; for example, it can be set to two levels, high speed and low speed, to slow down when encountering complex obstacles and ensure safety. It's understandable that the speed setting is for the AUV to slow down when facing obstacles to ensure its safety.

[0038] Here, the navigation control model is trained based on sample camera images, sample sonar images, and sample position information from the underwater autonomous vehicle (AUV), as well as the AUV's labeled yaw angle, labeled pitch angle, and labeled velocity. Specifically, a dataset containing a large amount of sample data needs to be constructed first. Each data point in the dataset contains a set of inputs and a corresponding set of expected outputs. The sample camera images, sample sonar images, and sample position information constitute the input samples for the navigation control model. This sample data can be collected by experts manually controlling the AUV to navigate in real or simulated underwater environments. During the collection process, the image data acquired by the vehicle's sensors and its own position information are recorded periodically.

[0039] The tag yaw angle, tag pitch angle, and tag velocity constitute the output targets that the navigation control model aims to learn. These tag data represent the actual control commands issued by experts when operating the aircraft.

[0040] By providing a large number of sample inputs and label output pairs to the navigation control model and employing specific optimization algorithms, such as gradient descent, to continuously adjust the internal parameters of the navigation control model, the model's output for any given sample input can increasingly approximate the corresponding label output. After sufficient training, the resulting navigation control model possesses the ability to mimic expert navigation control.

[0041] The method provided in this invention inputs the position information, camera images, and sonar images of an underwater autonomous vehicle (AUV) into a navigation control model to obtain the yaw angle, pitch angle, and velocity of the AUV output by the navigation control model. The navigation control model is trained based on sample camera images, sample sonar images, and sample position information of the AUV, as well as the labeled yaw angle, labeled pitch angle, and labeled velocity of the AUV. By fusing heterogeneous sensor information from different physical principles—position information, camera images, and sonar images—the navigation control model can obtain a more comprehensive environmental perception capability than any single sensor. Thus, when facing complex and dynamic underwater environments, the navigation control model can still make comprehensive decisions based on the local fine visual information and position information provided by the camera images, effectively compensating for the deficiencies of a single sensor, and ultimately outputting stable yaw angle, pitch angle, and velocity control commands, significantly improving the navigation reliability and robustness of the AUV in complex and dynamic environments.

[0042] Based on the above embodiments, the training steps of the navigation control model include: Step 210: Obtain the initial navigation control model; Step 220: Input the sample camera image, the sample sonar image, and the sample position information into the initial navigation control model to obtain the predicted yaw angle, predicted pitch angle, and predicted velocity output by the initial navigation control model; Step 230: Determine the yaw loss based on the difference between the predicted yaw angle and the tag yaw angle; Step 240: Determine the pitch angle loss based on the difference between the predicted pitch angle and the tag pitch angle; Step 250: Determine the speed loss based on the difference between the predicted speed and the label speed; Step 260: Based on the yaw loss, pitch angle loss and velocity loss, determine the target loss, and based on the target loss, perform parameter iteration on the initial navigation control model to obtain the navigation control model.

[0043] Specifically, firstly, an initial navigation control model can be obtained. The parameters of the initial navigation control model can be preset or randomly generated. This embodiment of the invention does not impose any specific limitations on this.

[0044] Then, the sample camera images, sample sonar images, and sample position information can be input into the initial navigation control model to obtain the predicted yaw angle, predicted pitch angle, and predicted velocity output by the initial navigation control model.

[0045] Furthermore, the yaw loss can be determined based on the difference between the predicted yaw angle and the labeled yaw angle. Based on the difference between the predicted pitch angle and the label pitch angle, the pitch angle loss is determined. Based on the difference between the predicted speed and the labeled speed, the speed loss is determined. .

[0046] Here, yaw loss Pitch angle loss and speed loss Both methods use cross-entropy loss, and the formula for cross-entropy loss is as follows: in, Represents cross-entropy loss, Indicates the number of categories, Represents the true label of class K. This represents the predicted probability that the input belongs to category K.

[0047] Understandably, the greater the difference between the predicted yaw angle and the tag yaw angle, the greater the yaw loss; conversely, the smaller the difference between the predicted yaw angle and the tag yaw angle, the smaller the yaw loss.

[0048] Understandably, the greater the difference between the predicted pitch angle and the label pitch angle, the greater the pitch angle loss; conversely, the smaller the difference between the predicted pitch angle and the label pitch angle, the smaller the pitch angle loss.

[0049] Understandably, the greater the difference between the prediction speed and the label speed, the greater the speed loss; the smaller the difference between the prediction speed and the label speed, the smaller the speed loss.

[0050] Here, the target loss can be determined based on the sum of yaw loss, pitch loss, and speed loss, or based on the weighted sum of yaw loss, pitch loss, and speed loss. Based on the target loss, the parameters of the initial navigation control model are iterated, and the initial navigation control model after parameter iteration is used as the navigation control model.

[0051] Here, the formula for the target loss is as follows: in, Indicates target loss. Indicates yaw loss, Indicates pitch angle loss. The velocity loss is represented by the fact that navigation control models are complex and prone to overfitting during training; therefore, L2 regularization can be introduced. This is the regularization strength parameter of L2 regularization, which is generally taken as... , This represents all the weight parameters of the initial navigation control model.

[0052] The method provided in this invention inputs sample camera images, sample sonar images, and sample position information of an underwater autonomous vehicle into an initial navigation control model. The predicted yaw angle, predicted pitch angle, and predicted velocity output from this model are compared with the corresponding tag yaw angle, tag pitch angle, and tag velocity, respectively, to construct yaw loss, pitch loss, and velocity loss sequentially. A target loss is then constructed by combining these three losses, and the parameters of the initial navigation control model are iteratively optimized based on this target loss. This allows the finally trained navigation control model to simultaneously and accurately learn the intrinsic relationship between the vehicle's attitude angles and motion velocity. This multi-task joint supervised training method enables the navigation control model to no longer understand attitude or velocity in isolation, but to establish a unified and coordinated perception capability of the underwater vehicle's motion state. Therefore, when facing complex underwater environments, it can output more coordinated and consistent control commands in terms of spatial attitude and motion trajectory, significantly enhancing the overall stability and accuracy of navigation control.

[0053] Based on the above embodiments, the navigation control model includes a feature extraction layer, a feature fusion layer, and a prediction layer; The feature extraction layer includes a location information extraction layer, a camera feature extraction layer, and a sonar feature extraction layer; the location information extraction layer is used to extract location features from the location information, the camera feature extraction layer is used to extract camera features from the camera image, and the sonar feature extraction layer is used to extract sonar features from the sonar image. The feature fusion layer is used to perform self-attention calculation on the location features, camera features, and sonar features, determine the correlation between the location features, camera features, and sonar features, and perform feature fusion on the location features, camera features, and sonar features based on the correlation to obtain fused features; The prediction layer is used to perform navigation control prediction based on the fused features to obtain the yaw angle, the pitch angle, and the speed.

[0054] Specifically, the navigation control model includes a feature extraction layer, a feature fusion layer, and a prediction layer. The feature extraction layer includes a location information extraction layer, a camera feature extraction layer, and a sonar feature extraction layer. The location information extraction layer is used to extract location features from the location information, the camera feature extraction layer is used to extract camera features from the camera image, and the sonar feature extraction layer is used to extract sonar features from the sonar image.

[0055] Here, the camera feature extraction layer and the sonar feature extraction layer can be a Res2Net network, a cascaded multilayer convolutional neural network (CNN), a deep neural network (DNN), or a combination of CNN and DNN, etc. The embodiments of the present invention do not specifically limit this.

[0056] It should be understood that the Res2Net network can enhance the extraction of multi-scale features by constructing hierarchical residual connections without significantly increasing computational costs, making it very suitable for handling complex visual scenes.

[0057] Here, location information, such as a 1×9 vector, can be input into one or more fully connected dense layers and processed using a non-linear activation function, such as ReLU (Rectified Linear Unit). The final output is a location feature with the same dimensions as the other modal features. This location feature can be represented by f. p ∈R 1 ×2048 express.

[0058] Here, camera features and sonar features can be represented by f respectively. i ∈ R 1×2048 and fs ∈ R 1×2048 express.

[0059] Here, the feature fusion layer is used to perform self-attention calculation on location features, camera features, and sonar features, determine the correlation between location features, camera features, and sonar features, and perform feature fusion on location features, camera features, and sonar features based on the correlation to obtain fused features.

[0060] One specific implementation is to combine the three feature vectors [f] i , f s , f p The features are concatenated into a longer feature vector f. c ∈ R 1×6144 Then, f c The input is fed into a dense layer and a softmax activation function, and three weight coefficients corresponding to camera, sonar, and location features are calculated [a]. i , a s , a p ], and a i + a s +a p = 1. These three weighting coefficients indicate which modality's information the navigation control model considers more important, i.e., more relevant, in the current decision-making process.

[0061] Then, based on the relevance, i.e., the weight coefficients, feature fusion is performed on the location features, camera features, and sonar features to obtain the fused features. The specific integration method is as follows: This approach allows the navigation control model to dynamically adjust its level of trust in different sensors based on the input data.

[0062] Here, the prediction layer is used to predict navigation control based on fused features, obtaining yaw angle, pitch angle, and velocity. Specifically, the fused features are input into one or more fully connected dense layers and processed by an appropriate activation function, such as ReLU, to finally output the predicted yaw angle, pitch angle, and velocity.

[0063] The method provided in this invention, through the design of a layered and decoupled model structure, makes the navigation control model's functions clear. Based on this, a self-attention mechanism is innovatively introduced into the feature fusion layer, enabling the navigation control model to dynamically and adaptively adjust the weight ratio of different sensor information in the fusion decision based on real-time perceived environmental characteristics. For example, in open water with good visibility, the navigation control model can assign higher weights to camera image features to utilize their rich texture details; while in narrow areas with turbid water or complex acoustic reflections, it automatically enhances the importance of sonar features to overcome the limitations of optical sensors. This adaptive fusion mechanism effectively improves the perception robustness and decision-making adaptability of the navigation control model in variable underwater environments, thereby significantly enhancing the overall accuracy and reliability of navigation control.

[0064] Based on the above embodiments, Figure 2 This is a schematic diagram of the navigation control model provided by the present invention, as shown below. Figure 2 As shown, firstly, the navigation control model receives three parallel inputs: position information, camera images, and sonar images. The position information is processed by a fully connected layer to obtain position features f. p Camera images are used to extract camera features f using a Res2Net network. i Sonar images are also processed using a Res2net network to extract sonar features f. s Subsequently, the aforementioned feature f c f i f s The data is fed into the feature fusion module, which innovatively introduces a self-attention mechanism for weighted integration, and then completes deep fusion through a fully connected layer, outputting the fused unified feature f. f Finally, in the output prediction stage, feature f f Through final fully connected layer decoding, the control actions of the autonomous vehicle are generated. These actions specifically include three key parameters: yaw angle, speed, and pitch angle, thereby achieving precise navigation control of the underwater vehicle.

[0065] Based on the above embodiments, the positional features are determined based on the starting position information, current position information, and ending position information of the underwater autonomous vehicle.

[0066] Specifically, in order to enable the navigation control model to not only perform local reactive obstacle avoidance but also global task-oriented path planning, the starting and ending points of the task were added to the input location information.

[0067] The starting position information refers to the three-dimensional coordinates of the underwater autonomous vehicle at the beginning of this navigation mission. The current position information refers to the real-time three-dimensional coordinates of the underwater autonomous vehicle at the current moment. The ending position information refers to the three-dimensional coordinates of the target position that the underwater autonomous vehicle needs to reach in this navigation mission.

[0068] These three types of location information can be combined into a single vector. For example, if each location is represented by three coordinate values ​​(x, y, z), they can be concatenated into a 1×9 vector: [start_x, start_y, start_z, current_x, current_y, current_z, end_x, end_y, end_z]. This location feature vector is then fed into a location information extraction layer to extract location features that include the global path planning context.

[0069] The method provided in this invention uses three types of information—starting point position information, current position information, and ending point position information—of an underwater autonomous vehicle to construct position features. By introducing starting point and ending point position information, the navigation control model can perceive the global path planning of the navigation mission, thereby establishing a clear navigation target orientation in decision-making. Through real-time correlation calculation of the current position information with the starting and ending point position information, the navigation control model can dynamically obtain the vehicle's pose state relative to the global path, and then adaptively adjust control commands according to mission stages and environmental changes. This design enables the navigation control model to combine local obstacle avoidance with global trajectory tracking, effectively avoiding the problem of deviation from the overall mission target due to a lack of global path awareness in local motion optimization, significantly improving the accuracy and mission completion efficiency of long-range autonomous navigation in complex underwater environments.

[0070] Based on the above embodiments, the sample camera image, the sample sonar image, and the sample location information are obtained by controlling the underwater autonomous vehicle to navigate in an underwater simulator and periodically collecting the camera image, sonar image, and location information captured by the underwater autonomous vehicle according to preset acquisition rules. The underwater simulator is equipped with preset seabed topography and multiple obstacles, and visual occlusion is provided to limit the visible distance of the camera images to a preset range.

[0071] Specifically, in order to obtain a large amount of sample data for training navigation and control models efficiently, safely, and at low cost, this embodiment of the invention adopts a data acquisition method in an underwater simulator.

[0072] Specifically, underwater simulators can utilize mature simulation platforms built on Robot Operating System (ROS), such as UUV-Simulator. This platform can realistically simulate the dynamic characteristics of autonomous underwater vehicles, the underwater environment, and sensor data.

[0073] To make the training data more realistic and challenging, the simulation environment was specially set up, with preset seabed topography and multiple obstacles. 3D modeling software, such as Blender, was used to construct seabed topography with different features. Figure 3 The location provided by this invention Figure 3 A schematic diagram of the D model, as shown below Figure 3 As shown, for example, a world with flat terrain but dense obstacles (a flat world) can be constructed, such as... Figure 3 (a), and a world with undulating terrain but relatively sparse obstacles (undulating world), such as Figure 3 (b). In these worlds, obstacles of various shapes are scattered throughout, such as cylinders, rectangular prisms, triangular pyramids, and cones.

[0074] Figure 4 This is a schematic diagram illustrating the setup of the simulation environment and the visualization effects of the sensors provided by the present invention, as shown below. Figure 4 As shown, to simulate the visual limitations caused by light attenuation and water turbidity in a real underwater environment, a blue visual occlusion effect was set in the simulator to limit the visible distance of the camera image to a preset range, such as 5 to 8 meters. The farther the distance, the more blurred the image. In addition, the data collected by the built-in BlueView P900 majority-wave sonar was visualized in the form of images, and the visualized images were used as sonar images.

[0075] The acquisition process for sample camera images, sample sonar images, and sample location information is as follows: The underwater autonomous vehicle is manually controlled to navigate in the underwater simulator set up above, and data is collected at preset times according to preset collection rules, such as once every 1 second, in order to balance the continuity and variability of data. The camera images and sonar images output by various sensors on the underwater autonomous vehicle, as well as the vehicle's position information and the control commands issued by the human (i.e., tag yaw angle, tag pitch angle and tag speed) constitute a complete training sample.

[0076] Based on the above embodiments, the preset collection rules include a first rule, a second rule, and a third rule; The first rule is to avoid collisions between the autonomous underwater vehicle and the obstacle or the preset seabed topography; The second rule is to maintain the navigation distance between the underwater autonomous vehicle and the preset seabed topography within a preset safe distance range; The third rule is to prioritize avoiding obstacles by using a smaller yaw angle, provided that the first rule is met.

[0077] Specifically, when manually controlling an underwater autonomous vehicle to collect sample data, in order to implicitly encode the expert's driving experience and excellent navigation strategies into the dataset, the operator needs to strictly adhere to the following preset data collection rules: The first rule, or safety principle, is to avoid collisions between autonomous underwater vehicles and obstacles or pre-defined seabed topography. This is the highest priority rule; any form of collision is unacceptable because safety is the primary prerequisite for underwater navigation.

[0078] The second rule, the stability principle, requires maintaining an autonomous underwater vehicle (AUV) within a predetermined safe distance from the seabed topography. For example, during operation, the distance between the AUV and the seabed should ideally be between 5 and 6 meters. This rule encourages AUVs to avoid obstacles more through horizontal yaw rather than through sharp pitching and rolling, which helps maintain navigation stability and energy efficiency.

[0079] The third rule, or the efficiency principle, prioritizes obstacle avoidance using smaller yaw angles, provided the first rule is met. This rule encourages aircraft to choose smoother, more direct obstacle avoidance paths to pass through obstacle areas as quickly as possible, thereby improving navigation efficiency.

[0080] In addition, rule four: In order to avoid a large amount of identical repetitive data, sensor data can be collected at 1-second intervals to ensure a balance between data continuity and sufficient variability between individual data points. This step can also avoid hindering the training of the navigation control model.

[0081] Through the above steps, we completed 70 data collection operations, generating 24,012 sets of camera images. We manually excluded 5,682 sets of low-quality data and randomly excluded 6,330 sets of data with similar labels, ultimately retaining 12,000 sets of data for training and validation of the navigation control model.

[0082] The method provided in this invention introduces expert knowledge and superior driving strategies as acquisition rules during the data acquisition phase. This makes the training dataset not just a simple environment-action mapping, but a high-level navigation behavior criterion that includes first, second, and third rules. The navigation control model trained on such a dataset can naturally learn and imitate these excellent strategies, enabling it to exhibit safer, more stable, and more efficient behavior patterns during autonomous navigation, thereby achieving intelligent navigation performance that surpasses simple imitation.

[0083] Here, a Res2Net network with a single visual sensor is used as a comparison of the navigation control model in this embodiment of the invention. Both networks are set to have the same training parameters, namely 300 training epochs, a batch size of 16, and an initial learning rate set to... Both networks employ a learning rate decay mechanism during training. This means that if the validation set shows only slight improvement within 10 iterations, the learning rate is reduced by 90%, and when the learning rate decreases to a certain level... When the model has converged to its optimal performance, training terminates automatically. Training results show that the navigation control model in this embodiment stops training after 135 iterations, while the Res2Net network stops training after 142 iterations. Our network's loss function decreases faster and has a smaller final value compared to the Res2Net network.

[0084] In addition, 20 sets of control experiments were conducted in the simulation environment using different maps, and the experimental results were analyzed. For the navigation control model and the Res2Net network model, we conducted simulation experiments 5 times each for two different terrains, totaling 20 times. During the experiments, we defined collisions as including two types: collisions with obstacles and collisions with the seabed. If the underwater vehicle could not complete subsequent motion due to a collision, or could not rotate at the pitch angle due to being stuck to the seabed, the voyage was considered a failure. Figure 5 This is a schematic diagram of a portion of the experimental trajectory provided by the present invention, such as... Figure 5As shown, the red path represents the underwater vehicle slowing down. Experimental results show that the Res2Net network model achieved an 80% success rate in a flat world, with 4 collisions and an average distance of 3.24 meters from the seabed; while the navigation control model achieved a 100% success rate in the flat world, with 0 collisions and an average distance of 4.54 meters from the seabed. In an undulating world, the Res2Net network model achieved a 20% success rate, with 11 collisions and an average distance of 5.34 meters from the seabed; while the navigation control model achieved an 80% success rate in the undulating world, with 1 collision and an average distance of 5.74 meters from the seabed. It is worth noting that although the average distances from the seabed are similar for both network models in the undulating world, the navigation control model's maximum distance from the seabed is 5.96 meters, and its minimum distance is 5.07 meters; while the Res2Net network model's maximum distance from the seabed is 9.16 meters, and its minimum distance is 2.07 meters. These simulation data are sufficient to demonstrate that the navigation control model has superior reliability and accuracy.

[0085] The navigation control device for an underwater autonomous vehicle provided by the present invention is described below. The navigation control device for an underwater autonomous vehicle described below can be referred to in correspondence with the navigation control method for an underwater autonomous vehicle described above.

[0086] Based on any of the above embodiments, the present invention provides a navigation and control device for an underwater autonomous vehicle. Figure 6 This is a schematic diagram of the navigation and control device for the underwater autonomous vehicle provided by the present invention, as shown below. Figure 6 As shown, the device includes: The acquisition unit 610 is used to acquire camera images and sonar images captured by the underwater autonomous vehicle; The input unit 620 is used to input the position information of the underwater autonomous vehicle, the camera image and the sonar image into the navigation control model to obtain the yaw angle, pitch angle and speed of the underwater autonomous vehicle output by the navigation control model; The navigation control model is trained based on sample camera images, sample sonar images, and sample position information of the underwater autonomous vehicle, as well as the tag yaw angle, tag pitch angle, and tag velocity of the underwater autonomous vehicle.

[0087] The apparatus provided in this invention inputs the position information, camera images, and sonar images of an underwater autonomous vehicle (AUV) into a navigation control model to obtain the yaw angle, pitch angle, and velocity of the AUV output by the navigation control model. The navigation control model is trained based on sample camera images, sample sonar images, and sample position information of the AUV, as well as the tagged yaw angle, tagged pitch angle, and tagged velocity of the AUV. By fusing heterogeneous sensor information from different physical principles—position information, camera images, and sonar images—the navigation control model can obtain a more comprehensive environmental perception capability than any single sensor. Thus, when facing complex and dynamic underwater environments, the navigation control model can still make comprehensive decisions based on the local fine visual information and position information provided by the camera images, effectively compensating for the deficiencies of a single sensor, and ultimately outputting stable yaw angle, pitch angle, and velocity control commands, significantly improving the navigation reliability and robustness of the AUV in complex and dynamic environments.

[0088] Based on any of the above embodiments, a training unit is further included, wherein the training unit is specifically used for: Obtain the initial navigation control model; The sample camera image, the sample sonar image, and the sample position information are input into the initial navigation control model to obtain the predicted yaw angle, predicted pitch angle, and predicted velocity output by the initial navigation control model. The yaw loss is determined based on the difference between the predicted yaw angle and the tag yaw angle; The pitch angle loss is determined based on the difference between the predicted pitch angle and the tag pitch angle; Based on the difference between the predicted speed and the label speed, determine the speed loss; Based on the yaw loss, pitch angle loss, and velocity loss, the target loss is determined, and based on the target loss, the parameters of the initial navigation control model are iterated to obtain the navigation control model.

[0089] Based on any of the above embodiments, the navigation control model includes a feature extraction layer, a feature fusion layer, and a prediction layer; The feature extraction layer includes a location information extraction layer, a camera feature extraction layer, and a sonar feature extraction layer; the location information extraction layer is used to extract location features from the location information, the camera feature extraction layer is used to extract camera features from the camera image, and the sonar feature extraction layer is used to extract sonar features from the sonar image. The feature fusion layer is used to perform self-attention calculation on the location features, camera features, and sonar features, determine the correlation between the location features, camera features, and sonar features, and perform feature fusion on the location features, camera features, and sonar features based on the correlation to obtain fused features; The prediction layer is used to perform navigation control prediction based on the fused features to obtain the yaw angle, the pitch angle, and the speed.

[0090] Based on any of the above embodiments, the positional feature is determined based on the starting position information, current position information, and ending position information of the underwater autonomous vehicle.

[0091] Based on any of the above embodiments, the sample camera image, the sample sonar image, and the sample location information are obtained by controlling the underwater autonomous vehicle to navigate in an underwater simulator and periodically collecting camera images, sonar images, and location information captured by the underwater autonomous vehicle according to preset acquisition rules; The underwater simulator is equipped with preset seabed topography and multiple obstacles, and visual occlusion is provided to limit the visible distance of the camera images to a preset range.

[0092] Based on any of the above embodiments, the preset collection rules include a first rule, a second rule, and a third rule; The first rule is to avoid collisions between the autonomous underwater vehicle and the obstacle or the preset seabed topography; The second rule is to maintain the navigation distance between the underwater autonomous vehicle and the preset seabed topography within a preset safe distance range; The third rule is to prioritize avoiding obstacles by using a smaller yaw angle, provided that the first rule is met.

[0093] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 7As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740. The processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a navigation control method for the underwater autonomous vehicle. This method includes: acquiring camera images and sonar images captured by the underwater autonomous vehicle; inputting the position information of the underwater autonomous vehicle, the camera images, and the sonar images into a navigation control model to obtain the yaw angle, pitch angle, and velocity of the underwater autonomous vehicle output by the navigation control model; the navigation control model is trained based on sample camera images, sample sonar images, and sample position information of the underwater autonomous vehicle, as well as the tag yaw angle, tag pitch angle, and tag velocity of the underwater autonomous vehicle.

[0094] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0095] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the navigation control method for an underwater autonomous vehicle provided by the above methods. The method includes: acquiring camera images and sonar images captured by the underwater autonomous vehicle; inputting the position information of the underwater autonomous vehicle, the camera images, and the sonar images into a navigation control model to obtain the yaw angle, pitch angle, and velocity of the underwater autonomous vehicle output by the navigation control model; the navigation control model is trained based on sample camera images, sample sonar images, and sample position information of the underwater autonomous vehicle, as well as the tag yaw angle, tag pitch angle, and tag velocity of the underwater autonomous vehicle.

[0096] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a navigation control method for an underwater autonomous vehicle provided by the methods described above. This method includes: acquiring camera images and sonar images captured by the underwater autonomous vehicle; inputting the position information of the underwater autonomous vehicle, the camera images, and the sonar images into a navigation control model to obtain the yaw angle, pitch angle, and velocity of the underwater autonomous vehicle output by the navigation control model; the navigation control model is trained based on sample camera images, sample sonar images, and sample position information of the underwater autonomous vehicle, as well as the tag yaw angle, tag pitch angle, and tag velocity of the underwater autonomous vehicle.

[0097] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A navigation and control method for an underwater autonomous vehicle, characterized in that, include: Acquire camera and sonar images captured by an underwater autonomous vehicle; The position information of the underwater autonomous vehicle, the camera image, and the sonar image are input into the navigation control model to obtain the yaw angle, pitch angle, and velocity of the underwater autonomous vehicle output by the navigation control model. The navigation control model is trained based on sample camera images, sample sonar images, and sample position information of the underwater autonomous vehicle, as well as the tag yaw angle, tag pitch angle, and tag velocity of the underwater autonomous vehicle.

2. The navigation and control method for an underwater autonomous vehicle according to claim 1, characterized in that, The training steps of the navigation control model include: Obtain the initial navigation control model; The sample camera image, the sample sonar image, and the sample position information are input into the initial navigation control model to obtain the predicted yaw angle, predicted pitch angle, and predicted velocity output by the initial navigation control model. The yaw loss is determined based on the difference between the predicted yaw angle and the tag yaw angle; The pitch angle loss is determined based on the difference between the predicted pitch angle and the tag pitch angle; Based on the difference between the predicted speed and the label speed, determine the speed loss; Based on the yaw loss, pitch angle loss, and velocity loss, the target loss is determined, and based on the target loss, the parameters of the initial navigation control model are iterated to obtain the navigation control model.

3. The navigation and control method for an underwater autonomous vehicle according to claim 1, characterized in that, The navigation control model includes a feature extraction layer, a feature fusion layer, and a prediction layer; The feature extraction layer includes a location information extraction layer, a camera feature extraction layer, and a sonar feature extraction layer; the location information extraction layer is used to extract location features from the location information, the camera feature extraction layer is used to extract camera features from the camera image, and the sonar feature extraction layer is used to extract sonar features from the sonar image. The feature fusion layer is used to perform self-attention calculation on the location features, camera features, and sonar features, determine the correlation between the location features, camera features, and sonar features, and perform feature fusion on the location features, camera features, and sonar features based on the correlation to obtain fused features; The prediction layer is used to perform navigation control prediction based on the fused features to obtain the yaw angle, the pitch angle, and the speed.

4. The navigation and control method for an underwater autonomous vehicle according to claim 3, characterized in that, The location features are determined based on the starting position information, current position information, and ending position information of the underwater autonomous vehicle.

5. The navigation and control method for an underwater autonomous vehicle according to any one of claims 1 to 4, characterized in that, The sample camera images, the sample sonar images, and the sample location information are obtained by controlling the underwater autonomous vehicle to navigate in an underwater simulator and periodically collecting camera images, sonar images, and location information captured by the underwater autonomous vehicle according to preset acquisition rules. The underwater simulator is equipped with preset seabed topography and multiple obstacles, and visual occlusion is provided to limit the visible distance of the camera images to a preset range.

6. The navigation and control method for an underwater autonomous vehicle according to claim 5, characterized in that, The preset data collection rules include a first rule, a second rule, and a third rule; The first rule is to avoid collisions between the autonomous underwater vehicle and the obstacle or the preset seabed topography; The second rule is to maintain the navigation distance between the underwater autonomous vehicle and the preset seabed topography within a preset safe distance range; The third rule is to prioritize avoiding obstacles by using a smaller yaw angle, provided that the first rule is met.

7. A navigation and control device for an underwater autonomous vehicle, characterized in that, include: The acquisition unit is used to acquire camera and sonar images captured by the underwater autonomous vehicle; The input unit is used to input the position information of the underwater autonomous vehicle, the camera image and the sonar image into the navigation control model to obtain the yaw angle, pitch angle and velocity of the underwater autonomous vehicle output by the navigation control model; The navigation control model is trained based on sample camera images, sample sonar images, and sample position information of the underwater autonomous vehicle, as well as the tag yaw angle, tag pitch angle, and tag velocity of the underwater autonomous vehicle.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the navigation control method for the underwater autonomous vehicle as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the navigation control method for the underwater autonomous vehicle as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the navigation control method for the underwater autonomous vehicle as described in any one of claims 1 to 6.

Images