Underwater vehicle docking control method and device based on imaging sonar

Abstract

The present application discloses a method and device for underwater vehicle docking control based on sonar imaging, which belongs to the field of vehicle automatic control technology. The present application decomposes the underwater vehicle docking control into depth tracking control and horizontal surface docking control, designs corresponding reinforcement learning cost functions to train the depth tracking controller and servo docking controller based on the deep network, designs the position error and field of view angle in the nonlinear weight balance cost function, and enables the controller to learn the optimal control strategy through reinforcement learning, quickly eliminate the position error and maintain the recovery device characteristics in the imaging sonar field of view. The present application can effectively avoid docking failures caused by the loss of target features and improve the docking success rate.

Description

Technical Field

[0001] The present application belongs to the technical field of automatic control of aircraft, and more specifically, to a method and device for docking control of underwater aircraft based on imaging sonar. Background Art

[0002] At present, autonomous underwater vehicles have been widely used in various underwater operation fields, such as resource exploration, seabed mapping, and marine structure maintenance. Among them, underactuated autonomous underwater vehicles, due to their good hydrodynamic characteristics and simple configuration, occupy an important position in high-speed, long-range tasks such as underwater inspection and search. Due to the limited onboard energy of underwater vehicles, when facing long-range tasks, underwater vehicles need to autonomously complete docking and docking during the operation to replenish energy and exchange data to indirectly improve endurance. Therefore, autonomous docking is a key technology to improve the operating efficiency of underwater vehicles, among which autonomous docking control is an important part of ensuring the success rate of autonomous docking and operation safety of underwater vehicles. However, due to the limited sensor perception range, such as underwater cameras and imaging sonars, the autonomous docking control of underwater vehicles needs to consider the limited perception range constraints while eliminating position and attitude errors, so as to avoid the loss of recovery device characteristics during movement, resulting in autonomous docking failures and safety hazards, especially for underactuated underwater vehicles, which also need to be combined with kinematic constraints. Therefore, whether the underwater vehicle docking control can achieve high-precision docking while maintaining the feature capture of the target recovery device under the constraints of limited perception range is a key issue to improve the docking success rate and vehicle safety.

[0003] The current docking technology of underactuated underwater vehicles is difficult to effectively combine the perception constraints of the recovery device and the docking motion constraints. Therefore, the docking target features are easily lost during the docking process, resulting in the ultimate failure of the docking. Summary of the invention

[0004] In response to the above defects or improvement needs of the prior art, the present application provides an underwater vehicle docking control method, device, equipment and storage medium, which aims to solve the problem that the docking target features are easily lost during the docking movement of an under-actuated underwater vehicle.

[0005] To achieve the above objectives, in a first aspect, the present application provides a servo docking control method for an underwater vehicle, comprising:

[0006] The depth error of the spacecraft and the recovery device, the pitch angle, velocity and angular velocity of the spacecraft are used as the input state vector of the depth tracking controller, and a reinforcement learning cost function is designed to train the deep network-based deep tracking controller to achieve the depth tracking control of the spacecraft.

[0007] The relative position of the recovery device, the difference in heading angles between the recovery device and the vehicle, the speed of the vehicle, the angular velocity and the rudder angle of the steering rudder are taken as the input state vectors of the servo docking controller. Three docking states are divided according to the relative position and the difference in heading angle. Reinforcement learning cost functions are designed for different docking states to train the servo docking controller based on the deep network to realize the horizontal plane docking control of the vehicle.

[0008] Preferably, the input state vector of the depth tracking controller is specifically:

[0009] {e z ,θ,u,v,w,p,q,r}

[0010] Among them, e z is the depth error between the spacecraft and the recovery device, θ is the spacecraft pitch angle, u is the spacecraft forward linear velocity, v is the spacecraft lateral linear velocity, w is the spacecraft vertical linear velocity, p is the spacecraft rolling angular velocity, q is the spacecraft pitch angular velocity, and r is the spacecraft bow angular velocity.

[0011] Preferably, the reinforcement learning cost function of the depth tracking controller is:

[0012] rws=-k z |e z |-k θ |0|-k p |p|

[0013] Among them, rws is the total generation value of deep reinforcement learning, k z is the depth error weight, k θ is the pitch angle weight, and p is the roll angular velocity weight.

[0014] Preferably, the input state vector of the servo docking controller is specifically:

[0015] {x,y,e ψ ,u,v,w,p,q,Δr,δ r}

[0016] Where x and y are the forward and lateral coordinates of the recovery device relative to the spacecraft, respectively, and e ψ is the difference between the heading angles of the recovery device and the aircraft, u is the forward linear velocity of the aircraft, v is the lateral linear velocity of the aircraft, w is the vertical linear velocity of the aircraft, p is the rolling angular velocity of the aircraft, q is the pitch angular velocity of the aircraft, Δr is the difference in the heading angular velocity of the aircraft within the control time interval, δ r is the rudder angle of the aircraft steering rudder.

[0017] Preferably, three docking states are divided according to the difference between the relative position and the heading angle, specifically:

[0018] If satisfied: The docking state is that the bow is adjusted to the right;

[0019] If satisfied: The docking state is that the bow is adjusted to the left;

[0020] Otherwise, the docking state is error adjustment;

[0021] in, is the position deflection angle,

[0022] Preferably, the reinforcement learning cost function of the servo docking controller is:

[0023] The reinforcement learning cost function when the bow is adjusted to the right is:

[0024]

[0025] Among them, rws is the total generation value of the docking reinforcement learning, k χ is the docking path offset weight, is the position angle weight, p χ (χ) is the harmonic function of the docking path offset, is the harmonic function of position deflection angle, k r is the heading angle difference weight, is the rudder angle weight, is the rudder angle amplitude of the aircraft steering rudder,

[0026] The reinforcement learning cost function when the bow is adjusted to the left is:

[0027]

[0028] The reinforcement learning cost function under the error adjustment state is:

[0029]

[0030] Preferably, the docking path offset harmonic function p χ (χ) is:

[0031]

[0032] Where, χ is the docking path offset, χ max is half the width of the sonar field of view, and e is a natural constant;

[0033] The position deflection harmonic function for:

[0034]

[0035] Where ε is a constant exponential coefficient.

[0036] In a second aspect, the present application provides an underwater vehicle docking control device, comprising:

[0037] The depth tracking unit is used to take the depth error of the spacecraft and the recovery device, the pitch angle, speed and angular velocity of the spacecraft as the input state vector of the depth tracking controller, design a reinforcement learning cost function to train the deep network-based depth tracking controller, and realize the depth tracking control of the spacecraft;

[0038] The horizontal docking unit is used to take the relative position of the recovery device, the difference between the heading angles of the recovery device and the aircraft, the speed of the aircraft, the angular velocity and the rudder angle of the steering rudder as the input state vector of the servo docking controller, divide the docking states into three states according to the relative position and the difference in the heading angle, and design reinforcement learning cost functions for different docking states to train the servo docking controller based on the deep network to realize the horizontal docking control of the aircraft.

[0039] In a third aspect, the present application provides an electronic device, comprising: a memory for storing programs; a processor for executing the programs stored in the memory, when the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0041] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art:

[0042] (1) This application implements servo docking control based on reinforcement learning training of deep networks, decomposes underwater vehicle docking control into depth tracking control and horizontal surface docking control, designs a corresponding reinforcement learning cost function to train the controller, balances the position error and field of view angle in the cost function through nonlinear weights, and enables the controller to learn an optimized control strategy through reinforcement learning, quickly eliminates the position error and maintains the characteristics of the recovery device in the imaging sonar field of view, improves the docking success rate and avoids control oscillation caused by target loss;

[0043] (2) The technical solution of the present application can be deployed on an industrial computer onboard an underwater vehicle to read the status data fed back by the sensor and the output of the forward-looking imaging sonar rear end, control the elevator and steering rudder actuators, form an acoustic-visual servo docking control system, and realize autonomous docking. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of an underwater vehicle servo docking control method provided in an embodiment of the present application;

[0045] Figure 2 is a schematic diagram of a docking control algorithm provided in an embodiment of the present application;

[0046] Figure 3a This is a schematic diagram of the environment of the underwater vehicle docking simulation experiment provided by the embodiment of the present application;

[0047] Figure 3b It is a schematic diagram of the fan-shaped field of view area of ​​the simulated imaging sonar provided in an embodiment of the present application;

[0048] Figure 4a is a track curve diagram of a static docking simulation test provided in an embodiment of the present application;

[0049] Figure 4b is a graph of aircraft status in a static docking simulation test provided in an embodiment of the present application;

[0050] Figure 4c is a track curve diagram of a comparative simulation test provided in an embodiment of the present application;

[0051] Figure 4d is a graph of aircraft status in a comparative simulation test provided in an embodiment of the present application;

[0052] Figure 5a is a track curve diagram of a dynamic docking simulation test provided in an embodiment of the present application;

[0053] Figure 5b is a graph of aircraft status in a dynamic docking simulation test provided in an embodiment of the present application;

[0054] Figure 6 This is a diagram of the servo docking control system architecture provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] Figure 1 A flowchart of a method for controlling an underwater vehicle docking based on sonar imaging is provided in an embodiment of the present application, such as Figure 1 As shown, the specific steps include:

[0057] (1) The information of the recovery device is collected by the forward-looking imaging sonar mounted on the bow of the underactuated underwater vehicle. The sonar image is processed by the back-end to determine the relative position of the underwater vehicle and the recovery device and the attitude information of the recovery device. The information includes: forward distance, lateral distance and the heading angle of the recovery device;

[0058] The underwater vehicle obtains its own status information and receives status information of the recovery device, wherein the status information includes: depth of the recovery device, depth of the vehicle, speed of the vehicle, attitude angle of the vehicle and angular velocity of the vehicle;

[0059] (2) Determine the depth tracking error based on the depth of the recovery device and the current depth, use the depth tracking error, pitch angle, navigation speed and angular velocity as the observation state of the depth tracking controller, and design a reinforcement learning cost function to train the deep network controller to achieve target depth tracking of the underactuated underwater vehicle;

[0060] (3) The heading tracking error is determined according to the heading angle of the recovery device and the current heading angle of the underwater vehicle. The relative position, heading tracking error, navigation speed and angular velocity are used as the observation states of the horizontal plane controller. The docking scenario is divided into three situations: rightward heading adjustment, leftward heading adjustment and position error adjustment. Reinforcement learning cost functions are designed to train deep network controllers to realize the horizontal plane servo control of the underactuated underwater vehicle based on imaging sonar positioning.

[0061] (4) Combining the depth tracking controller and the servo docking controller trained based on the designed reinforcement learning cost function, the underwater vehicle can stably navigate at the desired depth and keep the forward-looking imaging sonar and the center of the recovery device in the vertical position consistent; realize the autonomous docking position servo control of the underactuated underwater vehicle, maintain the feature capture of the recovery device in the forward-looking sonar field of view under the kinematic constraints and limited field of view constraints, improve the docking success rate and avoid potential control oscillations and safety risks caused by the loss of the target recovery device features.

[0062] Figure 2 This is a schematic diagram of the docking control algorithm provided by the present application for the embodiment of the present application; Figure 1 As shown, specifically:

[0063] The underwater vehicle collects information about the recovery device through the forward-looking imaging sonar carried on the bow. At the same time, the underwater vehicle collects the motion status of the vehicle itself and the depth information of the vehicle and the recovery device.

[0064] Use the above information to train a deep network-based depth tracking controller and servo docking controller:

[0065] For a depth tracking controller, the recovery device depth z d Subtract the current depth z of the underwater vehicle to obtain the depth tracking error ez , and obtain the input state vector of the depth tracking controller:

[0066] {e z ,θ,u,v,w,p,q,r}

[0067] Wherein, θ is the pitch angle of the underwater vehicle, u is the forward linear velocity of the underwater vehicle, v is the lateral linear velocity of the underwater vehicle, w is the vertical linear velocity of the underwater vehicle, p is the rolling angular velocity of the underwater vehicle, q is the pitch angular velocity of the underwater vehicle, and r is the bow angular velocity of the underwater vehicle;

[0068] The benefit value is calculated based on the deep tracking controller reinforcement learning cost function:

[0069] rws=-k z |e z |-k θ |θ|-k p |p|

[0070] Among them, rws is the total generation value of deep reinforcement learning, k z is the depth tracking error weight, k θ is the pitch angle weight, p is the roll angular velocity weight;

[0071] The deep network-based deep tracking controller is iteratively trained according to the calculated benefit value during the reinforcement learning process until the deep tracking converges.

[0072] For the servo docking controller, set the heading angle of the recovery device to d The heading tracking error e is obtained by subtracting the current heading angle ψ of the underwater vehicle. ψ , and obtain the servo docking controller input state vector:

[0073] {x,y,e ψ ,u,v,w,p,q,Δr,δ r}

[0074] Where x is the forward coordinate of the recovery device feature in the forward imaging sonar field of view relative to the origin of the field of view, y is the lateral coordinate; u is the forward linear velocity of the underwater vehicle, v is the lateral linear velocity of the underwater vehicle, w is the vertical linear velocity of the underwater vehicle, p is the rolling angular velocity of the underwater vehicle, q is the pitch angular velocity of the underwater vehicle, Δr is the difference in the bow angular velocity of the underwater vehicle within the continuous control time interval, δ r is the rudder angle of the underwater vehicle steering rudder;

[0075] Calculate the position deflection angle separately And the relative distance R:

[0076]

[0077]

[0078] Calculate the underwater vehicle docking path offset χ:

[0079]

[0080] Calculate the position offset and harmonic weight p χ (χ):

[0081]

[0082] In the formula, χ max It is 1 / 2 of the width of the field of view of the forward-looking imaging sonar; e is a natural constant;

[0083] Calculate the position deflection and harmonic weight

[0084]

[0085] Where ε is the constant exponential coefficient;

[0086] The docking status is determined based on the relative position between the recovery device and the underwater vehicle. When , the docking state is judged as the bow is adjusted to the right; the benefit value is calculated according to the reinforcement learning cost function of the servo docking controller in the right adjustment state:

[0087]

[0088] In the formula, rws is the total generation value of reinforcement learning, is the current rudder angle weight, is the rudder angle amplitude of the underwater vehicle steering rudder; k χ is the docking path offset weight, is the position angle weight; k r is the heading angle difference weight;

[0089] When the conditions are met When , the docking state is judged as the bow is adjusted to the left; the benefit value is calculated according to the reinforcement learning cost function of the servo docking controller in the left adjustment state:

[0090]

[0091] When the state condition does not satisfy the bow adjustment to the right or the bow adjustment to the left, the docking state is judged as the position error adjustment; the benefit value is calculated according to the reinforcement learning cost function of the servo docking controller under the position error state:

[0092]

[0093] The deep network of the servo docking controller is iteratively trained according to the calculated benefit value during the reinforcement learning process until the position error and heading error converge.

[0094] The depth tracking controller and servo docking controller based on the deep network are trained by the above method and deployed in the underwater vehicle;

[0095] When the underwater vehicle performs the docking and recovery mission, it collects the recovery device information, the vehicle's own motion state, and the depth information of the vehicle and the recovery device in real time; and inputs the above data into the depth tracking controller and the servo docking controller after processing. The depth tracking controller and the servo docking controller output elevator commands and steering commands to control the execution steering gear, so that the underwater vehicle can stably navigate at the desired depth and keep the forward-looking imaging sonar and the recovery device center in the same vertical position, and keep the feature capture of the recovery device under the constraint of the limited field of view of the imaging sonar, so as to improve the docking success rate and avoid potential control oscillation and safety risks caused by the loss of the target recovery device features.

[0096] In order to verify the reliability and practicality of the method of the present invention, the following Figure 3a A typical underwater vehicle acoustic vision guided docking simulation environment as shown in the figure is used as a test platform. In this simulation environment, the underwater vehicle shown in ① has an underactuated characteristic, and a forward-looking imaging sonar simulation plug-in as shown in ② is configured at the bow of the model. As shown in ③, a typical underactuated underwater vehicle recovery device is shown, and as shown in ④, the forward-looking sonar field of view. The simulation platform runs the forward-looking imaging sonar backend processing program, and the forward-looking imaging sonar backend processing program is used to simulate the forward-looking imaging sonar backend. Figure 3b After processing the sonar image shown in the figure, the recovery device features shown in ① are extracted and converted into actual relative positions and heading angles. ② shows the fan-shaped field of view of the simulated imaging sonar on the horizontal projection plane, with a maximum distance of 40m, a horizontal field of view of 80°, and a vertical field of view of 12°. The depth tracking controller and servo docking controller are deployed on the underwater vehicle. The physical controller obtains the underwater vehicle status information and the recovery device status information from the simulation platform, and trains the depth tracking controller and servo docking controller deep network through reinforcement learning.

[0097] During the test, the north coordinates of the initial position of the underwater vehicle were randomly sampled from -40m to -10m, the east coordinates were randomly sampled from -25m to 25m, the heading angles were randomly sampled from -45° to 45°, and the depth was randomly sampled from 5-15m. The target recovery device depth was 10m. When the recovery device was set to a stationary state, the test results are shown in Figure 4a-4d .

[0098] Figure 4aThe three-dimensional space track of the aircraft in some test samples is shown. The average docking success rate of the aircraft exceeds 87%, the average depth error is ≯0.0003m, and the average pitch angle of the aircraft in the process of maintaining deep tracking is ≯3°, which meets the constraints of the forward imaging sonar in the vertical field of view. High-precision deep tracking is guaranteed by the designed reinforcement learning cost function. Figure 4b The state parameter curves of the vehicle servo docking control are displayed. The average horizontal position deviation is ≯0.3m, the average heading error is ≯3°, and the field of view deviation angle is ≯40° throughout the test, indicating that there is no loss of target features during the test. Figure 4c The three-dimensional space track of the aircraft in some comparison samples is shown. The comparison test samples use the underwater aircraft based on the classic path following control method to carry out the docking task. It can be seen that two of the three test samples provided cannot complete the docking. Figure 4d From the curves of position offset, field of view deviation and heading angle, it can be seen that when the position offset of the classic path following algorithm is large, the expected heading angle given by the guidance algorithm will cause the vehicle to turn more significantly, causing the field of view deviation of the recovery device feature to exceed 40°, resulting in feature loss and ultimately docking failure. Therefore, the stability of target feature capture and high-precision docking performance are guaranteed by the reinforcement learning cost function designed for each scene state.

[0099] Figure 5a The three-dimensional space track of the vehicle and the recovery device when the recovery device is set to motion is shown. The average docking success rate of the vehicle exceeds 80%, the average depth error is ≯0.001m, and the average pitch angle of the vehicle in the process of maintaining deep tracking is ≯5°, which meets the constraints of the forward imaging sonar in the vertical field of view. High-precision dynamic target depth tracking is guaranteed by the designed reinforcement learning cost function. Figure 5b The state parameter curves of the vehicle servo docking control are shown. The average horizontal position deviation is ≯0.35m, the average heading error is ≯5°, and the field of view deviation angle is ≯40° throughout the test, indicating that there is no loss of target features during the test. The stability of target feature capture and high-precision docking performance in dynamic docking scenarios are guaranteed by the reinforcement learning cost function designed for different scene states.

[0100] Figure 6 This is a diagram of the architecture of an underwater vehicle docking control device based on sonar imaging provided in an embodiment of the present application; Figure 6 As shown, including:

[0101] The sonar backend processing unit 610 is used to extract the recovery device features in the imaging sonar feedback image and determine the relative position and attitude information of the underwater vehicle and the recovery device. The information includes: forward distance, lateral distance and the heading angle of the recovery device;

[0102] A state information determination unit 611 is used to determine the state of the underwater vehicle and the depth state information of the recovery device, wherein the state information includes: the depth of the recovery device, the current depth of the underwater vehicle, the navigation speed, the attitude angle and the angular velocity;

[0103] The servo docking control unit 612 determines the heading tracking error according to the heading angle of the recovery device and the current heading angle of the underwater vehicle, takes the relative position, heading tracking error, navigation speed and angular velocity as the observation state of the horizontal plane controller, and divides the docking scenario into three cases: rightward heading adjustment, leftward heading adjustment and position error adjustment. The deep network controller is trained based on different reinforcement learning cost functions to realize the horizontal plane servo control of the underactuated underwater vehicle based on imaging sonar positioning;

[0104] The depth tracking control unit 613 determines the depth tracking error according to the depth of the recovery device and the current depth, takes the depth tracking error, the pitch angle, the navigation speed and the angular velocity as the observation state of the depth tracking controller, and trains the deep network controller based on the reinforcement learning cost function to realize the target depth tracking of the underactuated underwater vehicle;

[0105] Actuator determination unit 614: Combines the rudder angle output by the depth tracking controller trained based on the designed reinforcement learning cost function and the servo docking controller, so that the elevator and steering rudder of the underactuated underwater vehicle are executed according to the rudder angle, so that the underwater vehicle can stably navigate at the desired depth and keep the forward-looking imaging sonar and the center of the recovery device in the same vertical position, and keep the feature capture of the recovery device under the constraint of the limited field of view of the imaging sonar, so as to improve the docking success rate and avoid potential control oscillation and safety risks caused by the loss of the target recovery device features.

[0106] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0107] Based on the method in the above embodiment, an embodiment of the present application provides an electronic device, which includes: a processor (processor) 810, a communication interface (Communications Interface) 820, a memory (memory) 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call the logic instructions in the memory 830 to execute the method in the above embodiment.

[0108] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0109] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0110] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0111] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0112] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0113] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a storage medium or transmitted through the storage medium. The computer instructions may be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0114] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0115] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0116] In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more than two. For example, "plurality" means two or more than two aircraft, etc.

[0117] The above content is easily understood by those skilled in the art. The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for docking control of an underwater vehicle based on imaging sonar, characterized in that: include: The depth error of the spacecraft and the recovery device, the pitch angle, velocity and angular velocity of the spacecraft are used as the input state vector of the depth tracking controller, and a reinforcement learning cost function is designed to train the deep network-based deep tracking controller to achieve the depth tracking control of the spacecraft. The relative position of the recovery device, the difference between the heading angles of the recovery device and the aircraft, the speed of the aircraft, the angular velocity and the steering angle of the steering rudder are used as the input state vector of the servo docking controller. Three docking states are divided according to the relative position and the difference in the heading angle. A reinforcement learning cost function is designed for each docking state to train the servo docking controller based on the deep network, so as to realize the horizontal docking control of the aircraft. The input state vector of the depth tracking controller is specifically: in, is the depth error of the spacecraft and the recovery device, is the aircraft pitch angle, is the forward linear velocity of the spacecraft, is the lateral linear velocity of the vehicle, is the vertical velocity of the vehicle, is the rolling angular velocity of the vehicle, is the vehicle's pitch angular velocity, is the heading angular velocity of the vehicle; The reinforcement learning cost function of the depth tracking controller is: in, is the total generation value of deep reinforcement learning, is the depth error weight, is the pitch angle weight, is the rolling angular velocity weight; The reinforcement learning cost function of the servo docking controller is: The reinforcement learning cost function when the bow is adjusted to the right is: in, To connect the total generation value of reinforcement learning, is the docking path offset weight, is the position angle weight, is the harmonic function of docking path offset, is the harmonic function of position deflection angle, is the heading angle difference weight, is the rudder angle weight, is the rudder angle amplitude of the aircraft steering rudder, is the position deflection angle, is the heading angular velocity difference of the vehicle during the control time interval, is the rudder angle of the aircraft steering rudder; The reinforcement learning cost function when the bow is adjusted to the left is: The reinforcement learning cost function under the error adjustment state is: 。 2. The method according to claim 1, characterized in that The input state vector of the servo docking controller is specifically: in, , are the forward and lateral coordinates of the recovery device relative to the spacecraft, is the difference between the heading angles of the recovery device and the vehicle, is the forward linear velocity of the spacecraft, is the lateral linear velocity of the vehicle, is the vertical velocity of the vehicle, is the rolling angular velocity of the vehicle, is the vehicle's pitch angular velocity, is the heading angular velocity difference of the vehicle during the control time interval, is the rudder angle of the aircraft steering rudder.

3. The method according to claim 2, characterized in that According to the difference between the relative position and the heading angle, three docking states are divided, specifically: If satisfied: , then the docking state is bow adjustment to right; If satisfied: , then the docking state is bow adjustment to left; Otherwise, the docking state is error adjustment; in, is the position deflection angle, .

4. The method according to claim 1, characterized in that: The docking path offset harmonic function for: in, is the docking path offset, , , is half the width of the sonar field of view, is a natural constant; is the difference between the heading angles of the recovery device and the vehicle, , are the forward and lateral coordinates of the recovery device relative to the spacecraft, is the relative distance; The position deflection harmonic function for: in, is the constant exponential coefficient.

5. An underwater vehicle docking control device based on imaging sonar, characterized in that: include: The depth tracking unit is used to take the depth error of the spacecraft and the recovery device, the pitch angle, speed and angular velocity of the spacecraft as the input state vector of the depth tracking controller, design a reinforcement learning cost function to train the deep network-based depth tracking controller, and realize the depth tracking control of the spacecraft; A horizontal docking unit is used to use the relative position of the recovery device, the difference between the heading angles of the recovery device and the aircraft, the speed of the aircraft, the angular velocity and the steering angle of the steering rudder as the input state vector of the servo docking controller, divide the docking states into three states according to the relative position and the difference between the heading angles, and respectively design a reinforcement learning cost function for different docking states to train the servo docking controller based on the deep network to realize the horizontal docking control of the aircraft; The input state vector of the depth tracking controller is specifically: in, is the depth error of the spacecraft and the recovery device, is the aircraft pitch angle, is the forward linear velocity of the spacecraft, is the lateral linear velocity of the vehicle, is the vertical velocity of the vehicle, is the rolling angular velocity of the vehicle, is the vehicle's pitch angular velocity, is the heading angular velocity of the vehicle; The reinforcement learning cost function of the depth tracking controller is: in, is the total generation value of deep reinforcement learning, is the depth error weight, is the pitch angle weight, is the rolling angular velocity weight; The reinforcement learning cost function of the servo docking controller is: The reinforcement learning cost function when the bow is adjusted to the right is: in, To connect the total generation value of reinforcement learning, is the docking path offset weight, is the position angle weight, is the harmonic function of docking path offset, is the harmonic function of position deflection angle, is the heading angle difference weight, is the rudder angle weight, is the rudder angle amplitude of the aircraft steering rudder, is the position deflection angle, is the heading angular velocity difference of the vehicle during the control time interval, is the rudder angle of the aircraft steering rudder; The reinforcement learning cost function when the bow is adjusted to the left is: The reinforcement learning cost function under the error adjustment state is: 。 6. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the program stored in the memory. When the program stored in the memory is executed, the processor is configured to execute the method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 4.

Images