An adaptive unmanned boat path planning method based on fuzzy sets and deep reinforcement learning

Abstract

The present invention belongs to the technical field of unmanned boat navigation, and specifically relates to an adaptive unmanned boat path planning method based on fuzzy sets and deep reinforcement learning; the method comprises generating an obstacle environment and recording information, including the positions of obstacles and target points; constructing fuzzy rules for fuzzy control of the unmanned boat; during the navigation of the unmanned boat, calculating in real time the distances of the unmanned boat relative to obstacles and target points, as well as the yaw angle of the unmanned boat; processing the results of the real-time calculation using the unmanned boat fuzzy rules to output a fuzzy coefficient; a reward function in deep reinforcement learning adaptively rewards the unmanned boat according to the fuzzy coefficient; training an unmanned boat path planning model according to the adaptive rewards of the unmanned boat in different states; and autonomously planning an optimal path using the trained unmanned boat path planning model. The present invention realizes unmanned boat path planning, ensures the safety of the unmanned boat, and improves the efficiency of the unmanned boat in executing tasks.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned boat navigation, and in particular relates to an adaptive unmanned boat path planning method based on fuzzy sets and deep reinforcement learning. Background Art

[0002] Research on unmanned aerial vehicles (UAVs) began in the late 20th century. Their advantages, such as small size, high maneuverability, and minimal casualties, have led to promising development in marine environmental monitoring, maritime search and rescue, and patrol operations. UAV technology remains a key research area. However, due to the unpredictable nature of the ocean and the increasing complexity of navigation environments, improving the autonomous navigation capabilities of UAVs is crucial. To ensure that UAVs quickly and safely reach their designated destinations and complete their missions, path planning is a key technology for ensuring their autonomous intelligence.

[0003] Path planning is a decision-making process in autonomous intelligent navigation. Its goal is to establish an optimal path from a starting point to a destination, minimizing the cost for the unmanned vehicle while ensuring safety. Currently, traditional path planning algorithms include the A* algorithm, derived from the Dijkstra algorithm for shortest paths in graph theory, and its variants, as well as intelligent optimization algorithms such as genetic algorithms, ant colony algorithms, simulated annealing algorithms, particle swarm algorithms, and combinations of these algorithms. However, these traditional algorithms rely heavily on environmental models and global environmental information, resulting in significant limitations in their application scenarios. The complex and volatile marine environment requires unmanned vehicles to possess autonomous learning capabilities.

[0004] Reinforcement learning is a key area of ​​machine learning that emphasizes maximizing rewards through interaction between an agent and its environment. It aims to solve decision-making problems, particularly continuous ones. The path planning problem for an unmanned vehicle can also be viewed as a continuous decision-making problem, requiring optimal navigation maneuvers under different conditions. Deep reinforcement learning has garnered significant attention in the field of artificial intelligence in recent years. It combines the perceptual capabilities of deep learning with the decision-making capabilities of reinforcement learning, directly controlling the behavior of an agent through high-dimensional perceptual input. In reinforcement learning, an agent measures the quality of its actions at a given moment solely through reward signals received from interacting with the environment. This feedback is used to continuously optimize the agent's decisions, maximizing the expected reward of the target. Therefore, in deep reinforcement learning, the design of the reward function directly impacts model training.

[0005] In path planning tasks, the design of reward functions naturally encounters the problem of sparse rewards. The agent receives positive rewards only when it reaches the target point, and negative rewards when it reaches a boundary or collides with an obstacle. Normal navigation also results in negative rewards due to energy consumption. The agent fails to receive sufficient positive rewards during interactions, resulting in slow or even impossible learning. Currently, most deep reinforcement learning-based unmanned vehicle path planning algorithms use the inverse of the distance between the unmanned vehicle and the target point as the reward function. However, this reward function results in slow convergence, long training cycles, and even incorrect learning decisions. Therefore, it is necessary to rationally design the reward function to achieve the optimal path decision. By adaptively providing the agent with rewards based on its state at different times, and then optimizing the network parameters using the reward signal, the model's decision-making ability can be improved, resulting in the optimal path decision. Summary of the Invention

[0006] To solve the above problems, the present invention provides an adaptive unmanned boat path planning method based on fuzzy sets and deep reinforcement learning, comprising the following steps:

[0007] S1. Generate an obstacle environment for unmanned boat training and record the obstacle environment information, including the location of the obstacle and the location of the target point;

[0008] S2. Fuzzy logic is introduced to construct fuzzy rules for the UAV, and membership functions are used to describe the distance between the UAV and obstacles, target points, and the yaw angle of the UAV.

[0009] S3. During the navigation process of the unmanned boat, calculate in real time the distance of the unmanned boat relative to the obstacle and the target point, as well as the yaw angle of the unmanned boat;

[0010] S4. Process the result of the real-time calculation in step S3 using the unmanned boat fuzzy rule and output the fuzzy coefficient in the interval [0,1] in real time;

[0011] S5. Design a reward function for the unmanned boat, which adaptively rewards the unmanned boat based on the fuzzy coefficient;

[0012] S6. Build a UAV path planning model based on deep reinforcement learning. Train the model based on the adaptive rewards of the UAV in different states. Use the trained UAV path planning model to autonomously plan the optimal path.

[0013] Furthermore, Python's graphical interface module Tkinter is used to randomly generate an obstacle environment, and the number of obstacles in the obstacle environment is also random.

[0014] Furthermore, the fuzzy rules of the unmanned boat include obstacle fuzzy rules and target fuzzy rules, specifically:

[0015] Obtain the distance between the unmanned boat and the obstacle, the distance between the unmanned boat and the target point, and the yaw angle of the unmanned boat. Use the distance between the unmanned boat and the obstacle and the yaw angle of the unmanned boat as input variables of the obstacle fuzzy rule, and use the distance between the unmanned boat and the target point and the yaw angle of the unmanned boat as input variables of the target fuzzy rule. The output variables of the obstacle fuzzy rule and the target fuzzy rule are the penalty fuzzy coefficient and the reward fuzzy coefficient, respectively.

[0016] Obstacle fuzzy rules: The input variables are fuzzified, that is, the distance between the unmanned boat and the obstacle is divided into 5 segments, BVN means the distance to the obstacle is very close, BN means the distance to the obstacle is close, BA means the distance to the obstacle is moderate, BF means the distance to the obstacle is far, and BVF means the distance to the obstacle is very far; the yaw angle of the unmanned boat is divided into 5 segments, NRB means a large right angle, NRS means a small right angle, Z means zero, PLS means a small left angle, and PLB means a large left angle; the output variable penalty fuzzy coefficient is divided into 5 segments, PVS means a very small penalty fuzzy coefficient, PS means a small penalty fuzzy coefficient, PM means a medium penalty fuzzy coefficient, PB means a large penalty fuzzy coefficient, and PVB means a very large penalty fuzzy coefficient. 25 penalty fuzzy rules are established; the closer the distance between the unmanned boat and the obstacle and the smaller the yaw angle of the unmanned boat, the larger the penalty fuzzy coefficient; the farther the distance between the unmanned boat and the obstacle and the larger the yaw angle of the unmanned boat, the smaller the penalty fuzzy coefficient;

[0017] Target fuzzy rule: The input variable is fuzzified, that is, the distance between the UAV and the target point is divided into five segments: TVN indicates a very close distance to the target point, TN indicates a close distance to the target point, TA indicates a moderate distance to the target point, TF indicates a far distance to the target point, and TVF indicates a very far distance to the target point. The yaw angle of the UAV is divided into five segments, which is the same as the division method of the obstacle fuzzy rule: NRB indicates a large right angle, NRS indicates a small right angle, Z indicates zero, PLS indicates a small left angle, and PLB indicates a large left angle. The output variable reward fuzzy coefficient is divided into five segments: RVS indicates a very small reward fuzzy coefficient, RS indicates a small reward fuzzy coefficient, RM indicates a medium reward fuzzy coefficient, RB indicates a large reward fuzzy coefficient, and RVB indicates a very large reward. 25 reward fuzzy rules are established. The closer the distance between the UAV and the target point and the smaller the yaw angle of the UAV, the larger the reward fuzzy coefficient. The farther the distance between the UAV and the target point and the larger the yaw angle of the UAV, the smaller the reward fuzzy coefficient.

[0018] Furthermore, the reward function includes three parts: normal navigation, obstacle avoidance, and reaching the target point. Normal navigation means that there are no obstacles within the detection range of the unmanned boat. The reward function R for normal navigation is n Expressed as:

[0019]

[0020] Where: goal It represents the target point fuzzy coefficient obtained by inputting the current distance between the unmanned boat and the target point and the current yaw angle of the unmanned boat into the fuzzy logic controller, d goal Indicates the distance between the unmanned boat and the target point at the current moment, d max Indicates the distance between the initial position of the unmanned boat and the target point;

[0021] Obstacle avoidance means that there are obstacles within the detection range of the unmanned boat. The reward function R for obstacle avoidance is c Expressed as:

[0022]

[0023] Among them, ρ obs The obstacle fuzzy coefficient is obtained by inputting the current distance between the unmanned boat and the obstacle and the current yaw angle of the unmanned boat into the fuzzy logic controller, r det Indicates the maximum radius of the detection range, d obs Indicates the current distance between the unmanned boat and the obstacle.

[0024] Furthermore, the reward function R for reaching the target point end is a constant value, where when the unmanned boat reaches the target point, the feedback reward is R end ; When the unmanned boat reaches the boundary of the environment, the feedback penalty is R end ; When the unmanned boat collides with an obstacle, the feedback penalty is R end .

[0025] Beneficial effects of the present invention:

[0026] 1. For real-world path planning problems, traditional path planning algorithms require a pre-built model of the environment before performing path planning. However, obtaining environmental information in advance is extremely difficult in real-world situations. Therefore, this invention utilizes reinforcement learning methods, which can work in unknown environments and enable path planning tasks to autonomously adapt to the environment.

[0027] 2. In the DQN network model, a convolutional neural network is used to approximate the fitting value function to solve the dimensionality explosion problem in the Q-learning model. The experience replay mechanism is used to store sequence samples to improve sample utilization and reduce sample correlation. A dual-network mechanism is adopted to optimize the model with the current network and the target network to improve the stability of the training process.

[0028] 3. This invention introduces fuzzy logic to design the reward function, so that the reward feedback in reinforcement learning can be adaptively adjusted according to the state, obtaining high-quality experience samples to shorten the training time, and at the same time better guiding the intelligent agent to the correct target position. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a step diagram of the adaptive unmanned boat path planning method of the present invention;

[0030] Figure 2 Schematic diagram of the DQN network model of the present invention;

[0031] Figure 3 A schematic diagram of the reward function design of the present invention;

[0032] Figure 4 This is a schematic diagram of the unmanned boat path planning system used in the present invention;

[0033] Figure 5 Schematic diagram of fuzzy logic control of the present invention;

[0034] Figure 6 is the yaw angle membership function of the present invention;

[0035] Figure 7 is the obstacle distance membership function of the present invention;

[0036] Figure 8 is the penalty fuzzy coefficient membership function of the present invention;

[0037] Figure 9 is the target distance membership function of the present invention;

[0038] Figure 10 It is the reward fuzzy coefficient membership function of the present invention. DETAILED DESCRIPTION

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0040] The present invention discloses an adaptive unmanned boat path planning method based on fuzzy sets and deep reinforcement learning, such as Figure 1 Shown, including:

[0041] S1. Generate an obstacle environment for unmanned boat training and record the obstacle environment information, including the location of the obstacle and the location of the target point;

[0042] S2. Fuzzy logic is introduced to construct fuzzy rules for the UAV, and membership functions are used to describe the distance between the UAV and obstacles, target points, and the yaw angle of the UAV.

[0043] S3. During the navigation process of the unmanned boat, calculate in real time the distance of the unmanned boat relative to the obstacle and the target point, as well as the yaw angle of the unmanned boat;

[0044] S4. Process the result of the real-time calculation in step S3 using the unmanned boat fuzzy rule and output the fuzzy coefficient in the interval [0,1] in real time;

[0045] like Figure 5 As shown in FIG, the distance and yaw angle are input into the fuzzy controller, and the input data is first fuzzified. The fuzzy data is defuzzified by the inference engine formed by the rule base designed for the fuzzy rules of the unmanned boat in step S2 and the database for processing fuzzy data, and finally the fuzzy coefficient in the interval [0,1] is output.

[0046] S5. Design a reward function for the unmanned boat, which adaptively rewards the unmanned boat based on the fuzzy coefficient;

[0047] S6. Build a UAV path planning model based on deep reinforcement learning. Train the model based on the adaptive rewards of the UAV in different states. Use the trained UAV path planning model to autonomously plan the optimal path.

[0048] Specifically, the fuzzy rules of the unmanned boat include obstacle fuzzy rules and target fuzzy rules, which are as follows:

[0049] Obtain the distance between the unmanned boat and the obstacle, the distance between the unmanned boat and the target point, and the yaw angle of the unmanned boat.

[0050] Table 1 Obstacle fuzzy rules

[0051]

[0052] The obstacle fuzzy rules are shown in Table 1. The distance between the unmanned boat and the obstacle is divided into 5 segments using fuzzy language variables, and the distance is standardized to the range of [0, 20]. Figure 7As shown in the figure, the precise distance is converted into membership within different ranges, and the value is fuzzified. [0,5] is BVN, which means the distance to the obstacle is very close; [0,10] is BN, which means the distance to the obstacle is close; [5,15] is BA, which means the distance to the obstacle is moderate; [10,20] is BF, which means the distance to the obstacle is far; [15,20] is BVF, which means the distance to the obstacle is very far. Fuzzy language variables are also used to divide the yaw angle of the unmanned boat into 5 segments. The yaw angle is Variations within the range Figure 6 As shown, NRB means a large right angle. NRS means a small right angle. Z represents zero, PLS indicates a small left angle. PLB represents a large left angle; the penalty fuzzy coefficient is divided into 5 segments using fuzzy language variables. Figure 8 As shown in the figure, [0,0.25] is PVS, indicating a very small penalty fuzzy coefficient; [0,0.5] is PS, indicating a small penalty fuzzy coefficient; [0.25,0.75] is PM, indicating a medium penalty fuzzy coefficient; [0.5,1] is PB, indicating a large penalty fuzzy coefficient; and [0.75,1] is PVB, indicating a very large penalty fuzzy coefficient. Based on human expert experience, obstacle fuzzy rules are constructed: when the distance between the UAV and the obstacle is closer and the UAV's yaw angle is smaller, it indicates that there are obstacles on the UAV's path toward the target point and the UAV is approaching the obstacle. In this case, the output penalty fuzzy coefficient is larger, and the subsequent penalty is larger. When the distance between the UAV and the obstacle is farther and the UAV's yaw angle is larger, it indicates that the UAV is moving away from the obstacle but has deviated from the target point. In this case, the output penalty fuzzy coefficient is smaller, and the penalty is smaller.

[0053] Table 2 Target fuzzy rules

[0054]

[0055] The target fuzzy rules are shown in Table 2. The distance between the unmanned boat and the target point is divided into 5 segments using fuzzy language variables, and the distance is standardized to the range of [0, 20]. Figure 9 As shown in the figure, [0,5] is TVN, which means the distance to the target point is very close; [0,10] is TN, which means the distance to the target point is close; [5,15] is TA, which means the distance to the target point is moderate; [10,20] is TF, which means the distance to the target point is far; [15,20] is TVF, which means the distance to the target point is very far; the yaw angle of the unmanned boat is divided into 5 segments, which is the same as the division method of the obstacle fuzzy rule. NRB means a large right angle. NRS means a small right angle. Z represents zero, PLS indicates a small left angle. PLB represents a large left angle; the reward fuzzy coefficient is divided into 5 segments using fuzzy language variables. Figure 10 As shown in the figure, [0, 0.25] represents RVS, indicating a very small reward fuzziness coefficient; [0, 0.5] represents RS, indicating a small reward fuzziness coefficient; [0.25, 0.75] represents RM, indicating a medium reward fuzziness coefficient; [0.5, 1] ​​represents RB, indicating a large reward fuzziness coefficient; and [0.75, 1] ​​represents RVB, indicating a very large reward fuzziness coefficient. Target fuzzy rules are constructed based on human expert experience: if the distance between the UAV and the target point is closer and the UAV's yaw angle is smaller, indicating that the UAV is heading correctly and approaching the target point, then a larger reward fuzziness coefficient and a larger reward value will be generated. If the distance between the UAV and the target point is farther and the yaw angle is larger, indicating that the UAV is moving away from the target point, then the reward fuzziness coefficient will be reduced, resulting in a smaller reward in this state.

[0056] According to the fuzzy rules, the membership values ​​of the fuzzy coefficients belonging to different fuzzy linguistic variables are obtained, and the final output value is obtained by the area centroid method.

[0057] Specifically, the reward function includes three parts: normal navigation, obstacle avoidance, and reaching the target point, which can be expressed as:

[0058]

[0059] When calculating the reward for normal navigation, we first determine whether the detection range of the unmanned boat contains obstacles. If there are no obstacles, it means that the unmanned boat is in a safe area. The reward function for normal navigation is used for calculation:

[0060]

[0061] Where: goal It represents the reward fuzzy coefficient obtained by inputting the current distance between the unmanned boat and the target point and the current yaw angle of the unmanned boat into the fuzzy logic controller. As the unmanned boat moves, the distance between the unmanned boat and the target point changes, and the reward fuzzy coefficient also changes; d goal Indicates the distance between the unmanned boat and the target point at the current moment, d max Indicates the distance between the initial position of the unmanned boat and the target point.

[0062] Obstacle avoidance means that there is an obstacle within the detection range of the unmanned boat. At this time, the detection range is not a safe area, and there is a risk of collision between the unmanned boat and the obstacle. This means that the unmanned boat is not within the safe area. The reward function R for obstacle avoidance is c Expressed as:

[0063]

[0064] Among them, ρ obs It represents the penalty fuzzy coefficient obtained by inputting the current distance between the unmanned boat and the obstacle and the yaw angle of the unmanned boat into the fuzzy logic controller. The unmanned boat is constantly moving, and the distance between the unmanned boat and the obstacle is constantly changing during the movement, so the reward fuzzy coefficient is also different; r det Indicates the maximum radius of the detection range, d obs Indicates the distance between the unmanned boat and the obstacle at the current moment.

[0065] Specifically, a path planning model for an unmanned boat is constructed based on deep reinforcement learning, and the path planning model for the unmanned boat is trained according to the adaptive rewards of the unmanned boat in different states, such as Figure 2 As shown in the figure, the DQN network model solves the sample correlation problem by introducing the experience replay pool mechanism and improves the algorithm stability through the dual network mechanism of the target network and the current network. The agent interacts with the environment once at time t to obtain the reward and the state at time t+1. t ,a t ,r t ,s t+1 ) The experience information is stored in the experience replay pool, and the sampled experience data in the experience replay pool is sent to the target network to predict the target Q value y t =r t +λmax a Q(s t+1 ,a;θ). In the dual network mechanism of the DQN model, networks with the same structure but different parameters are used. The current network is used to predict the Q estimate, and the target network predicts the target Q value, and the loss function L(θ)=(y t -Q(s t ,a t ;θ)) 2 Update the current network parameters and update the parameters of the current network to the target network after time step C.

[0066] In the above example, Python's Tkinter module was used to construct a simulated training environment for an unmanned vehicle (UAV), which includes three main components: the UAV, obstacles, and target locations. In this example, the UAV serves as the intelligent agent controlled in reinforcement learning, and its detection range is a 135° sector with a radius of R. The number and size of obstacles in the randomly generated simulation training environment are also random. The simulation training environment information is saved for subsequent model training.

[0067] Among them, the position information of the unmanned boat is given in the form of a plane rectangular coordinate system, (x t ,y t) is the position of the unmanned boat at time t, θ t represents the navigation direction of the unmanned boat at time t, α t is the angle between the line connecting the unmanned boat and the target position at time t and the positive direction of the x-axis, β t is the angle between the line connecting the unmanned boat and the obstacle position scanned at time t and the positive direction of the x-axis, d obs is the distance from the unmanned boat to the obstacle at time t, d goal is the distance from the UAV to the target position at time t, and the yaw angle of the UAV is Δθ=θ t -α t If there are no obstacles in the unmanned boat scanning area, then d obs =R.

[0068] Specifically, the adaptive unmanned boat path planning process based on fuzzy sets and deep reinforcement, such as Figure 3 Shown, including:

[0069] S11. Calculate the obstacle distance between the UAV and the nearest obstacle within the scanning range at time t, the distance between the UAV and the target point, and the yaw angle of the UAV;

[0070] S12. Determine whether the obstacle distance is less than the detection radius R of the unmanned boat detection range. If so, execute step S13; otherwise, execute step S15;

[0071] S13. Determine whether the obstacle distance is less than the safe distance, i.e., the minimum distance at which the UAV will not collide. If so, it indicates that the UAV has collided with the obstacle or exceeded the environmental boundary, and the UAV is penalized. Otherwise, execute step S14.

[0072] S14. Determine whether the target distance is less than the safe distance. If so, the UAV has reached the target position, and a reward is given to the UAV, completing this round of training. Otherwise, the UAV has detected an obstacle within its detection range, and a penalty is calculated using the obstacle avoidance reward function. t = t + 1, and the process continues with step S11.

[0073] S15. Determine whether the target distance is less than the safe distance. If so, it means that the unmanned boat has reached the target position, and then a reward is fed back to the unmanned boat, and this round of training is completed; otherwise, it means that the unmanned boat is now in the safe area, and the reward is calculated using the normal navigation reward function, t = t + 1, and step S11 is executed.

[0074] The traditional reward function takes the inverse of the distance between the UAV and the target as a reward or penalizes it based on whether a collision occurs, which results in slow algorithm convergence, long training cycles, and even training failure. The present invention constructs a reward function based on fuzzy coefficients, such as Figure 4As shown in the figure, the unmanned boat perceives environmental information and obtains a series of historical experience information by interacting with the environment. The historical information includes the state information and decision information at time t and the reward information obtained by executing the decision. The improved DQN model based on the Q-learning algorithm is used to train the historical experience data. Finally, the training model is used to plan a path with low time cost and high safety for the unmanned boat in the obstacle environment.

[0075] To sum up, the path planning method for unmanned boats at sea of ​​the present invention realizes the path planning of unmanned boats, which can be used for tasks such as ocean patrol, water surface monitoring, and water surface rescue, and the path is safe and fast; it avoids the situation where the unmanned boats collide with obstacles during the execution of tasks, ensures the safety of the unmanned boats, and improves the efficiency of the unmanned boats in performing tasks.

[0076] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An adaptive unmanned boat path planning method based on fuzzy sets and deep reinforcement learning, characterized in that: The following steps are involved: S1. Generate an obstacle environment for unmanned boat training and record the obstacle environment information, including the location of the obstacle and the location of the target point; S2. Fuzzy logic is introduced to construct fuzzy rules for the UAV, and membership functions are used to describe the distance between the UAV and obstacles, target points, and the yaw angle of the UAV. The fuzzy rules of the unmanned boat include obstacle fuzzy rules and target fuzzy rules, specifically: Obtain the distance between the unmanned boat and the obstacle, the distance between the unmanned boat and the target point, and the yaw angle of the unmanned boat. Use the distance between the unmanned boat and the obstacle and the yaw angle of the unmanned boat as input variables of the obstacle fuzzy rule, and use the distance between the unmanned boat and the target point and the yaw angle of the unmanned boat as input variables of the target fuzzy rule. The output variables of the obstacle fuzzy rule and the target fuzzy rule are the penalty fuzzy coefficient and the reward fuzzy coefficient, respectively. Obstacle fuzzy rules: The input variables are fuzzified, that is, the distance between the unmanned boat and the obstacle is divided into 5 segments, BVN means the distance to the obstacle is very close, BN means the distance to the obstacle is close, BA means the distance to the obstacle is moderate, BF means the distance to the obstacle is far, and BVF means the distance to the obstacle is very far; the yaw angle of the unmanned boat is divided into 5 segments, NRB means a large right angle, NRS means a small right angle, Z means zero, PLS means a small left angle, and PLB means a large left angle; the output variable penalty fuzzy coefficient is divided into 5 segments, PVS means a very small penalty fuzzy coefficient, PS means a small penalty fuzzy coefficient, PM means a medium penalty fuzzy coefficient, PB means a large penalty fuzzy coefficient, and PVB means a very large penalty fuzzy coefficient. 25 penalty fuzzy rules are established; the closer the distance between the unmanned boat and the obstacle and the smaller the yaw angle of the unmanned boat, the larger the penalty fuzzy coefficient; the farther the distance between the unmanned boat and the obstacle and the larger the yaw angle of the unmanned boat, the smaller the penalty fuzzy coefficient; Target fuzzy rule: The input variable is fuzzified, that is, the distance between the UAV and the target point is divided into 5 segments, TVN means the distance to the target point is very close, TN means the distance to the target point is close, TA means the distance to the target point is moderate, TF means the distance to the target point is far, and TVF means the distance to the target point is very far; the yaw angle of the UAV is divided into 5 segments, which is the same as the division method of the obstacle fuzzy rule, NRB means a large right angle, NRS means a small right angle, Z means zero, PLS means a small left angle, and PLB means a large left angle; the output variable reward fuzzy coefficient is divided into 5 segments, RVS means a very small reward fuzzy coefficient, RS means a small reward fuzzy coefficient, RM means a medium reward fuzzy coefficient, RB means a large reward fuzzy coefficient, and RVB means a very large reward fuzzy coefficient. 25 reward fuzzy rules are established; the closer the distance between the UAV and the target point and the smaller the yaw angle of the UAV, the larger the reward fuzzy coefficient; the farther the distance between the UAV and the target point and the larger the yaw angle of the UAV, the smaller the reward fuzzy coefficient; S3. During the navigation process of the unmanned boat, calculate in real time the distance of the unmanned boat relative to the obstacle and the target point, as well as the yaw angle of the unmanned boat; S4. Process the result of the real-time calculation in step S3 using the unmanned boat fuzzy rule and output the fuzzy coefficient in the interval [0,1] in real time; S5. Design a reward function for the unmanned boat, which adaptively rewards the unmanned boat based on the fuzzy coefficient; The reward function includes three parts: normal navigation, obstacle avoidance and reaching the target point. Normal navigation means that there are no obstacles within the detection range of the unmanned boat. The reward function for normal navigation is R n Expressed as: in; ρ goal It represents the reward fuzzy coefficient obtained by inputting the distance between the unmanned boat and the target point at the current moment and the yaw angle of the unmanned boat at the current moment into the fuzzy logic controller, d goal Indicates the distance between the unmanned boat and the target point at the current moment, d max Indicates the distance between the initial position of the unmanned boat and the target point; Obstacle avoidance means that there are obstacles within the detection range of the unmanned boat. The reward function R for obstacle avoidance is c Expressed as: Among them, ρ obs It represents the penalty fuzzy coefficient obtained by inputting the current distance between the unmanned boat and the obstacle and the current yaw angle of the unmanned boat into the fuzzy logic controller, r det Indicates the maximum radius of the detection range, d obs Indicates the current distance between the unmanned boat and the obstacle; The reward function R for reaching the target point end is a constant value, where when the unmanned boat reaches the target point, the feedback reward is R end ; When the unmanned boat reaches the boundary of the environment, the feedback penalty is R end ; When the unmanned boat collides with an obstacle, the feedback penalty is R end ; S6. Build a path planning model for the unmanned vehicle based on deep reinforcement learning. Train the model based on adaptive rewards for the unmanned vehicle in different states. Use the trained model to autonomously plan the optimal path. The adaptive unmanned boat path planning process based on fuzzy sets and deep reinforcement includes: S11. Calculate the obstacle distance between the UAV and the nearest obstacle within the scanning range at time t, the distance between the UAV and the target point, and the yaw angle of the UAV; S12. Determine whether the obstacle distance is less than the detection radius of the unmanned boat detection range. If so, execute step S13; otherwise, execute step S15; S13. Determine whether the obstacle distance is less than the safe distance, i.e., the minimum distance at which the UAV will not collide. If so, it indicates that the UAV has collided with the obstacle or exceeded the environmental boundary, and the UAV is penalized. Otherwise, execute step S14. S14. Determine whether the target distance is less than the safe distance. If so, the UAV has reached the target position, and a reward is given to the UAV, completing this round of training. Otherwise, the UAV has detected an obstacle within its detection range, and a penalty is calculated using the obstacle avoidance reward function. t = t + 1, and the process continues with step S11. S15. Determine whether the target distance is less than the safe distance. If so, it means that the unmanned boat has reached the target position, and then a reward is fed back to the unmanned boat, and this round of training is completed; otherwise, it means that the unmanned boat is now in the safe area, and the reward is calculated using the normal navigation reward function, t = t + 1, and step S11 is executed.

2. The adaptive unmanned boat path planning method based on fuzzy sets and deep reinforcement learning according to claim 1 is characterized in that: The obstacle environment is randomly generated using Python's graphical interface module Tkinter, and the number of obstacles in the obstacle environment is also random.

Images