Autonomous navigation control method and system for unmanned ships based on artificial intelligence

Abstract

The present invention provides an artificial intelligence-based autonomous navigation control method and system for an unmanned ship, which relates to the field of control technology, including collecting real-time navigation data of an unmanned aircraft by multi-source sensors, inputting navigation environment feature data into a pre-trained reinforcement learning model, wherein the reinforcement learning model adopts a dual Q network structure, iteratively optimizes the navigation strategy based on an environmental reward function, and generates navigation strategy optimization parameters; constructs a multi-objective route optimization model according to the navigation strategy optimization parameters, adopts an improved ant colony algorithm to globally optimize the route, and outputs optimal navigation path data; establishes a hierarchical navigation control model according to the optimal navigation path data, wherein the hierarchical navigation control model includes a strategic layer, a tactical layer, and an execution layer, and outputs navigation control instructions through the hierarchical navigation control model to realize autonomous navigation control of the unmanned aircraft.

Description

Technical Field

[0001] The present invention relates to control technology, and in particular to an autonomous navigation control method and system for an unmanned ship based on artificial intelligence. Background Art

[0002] With the continuous deepening of ocean development and utilization, unmanned aerial vehicles are playing an increasingly important role in the fields of marine resource exploration, marine environment monitoring, maritime search and rescue, etc. Realizing autonomous navigation control of unmanned aerial vehicles is a key technology to ensure their safe and efficient execution of tasks.

[0003] At present, the autonomous navigation control of unmanned aerial vehicles mainly has the following technical problems:

[0004] Traditional navigation control methods usually use a single control strategy, which is difficult to adapt to the complex and changing marine environment. When encountering complex sea conditions, dynamic obstacles, etc., the control performance is significantly reduced, affecting the safety and reliability of navigation. A single control strategy cannot be optimized according to the characteristics of different navigation stages, and it is difficult to achieve global optimal control.

[0005] Most existing route planning methods are based on static environment assumptions and lack the ability to respond to dynamic environments in real time. When the navigation environment changes, the pre-planned route may not meet the actual navigation needs, resulting in low navigation efficiency and even collision risks.

[0006] Conventional trajectory tracking control algorithms often use simplified vehicle motion models and fail to fully consider the dynamic characteristics and motion constraints of the vehicle, which leads to insufficient control accuracy and easily produces drastic control outputs, increasing the load on the actuator.

[0007] Existing technical solutions generally lack a hierarchical control architecture design, and the coordination between the various control modules is poor, making it difficult to achieve an effective combination of the global and local aspects. This limits the overall performance of the system and makes it impossible to achieve the optimal control effect.

[0008] Traditional methods often use a simple weighted summation method to deal with multi-objective optimization problems, which makes it difficult to balance multiple performance indicators such as navigation safety and economy. This approach is prone to fall into local optimality, and the selection of weights lacks a theoretical basis.

[0009] In response to the above technical problems, it is urgent to propose an unmanned autonomous navigation control method based on strategy optimization and route optimization. Summary of the invention

[0010] The embodiments of the present invention provide an unmanned ship autonomous navigation control method and system based on artificial intelligence, which can solve the problems in the prior art.

[0011] According to a first aspect of the embodiments of the present invention,

[0012] Provides an artificial intelligence-based autonomous navigation control method for unmanned ships, including:

[0013] The real-time navigation data of the unmanned aerial vehicle is collected by multi-source sensors, wherein the multi-source sensors include inertial navigation sensors, visual sensors, lidar sensors, and millimeter-wave radar sensors; the real-time navigation data is subjected to feature fusion processing based on a deep neural network to obtain navigation environment feature data; the navigation environment feature data is input into a pre-trained reinforcement learning model, wherein the reinforcement learning model adopts a dual Q network structure, and the navigation strategy is iteratively optimized based on the environmental reward function to generate navigation strategy optimization parameters;

[0014] A multi-objective route optimization model is constructed according to the navigation strategy optimization parameters, wherein the multi-objective route optimization model takes the shortest navigation time and the minimum energy consumption as optimization objectives, and uses an improved ant colony algorithm to globally optimize the route, wherein the improved ant colony algorithm introduces an adaptive pheromone volatilization factor and a dynamic heuristic factor to improve the route search efficiency; and optimal navigation path data is output based on the multi-objective route optimization model;

[0015] A hierarchical navigation control model is established according to the optimal navigation path data, and the hierarchical navigation control model includes a strategic layer, a tactical layer and an execution layer, wherein the strategic layer performs global path planning based on the navigation strategy optimization parameters, the tactical layer performs local route optimization based on the optimal navigation path data, and the execution layer realizes the trajectory tracking of the aircraft based on the model predictive control algorithm; the navigation control instructions are output through the hierarchical navigation control model to realize autonomous navigation control of the unmanned aircraft.

[0016] In an optional embodiment,

[0017] The navigation environment characteristic data is input into a pre-trained reinforcement learning model, the reinforcement learning model adopts a dual Q network structure, and iteratively optimizes the navigation strategy based on the environmental reward function, and generates navigation strategy optimization parameters including:

[0018] Acquire real-time navigation data, the real-time navigation data including the current position coordinates of the aircraft, the current speed vector of the aircraft, the current attitude angle of the aircraft, the relative position matrix of surrounding obstacles, and the relative position vector of the target point; construct a state space based on the real-time navigation data, and construct an action space based on the speed adjustment amount, heading angle adjustment amount, and pitch angle adjustment amount that the aircraft can execute;

[0019] A multi-objective combined reward function is constructed based on the state space and the action space, wherein the multi-objective combined reward function includes a distance reward item, an obstacle avoidance reward item, an energy consumption reward item, and a smoothness reward item, wherein the distance reward item is calculated by the Euclidean distance between the current position of the aircraft and the target position, the obstacle avoidance reward item is calculated by the difference between the minimum Euclidean distance between the current position of the aircraft and the obstacle position and the safety distance threshold, the energy consumption reward item is calculated by the weighted square sum of the speed adjustment amount, the heading angle adjustment amount, and the pitch angle adjustment amount, and the smoothness reward item is calculated by the Euclidean distance between the current action vector and the action vector at the previous moment;

[0020] Constructing a dual Q network structure, the dual Q network structure includes an online network and a target network, the online network and the target network both include an input layer, three hidden layers and an output layer, the dimension of the input layer is the same as the dimension of the state space, the number of neurons in the three hidden layers is 256, 128 and 64 respectively, the hidden layer uses a ReLU activation function, the dimension of the output layer is the same as the dimension of the action space and uses a linear activation function; constructing a loss function based on the online network and the target network, the loss function includes an immediate reward term and a minimum Q value term predicted by the target network;

[0021] The online network and the target network are trained by an experience replay method, and training samples are collected and stored in an experience replay pool through an ε-greedy strategy. Random sampling is performed from the experience replay pool for network training, wherein the online network updates parameters at each training step, and the target network updates parameters by a soft update method at each preset number of steps; and strategy optimization parameters are output based on the trained online network and the target network, and the strategy optimization parameters include a network weight matrix, a network bias vector, a state normalization parameter, and an action normalization parameter.

[0022] In an optional embodiment,

[0023] A multi-objective route optimization model is constructed according to the navigation strategy optimization parameters. The multi-objective route optimization model takes the shortest navigation time and the minimum energy consumption as optimization objectives, and uses an improved ant colony algorithm to perform global optimization on the route. The optimal navigation path data output based on the multi-objective route optimization model includes:

[0024] Constructing a multi-objective function for route optimization, the multi-objective function includes a time optimization objective function and an energy consumption optimization objective function, wherein the time optimization objective function is calculated by summing the ratios of route segment distances to corresponding navigation speeds, and the energy consumption optimization objective function is calculated by weighted summing of speed energy consumption items, steering energy consumption items, and lifting energy consumption items;

[0025] Constructing route optimization constraint conditions based on the multi-objective function, the route optimization constraint conditions include speed constraint, steering constraint, climb constraint and obstacle constraint, the speed constraint is used to limit the value range of the navigation speed, the steering constraint is used to limit the value range of the heading angle change, the climb constraint is used to limit the value range of the height change, and the obstacle constraint is used to ensure a safe distance between the aircraft and the obstacle;

[0026] The route is encoded using a waypoint sequence, each waypoint contains position coordinates and navigation speed information, a state transition probability is constructed based on pheromone concentration and heuristic information, and the next feasible waypoint is determined by the state transition probability; wherein the heuristic information is obtained by weighted calculation of the distance between waypoints and energy consumption estimation;

[0027] Introducing an adaptive pheromone volatilization mechanism, the adaptive pheromone volatilization mechanism adopts a time-varying pheromone volatilization factor, the time-varying pheromone volatilization factor changes exponentially with the increase of the number of iterations, and the pheromone concentration is dynamically updated by the time-varying pheromone volatilization factor;

[0028] A dynamic heuristic factor is introduced, and the dynamic heuristic factor changes exponentially with the increase of the number of iterations. The weight of the heuristic information in the state transition probability is adjusted by the dynamic heuristic factor; iterative optimization is performed based on the adaptive pheromone volatilization mechanism and the dynamic heuristic factor, and the time cost and energy cost of the route generated by each iteration are evaluated, and the non-dominated solution is added to the Pareto frontier solution set;

[0029] An optimal solution that satisfies the time-energy consumption trade-off is selected from the Pareto frontier solution set as the optimal route, and the optimal route is subjected to spline interpolation smoothing processing to generate an optimal waypoint sequence and corresponding navigation control parameters.

[0030] In an optional embodiment,

[0031] The strategic layer performs global path planning based on the navigation strategy optimization parameters, including:

[0032] A parameterized curve is used to describe the navigation path, and the navigation path is expressed as a function of a navigation path parameter, wherein the value range of the navigation path parameter is 0 to 1, and the navigation path includes an X-axis coordinate component, a Y-axis coordinate component, and a Z-axis coordinate component;

[0033] The navigation path is described based on a cubic B-spline curve, and the spatial coordinates of the navigation path are calculated by summing the product of the combination coefficient and the B-spline basis function, and the B-spline basis function is calculated by the power function of the combination number and the navigation path parameter;

[0034] Constructing navigation path constraints, wherein the navigation path constraints include position constraints, speed constraints, acceleration constraints, and curvature constraints, wherein the position constraints are used to limit the coordinate range of the navigation path, the speed constraints are used to limit the first-order derivative range of the navigation path, the acceleration constraints are used to limit the second-order derivative range of the navigation path, and the curvature constraints are used to limit the curvature variation range of the navigation path;

[0035] Constructing a global multi-objective optimization function, the global multi-objective optimization function includes a path length term, a curvature integral term, a curvature change rate integral term, and an obstacle distance measurement term, and performing a weighted combination of each term in the global multi-objective optimization function by a weighting coefficient;

[0036] Discretize the navigation path interval into multiple path segments, discretize the global multi-objective optimization function, and construct a global discretization objective function, wherein the global discretization objective function includes path segment length, path segment curvature, curvature change, and minimum obstacle distance;

[0037] The global discretized objective function is iteratively optimized using a sequential quadratic programming method, a quadratic approximation of the global multi-objective optimization function and linear constraints are constructed to solve the quadratic programming problem to obtain a position increment of the control point, and the position coordinates of the control point are updated based on the position increment;

[0038] The optimized navigation path is smoothed by quintic spline interpolation, and a smooth and continuous navigation path is generated by maintaining the continuity of the position derivative, velocity derivative and acceleration derivative at the interpolation endpoints. A waypoint sequence and a corresponding navigation speed are generated based on the smooth and continuous navigation path.

[0039] In an optional embodiment,

[0040] The tactical layer performs local route optimization based on the optimal navigation path data, including:

[0041] A local planning window is constructed based on the current position, current heading angle and current speed of the aircraft, and the size of the local planning window is adaptively adjusted through a speed adaptive coefficient, so as to establish a positive correlation between the size of the local planning window and the current speed;

[0042] A local environment model is constructed using multi-source sensor data, the sensor data is transformed into position data in a local coordinate system, the occupancy probability of the grid map is updated based on the position data, and the occupancy probability value of each grid is calculated by a logarithmic probability update method;

[0043] The Kalman filter algorithm is used to track dynamic obstacles, and the state vector of the obstacle is predicted through the state prediction equation. The predicted state is updated based on the measurement data to obtain the precise position and motion information of the obstacle.

[0044] Constructing a route optimization model, taking the kinematic equation of the aircraft as the state equation, the state equation including the position coordinates, velocity and heading angle, constructing state constraints and dynamic constraints, the state constraints are used to limit the value range of the position coordinates and the value range of the velocity, and the dynamic constraints are used to limit the value range of the angular velocity, acceleration and the rate of change of the heading angle;

[0045] Constructing a multi-objective cost function, the multi-objective cost function includes a reference trajectory tracking term, an obstacle avoidance term, a trajectory smoothing term, and an energy consumption term, and performing a weighted combination of each term in the multi-objective cost function by a weighting coefficient;

[0046] The multi-objective cost function is optimized and solved by using a gradient projection method, the gradient of the cost function with respect to the control amount is calculated, the updated control amount is projected to a feasible domain, and the optimal step length is determined by backtracking line search.

[0047] In an optional embodiment,

[0048] The execution layer realizes the trajectory tracking of the aircraft based on the model predictive control algorithm; outputs the navigation control instructions through the hierarchical navigation control model to realize the autonomous navigation control of the unmanned aircraft, including:

[0049] Establishing a six-degree-of-freedom dynamics model of the aircraft, wherein the six-degree-of-freedom dynamics model includes a translation equation and a rotation equation, wherein the translation equation includes a thrust term, a drag term, and a gravity term, and the rotation equation includes a torque term and a gyroscopic torque term;

[0050] The six-degree-of-freedom dynamics model is constructed as a state space expression, wherein the state vector of the state space expression includes position coordinates, velocity, attitude angle and angular velocity, and the control vector of the state space expression includes thrust and three-axis torque;

[0051] Linearizing the state space expression, calculating the Jacobian matrix of the system state equation with respect to the state vector and the control vector, and constructing a linearized prediction model;

[0052] Constructing a trajectory tracking prediction cost function, wherein the prediction cost function includes a tracking error term, a control amount penalty term, and a control increment penalty term, and performing a weighted combination of each penalty term through a weight matrix;

[0053] Constructing state constraints, the state constraints include position coordinate constraints and attitude angle constraints, constructing control constraints, the control constraints include thrust constraints and torque constraints, and constructing control increment constraints, the control increment constraints include thrust increment constraints and torque increment constraints;

[0054] Converting the prediction cost function into a standard form of a quadratic programming problem, calculating a quadratic matrix and linear term coefficients, and constructing an inequality constraint matrix and an equality constraint matrix;

[0055] The quadratic programming problem is solved by adopting the interior point method, the Newton step is obtained by solving the Karush-Kuhn-Tucker system, the optimal step is determined based on the complementary slackness, and the primal variables and the dual variables are iteratively updated;

[0056] Based on the optimization solution results, control quantity mapping is performed, the total thrust is distributed to each actuator through the thrust distribution matrix, and the three-axis torque is distributed to each actuator through the torque distribution matrix;

[0057] The output of the actuator is limited, the motor speed is limited according to the motor characteristics, the rudder angle is limited according to the rudder characteristics, and the final control command is generated.

[0058] According to a second aspect of the embodiments of the present invention,

[0059] Provides an AI-based autonomous navigation control system for unmanned ships, including:

[0060] The first unit is used to collect real-time navigation data of the unmanned aerial vehicle through multi-source sensors, wherein the multi-source sensors include inertial navigation sensors, visual sensors, lidar sensors, and millimeter-wave radar sensors; perform feature fusion processing on the real-time navigation data based on a deep neural network to obtain navigation environment feature data; input the navigation environment feature data into a pre-trained reinforcement learning model, wherein the reinforcement learning model adopts a dual Q network structure, iteratively optimizes the navigation strategy based on the environmental reward function, and generates navigation strategy optimization parameters;

[0061] The second unit is used to construct a multi-objective route optimization model according to the navigation strategy optimization parameters, wherein the multi-objective route optimization model takes the shortest navigation time and the minimum energy consumption as optimization objectives, and uses an improved ant colony algorithm to globally optimize the route, wherein the improved ant colony algorithm introduces an adaptive pheromone volatilization factor and a dynamic heuristic factor to improve the route search efficiency; and output optimal navigation path data based on the multi-objective route optimization model;

[0062] The third unit is used to establish a hierarchical navigation control model according to the optimal navigation path data, wherein the hierarchical navigation control model includes a strategic layer, a tactical layer and an execution layer, wherein the strategic layer performs global path planning based on the navigation strategy optimization parameters, the tactical layer performs local route optimization based on the optimal navigation path data, and the execution layer realizes the trajectory tracking of the aircraft based on the model predictive control algorithm; and the navigation control instructions are output through the hierarchical navigation control model to realize the autonomous navigation control of the unmanned aircraft.

[0063] According to a third aspect of the embodiments of the present invention,

[0064] An electronic device is provided, comprising:

[0065] processor;

[0066] a memory for storing processor-executable instructions;

[0067] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0068] According to a fourth aspect of the embodiments of the present invention,

[0069] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0070] The present invention has the following beneficial effects through the above technical solution:

[0071] The present invention uses a combination of multi-source sensors and deep neural networks to obtain navigation environment characteristics, and introduces a reinforcement learning model with a dual Q network structure for strategy optimization, which significantly improves the accuracy of environmental perception and strategy optimization. Through the fusion processing of multi-source sensor data, the system can fully perceive the dynamic changes of the navigation environment; and the feature extraction method based on deep neural networks effectively extracts high-dimensional feature information of the environment. The introduction of the dual Q network structure solves the over-estimation problem of traditional Q learning, making strategy optimization more stable and reliable, and effectively improving the adaptability of the navigation strategy.

[0072] The improved ant colony algorithm proposed in the present invention introduces adaptive pheromone volatilization factors and dynamic heuristic factors in multi-objective route optimization, which greatly improves the efficiency and optimization effect of route planning. The adaptive pheromone volatilization factor can dynamically adjust the volatilization rate of pheromone according to the search process to prevent the algorithm from falling into the local optimum; the dynamic heuristic factor enhances the exploration ability of the algorithm and accelerates the convergence speed. Through the multi-objective optimization of navigation time and energy consumption, the system can achieve rational use of energy while ensuring navigation efficiency.

[0073] The hierarchical navigation control model constructed by the present invention realizes the organic combination of global planning and local control, and significantly enhances the control performance of the system. The global path planning of the strategic layer ensures the overall optimality of the navigation mission; the local route optimization of the tactical layer provides the ability to respond quickly to the dynamic environment; the execution layer uses the model predictive control algorithm to achieve accurate trajectory tracking, ensuring the stability and reliability of the control instructions. The synergy of the three control levels enables the system to have stronger environmental adaptability and control accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 The figure is a flow chart of an autonomous navigation control method for an unmanned ship based on artificial intelligence according to an embodiment of the present invention;

[0075] Figure 2 The figure is a schematic diagram of the structure of an unmanned ship autonomous navigation control system based on artificial intelligence according to an embodiment of the present invention. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0078] Figure 1 FIG. 1 is a flow chart of an autonomous navigation control method for an unmanned ship based on artificial intelligence according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0079] S101. Collecting real-time navigation data of the unmanned aerial vehicle through multi-source sensors, wherein the multi-source sensors include inertial navigation sensors, visual sensors, lidar sensors, and millimeter-wave radar sensors; performing feature fusion processing on the real-time navigation data based on a deep neural network to obtain navigation environment feature data; inputting the navigation environment feature data into a pre-trained reinforcement learning model, wherein the reinforcement learning model adopts a dual Q network structure, iteratively optimizes the navigation strategy based on the environmental reward function, and generates navigation strategy optimization parameters;

[0080] S102. Construct a multi-objective route optimization model according to the navigation strategy optimization parameters, wherein the multi-objective route optimization model takes the shortest navigation time and the minimum energy consumption as optimization objectives, and uses an improved ant colony algorithm to globally optimize the route, wherein the improved ant colony algorithm introduces an adaptive pheromone volatilization factor and a dynamic heuristic factor to improve the route search efficiency; output optimal navigation path data based on the multi-objective route optimization model;

[0081] S103. Establish a hierarchical navigation control model based on the optimal navigation path data, the hierarchical navigation control model includes a strategic layer, a tactical layer and an execution layer, wherein the strategic layer performs global path planning based on the navigation strategy optimization parameters, the tactical layer performs local route optimization based on the optimal navigation path data, and the execution layer realizes the trajectory tracking of the aircraft based on the model predictive control algorithm; outputs navigation control instructions through the hierarchical navigation control model to realize autonomous navigation control of the unmanned aircraft.

[0082] The specific implementation method of the unmanned ship autonomous navigation control method based on artificial intelligence is as follows:

[0083] First, the real-time navigation data of the unmanned aerial vehicle is collected through multi-source sensors. Specifically, the inertial navigation sensor collects the position, speed, attitude and other motion state data of the aircraft; the visual sensor collects image data of the surrounding environment; the lidar sensor collects three-dimensional point cloud data of surrounding obstacles; and the millimeter-wave radar sensor collects the position and speed information of distant targets. For example, the position data collected by the inertial navigation sensor is longitude and latitude (120.5°E, 30.2°N), the speed is 10 knots, and the heading angle is 45°; the image resolution collected by the visual sensor is 1920×1080 pixels; the point cloud data density collected by the lidar is 100 points per square meter; and the detection distance of the millimeter-wave radar is 200 meters.

[0084] Next, a deep neural network is used to perform feature fusion processing on multi-source sensor data. First, the sensor data is preprocessed, such as filtering and denoising the image data and downsampling the point cloud data. Then a multimodal deep neural network is designed, which includes convolutional layers, pooling layers, and fully connected layers to extract the features of each type of sensor data. Finally, the features of different modes are fused through the feature fusion layer to obtain a unified feature representation of the navigation environment. For example, the fused feature dimension is 256, which contains information such as obstacle location, type, and motion status.

[0085] Then, the fused environmental features are input into the pre-trained reinforcement learning model. The model adopts a dual Q network structure, including a main network and a target network. The main network is used to estimate the Q value of each action in the current state, and the target network is used to calculate the target Q value. The state space of the model includes the state of the aircraft position, speed, heading, and the surrounding environment characteristics; the action space includes control quantities such as heading and speed; and the reward function considers factors such as navigation time, energy consumption, and collision risk. By interacting with the simulation environment, the network parameters are continuously updated to optimize the navigation strategy. Finally, the navigation strategy optimization parameters are output, such as the optimal heading angle of 60° and the optimal speed of 8 knots.

[0086] Based on the optimization parameters of navigation strategy, a multi-objective route optimization model is constructed. The model takes the shortest navigation time and the minimum energy consumption as the optimization objectives, and uses the improved ant colony algorithm for global route optimization. The improved ant colony algorithm introduces an adaptive pheromone volatilization factor to dynamically adjust the pheromone volatilization rate according to the number of iterations; at the same time, a dynamic heuristic factor is introduced to dynamically adjust the influence of the heuristic information according to the current optimal solution. These improvements improve the search efficiency and convergence speed of the algorithm. Finally, the optimal navigation path data is output, including a series of waypoint coordinates.

[0087] According to the optimal navigation path data, a hierarchical navigation control model is established. The model includes three levels: strategic layer, tactical layer and execution layer. The strategic layer performs global path planning based on the navigation strategy optimization parameters to generate the overall navigation strategy. The tactical layer receives the planning results of the strategic layer, combines the real-time environmental information to optimize the local route, and generates the local optimal path. The execution layer converts the optimized path of the tactical layer into specific control instructions based on the model predictive control algorithm. The model predictive control algorithm solves the optimal control problem online, predicts the system state in the future, and selects the optimal control sequence. Finally, the execution layer outputs specific control instructions such as rudder angle and propeller speed to achieve accurate trajectory tracking of the unmanned vehicle.

[0088] Through the above-mentioned hierarchical navigation control model, closed-loop autonomous navigation from global path planning to local trajectory optimization and then to underlying control is realized. This method makes full use of artificial intelligence technology, improves the environmental perception, decision-making planning and control execution capabilities of unmanned ships, and provides an effective solution for achieving safe and efficient autonomous navigation in complex environments.

[0089] In an optional implementation, the navigation environment characteristic data is input into a pre-trained reinforcement learning model, the reinforcement learning model adopts a dual Q network structure, and the navigation strategy is iteratively optimized based on the environmental reward function, and the navigation strategy optimization parameters are generated, including:

[0090] Acquire real-time navigation data, the real-time navigation data including the current position coordinates of the aircraft, the current speed vector of the aircraft, the current attitude angle of the aircraft, the relative position matrix of surrounding obstacles, and the relative position vector of the target point; construct a state space based on the real-time navigation data, and construct an action space based on the speed adjustment amount, heading angle adjustment amount, and pitch angle adjustment amount that the aircraft can execute;

[0091] A multi-objective combined reward function is constructed based on the state space and the action space, wherein the multi-objective combined reward function includes a distance reward item, an obstacle avoidance reward item, an energy consumption reward item, and a smoothness reward item, wherein the distance reward item is calculated by the Euclidean distance between the current position of the aircraft and the target position, the obstacle avoidance reward item is calculated by the difference between the minimum Euclidean distance between the current position of the aircraft and the obstacle position and the safety distance threshold, the energy consumption reward item is calculated by the weighted square sum of the speed adjustment amount, the heading angle adjustment amount, and the pitch angle adjustment amount, and the smoothness reward item is calculated by the Euclidean distance between the current action vector and the action vector at the previous moment;

[0092] Constructing a dual Q network structure, the dual Q network structure includes an online network and a target network, the online network and the target network both include an input layer, three hidden layers and an output layer, the dimension of the input layer is the same as the dimension of the state space, the number of neurons in the three hidden layers is 256, 128 and 64 respectively, the hidden layer uses a ReLU activation function, the dimension of the output layer is the same as the dimension of the action space and uses a linear activation function; constructing a loss function based on the online network and the target network, the loss function includes an immediate reward term and a minimum Q value term predicted by the target network;

[0093] The online network and the target network are trained by an experience replay method, and training samples are collected and stored in an experience replay pool through an ε-greedy strategy. Random sampling is performed from the experience replay pool for network training, wherein the online network updates parameters at each training step, and the target network updates parameters by a soft update method at each preset number of steps; and strategy optimization parameters are output based on the trained online network and the target network, and the strategy optimization parameters include a network weight matrix, a network bias vector, a state normalization parameter, and an action normalization parameter.

[0094] This embodiment provides a navigation strategy optimization method based on reinforcement learning, and the specific steps are as follows:

[0095] First, obtain the navigation environment feature data, including the current position coordinates of the aircraft, the current velocity vector, the current attitude angle, the relative position matrix of the surrounding obstacles, and the relative position vector of the target point. For example, the current position coordinates of the aircraft are (100, 200, 50), the velocity vector is (5, 3, 1), the attitude angle is (30°, 45°, 0°), the relative position matrix of the obstacles is [[50, 30, 20], [-30, 40, 10]], and the relative position vector of the target point is (500, 300,100).

[0096] Then, the state space is constructed based on the acquired real-time navigation data. The state space consists of the current position, speed, attitude angle of the aircraft, and the relative positions of obstacles and target points, and the dimension is 15. At the same time, the action space is constructed according to the speed adjustment, heading adjustment, and pitch adjustment that the aircraft can perform. For example, the speed adjustment range is [-2, 2] m / s, the heading adjustment range is [-30°, 30°], and the pitch adjustment range is [-15°, 15°], and the action space dimension is 3.

[0097] Next, a multi-objective combined reward function is constructed, including four sub-items: distance reward, obstacle avoidance reward, energy consumption reward, and smoothness reward. The distance reward is obtained by calculating the Euclidean distance between the current position of the aircraft and the target position. The closer the distance, the higher the reward. The obstacle avoidance reward is obtained by calculating the difference between the distance between the aircraft and the nearest obstacle and the safety threshold. A positive reward is given when the distance is greater than the threshold, otherwise a negative reward is given. The energy consumption reward is calculated based on the adjustment amount of the speed, heading angle, and pitch angle. The smaller the adjustment amount, the higher the reward. The smoothness reward is obtained by calculating the Euclidean distance between the current action and the previous action. The smaller the distance, the higher the reward. The four sub-items are combined according to certain weights to obtain the final reward.

[0098] Then, a dual Q network structure is constructed, including an online network and a target network. The two networks have the same structure, both containing an input layer, three hidden layers, and an output layer. The dimension of the input layer is 15, which is consistent with the state space. The number of neurons in the three hidden layers is 256, 128, and 64, respectively, and the activation function is ReLU. The dimension of the output layer is 3, which is consistent with the action space, and a linear activation function is used. The loss function is constructed based on the online network and the target network, including the immediate reward term and the minimum Q value term predicted by the target network.

[0099] Next, the network is trained using the experience replay method. The training samples are collected using the ε-greedy strategy and stored in an experience replay pool with a capacity of 10,000. 32 samples are randomly sampled from the replay pool for training each time. The online network updates parameters at each training step, and the target network updates parameters once every 100 steps through soft update, with a soft update coefficient of 0.01. During the training process, the ε value decays linearly from 1 to 0.1, and the total number of training steps is 100,000.

[0100] Finally, the trained network output strategy optimization parameters are obtained, including network weight matrix, bias vector, state normalization parameter and action normalization parameter. These parameters can be used for subsequent online decision making to achieve optimization of navigation strategy.

[0101] This method optimizes the navigation strategy in complex navigation environments by constructing a dual Q network structure and a multi-objective combined reward function. The training stability and efficiency are improved by using techniques such as experience replay and soft update. The output strategy optimization parameters can be directly applied to actual navigation control to improve navigation safety and efficiency.

[0102] In an optional implementation, a multi-objective route optimization model is constructed according to the navigation strategy optimization parameters, the multi-objective route optimization model takes the shortest navigation time and the minimum energy consumption as optimization objectives, and uses an improved ant colony algorithm to perform global optimization on the route. Outputting the optimal navigation path data based on the multi-objective route optimization model includes:

[0103] Constructing a multi-objective function for route optimization, the multi-objective function includes a time optimization objective function and an energy consumption optimization objective function, wherein the time optimization objective function is calculated by summing the ratios of route segment distances to corresponding navigation speeds, and the energy consumption optimization objective function is calculated by weighted summing of speed energy consumption items, steering energy consumption items, and lifting energy consumption items;

[0104] Constructing route optimization constraint conditions based on the multi-objective function, the route optimization constraint conditions include speed constraint, steering constraint, climb constraint and obstacle constraint, the speed constraint is used to limit the value range of the navigation speed, the steering constraint is used to limit the value range of the heading angle change, the climb constraint is used to limit the value range of the height change, and the obstacle constraint is used to ensure a safe distance between the aircraft and the obstacle;

[0105] The route is encoded using a waypoint sequence, each waypoint contains position coordinates and navigation speed information, a state transition probability is constructed based on pheromone concentration and heuristic information, and the next feasible waypoint is determined by the state transition probability; wherein the heuristic information is obtained by weighted calculation of the distance between waypoints and energy consumption estimation;

[0106] Introducing an adaptive pheromone volatilization mechanism, the adaptive pheromone volatilization mechanism adopts a time-varying pheromone volatilization factor, the time-varying pheromone volatilization factor changes exponentially with the increase of the number of iterations, and the pheromone concentration is dynamically updated by the time-varying pheromone volatilization factor;

[0107] A dynamic heuristic factor is introduced, and the dynamic heuristic factor changes exponentially with the increase of the number of iterations. The weight of the heuristic information in the state transition probability is adjusted by the dynamic heuristic factor; iterative optimization is performed based on the adaptive pheromone volatilization mechanism and the dynamic heuristic factor, and the time cost and energy cost of the route generated by each iteration are evaluated, and the non-dominated solution is added to the Pareto frontier solution set;

[0108] An optimal solution that satisfies the time-energy consumption trade-off is selected from the Pareto frontier solution set as the optimal route, and the optimal route is subjected to spline interpolation smoothing processing to generate an optimal waypoint sequence and corresponding navigation control parameters.

[0109] A multi-objective route optimization model is constructed based on the navigation strategy optimization parameters, and the model takes the shortest navigation time and the minimum energy consumption as the optimization objectives. The improved ant colony algorithm is used to globally optimize the route, and the optimal navigation path data is output based on the multi-objective route optimization model.

[0110] First, a multi-objective function for route optimization is constructed, including a time optimization objective function and an energy consumption optimization objective function. The time optimization objective function is calculated by summing the ratio of the route segment distance to the corresponding navigation speed. Specifically, the entire route is divided into N segments, the distance of each segment is di, and the corresponding navigation speed is vi, then the time optimization objective function can be expressed in the form of d1 / v1 + d2 / v2 + ... + dN / vN. The energy consumption optimization objective function is calculated by weighted summation of speed energy consumption items, steering energy consumption items, and lifting energy consumption items. The speed energy consumption item is proportional to the square of the navigation speed, the steering energy consumption item is proportional to the square of the steering angle, and the lifting energy consumption item is proportional to the square of the altitude change. The weight coefficients of each item can be adjusted according to the actual situation. For example, the weight of the speed energy consumption item can be 0.5, the weight of the steering energy consumption item can be 0.3, and the weight of the lifting energy consumption item can be 0.2.

[0111] Next, based on the multi-objective function, the route optimization constraints are constructed, including speed constraints, steering constraints, climb constraints and obstacle constraints. The speed constraint is used to limit the range of the navigation speed, for example, the minimum speed can be set to 10m / s and the maximum speed can be set to 50m / s. The steering constraint is used to limit the range of the heading angle change, for example, the maximum steering angle can be set to 30°. The climb constraint is used to limit the range of the height change, for example, the maximum climb rate can be set to 5m / s. The obstacle constraint is used to ensure the safe distance between the aircraft and the obstacle, for example, the minimum safe distance can be set to 100m.

[0112] Then, the route is encoded using a waypoint sequence, each of which contains location coordinates and navigation speed information. The state transition probability is constructed based on the pheromone concentration and heuristic information, and the next feasible waypoint is determined by the state transition probability. The heuristic information is obtained by weighted calculation of the distance between waypoints and the energy consumption estimate. Specifically, the distance weight can be set to 0.6 and the energy consumption weight can be set to 0.4. Then the heuristic information can be expressed in the form of 0.6 / dij + 0.4 / Eij, where dij is the distance from waypoint i to j, and Eij is the estimated energy consumption from i to j.

[0113] In order to improve the convergence performance of the algorithm, an adaptive pheromone volatilization mechanism is introduced. This mechanism uses a time-varying pheromone volatilization factor, which changes exponentially with the increase of the number of iterations. For example, the initial volatilization factor can be set to 0.9 and the minimum volatilization factor can be set to 0.1. Then the volatilization factor of the tth iteration can be expressed in the form of 0.9×(0.1 / 0.9)^(t / T), where T is the total number of iterations. By dynamically updating the pheromone concentration through the time-varying pheromone volatilization factor, a large global search capability can be maintained in the early stage of the iteration, and the local search capability can be enhanced in the later stage of the iteration.

[0114] At the same time, a dynamic heuristic factor is introduced, which changes exponentially with the number of iterations. For example, the initial heuristic factor can be set to 1 and the maximum heuristic factor can be set to 5. Then the heuristic factor of the tth iteration can be expressed in the form of 1+(5-1)×(t / T)^2. By adjusting the weight of the heuristic information in the state transition probability through the dynamic heuristic factor, the importance of the heuristic information can be gradually increased during the iteration process, thereby improving the convergence speed of the algorithm.

[0115] Based on the adaptive pheromone volatilization mechanism and dynamic heuristic factors, iterative optimization is performed, and the time cost and energy cost of the routes generated in each iteration are evaluated, and the non-dominated solutions are added to the Pareto frontier solution set. Specifically, the population size can be set to 100 and the number of iterations can be set to 1000. In each iteration, 100 candidate routes are generated, the time cost and energy cost of each route are calculated, and the non-inferior solutions are added to the Pareto frontier solution set. After 1000 iterations, the Pareto frontier solution set may contain hundreds of non-inferior solutions.

[0116] Finally, the optimal solution that satisfies the time-energy trade-off is selected from the Pareto frontier solution set as the optimal route. The weighted sum method can be used for selection. For example, the time weight is set to 0.6 and the energy weight is set to 0.4. The weighted sum score of each solution is calculated, and the solution with the lowest score is selected as the optimal solution. The optimal route is smoothed by spline interpolation to generate the optimal waypoint sequence and the corresponding navigation control parameters. Specifically, the cubic spline interpolation method can be used to insert several smooth transition points between the original waypoints to make the route smoother and more continuous. The final output optimal waypoint sequence may contain hundreds of points, each of which contains three-dimensional coordinate information and navigation speed information.

[0117] The above method can effectively solve the UAV route planning problem and realize the multi-objective optimization of flight time and energy consumption under the premise of meeting various constraints. This method has strong adaptability and flexibility, and various parameters can be adjusted according to actual application needs to achieve the best planning effect.

[0118] In an optional implementation, the strategic layer performs global path planning based on the navigation strategy optimization parameters, including:

[0119] A parameterized curve is used to describe the navigation path, and the navigation path is expressed as a function of a navigation path parameter, wherein the value range of the navigation path parameter is 0 to 1, and the navigation path includes an X-axis coordinate component, a Y-axis coordinate component, and a Z-axis coordinate component;

[0120] The navigation path is described based on a cubic B-spline curve, and the spatial coordinates of the navigation path are calculated by summing the product of the combination coefficient and the B-spline basis function, and the B-spline basis function is calculated by the power function of the combination number and the navigation path parameter;

[0121] Constructing navigation path constraints, wherein the navigation path constraints include position constraints, speed constraints, acceleration constraints, and curvature constraints, wherein the position constraints are used to limit the coordinate range of the navigation path, the speed constraints are used to limit the first-order derivative range of the navigation path, the acceleration constraints are used to limit the second-order derivative range of the navigation path, and the curvature constraints are used to limit the curvature variation range of the navigation path;

[0122] Constructing a global multi-objective optimization function, the global multi-objective optimization function includes a path length term, a curvature integral term, a curvature change rate integral term, and an obstacle distance measurement term, and performing a weighted combination of each term in the global multi-objective optimization function by a weighting coefficient;

[0123] Discretize the navigation path interval into multiple path segments, discretize the global multi-objective optimization function, and construct a global discretization objective function, wherein the global discretization objective function includes path segment length, path segment curvature, curvature change, and minimum obstacle distance;

[0124] The global discretized objective function is iteratively optimized using a sequential quadratic programming method, a quadratic approximation of the global multi-objective optimization function and linear constraints are constructed to solve the quadratic programming problem to obtain a position increment of the control point, and the position coordinates of the control point are updated based on the position increment;

[0125] The optimized navigation path is smoothed by quintic spline interpolation, and a smooth and continuous navigation path is generated by maintaining the continuity of the position derivative, velocity derivative and acceleration derivative at the interpolation endpoints. A waypoint sequence and a corresponding navigation speed are generated based on the smooth and continuous navigation path.

[0126] In a specific implementation, a parameterized curve is first used to describe the navigation path. The navigation path is represented as a function of the navigation path parameter t, where the value of t ranges from 0 to 1. The navigation path includes three coordinate components: the X-axis, the Y-axis, and the Z-axis. For example, the component functions of the navigation path on the three coordinate axes can be represented by X(t), Y(t), and Z(t), respectively.

[0127] Next, a cubic B-spline curve is used to describe the navigation path more accurately. The cubic B-spline curve consists of a series of control points and basis functions. By combining the product of the coefficients and the B-spline basis functions and summing them, the spatial coordinates of the navigation path at any parameter t can be calculated. The B-spline basis function is calculated by combining the number of combinations and the power function of t. For example, for the four control points P0, P1, P2, and P3, the coordinates of any point on the navigation path can be expressed as: P(t) = P0B0(t) +P1B1(t) + P2B2(t) + P3B3(t), where Bi(t) is the corresponding basis function.

[0128] Then, the constraints of the navigation path are constructed, including position constraints, speed constraints, acceleration constraints and curvature constraints. Position constraints are used to limit the coordinate range of the navigation path, such as the X-axis coordinate range is [-100, 100]. Speed ​​constraints limit the range of the first-order derivative of the path, such as the absolute value of the speed does not exceed 30m / s. Acceleration constraints limit the range of the second-order derivative, such as the absolute value of the acceleration does not exceed 5m / s2. Curvature constraints limit the range of the curvature of the path, such as the curvature radius is not less than 50m.

[0129] Then, a global multi-objective optimization function is constructed, which includes the path length term, the curvature integral term, the curvature change rate integral term, and the obstacle distance measurement term. These terms are combined by weighting coefficients. For example, the weight of the path length term can be 0.4, the weight of the curvature integral term can be 0.3, the weight of the curvature change rate integral term can be 0.2, and the weight of the obstacle distance measurement term can be 0.1.

[0130] The navigation path interval is discretized into multiple path segments, the global multi-objective optimization function is discretized, and the global discretized objective function is constructed. The function includes the path segment length, path segment curvature, curvature change, and the minimum distance to obstacles. For example, the navigation path can be divided into 100 segments, and the above four index values ​​of each segment are calculated.

[0131] The global discretized objective function is iteratively optimized using the sequential quadratic programming method. By constructing the quadratic approximation of the global multi-objective optimization function and the linear constraints, the quadratic programming problem is solved to obtain the position increment of the control point, and the position coordinates of the control point are updated based on the position increment. For example, the maximum number of iterations can be set to 50, the convergence threshold to 0.001, and the quadratic programming problem is solved and the control point position is updated in each iteration.

[0132] Finally, the optimized navigation path is smoothed by quintic spline interpolation. By maintaining the continuity of the position derivative, velocity derivative, and acceleration derivative at the interpolation endpoints, a smooth and continuous navigation path is generated. Based on the smooth path, a waypoint sequence and the corresponding navigation speed are generated. For example, a waypoint can be taken every 10m on the optimized path, and the corresponding navigation speed can be calculated based on the curvature.

[0133] Through the above steps, global path planning based on navigation strategy optimization parameters can be achieved to obtain a smooth navigation path that meets various constraints and is globally optimal. This method comprehensively considers multiple factors such as path length, curvature change, obstacle avoidance, etc., and can generate a safe and efficient navigation trajectory.

[0134] In an optional implementation, the tactical layer performs local route optimization based on the optimal navigation path data, including:

[0135] A local planning window is constructed based on the current position, current heading angle and current speed of the aircraft, and the size of the local planning window is adaptively adjusted through a speed adaptive coefficient, so as to establish a positive correlation between the size of the local planning window and the current speed;

[0136] A local environment model is constructed using multi-source sensor data, the sensor data is transformed into position data in a local coordinate system, the occupancy probability of the grid map is updated based on the position data, and the occupancy probability value of each grid is calculated by a logarithmic probability update method;

[0137] The Kalman filter algorithm is used to track dynamic obstacles, and the state vector of the obstacle is predicted through the state prediction equation. The predicted state is updated based on the measurement data to obtain the precise position and motion information of the obstacle.

[0138] Constructing a route optimization model, taking the kinematic equation of the aircraft as the state equation, the state equation including the position coordinates, velocity and heading angle, constructing state constraints and dynamic constraints, the state constraints are used to limit the value range of the position coordinates and the value range of the velocity, and the dynamic constraints are used to limit the value range of the angular velocity, acceleration and the rate of change of the heading angle;

[0139] Constructing a multi-objective cost function, the multi-objective cost function includes a reference trajectory tracking term, an obstacle avoidance term, a trajectory smoothing term, and an energy consumption term, and performing a weighted combination of each term in the multi-objective cost function by a weighting coefficient;

[0140] The multi-objective cost function is optimized and solved by using a gradient projection method, the gradient of the cost function with respect to the control amount is calculated, the updated control amount is projected to a feasible domain, and the optimal step length is determined by backtracking line search.

[0141] The method for optimizing the local route according to the optimal navigation path data can be implemented in the following steps:

[0142] First, a local planning window is constructed based on the current position, current heading angle, and current speed of the aircraft. Specifically, a rectangular planning window is constructed with the current position of the aircraft as the center and the current heading as the reference direction. The length and width of the planning window are related to the current speed of the aircraft. By introducing the speed adaptation coefficient k, the planning window size is adaptively adjusted as the speed changes. For example, the planning window length L=k*v and the width W=0.5*k*v can be set, where v is the current speed. When the speed is higher, the planning window becomes larger accordingly to take into account the maneuverability of the aircraft.

[0143] Next, the local environment model is constructed using multi-source sensor data. First, the obstacle position data obtained by radar, vision and other sensors are transformed into a local coordinate system with the aircraft as the origin. Then, based on these position data, the occupancy probability of each grid in the grid map is updated using the logarithmic probability update method. Specifically, the grid map is divided into m×n grids. For each grid (i, j), the posterior probability p(mi, j|z) of the grid being occupied is calculated based on the sensor measurement value z. Through multiple measurement and iterative updates, an accurate local environment occupancy probability map is finally obtained.

[0144] For dynamic obstacles, the Kalman filter algorithm is used for tracking. The position and speed of the obstacle are taken as the state vector x, and the state prediction equation x(k)=Fx(k-1)+w(k-1) is established, where F is the state transfer matrix and w is the process noise. Then, based on the new observation data z(k), the state is updated using the observation equation z(k)=Hx(k)+v(k), where H is the observation matrix and v is the observation noise. By iterating the two steps of prediction and update, the accurate position and speed information of the obstacle can be obtained.

[0145] When constructing the route optimization model, the kinematic equation of the aircraft is used as the state equation. The state vector includes the position coordinates (x, y), the velocity v, and the heading angle θ. At the same time, state constraints are constructed, such as xmin≤x≤xmax, ymin≤y≤ymax to limit the value range of the position coordinates, and vmin≤v≤vmax to limit the value range of the velocity. The dynamic constraints include |ω|≤ωmax to limit the angular velocity, |a|≤amax to limit the acceleration, and |dθ / dt|≤θmax to limit the heading angle change rate. These constraints ensure that the generated trajectory meets the motion characteristics of the aircraft.

[0146] In the construction of the multi-objective cost function, the following items are included:

[0147] 1) The reference trajectory tracking term Jref is used to measure the deviation between the optimized trajectory and the reference trajectory;

[0148] 2) Obstacle avoidance items Jobs, used to ensure a safe distance from obstacles;

[0149] 3) The trajectory smoothing term Jsmooth is used to avoid drastic changes in the heading angle;

[0150] 4) Energy consumption term Jenergy, used to minimize the changes in speed and acceleration.

[0151] By weighting each item through weighted combination of weighting coefficients α, β, γ, and δ, we get the total cost function J=αJref+βJobs+γJsmooth+δJenergy.

[0152] Finally, the gradient projection method is used to optimize and solve the multi-objective cost function. First, calculate the gradient ∇J of the cost function J with respect to the control variable u, and then update the control variable along the gradient direction as u(k + 1) = u(k) - η∇J(k), where η is the step size. Project the updated u(k + 1) into the feasible region that satisfies the constraint conditions. Determine the optimal step size η through backtracking line search such that J(u(k + 1)) < J(u(k)). Repeat the above process until convergence to obtain the optimal control sequence, and then generate the locally optimal flight path.

[0153] Through the above steps, based on the globally optimal navigation path data and combined with the locally perceived environmental information in real time, a locally optimal flight path that satisfies multiple constraint conditions can be generated, thereby realizing the dynamic optimization and adjustment of the global path. This method can effectively cope with complex dynamic environments and ensure that the vehicle completes the navigation task safely and efficiently.

[0154] In an alternative embodiment, the execution layer implements the trajectory tracking of the vehicle based on the model predictive control algorithm; the autonomous navigation control of the unmanned vehicle is realized by outputting navigation control instructions through the hierarchical navigation control model, including:

[0155] Establish a six-degree-of-freedom dynamic model of the vehicle. The six-degree-of-freedom dynamic model includes translational equations and rotational equations. The translational equations include thrust terms, drag terms, and gravity terms, and the rotational equations include torque terms and gyroscopic torque terms;

[0156] Construct the six-degree-of-freedom dynamic model into a state-space expression. The state vector of the state-space expression includes position coordinates, velocity, attitude angles, and angular velocity, and the control vector of the state-space expression includes thrust and three-axis torques;

[0157] Linearize the state-space expression, calculate the Jacobian matrix of the system state equation with respect to the state vector and the control vector, and construct a linearized prediction model;

[0158] Construct a trajectory tracking prediction cost function. The prediction cost function includes a tracking error term, a control variable penalty term, and a control increment penalty term, and the weight matrix is used to weight and combine each penalty term;

[0159] Construct state constraint conditions, which include position coordinate constraints and attitude angle constraints, construct control constraint conditions, which include thrust constraints and torque constraints, and construct control increment constraint conditions, which include thrust increment constraints and torque increment constraints;

[0160] Converting the prediction cost function into a standard form of a quadratic programming problem, calculating a quadratic matrix and linear term coefficients, and constructing an inequality constraint matrix and an equality constraint matrix;

[0161] The quadratic programming problem is solved by adopting the interior point method, the Newton step is obtained by solving the Karush-Kuhn-Tucker system, the optimal step is determined based on the complementary slackness, and the primal variables and the dual variables are iteratively updated;

[0162] Based on the optimization solution results, control quantity mapping is performed, the total thrust is distributed to each actuator through the thrust distribution matrix, and the three-axis torque is distributed to each actuator through the torque distribution matrix;

[0163] The output of the actuator is limited, the motor speed is limited according to the motor characteristics, the rudder angle is limited according to the rudder characteristics, and the final control command is generated.

[0164] According to the hierarchical navigation control model, the execution layer implements the trajectory tracking of the aircraft based on the model predictive control algorithm. The specific implementation method is as follows:

[0165] First, a six-degree-of-freedom dynamic model of the aircraft is established. The model consists of two parts: the translation equation and the rotation equation. The translation equation describes the movement of the center of mass of the aircraft, including thrust terms, drag terms, and gravity terms. The thrust term represents the resultant force generated by each actuator, the drag term represents the air resistance, and the gravity term represents the gravity of the aircraft. The rotation equation describes the rotation of the aircraft around the center of mass, including torque terms and gyroscopic torque terms. The torque term represents the resultant torque generated by each actuator, and the gyroscopic torque term represents the additional torque generated by the rotation. For example, for a fixed-wing UAV, the thrust term in the translation equation is the engine thrust, the drag term is the aerodynamic drag, and the torque term in the rotation equation is the aerodynamic torque generated by the deflection of the rudder.

[0166] Next, the six-degree-of-freedom dynamics model is constructed as a state space expression. The state vector includes position coordinates, velocity, attitude angle, and angular velocity, and the control vector includes thrust and three-axis torque. Taking a fixed-wing drone as an example, the state vector is 12-dimensional, including 3 position coordinates, 3 velocity components, 3 attitude angles, and 3 angular velocity components; the control vector is 4-dimensional, including 1 thrust and 3 torque components. The state space expression describes the change of the state vector over time.

[0167] Then the state space expression is linearized. The Jacobian matrix of the system state equation with respect to the state vector and the control vector is calculated, and a linearized prediction model is constructed. The linearization process can approximate the nonlinear model to a linear model, which is convenient for subsequent optimization and solution. Taking the fixed-wing UAV as an example, after linearization, a 12-order state matrix and a 4×12-order input matrix are obtained.

[0168] On this basis, the trajectory tracking prediction cost function is constructed. This function includes tracking error term, control amount penalty term and control increment penalty term, and each penalty term is weighted and combined through the weight matrix. The tracking error term is used to measure the deviation between the actual trajectory and the expected trajectory, the control amount penalty term is used to limit the amplitude of the control amount, and the control increment penalty term is used to suppress the drastic change of the control amount. For example, for fixed-wing UAVs, the position error weight can be set to 1, the attitude error weight to 0.1, the thrust penalty weight to 0.01, and the torque penalty weight to 0.001.

[0169] At the same time, constraints are constructed, including state constraints, control constraints, and control increment constraints. State constraints include position coordinate constraints and attitude angle constraints, control constraints include thrust constraints and torque constraints, and control increment constraints include thrust increment constraints and torque increment constraints. These constraints are used to ensure the feasibility of the optimization results. Taking a fixed-wing drone as an example, the altitude constraint can be set to 0-5000 meters, the pitch angle constraint to ±30 degrees, the thrust constraint to 0-1000 Newtons, and the torque constraint to ±100 Newton meters.

[0170] The prediction cost function is converted into the standard form of the quadratic programming problem, the quadratic matrix and the linear term coefficients are calculated, and the inequality constraint matrix and the equality constraint matrix are constructed. The standard form of the quadratic programming problem includes a quadratic objective function, a linear equality constraint, and a linear inequality constraint. During the conversion process, the state prediction equation needs to be substituted into the objective function, and the constraints need to be organized into a standard form.

[0171] The interior point method is used to solve the quadratic programming problem. The Newton step size is obtained by solving the Karush-Kuhn-Tucker system, and the optimal step size is determined based on complementary slackness. The primal and dual variables are iteratively updated. The advantage of the interior point method is that it converges quickly and is suitable for solving large-scale optimization problems. During the iteration process, the maximum number of iterations can be set to 100 and the convergence threshold can be set to 0.001.

[0172] The control quantity is mapped based on the optimization solution. The total thrust is distributed to each actuator through the thrust distribution matrix, and the three-axis torque is distributed to each actuator through the torque distribution matrix. Taking a quad-rotor drone as an example, the total thrust can be evenly distributed to the four motors, the yaw torque can be distributed to the four motors through differential, and the pitch torque and roll torque can be realized through the differential of the diagonal motors.

[0173] Finally, the output of the actuator is limited. The motor speed is limited according to the motor characteristics, and the rudder angle is limited according to the rudder characteristics to generate the final control command. For example, for a fixed-wing UAV, the motor speed can be limited to 1000-6000 rpm, and the rudder angle can be limited to ±30 degrees. Through the above steps, the trajectory tracking algorithm based on model predictive control can be implemented, the navigation control command can be output, and the autonomous navigation control of the unmanned aircraft can be realized.

[0174] Figure 2 FIG. 1 is a schematic diagram of the structure of an unmanned ship autonomous navigation control system based on artificial intelligence according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0175] The first unit is used to collect real-time navigation data of the unmanned aerial vehicle through multi-source sensors, wherein the multi-source sensors include inertial navigation sensors, visual sensors, lidar sensors, and millimeter-wave radar sensors; perform feature fusion processing on the real-time navigation data based on a deep neural network to obtain navigation environment feature data; input the navigation environment feature data into a pre-trained reinforcement learning model, wherein the reinforcement learning model adopts a dual Q network structure, iteratively optimizes the navigation strategy based on the environmental reward function, and generates navigation strategy optimization parameters;

[0176] The second unit is used to construct a multi-objective route optimization model according to the navigation strategy optimization parameters, wherein the multi-objective route optimization model takes the shortest navigation time and the minimum energy consumption as optimization objectives, and uses an improved ant colony algorithm to globally optimize the route, wherein the improved ant colony algorithm introduces an adaptive pheromone volatilization factor and a dynamic heuristic factor to improve the route search efficiency; and output optimal navigation path data based on the multi-objective route optimization model;

[0177] The third unit is used to establish a hierarchical navigation control model according to the optimal navigation path data, wherein the hierarchical navigation control model includes a strategic layer, a tactical layer and an execution layer, wherein the strategic layer performs global path planning based on the navigation strategy optimization parameters, the tactical layer performs local route optimization based on the optimal navigation path data, and the execution layer realizes the trajectory tracking of the aircraft based on the model predictive control algorithm; and the navigation control instructions are output through the hierarchical navigation control model to realize the autonomous navigation control of the unmanned aircraft.

[0178] According to a third aspect of the embodiments of the present invention,

[0179] An electronic device is provided, comprising:

[0180] processor;

[0181] a memory for storing processor-executable instructions;

[0182] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0183] According to a fourth aspect of the embodiments of the present invention,

[0184] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0185] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An autonomous navigation control method for an unmanned ship based on artificial intelligence, characterized in that: include: The real-time navigation data of the unmanned aerial vehicle is collected by multi-source sensors, wherein the multi-source sensors include inertial navigation sensors, visual sensors, lidar sensors, and millimeter-wave radar sensors; the real-time navigation data is subjected to feature fusion processing based on a deep neural network to obtain navigation environment feature data; the navigation environment feature data is input into a pre-trained reinforcement learning model, wherein the reinforcement learning model adopts a dual Q network structure, and the navigation strategy is iteratively optimized based on the environmental reward function to generate navigation strategy optimization parameters; A multi-objective route optimization model is constructed according to the navigation strategy optimization parameters, wherein the multi-objective route optimization model takes the shortest navigation time and the minimum energy consumption as optimization objectives, and uses an improved ant colony algorithm to globally optimize the route, wherein the improved ant colony algorithm introduces an adaptive pheromone volatilization factor and a dynamic heuristic factor to improve the route search efficiency; and optimal navigation path data is output based on the multi-objective route optimization model; A hierarchical navigation control model is established according to the optimal navigation path data, and the hierarchical navigation control model includes a strategic layer, a tactical layer and an execution layer, wherein the strategic layer performs global path planning based on the navigation strategy optimization parameters, the tactical layer performs local route optimization based on the optimal navigation path data, and the execution layer realizes the trajectory tracking of the aircraft based on the model predictive control algorithm; the navigation control instructions are output through the hierarchical navigation control model to realize autonomous navigation control of the unmanned aircraft.

2. The method according to claim 1, characterized in that The navigation environment characteristic data is input into a pre-trained reinforcement learning model, the reinforcement learning model adopts a dual Q network structure, and iteratively optimizes the navigation strategy based on the environmental reward function, and generates navigation strategy optimization parameters including: Acquire real-time navigation data, the real-time navigation data including the current position coordinates of the aircraft, the current speed vector of the aircraft, the current attitude angle of the aircraft, the relative position matrix of surrounding obstacles, and the relative position vector of the target point; construct a state space based on the real-time navigation data, and construct an action space based on the speed adjustment amount, heading angle adjustment amount, and pitch angle adjustment amount that the aircraft can execute; A multi-objective combined reward function is constructed based on the state space and the action space, wherein the multi-objective combined reward function includes a distance reward item, an obstacle avoidance reward item, an energy consumption reward item, and a smoothness reward item, wherein the distance reward item is calculated by the Euclidean distance between the current position of the aircraft and the target position, the obstacle avoidance reward item is calculated by the difference between the minimum Euclidean distance between the current position of the aircraft and the obstacle position and the safety distance threshold, the energy consumption reward item is calculated by the weighted square sum of the speed adjustment amount, the heading angle adjustment amount, and the pitch angle adjustment amount, and the smoothness reward item is calculated by the Euclidean distance between the current action vector and the action vector at the previous moment; Constructing a dual Q network structure, the dual Q network structure includes an online network and a target network, the online network and the target network both include an input layer, three hidden layers and an output layer, the dimension of the input layer is the same as the dimension of the state space, the number of neurons in the three hidden layers is 256, 128 and 64 respectively, the hidden layer uses a ReLU activation function, the dimension of the output layer is the same as the dimension of the action space and uses a linear activation function; constructing a loss function based on the online network and the target network, the loss function includes an immediate reward term and a minimum Q value term predicted by the target network; The online network and the target network are trained by an experience replay method, and training samples are collected and stored in an experience replay pool through an ε-greedy strategy. Random sampling is performed from the experience replay pool for network training, wherein the online network updates parameters at each training step, and the target network updates parameters by a soft update method at each preset number of steps; and strategy optimization parameters are output based on the trained online network and the target network, and the strategy optimization parameters include a network weight matrix, a network bias vector, a state normalization parameter, and an action normalization parameter.

3. The method according to claim 1, characterized in that A multi-objective route optimization model is constructed according to the navigation strategy optimization parameters. The multi-objective route optimization model takes the shortest navigation time and the minimum energy consumption as optimization objectives, and uses an improved ant colony algorithm to perform global optimization on the route. The optimal navigation path data output based on the multi-objective route optimization model includes: Constructing a multi-objective function for route optimization, the multi-objective function includes a time optimization objective function and an energy consumption optimization objective function, wherein the time optimization objective function is calculated by summing the ratios of route segment distances to corresponding navigation speeds, and the energy consumption optimization objective function is calculated by weighted summing of speed energy consumption items, steering energy consumption items, and lifting energy consumption items; Constructing route optimization constraint conditions based on the multi-objective function, the route optimization constraint conditions include speed constraint, steering constraint, climb constraint and obstacle constraint, the speed constraint is used to limit the value range of the navigation speed, the steering constraint is used to limit the value range of the heading angle change, the climb constraint is used to limit the value range of the height change, and the obstacle constraint is used to ensure a safe distance between the aircraft and the obstacle; The route is encoded using a waypoint sequence, each waypoint contains position coordinates and navigation speed information, a state transition probability is constructed based on pheromone concentration and heuristic information, and the next feasible waypoint is determined by the state transition probability; wherein the heuristic information is obtained by weighted calculation of the distance between waypoints and energy consumption estimation; Introducing an adaptive pheromone volatilization mechanism, the adaptive pheromone volatilization mechanism adopts a time-varying pheromone volatilization factor, the time-varying pheromone volatilization factor changes exponentially with the increase of the number of iterations, and the pheromone concentration is dynamically updated by the time-varying pheromone volatilization factor; A dynamic heuristic factor is introduced, and the dynamic heuristic factor changes exponentially with the increase of the number of iterations. The weight of the heuristic information in the state transition probability is adjusted by the dynamic heuristic factor; iterative optimization is performed based on the adaptive pheromone volatilization mechanism and the dynamic heuristic factor, and the time cost and energy cost of the route generated by each iteration are evaluated, and the non-dominated solution is added to the Pareto frontier solution set; An optimal solution that satisfies the time-energy consumption trade-off is selected from the Pareto frontier solution set as the optimal route, and the optimal route is subjected to spline interpolation smoothing processing to generate an optimal waypoint sequence and corresponding navigation control parameters.

4. The method according to claim 1, characterized in that: The strategic layer performs global path planning based on the navigation strategy optimization parameters, including: A parameterized curve is used to describe the navigation path, and the navigation path is expressed as a function of a navigation path parameter, wherein the value range of the navigation path parameter is 0 to 1, and the navigation path includes an X-axis coordinate component, a Y-axis coordinate component, and a Z-axis coordinate component; The navigation path is described based on a cubic B-spline curve, and the spatial coordinates of the navigation path are calculated by summing the product of the combination coefficient and the B-spline basis function, and the B-spline basis function is calculated by the power function of the combination number and the navigation path parameter; Constructing navigation path constraints, wherein the navigation path constraints include position constraints, speed constraints, acceleration constraints, and curvature constraints, wherein the position constraints are used to limit the coordinate range of the navigation path, the speed constraints are used to limit the first-order derivative range of the navigation path, the acceleration constraints are used to limit the second-order derivative range of the navigation path, and the curvature constraints are used to limit the curvature variation range of the navigation path; Constructing a global multi-objective optimization function, the global multi-objective optimization function includes a path length term, a curvature integral term, a curvature change rate integral term, and an obstacle distance measurement term, and performing a weighted combination of each term in the global multi-objective optimization function by a weighting coefficient; Discretize the navigation path interval into multiple path segments, discretize the global multi-objective optimization function, and construct a global discretization objective function, wherein the global discretization objective function includes path segment length, path segment curvature, curvature change, and minimum obstacle distance; The global discretized objective function is iteratively optimized using a sequential quadratic programming method, a quadratic approximation of the global multi-objective optimization function and linear constraints are constructed to solve the quadratic programming problem to obtain a position increment of the control point, and the position coordinates of the control point are updated based on the position increment; The optimized navigation path is smoothed by quintic spline interpolation, and a smooth and continuous navigation path is generated by maintaining the continuity of the position derivative, velocity derivative and acceleration derivative at the interpolation endpoints. A waypoint sequence and a corresponding navigation speed are generated based on the smooth and continuous navigation path.

5. The method according to claim 1, characterized in that The tactical layer performs local route optimization based on the optimal navigation path data, including: A local planning window is constructed based on the current position, current heading angle and current speed of the aircraft, and the size of the local planning window is adaptively adjusted through a speed adaptive coefficient, so as to establish a positive correlation between the size of the local planning window and the current speed; A local environment model is constructed using multi-source sensor data, the sensor data is transformed into position data in a local coordinate system, the occupancy probability of the grid map is updated based on the position data, and the occupancy probability value of each grid is calculated by a logarithmic probability update method; The Kalman filter algorithm is used to track dynamic obstacles, and the state vector of the obstacle is predicted through the state prediction equation. The predicted state is updated based on the measurement data to obtain the precise position and motion information of the obstacle. Constructing a route optimization model, taking the kinematic equation of the aircraft as the state equation, the state equation including the position coordinates, velocity and heading angle, constructing state constraints and dynamic constraints, the state constraints are used to limit the value range of the position coordinates and the value range of the velocity, and the dynamic constraints are used to limit the value range of the angular velocity, acceleration and the rate of change of the heading angle; Constructing a multi-objective cost function, the multi-objective cost function includes a reference trajectory tracking term, an obstacle avoidance term, a trajectory smoothing term, and an energy consumption term, and performing a weighted combination of each term in the multi-objective cost function by a weighting coefficient; The multi-objective cost function is optimized and solved by using a gradient projection method, the gradient of the cost function with respect to the control amount is calculated, the updated control amount is projected to a feasible domain, and the optimal step length is determined by backtracking line search.

6. The method according to claim 1, characterized in that The execution layer realizes the trajectory tracking of the aircraft based on the model predictive control algorithm; outputs the navigation control instructions through the hierarchical navigation control model to realize the autonomous navigation control of the unmanned aircraft, including: Establishing a six-degree-of-freedom dynamics model of the aircraft, wherein the six-degree-of-freedom dynamics model includes a translation equation and a rotation equation, wherein the translation equation includes a thrust term, a drag term, and a gravity term, and the rotation equation includes a torque term and a gyroscopic torque term; The six-degree-of-freedom dynamics model is constructed as a state space expression, wherein the state vector of the state space expression includes position coordinates, velocity, attitude angle and angular velocity, and the control vector of the state space expression includes thrust and three-axis torque; Linearizing the state space expression, calculating the Jacobian matrix of the system state equation with respect to the state vector and the control vector, and constructing a linearized prediction model; Constructing a trajectory tracking prediction cost function, wherein the prediction cost function includes a tracking error term, a control amount penalty term, and a control increment penalty term, and performing a weighted combination of each penalty term through a weight matrix; Constructing state constraints, the state constraints include position coordinate constraints and attitude angle constraints, constructing control constraints, the control constraints include thrust constraints and torque constraints, and constructing control increment constraints, the control increment constraints include thrust increment constraints and torque increment constraints; Converting the prediction cost function into a standard form of a quadratic programming problem, calculating a quadratic matrix and linear term coefficients, and constructing an inequality constraint matrix and an equality constraint matrix; The quadratic programming problem is solved by adopting the interior point method, the Newton step is obtained by solving the Karush-Kuhn-Tucker system, the optimal step is determined based on the complementary slackness, and the primal variables and the dual variables are iteratively updated; Based on the optimization solution results, control quantity mapping is performed, the total thrust is distributed to each actuator through the thrust distribution matrix, and the three-axis torque is distributed to each actuator through the torque distribution matrix; The output of the actuator is limited, the motor speed is limited according to the motor characteristics, the rudder angle is limited according to the rudder characteristics, and the final control command is generated.

7. An unmanned ship autonomous navigation control system based on artificial intelligence, used to implement the method described in any one of claims 1 to 6, characterized in that: include: The first unit is used to collect real-time navigation data of the unmanned aerial vehicle through multi-source sensors, wherein the multi-source sensors include inertial navigation sensors, visual sensors, lidar sensors, and millimeter-wave radar sensors; perform feature fusion processing on the real-time navigation data based on a deep neural network to obtain navigation environment feature data; input the navigation environment feature data into a pre-trained reinforcement learning model, wherein the reinforcement learning model adopts a dual Q network structure, iteratively optimizes the navigation strategy based on the environmental reward function, and generates navigation strategy optimization parameters; The second unit is used to construct a multi-objective route optimization model according to the navigation strategy optimization parameters, wherein the multi-objective route optimization model takes the shortest navigation time and the minimum energy consumption as optimization objectives, and uses an improved ant colony algorithm to globally optimize the route, wherein the improved ant colony algorithm introduces an adaptive pheromone volatilization factor and a dynamic heuristic factor to improve the route search efficiency; and output optimal navigation path data based on the multi-objective route optimization model; The third unit is used to establish a hierarchical navigation control model according to the optimal navigation path data, wherein the hierarchical navigation control model includes a strategic layer, a tactical layer and an execution layer, wherein the strategic layer performs global path planning based on the navigation strategy optimization parameters, the tactical layer performs local route optimization based on the optimal navigation path data, and the execution layer realizes the trajectory tracking of the aircraft based on the model predictive control algorithm; and the navigation control instructions are output through the hierarchical navigation control model to realize the autonomous navigation control of the unmanned aircraft.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images