Unmanned aerial vehicle target task autonomous area searching and distributing method based on field data
Through the autonomous region search and allocation method of drone target tasks based on field data, deep neural networks and reinforcement learning are used to optimize drone track planning, and the stability and efficiency of drone flight planning in complex environments are solved. The effect of fast arrival and efficient residency is achieved.
Patent Information
- Application Number
- CN202411089859.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
It is difficult to achieve stable and efficient mission flight planning in complex environments, and its wind resistance is limited, resulting in large energy consumption and difficulty in quickly reaching the mission target area and resident efficiently.
Adopting the autonomous region search and allocation method of drone target tasks based on field data, we design a deep neural network model to fuse drone state information and deep field information, combine long and short-term memory neural network and deep adversarial loop Q neural network to establish intelligence , use reinforcement learning to train strategies in simulation environments, maximize value functions to optimize track planning.
While saving drone energy consumption, quickly reach the mission target area, achieving efficient residency, supporting the task development, and showing a target point arrival rate of 99.8% through multiple tests.
Smart Images

Figure CN119004979A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of unmanned aerial vehicles and artificial intelligence, and in particular relates to an autonomous area search and allocation method for unmanned aerial vehicle target tasks based on field data. Background Art
[0002] Drones have the advantages of long flight time, large mission payload, and fixed-point stationing. They are ideal flight platforms for applications such as communications, remote sensing, and meteorological detection. In recent years, with the gradual development of drone platforms and mission payload technologies, the demand for drone-related applications has become more and more urgent, and the industry is eager for unmanned flight applications to be steadily implemented. The demand for drone applications has put forward higher and higher requirements on their capabilities, requiring them to perform tasks stably and reliably under complex environmental conditions and improve their autonomous flight capabilities. Mission flight planning has become a key link in improving the application capabilities of drones.
[0003] Under the urgent demand for applications, drone technology continues to develop, but the flight planning method of drone missions still needs to be improved. As a drone that is lighter than air and relies on buoyancy to fly, its flight is significantly affected by the depth field in the environment. Although drones have a certain wind resistance capability due to their propeller power, their wind resistance capability is ultimately limited due to their large size. Summary of the invention
[0004] 1. Technical issues to be resolved
[0005] The technical problem to be solved by the present invention is how to provide an autonomous area search and allocation method for UAV target tasks based on field data to solve the problem of UAV mission flight planning.
[0006] (II) Technical solution
[0007] In order to solve the above technical problems, the present invention proposes a method for autonomous area search and allocation of UAV target tasks based on field data, which comprises the following steps:
[0008] S1. According to the mission flight planning application data structure, a deep neural network model is designed to fuse the UAV state information and the depth field information; the fused depth field information and UAV state information are input into the long short-term memory neural network to sense the changing trend of the fused information;
[0009] S2. Consider the task decision process as a Markov decision process, design an intelligent agent based on the drone state change model and deep adversarial recurrent Q neural network, use the output of the long short-term memory neural network as the input of the intelligent agent, output the drone action decision, and cyclically make the decision to obtain the trajectory planning point;
[0010] S3. Establish a UAV flight simulation environment, including a depth field generation model, and set up a corresponding reward mechanism to enable the agent to obtain real-time feedback during interaction with the environment;
[0011] S4. In the simulation environment obtained in step S3, reinforcement learning is used to carry out training, and the UAV learns strategies in the interaction with the environment by maximizing the value function. The calculation formula for maximizing the value function is as follows:
[0012]
[0013] Among them, γ is the attenuation factor, r t is the reward for the current decision, s t is the current input of the agent, a t is the UAV action decision output at the current moment, s t+1 is the input of the agent at the next moment, a t+1 Output for the drone’s action decision at the next moment, For all possible s t+1 expectations;
[0014] S5. Use actual depth field data to test whether the obstacle avoidance decision flight is completed, and evaluate the performance of intelligent decision-making based on the test results.
[0015] (III) Beneficial effects
[0016] The present invention proposes a method for autonomous area search and allocation of UAV target tasks based on field data. The present invention adopts the above-mentioned method for autonomous area search and allocation of UAV target tasks based on field data. Through the use of multiple artificial intelligence models, it can save the energy consumption of the UAV while being able to quickly reach the mission target area and achieve efficient residence, thereby supporting the implementation of the mission. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is an intelligent agent design diagram of the method for autonomous area search and allocation of target tasks of unmanned aerial vehicles based on field data of the present invention;
[0018] Figure 2 It is a principle diagram of an intelligent agent of the method for autonomous area search and allocation of target tasks of unmanned aerial vehicles based on field data of the present invention;
[0019] Figure 3 A diagram showing a learning process of an intelligent planning method for autonomous area search and allocation of target tasks for unmanned aerial vehicles based on field data of the present invention;
[0020] Figure 4 This is a test process diagram of the intelligent planning method of the autonomous area search and allocation method for UAV target tasks based on field data of the present invention. DETAILED DESCRIPTION
[0021] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.
[0022] The purpose of the present invention is to provide a method for autonomous area search and allocation of UAV target tasks based on field data, which can quickly reach the mission target area and achieve efficient residence while saving the energy consumption of the UAV, thereby supporting the implementation of the mission.
[0023] The present invention discloses a method for autonomous regional search and allocation of target tasks of unmanned aerial vehicles based on field data. According to the mission flight planning application data structure, a deep neural network model is designed; an intelligent agent is established by combining the perceived information through a long short-term memory artificial network layer with a deep adversarial recurrent Q neural network; an intelligent agent is designed to integrate the state information of the unmanned aerial vehicle with the deep field information; a flight simulation environment for unmanned aerial vehicles is established, and a reward mechanism is set up so that the intelligent agent can obtain real-time feedback in the interaction with the environment; model training is carried out using the idea of reinforcement learning, and the optimal strategy is learned in the interaction between the unmanned aerial vehicle and the environment by maximizing the value function, and whether the target is completed is tested, and the performance evaluation of the intelligent decision-making is performed on the results. The present invention adopts the above-mentioned method for autonomous regional search and allocation of target tasks of unmanned aerial vehicles based on field data, which saves the energy consumption of unmanned aerial vehicles, can quickly reach the mission target area and efficiently reside there, and support the implementation of the mission.
[0024] To achieve the above object, the present invention provides a method for autonomous area search and allocation of target tasks of unmanned aerial vehicles based on field data, comprising the following steps:
[0025] S1. According to the mission flight planning application data structure, a deep neural network model is designed to fuse the UAV state information and the depth field information; the fused depth field information and UAV state information are input into the long short-term memory neural network to sense the changing trend of the fused information;
[0026] After sensing the changing trend of fusion information through the long short-term memory neural network, the present invention will combine the deep adversarial recurrent Q neural network to establish an intelligent agent that outputs action decisions, so that the intelligent agent has global memory and can obtain strategies in time-series decision-making;
[0027] S2. Consider the task decision process as a Markov decision process, design an intelligent agent based on the drone state change model and deep adversarial recurrent Q neural network, use the output of the long short-term memory neural network as the input of the intelligent agent, output the drone action decision, and cyclically make the decision to obtain the trajectory planning point;
[0028] S3. Establish a UAV flight simulation environment, including a depth field generation model, and set up a corresponding reward mechanism to enable the agent to obtain real-time feedback during interaction with the environment;
[0029] S4. In the simulation environment obtained in step S3, reinforcement learning is used to carry out training, and the UAV learns strategies in the interaction with the environment by maximizing the value function. The calculation formula for maximizing the value function is as follows:
[0030]
[0031] Among them, γ is the attenuation factor, r t is the reward for the current decision, s t is the current input of the agent, a t is the UAV action decision output at the current moment, s t+1 is the input of the agent at the next moment, a t+1 Output for the drone’s action decision at the next moment, For all possible s t+1 expectations;
[0032] S5. Use actual depth field data to test whether the obstacle avoidance decision flight is completed, and evaluate the performance of intelligent decision-making based on the test results.
[0033] Preferably, in step S1, designing a deep neural network model to fuse drone state information and depth field information includes:
[0034] Use convolutional neural networks to extract the current depth field information features, and then use fully connected layer neural networks to fuse the depth field and drone status information;
[0035] In step S1, the heterogeneous information of fused depth field information and drone status information is used as input, and the long short-term memory neural network is used to process the sequential decision-making problem, sense the changing trend of the fused information, and enable the intelligent agent to have memory function. The output of the long short-term memory neural network enters the deep adversarial recurrent Q neural network, which predicts the changing trend of the fused data through sequence data and outputs the expected estimate of the maximized value function.
[0036] The step S1 specifically includes:
[0037] S11, using a convolutional neural network to extract the depth field information features of the current moment in the two-dimensional time series depth field, and generate a one-dimensional vector;
[0038] S12. Concatenate the generated one-dimensional vector with the current state information of the drone using a fully connected layer, and sense the changing trend of the fused information through a long short-term memory neural network.
[0039] In step S1, when the drone status information and the depth field information are fused, the mission information may also be fused.
[0040] Preferably, step S2 specifically includes: the task decision process is a Markov decision process, the output of the long short-term memory neural network is used as the input of the intelligent agent, the drone action decision is obtained through a deep adversarial recurrent Q neural network, and after the drone takes an action decision, the drone state information at the next moment is updated according to the drone state change model, and the drone task planning result is obtained through loop iteration.
[0041] The UAV state change model only considers the kinematic model of the UAV. The UAV has 4 degrees of freedom: 3 position degrees of freedom and 1 rotational degree of freedom.
[0042] The action decision u of the drone is:
[0043]
[0044] The action decision u includes the increment of the heading angle And the horizontal velocity increment Δv, the height velocity increment Δτ.
[0045] The drone's position vector:
[0046]
[0047] The rate of change of position can be expressed as:
[0048]
[0049] Among them, x, y, and h are the horizontal coordinate, vertical coordinate, and height of the drone, respectively. is the yaw angle of the drone, which is the angle between the drone's heading (speed direction) and the north direction. Clockwise is positive. v is the drone's horizontal speed. τ is the drone's altitude speed. is the heading angle.
[0050] Preferably, step S2 specifically includes:
[0051] S21. Use the output of the long short-term memory neural network as the input of the agent at the current moment t , output the current UAV action decision a t ;
[0052] S22, applying the drone state change model to obtain the state information of the drone at the next moment, and fusion information of the obtained state information at the next moment and the depth field information at the next moment;
[0053] S23, after the fusion information is passed through the long short-term memory neural network, the input s of the agent at the next moment is obtained t+1 , the trajectory planning points are obtained through cyclic decision making.
[0054] Preferably, in step S3, the reward mechanism is an evaluation mechanism for the action decision adopted by the current UAV. After the UAV makes an action decision based on the fusion information at the current moment, it will transfer to the fusion information at the next moment. The reward mechanism makes an evaluation by comparing the changes in the fusion information at the current moment and the fusion information at the next moment.
[0055] The reward mechanism mainly consists of two parts, one is the distance function from the target:
[0056]
[0057] Where d is the estimated distance between the UAV and the target, and ε is 10 -6 . The second is the distance relationship with obstacles:
[0058]
[0059] where d n is the estimated distance between the nearest obstacle and the drone. The reward for each decision is:
[0060] R=R d +R s
[0061] Preferably, in step S5, the performance evaluation of intelligent decision-making includes:
[0062] S51, by capturing the actual depth field data, the starting state of the drone, the expected position coordinates and the sequence of the actual depth field data are simultaneously used as the input of the intelligent agent to realize future task decision-making;
[0063] S52. Control the UAV through intelligent decision-making actions and evaluate the UAV's target completion status.
[0064] Preferably, in step S52, the method for evaluating whether the drone has completed the target state is to determine the distance between the final position of the drone and the target point.
[0065] Embodiment 1
[0066] A method for autonomous area search and allocation of target tasks of unmanned aerial vehicles based on field data comprises the following steps:
[0067] S1. According to the mission flight planning application data structure, a deep neural network model is designed to fuse the UAV state information and the depth field information; the fused depth field information and UAV state information are input into the long short-term memory neural network to sense the changing trend of the fused information;
[0068] By passing the perceived information through a long short-term memory neural network and combining it with a deep adversarial recurrent Q network, an intelligent agent that outputs action decisions is established, so that the intelligent agent has global memory and can obtain the optimal strategy in time series decision-making; based on the deep adversarial recurrent Q neural network, the fused deep field and state heterogeneous information are used as input, and the long short-term memory neural network is used to process sequence decision problems, sense the changing trend of the fused information, and enable the model to have a memory function, and predict the changing trend of the fused data through the sequence data. Figure 1 shown.
[0069] S2. Consider the task decision process as a Markov decision process, design an intelligent agent based on the drone state change model and deep adversarial recurrent Q neural network, use the output of the long short-term memory neural network as the input of the intelligent agent, output the drone action decision, and recursively make the decision to obtain the trajectory planning point; the intelligent agent design process is as follows Figure 2 shown.
[0070] The design of the intelligent agent specifically includes: considering the task decision process as a Markov decision process, and deciding the fusion state of the drone action as s based on the fusion information of the drone state, task and environment. t ,After the UAV makes an action decision, the UAV state at the next moment is updated according to the UAV state change model.
[0071] Action decisions include the increments of speed and heading angle. Through loop iteration, the mission planning results of the drone during the entire mission are obtained to support decision making.
[0072] The fusion of drone status information and depth field information specifically includes:
[0073] S21. Use convolutional neural network to extract the depth field features of the current moment in the two-dimensional time series depth field; concatenate the generated one-dimensional vector with the current state information of the drone, and use it as the agent input s after passing through the long short-term memory neural network. t , output the current UAV action decision a t ;
[0074] S22, apply the drone state change model to obtain the drone state information at the next moment, and fuse the new state information with the depth field information at the next moment into s t+1 ;
[0075] S23, obtain the trajectory planning point through the agent's post-loop decision.
[0076] S3. Establish a UAV flight simulation environment, including a depth field generation model, and set up a corresponding reward mechanism to enable the agent to obtain real-time feedback during interaction with the environment;
[0077] S4. In the simulation environment obtained in step S3, the reinforcement learning idea is used to carry out training, and the UAV learns the optimal strategy in the interaction with the environment by maximizing the value function. The calculation formula of the maximization value function is as follows:
[0078]
[0079] Among them, γ is the attenuation factor, r t is the reward for the current decision, s t is the current input of the agent, a t is the UAV action decision output at the current moment, s t+1 is the input of the agent at the next moment, a t+1 Output for the drone’s action decision at the next moment, For all possible s t+1 expectations;
[0080] S5. Use actual depth field data to test whether the obstacle avoidance decision flight is completed, and evaluate the performance of intelligent decision-making based on the test results.
[0081] The reward mechanism is an evaluation mechanism for the action decision made by the current drone. After the drone makes an action decision based on the fusion information at the current moment, it will transfer to the fusion information at the next moment. The reward mechanism makes an evaluation by comparing the changes between the two.
[0082] In step S5, the performance evaluation of intelligent decision-making specifically includes:
[0083] S51. By capturing the actual depth field prediction data, the UAV's starting state, expected position coordinates, and the sequence of the actual depth field are simultaneously used as the input of the intelligent agent to achieve decision-making for at least the next 24 hours of tasks, and quickly update with the data update; the UAV's starting state includes the UAV's initial speed, heading, coordinates, current time, remaining energy ratio and other information.
[0084] S52. By using the intelligent decision-making results generated from the actual depth field data, it is evaluated whether the UAV can achieve the target state.
[0085] In step S52, the speed and heading of the UAV are updated according to the incremental decisions of the speed and heading angle generated at each moment, and the desired route is drawn according to the state change model, thereby drawing the path of the UAV. The method for evaluating whether the UAV can achieve the target state is to determine the distance between the final position of the UAV and the target point.
[0086] Embodiment 2
[0087] The following simulation test is carried out:
[0088] The training process is as follows Figure 3 As shown in the figure, the depth field used for training is a simulated depth field, and the data format is a 2*12*21 format tensor, where each unit represents a change of 0.25 degrees of longitude and latitude. The state information of the drone includes the initial coordinates (2.5, 5), the current time 0, the remaining energy ratio 55%, the current speed 5m / s, the heading 60°, and the coordinates of the mission target point (17.5, 9), where the depth field changes over time. The drone continues to learn in this environment until it can complete path planning under any initial data.
[0089] For the trained agent, real meteorological depth field data is used for testing. The testing process is as follows: Figure 4 As shown in , the depth field data format is also a 2*12*21 format tensor, where each unit represents a change of 0.25 degrees of longitude and latitude. The status information of the drone includes the initial coordinates (2.5, 4), the current time 8 o'clock, the remaining energy ratio 65%, the current speed 7m / s, the heading 15°, and the coordinates of the mission target point (17.7, 9.7), where the decision interval is 450s. For the convenience of display, the test process interval shown is 1h. 1000 sets of depth fields are used for testing, and the probability of reaching the target point is 99.8%.
[0090] The present invention adopts the above-mentioned method of autonomous area search and allocation of UAV target tasks based on field data. Through the use of multiple artificial intelligence models, it can quickly reach the mission target area and achieve efficient residence while saving the energy consumption of the UAV, thereby supporting the implementation of the mission.
[0091] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for autonomous area search and allocation of UAV target tasks based on field data, characterized in that: The method comprises the following steps: S1. According to the mission flight planning application data structure, a deep neural network model is designed to fuse the UAV state information and the depth field information; the fused depth field information and UAV state information are input into the long short-term memory neural network to sense the changing trend of the fused information; S2. Consider the task decision process as a Markov decision process, design an intelligent agent based on the drone state change model and deep adversarial recurrent Q neural network, use the output of the long short-term memory neural network as the input of the intelligent agent, output the drone action decision, and cyclically make the decision to obtain the trajectory planning point; S3. Establish a UAV flight simulation environment, including a depth field generation model, and set up a corresponding reward mechanism to enable the agent to obtain real-time feedback during interaction with the environment; S4. In the simulation environment obtained in step S3, reinforcement learning is used to carry out training, and the UAV learns strategies in the interaction with the environment by maximizing the value function. The calculation formula for maximizing the value function is as follows: Among them, γ is the attenuation factor, r t is the reward for the current decision, s t is the current input of the agent, a t is the UAV action decision output at the current moment, s t+1 is the input of the agent at the next moment, a t+1 Output for the drone’s action decision at the next moment, For all possible s t+1 expectations; S5. Use actual depth field data to test whether the obstacle avoidance decision flight is completed, and evaluate the performance of intelligent decision-making based on the test results.
2. The method for autonomous area search and allocation of UAV target tasks based on field data according to claim 1, characterized in that: In step S1, designing a deep neural network model to fuse the drone state information and the depth field information includes: using a convolutional neural network to extract the depth field information features at the current moment, and then using a fully connected layer neural network to fuse the depth field and the drone state information.
3. The method for autonomous area search and allocation of UAV target tasks based on field data as claimed in claim 2, characterized in that: The step S1 specifically includes: S11, using a convolutional neural network to extract the depth field information features of the current moment in the two-dimensional time series depth field, and generate a one-dimensional vector; S 12. Concatenate the generated one-dimensional vector with the current state information of the drone using a fully connected layer, and sense the changing trend of the fused information through a long short-term memory neural network.
4. The method for autonomous area search and allocation of target tasks of unmanned aerial vehicles based on field data as claimed in claim 3, characterized in that: In the step S1, when the drone status information and the depth field information are fused, the mission information is also fused.
5. The method for autonomous area search and allocation of target tasks of unmanned aerial vehicles based on field data according to any one of claims 1 to 4, characterized in that: The step S2 includes: the task decision process is a Markov decision process, the output of the long short-term memory neural network is used as the input of the intelligent agent, the drone action decision is obtained through a deep adversarial recurrent Q neural network, and after the drone takes the action decision, the drone state information at the next moment is updated according to the drone state change model, and the drone task planning result is obtained through cyclic iteration.
6. The method for autonomous area search and allocation of UAV target tasks based on field data as claimed in claim 5, characterized in that: The UAV state change model only considers the kinematic model of the UAV, which has 4 degrees of freedom: 3 position degrees of freedom and 1 rotational degree of freedom; The action decision u of the drone is: The action decision u includes the increment of the heading angle and the horizontal velocity increment Δv, the height velocity increment Δτ; The drone's position vector: The rate of change of position is expressed as: Among them, x, y, and h are the horizontal coordinate, vertical coordinate, and height of the drone, respectively. is the yaw angle of the drone, which is the angle between the drone’s direction and the north direction, and clockwise is positive. v is the drone’s horizontal speed, and τ is the drone’s altitude speed. is the heading angle.
7. The method for autonomous area search and allocation of UAV target tasks based on field data as claimed in claim 5, characterized in that: The step S2 specifically includes: S21. Use the output of the long short-term memory neural network as the input of the agent at the current moment t , output the current UAV action decision a t ; S22, applying the drone state change model to obtain the state information of the drone at the next moment, and fusion information of the obtained state information at the next moment and the depth field information at the next moment; S23, after the fusion information is passed through the long short-term memory neural network, the input s of the agent at the next moment is obtained t+ 1. Cycle decision to obtain the trajectory planning points.
8. The method for autonomous area search and allocation of target tasks of unmanned aerial vehicles based on field data as claimed in claim 5, characterized in that: The reward mechanism in step S3 is an evaluation mechanism for the action decision adopted by the current UAV. After the UAV makes an action decision based on the fusion information at the current moment, it will transfer to the fusion information at the next moment. The reward mechanism makes an evaluation by comparing the changes between the fusion information at the current moment and the fusion information at the next moment. The reward mechanism consists of two parts: One is the distance function to the target: Where d is the estimated distance between the UAV and the target, and ε is 10 -6 ; The second is the distance relationship with obstacles: where d n is the distance estimate between the nearest obstacle and the drone, and the reward for each decision is: R=R d +R s 。 9. The method for autonomous area search and allocation of target tasks of unmanned aerial vehicles based on field data as claimed in claim 8, characterized in that: In step S5, the performance evaluation of intelligent decision-making includes: S51, by capturing the actual depth field data, the starting state of the drone, the expected position coordinates and the sequence of the actual depth field data are simultaneously used as the input of the intelligent agent to realize future task decision-making; S52. Control the UAV through intelligent decision-making actions and evaluate the UAV's target completion status.
10. The method for autonomous area search and allocation of target tasks of unmanned aerial vehicles based on field data as claimed in claim 9, characterized in that: In step S52, evaluating whether the drone has completed the target state includes: determining the distance between the final position of the drone and the target point.
Citation Information
Patent Citations
Multi-drone cluster navigation method
CN109445456A
Unmanned airship intelligent flight planning method based on meteorological data
CN116522802A
Cited By
UWB-based finite time unmanned aerial vehicle cooperative control method
CN120371009A
