Flight decision making method and device, computer equipment, and storage medium

Abstract

The disclosed embodiments provide a flight decision generation method and apparatus, a computer device, and a storage medium, and belong to the field of UAV technology. The method includes: obtaining mission requirement data; constructing a flight decision model based on the data; wherein the flight decision model includes a plurality of decision tuple data, the decision tuple data includes an original hyperparameter group, and the original hyperparameter group set forms an original hyperparameter population; constructing a corresponding target learning function based on the flight decision model; updating and optimizing the original hyperparameter population according to the target learning function to obtain a target hyperparameter population; and obtaining the target flight decision of the UAV according to the target hyperparameter population. Based on the establishment of a flight decision model, the present application defines a target learning function, so that the UAV, on the basis of completing the autonomous navigation task, further updates and optimizes the original hyperparameter population through the optimization target of the target learning function to obtain the target flight decision, thereby improving the flexibility of the UAV's autonomous navigation task.

Description

Technical Field

[0001] The present application relates to the field of drone technology, and in particular to a flight decision-making method and device, computer equipment, and storage medium. Background Art

[0002] Currently, the goal of autonomous drone navigation is to enable the drone to avoid obstacles in its environment and safely fly from its starting point to its destination through autonomous decision-making. Typically, autonomous drone navigation is modeled as a flight decision-making process, and an optimal control strategy is trained to accomplish this task. However, in practical applications, autonomous drone navigation often requires considering multiple objectives. Existing flight decision-making processes are ineffective in addressing these multiple objectives and suffer from limited flexibility. Summary of the Invention

[0003] The main purpose of the embodiments of the present disclosure is to propose a flight decision generation method and device, computer equipment, and storage medium, which can improve the flexibility of UAV autonomous navigation tasks by generating flight decisions.

[0004] To achieve the above objectives, a first aspect of an embodiment of the present disclosure provides a flight decision generation method, comprising:

[0005] Obtain task requirement data;

[0006] Constructing a flight decision model based on the mission requirement data; wherein the flight decision model includes a plurality of decision tuple data for characterizing the original flight decision of the UAV, each of the decision tuple data includes an original hyperparameter group, and each of the original hyperparameter groups is aggregated to form an original hyperparameter population;

[0007] Based on the flight decision model, construct a corresponding target learning function;

[0008] Updating and optimizing the original hyperparameter population according to the target learning function to obtain a target hyperparameter population;

[0009] A target flight decision of the UAV is obtained according to the target hyperparameter population.

[0010] In some embodiments, each of the decision tuple data includes state data, action data, a state transition function, a reward function, and a discount factor;

[0011] The state data includes, at a certain moment, a first distance between the drone and the obstacle, a maximum detection range of the drone's sensor, a second distance and angle between the drone's current position and a preset end point, the drone's current flight speed and viewing direction, and the drone's maximum speed limit;

[0012] The motion data includes acceleration and deceleration data and steering data of the drone at a certain moment;

[0013] The state transfer function is used to generate the state data of the drone at the next moment;

[0014] The reward function includes the original hyperparameter group, and the reward function is used to evaluate the preliminary quality of the drone's execution of an action in a certain state;

[0015] The discount factor is used in combination with the reward function to calculate the quality of the drone's execution of an action in a certain state.

[0016] In some embodiments, the objective learning function includes a first objective function and a second objective function; and constructing a corresponding objective learning function according to the flight decision model includes:

[0017] Obtaining an optimization target according to the flight decision model; wherein the optimization target includes a flight time target and a flight risk target of the UAV;

[0018] The first objective function is constructed according to the flight time target, and the second objective function is constructed according to the flight risk target.

[0019] In some embodiments, updating and optimizing the original hyperparameter population according to the target learning function to obtain a target hyperparameter population includes:

[0020] Performing reinforcement learning training on each of the original hyperparameter groups in the original hyperparameter population through the flight decision model to obtain corresponding multiple original control strategies;

[0021] Performing a mutation operation on each of the original hyperparameter groups to generate multiple descendant hyperparameter groups;

[0022] Performing reinforcement learning training on each of the offspring hyperparameter groups through the flight decision model to obtain corresponding multiple offspring control strategies;

[0023] The target hyperparameter population is obtained through the multiple original control strategies and the multiple descendant control strategies.

[0024] In some embodiments, obtaining a target hyperparameter population using the multiple original control strategies and the multiple descendant control strategies includes:

[0025] Using each of the original control strategies to control the UAV to interact with the environment, so as to calculate a first objective function value of each of the original hyperparameter groups according to the target learning function;

[0026] Using each of the child control strategies to control the UAV to interact with the environment, so as to calculate a second objective function value of each of the child hyperparameter groups according to the objective learning function;

[0027] Adding the multiple child hyperparameter groups to the original hyperparameter population to form an updated original hyperparameter population;

[0028] According to the first objective function value and the second objective function value, an environment selection operation is performed on the updated original hyperparameter population to obtain the target hyperparameter population.

[0029] In some embodiments, performing an environment selection operation on the updated original hyperparameter population according to the first objective function value and the second objective function value to obtain the target hyperparameter population includes:

[0030] Dividing the updated original hyperparameter population to obtain multiple preliminary hyperparameter populations; wherein each preliminary hyperparameter group is the original hyperparameter group or the descendant hyperparameter group, each preliminary hyperparameter population includes multiple preliminary hyperparameter groups, and each preliminary hyperparameter group in each preliminary hyperparameter population is not Pareto dominated by each other;

[0031] performing non-dominated sorting on the plurality of preliminary hyperparameter populations according to numerical values ​​of the first objective function value and the second objective function value to obtain an environment selection order;

[0032] An environment selection operation is performed on each of the preliminary hyperparameter populations according to the environment selection order to obtain the target hyperparameter population.

[0033] In some embodiments, performing an environment selection operation on each of the preliminary hyperparameter populations according to the environment selection order to obtain the target hyperparameter population includes:

[0034] Get the preset individual quantity threshold;

[0035] According to the environment selection order, a plurality of the preliminary hyperparameter groups that meet the individual quantity threshold are selected in sequence from the plurality of preliminary hyperparameter populations to form the target hyperparameter population.

[0036] A second aspect of the embodiments of the present disclosure provides a flight decision generating device, comprising:

[0037] Data acquisition module: used to obtain task requirement data;

[0038] A model construction module is configured to construct a flight decision model based on the mission requirement data; wherein the flight decision model includes a plurality of decision tuple data for characterizing the original flight decision of the UAV, each of the decision tuple data includes an original hyperparameter group, and each of the original hyperparameter groups is aggregated to form an original hyperparameter population;

[0039] Function construction module: used to construct the corresponding target learning function based on the flight decision model;

[0040] Population optimization module: used to update and optimize the original hyperparameter population according to the target learning function to obtain a target hyperparameter population;

[0041] Decision acquisition module: used to obtain the target flight decision of the UAV according to the target hyperparameter population.

[0042] A third aspect of the embodiments of the present disclosure provides a computer device, comprising a memory and a processor, wherein the memory stores a program, and when the program is executed by the processor, the processor is used to execute the method described in any one of the embodiments of the first aspect of the application.

[0043] The fourth aspect of the embodiments of the present disclosure proposes a storage medium, which is a computer-readable storage medium. The storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method described in any one of the embodiments of the first aspect of the present application.

[0044] The flight decision generation method and apparatus, computer equipment, and storage medium proposed in the embodiments of the present disclosure obtain mission requirement data; construct a flight decision model based on the mission requirement data; wherein the flight decision model includes multiple decision tuple data for characterizing the original flight decision of the drone, each decision tuple data includes an original hyperparameter group, and each original hyperparameter group is aggregated to form an original hyperparameter population; based on the flight decision model, a corresponding target learning function is constructed; the original hyperparameter population is updated and optimized according to the target learning function to obtain a target hyperparameter population; and the target flight decision of the drone is obtained according to the target hyperparameter population. Based on the establishment of the flight decision model of the drone, the embodiments of the present disclosure define a target learning function for optimization. On the basis of enabling the drone to complete the autonomous navigation task through the target flight decision, the original hyperparameter population is further updated and optimized through the optimization objective of the target learning function to obtain the target flight decision, thereby improving the flexibility of the drone's autonomous navigation task. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of a flight decision making method provided by an embodiment of the present disclosure;

[0046] Figure 2 is a schematic diagram of a first state of a drone provided by an embodiment of the present disclosure;

[0047] Figure 3 is a schematic diagram of the second state of the drone provided by an embodiment of the present disclosure;

[0048] Figure 4 yes Figure 1 Flowchart of step S130 in FIG.

[0049] Figure 5 yes Figure 1 Flowchart of step S140 in FIG.

[0050] Figure 6 yes Figure 5 Flowchart of step S540 in FIG.

[0051] Figure 7 yes Figure 6 Flowchart of step S640 in FIG.

[0052] Figure 8 yes Figure 7 Flowchart of step S730 in FIG.

[0053] Figure 9 is a flowchart of a multi-objective deep reinforcement learning algorithm provided by an embodiment of the present disclosure;

[0054] Figure 10 This is a module structure diagram of the flight decision making device provided by an embodiment of the present disclosure;

[0055] Figure 11 It is a schematic diagram of the hardware structure of the computer device provided in the embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0057] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0059] In addition, the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present disclosure.

[0060] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0061] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0062] First, let’s analyze some of the terms used in this application:

[0063] Artificial intelligence (AI) is a new technical discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. A branch of computer science, AI seeks to understand the essence of intelligence and create new intelligent machines that can respond in a manner similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thinking. It also encompasses the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0064] A Markov Decision Process (MDP) is a mathematical model of sequential decision making, used to simulate the stochastic strategies and rewards achievable by an agent in an environment where the system state exhibits Markov properties. An MDP is constructed based on a set of interacting objects: the agent and the environment. Its elements include state, action, strategy, and reward. In an MDP simulation, the agent perceives the current system state and takes actions in the environment according to its strategy, thereby changing the state and receiving rewards. The accumulated rewards over time are called rewards.

[0065] Deep reinforcement learning: Combining the perception capabilities of deep learning with the decision-making capabilities of reinforcement learning, it can directly control itself based on input images, creating an AI approach closer to human thinking. Deep learning has strong perception capabilities but lacks certain decision-making capabilities; while reinforcement learning, while capable of decision-making, is unable to address perception issues. Therefore, combining the two, leveraging their complementary strengths, provides a solution to perception and decision-making issues in complex systems.

[0066] Asynchronous Advantage Actor-Critic (A3C): A3C's basic framework is the AC framework, but it utilizes multiple threads instead of a single thread. Each thread is equivalent to an agent performing random exploration. Multiple agents explore together, computing policy gradients in parallel to maintain a consistent update volume.

[0067] White Gaussian Noise: Gaussian refers to a normal probability distribution, while white noise refers to its uncorrelated second-order moments and a constant first-order moment, referring to the temporal correlation of successive signals. Gaussian white noise is an ideal model for analyzing additive channel noise, and thermal noise, a major noise source in communications, falls into this category.

[0068] Euclidean distance is the "ordinary" (i.e., straight-line) distance between two points in Euclidean space. Using this distance, Euclidean space becomes a metric space. This is a commonly used definition of distance, referring to the true distance between two points in m-dimensional space, or the natural length of a vector (i.e., the distance from that point to the origin). In two- and three-dimensional space, the Euclidean distance is the actual distance between two points.

[0069] Currently, the goal of autonomous drone navigation is to enable the drone to make autonomous decisions, avoid obstacles in its environment, and safely fly from its starting point to its destination. Typically, autonomous drone navigation is modeled as a flight decision-making process, and an optimal control strategy is trained to accomplish this task. However, in practical applications, autonomous drone navigation often requires considering more complex objectives. For example, users may desire the drone to complete its autonomous navigation mission with the shortest possible flight time and the lowest possible flight risk. However, in practice, it is often difficult to simultaneously achieve all of these objectives. Excessive pursuit of short flight times often results in higher flight risks, while excessive pursuit of low flight risks often results in longer flight times. Existing technologies are unable to effectively address this complex, multi-objective autonomous drone navigation task, resulting in limited flexibility.

[0070] Based on this, the embodiments of the present disclosure provide a flight decision generation method and apparatus, a computer device, and a storage medium, which obtain mission requirement data; construct a flight decision model based on the mission requirement data; wherein the flight decision model includes multiple decision tuple data for characterizing the original flight decision of the drone, each decision tuple data includes an original hyperparameter group, and each original hyperparameter group is aggregated to form an original hyperparameter population; based on the flight decision model, a corresponding target learning function is constructed; the original hyperparameter population is updated and optimized according to the target learning function to obtain a target hyperparameter population; and the target flight decision of the drone is obtained according to the target hyperparameter population. Based on the establishment of the flight decision model of the drone, the embodiments of the present disclosure define a target learning function for optimization. On the basis of enabling the drone to complete the autonomous navigation task through the target flight decision, the original hyperparameter population is further updated and optimized according to the optimization objective of the target learning function to obtain the target flight decision, thereby improving the flexibility of the drone's autonomous navigation task.

[0071] The embodiments of the present disclosure provide a flight decision-making method and apparatus, a computer device, and a storage medium, which are specifically illustrated by the following embodiments. First, the flight decision-making method in the embodiments of the present disclosure is described.

[0072] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0073] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0074] The flight decision generation method provided by the embodiment of the present disclosure relates to the field of drone technology and also to the field of artificial intelligence. The flight decision generation method provided by the embodiment of the present disclosure can be applied to a terminal, can be applied to a server, or can be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, or a smart watch, etc.; the server can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the flight decision generation method, etc., but is not limited to the above forms.

[0075] The disclosed embodiments can be used in a wide variety of general or specialized computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0076] Reference Figure 1 According to the flight decision generation method of the first aspect of the embodiment of the present disclosure, it includes but is not limited to steps S110 to S150.

[0077] Step S110, obtaining task requirement data;

[0078] Step S120, constructing a flight decision model based on the mission requirement data;

[0079] Step S130, constructing a corresponding target learning function based on the flight decision model;

[0080] Step S140, updating and optimizing the original hyperparameter population according to the target learning function to obtain a target hyperparameter population;

[0081] Step S150: Obtain the target flight decision of the UAV according to the target hyperparameter population.

[0082] In some embodiments, step S110 obtains mission requirement data for the drone. Specifically, the mission requirement data may include the drone's mission scenario and the specific flight requirements for the drone within the mission scenario. In practical applications, the mission scenario may involve the drone flying at a fixed altitude, starting from a starting point and, through autonomous decision-making, avoiding all obstacles to safely reach a destination.

[0083] In step S120 of some embodiments, a flight decision model is constructed based on the mission requirement data. The flight decision model is generated according to a Markov decision process, and the goal of the flight decision model is to find an optimal preliminary flight control strategy that maximizes the cumulative reward of the UAV autonomous navigation mission. The flight decision model includes multiple decision tuple data representing the original flight decisions of the UAV, each decision tuple data including an original hyperparameter set, and each original hyperparameter set is aggregated to form an original hyperparameter population.

[0084] In step S130 of some embodiments, a corresponding target learning function is constructed based on the flight decision model. Specifically, the target learning function is set through the flight decision model, and multiple optimization objectives of the UAV autonomous navigation task are defined through the target learning function.

[0085] In step S140 of some embodiments, the original hyperparameter population is updated and optimized according to the target learning function to obtain a target hyperparameter population, so that the drone can consider minimizing multiple optimization objectives on the basis of safely completing the autonomous navigation task, thereby further optimizing the original hyperparameter population.

[0086] In some embodiments, step S150 obtains a target flight decision for the drone based on the target hyperparameter population. Specifically, the target hyperparameter population can be used to obtain an optimal set of flight strategies. The user can select a target flight strategy from the set based on mission requirements and preferences, thereby improving the flexibility of the drone's autonomous navigation mission.

[0087] In some embodiments, each decision tuple data includes state data, action data, state transition function, reward function and discount factor. Specifically, the flight decision model of the present application, i.e., the Markov decision process, can be represented by a decision tuple, i.e., (S, A, P, r, γ, ρ0). The various data in the decision tuple are called decision tuple data, where S represents state data, A represents action data, P represents state transition function, r represents reward function, γ represents discount factor, and ρ0 represents the distribution of initial state s0, as follows:

[0088] In some embodiments, the state data is the state of the drone at a certain moment, such as time t. In the embodiment of the present application, the state data can be represented as a 12-dimensional vector in Indicates the first distance to the obstacle detected by the drone’s own sensor, d limit Indicates the maximum detection range of the sensor, and ξ t ∈[-π,π] represents the second distance and angle data between the current position of the drone at time t and the preset end point (relative to the north direction), v t ∈[0,v limit ] and φ t ∈[-π,π] represents the current flight speed and viewing direction of the UAV at time t, that is, the first viewing direction (relative to the north direction), v limit Indicates the maximum speed limit of the drone.

[0089] Specifically, the drone status mentioned in the embodiment of the present application can be referred to Figure 2 and Figure 3 , Figure 2 It represents the state plane diagram of the drone when the altitude is determined, where Figure 2 The 8 directions represent the 8 sensors on the drone, which can detect the distance to the surrounding obstacles (i.e. ), so the drone can detect 8 directions in total, front, back, left, and right. Figure 3 Indicates the relationship between the drone's first-person perspective and the end point. Figure 2 and Figure 3 As can be seen, the drone of the disclosed embodiment can sense obstacles in all directions and obtain the first distance between the sensor and the obstacle. However, if the drone cannot sense obstacles in a certain direction, it may collide with the obstacle, resulting in reduced safety. It should be noted that those skilled in the art can configure the drone with different numbers of sensors based on actual needs, and the disclosed embodiment does not impose any specific limitations.

[0090] In some embodiments, the action data a tIncluding the acceleration and deceleration data ρ of the drone at a certain moment, such as time t t ∈[ρ lb ,ρ up ] and steering data φ t ∈[-π,π], in summary, the action data can be expressed as a t =[ρ t ,φ t ].

[0091] In some embodiments, the state transfer function is used to generate the state data of the drone at the next moment. Specifically, at time t, the drone obtains its state s t , according to the UAV control strategy a~π(·|s), the output action a is obtained t , the environment will make the corresponding environmental state transfer according to the state transfer function, and get the state s at the next moment t+1 .

[0092] In some embodiments, the reward function includes an initial set of hyperparameters, and is used to evaluate the initial quality of the drone's actions in a certain state. Specifically, the reward function can be designed based on the following scenarios, for example:

[0093] Case 1: When the second distance between the drone and the preset endpoint becomes smaller, it will be rewarded. Otherwise, it will be punished. The definition is:

[0094] Case 2: When the drone is too close to an obstacle, it will receive a corresponding penalty, defined as: Where ds represents the safe distance between the drone and the obstacle.

[0095] Case 3: When the drone is very close to the finish line, it will be rewarded Where rd is a preset constant.

[0096] Case 4: The drone will receive a constant penalty r c (s t , a t )=-1, encouraging the drone to reach the destination as quickly as possible.

[0097] The four hyperparameters are combined to form an original hyperparameter group.

[0098] In some embodiments, the discount factor is used in combination with the reward function to calculate the quality of the drone's action in a certain state. In practical applications, the discount factor is: γ∈[0,1].

[0099] Furthermore, the cumulative reward of the UAV autonomous navigation task is defined as in It should be noted that the goal of the above Markov decision process is to find an optimal control strategy a~π(·|s) so that the cumulative reward of the UAV autonomous navigation task is maximized.

[0100] In some embodiments, the target learning function includes a first target function and a second target function, such as Figure 4 As shown, step S130 specifically includes but is not limited to steps S410 to S420.

[0101] Step S410, obtaining an optimization target according to the flight decision model;

[0102] Step S420: constructing a first objective function according to the flight time target, and constructing a second objective function according to the flight risk target.

[0103] In step S410 of some embodiments, an optimization target is obtained according to the flight decision model. Specifically, the optimization target includes the flight time target and the flight risk target of the UAV.

[0104] In step S420 of some embodiments, a first function is constructed based on the flight time target. Let the flight time target be f1, then the first function can be defined as f1=T, where T is the mission completion time, and the shorter the flight time, the better. A second function is constructed based on the flight risk target. Let the flight risk target be f2, then the second function can be defined as f2=1 / d min , where d min is the shortest distance between the UAV and the obstacle in the mission, that is Among them, the lower the flight risk, the better.

[0105] In some embodiments, as Figure 5 As shown, step S140 specifically includes but is not limited to steps S510 to S540.

[0106] Step S510, performing reinforcement learning training on each original hyperparameter group in the original hyperparameter population through the flight decision model to obtain corresponding multiple original control strategies;

[0107] Step S520, performing a mutation operation on each original hyperparameter group to generate multiple descendant hyperparameter groups;

[0108] Step S530 , performing reinforcement learning training on each offspring hyperparameter group through the flight decision model to obtain corresponding multiple offspring control strategies;

[0109] Step S540: Obtain a target hyperparameter population through multiple original control strategies and multiple descendant control strategies.

[0110] For the convenience of description, before step S510 to step S540, it is necessary to first initialize the original hyperparameter population, wherein the original hyperparameter population is set to P = {β 1 ,...,,β n},in Each individual in the population is composed of the original hyperparameter group mentioned above. The goal of learning in the embodiment of the present disclosure is to find a set of Pareto optimal individuals, each of which represents a set of optimal hyperparameters, that is, to realize the process of automatically optimizing hyperparameters.

[0111] Furthermore, it is necessary to initialize a loop count G=1 to count the number of times step S140 is executed. Each time step S140 is executed, G is incremented by 1 until G=G max , used to control step S140 to be executed a certain number of times.

[0112] In step S510 of some embodiments, a flight decision model is used to perform reinforcement learning training on each original hyperparameter group in the original hyperparameter population to obtain a corresponding plurality of original control strategies. Specifically, a deep reinforcement learning algorithm is applied to each individual in the population P to train the corresponding optimal control strategy {π 1 ,...,,π n}, namely, the multiple original control strategies mentioned in the embodiments of this disclosure. In practical applications, the A3C algorithm can be used to train each individual. Before training, each offspring individual first inherits the strategy of its parent individual and is trained based on its parent individual strategy, thereby accelerating the training process of the offspring individual. It should be noted that offspring individuals are generated by mutation of parent individuals. A parent individual mutates to produce a corresponding offspring, so each offspring has its own unique corresponding parent individual.

[0113] In step S520 of some embodiments, a mutation operation is performed on each original hyperparameter group to generate multiple offspring hyperparameter groups. Specifically, a mutation operation is performed on each individual in the population P to generate an offspring population Q = mutation (P), where the offspring population is represented by Q = {β 1′ ,...,,β n′}, where the offspring population is the offspring hyperparameter set mentioned in the embodiments of this disclosure. Specifically, a Gaussian white noise mutation operation is performed on each parent individual β to generate an offspring individual β′, that is, β′=β+Δβ, where Δβ is Gaussian white noise generated by sampling from a standard normal distribution N(0, I).

[0114] In step S530 of some embodiments, similarly, each generation hyperparameter group is subjected to reinforcement learning training through the flight decision model to obtain corresponding multiple generation control strategies, namely {π 1′ ,...,,πn′}.

[0115] In step S540 of some embodiments, a target hyperparameter population, ie, an optimal strategy set, is obtained through multiple original control strategies and multiple descendant control strategies.

[0116] In some embodiments, as Figure 6 As shown, step S540 specifically includes but is not limited to steps S610 to S640.

[0117] Step S610 , using each original control strategy to control the drone to interact with the environment, so as to calculate a first objective function value of each original hyperparameter group according to the target learning function;

[0118] Step S620 , using each child control strategy to control the UAV to interact with the environment, so as to calculate a second objective function value of each child hyperparameter set according to the objective learning function;

[0119] Step S630, adding the multiple child hyperparameter groups to the original hyperparameter population to form an updated original hyperparameter population;

[0120] Step S640: Perform an environment selection operation on the updated original hyperparameter population according to the first objective function value and the second objective function value to obtain a target hyperparameter population.

[0121] In some embodiments, in step S610, each original control strategy is used to control the drone to interact with the environment, thereby calculating a first objective function value for each original hyperparameter set according to the objective learning function. In other words, each original control strategy is used to control the drone to interact with the environment, and the objective learning function is used to evaluate the first objective function value of each individual in the original hyperparameter population P.

[0122] In step S620 of some embodiments, each child control policy is similarly used to control the drone's interaction with the environment to calculate a second objective function value for each child hyperparameter set based on the objective learning function. In other words, each child control policy is used to control the drone's interaction with the environment to evaluate the second objective function value for each individual in the child hyperparameter population Q formed by the child hyperparameter set.

[0123] In practical applications, the evaluation process of the original hyperparameter group and the offspring hyperparameter group using the target learning function is as follows: the drone is controlled to interact with the environment through the trained preliminary flight strategy a~π(·|s) until the drone completes the task. After the drone completes the task, the task completion time T and the closest distance d to the obstacle need to be recorded. min , get the first objective function value and the second objective function value (f1, f2) corresponding to the preliminary flight strategy. Set the maximum completion time Tmax , if the drone is in T max If the UAV cannot reach the destination within the time limit, the mission is considered failed and the first and second objective functions of the preliminary flight strategy need to be marked as positive infinity, i.e. (+∞, +∞). The task is judged to have failed, and the strategy objective function value is marked as (+∞, +∞).

[0124] In step S630 of some embodiments, multiple child hyperparameter groups are added to the original hyperparameter population to form an updated original hyperparameter population, where the updated original hyperparameter population P=P∪Q.

[0125] In step S640 of some embodiments, an environment selection operation is performed on the updated original hyperparameter population according to the first objective function value and the second objective function value to obtain a target hyperparameter population.

[0126] In some embodiments, as Figure 7 As shown, step S640 specifically includes but is not limited to steps S710 to S730.

[0127] Step S710, dividing the updated original hyperparameter population to obtain multiple preliminary hyperparameter populations;

[0128] Step S720 , performing non-dominated sorting on the multiple preliminary hyperparameter populations according to the numerical values ​​of the first objective function value and the second objective function value to obtain an environment selection order;

[0129] Step S730 , performing an environment selection operation on each preliminary hyperparameter population according to the environment selection order to obtain a target hyperparameter population.

[0130] In some embodiments, in step S710 and step S720, the updated original hyperparameter population is divided to obtain multiple preliminary hyperparameter populations; wherein each preliminary hyperparameter group is an original hyperparameter group or a descendant hyperparameter group, each preliminary hyperparameter population includes multiple preliminary hyperparameter groups, and each preliminary hyperparameter group in each preliminary hyperparameter population is not Pareto dominated by each other. According to the numerical values ​​of the first objective function value and the second objective function value, the multiple preliminary hyperparameter populations are non-dominated and sorted to obtain the environment selection order; specifically, the population P containing 2n individuals is divided into l subpopulations {P1, P2, ..., P l}, Individuals in each subpopulation do not Pareto dominate each other (i.e., no individual has both f1 and f2 objective function values ​​smaller than another individual), and P1 Pareto dominates {P2, ..., P l}, P2 Pareto dominates {P3, ..., Pl By analogy, P1 Pareto dominates P2, which means that each individual in P2 is Pareto dominated by an individual in P1 (i.e., the f1 and f2 objective function values ​​of the Pareto dominated individual are greater than those of the individual that Pareto dominates it).

[0131] In step S730 of some embodiments, an environment selection operation is performed on each preliminary hyperparameter population according to an environment selection order to obtain a target hyperparameter population.

[0132] In some embodiments, as Figure 8 As shown, step S730 specifically includes but is not limited to steps S810 to S820.

[0133] Step S810, obtaining a preset individual quantity threshold;

[0134] Step S820 , selecting multiple preliminary hyperparameter groups that meet the individual quantity threshold from the multiple preliminary hyperparameter populations according to the environment selection order, to form a target hyperparameter population.

[0135] In step S810 of some embodiments, a preset individual quantity threshold n is obtained.

[0136] In step S820 of some embodiments, the subpopulations {P1, P2, ..., P l}, and the environment selection is carried out in sequence until n preliminary hyperparameter groups are selected, and then further selection is carried out to form the target hyperparameter population.

[0137] Specifically, steps S810 to S820 are as follows:

[0138] Step 1: Start selecting the environment from P1. If the number of individuals in P1 is greater than n, proceed to the next step, otherwise continue to select P2. If the number of individuals in {P1, P2} is greater than n, proceed to the next step, otherwise continue to select P3. And so on, until P is selected. k , k≤l,{P1, P2, ..., P k}If the number of individuals in is greater than n, go to the next step.

[0139] Step 2: Assume {P1, P2, ..., P k} is n′, then we need to k Delete n′-n individuals from the equation. First calculate P k The Euclidean distance between individuals in the target space. Then find P kThe two closest individuals in the , and randomly delete one of them. This step is repeated n′Pn times, that is, from P k Delete n′-n individuals.

[0140] Step 3: Set P = {P1, P2, ..., P k} output as the next generation population.

[0141] A plurality of preliminary hyperparameter groups that meet a threshold of the number of individuals are sequentially selected from the plurality of preliminary hyperparameter populations to form a target hyperparameter population.

[0142] In some embodiments, as Figure 9 As shown, the specific process of the evolutionary multi-objective deep reinforcement learning algorithm of the embodiment of the present disclosure is as follows:

[0143] First, the population is initialized, which includes multiple individuals. Each individual is subjected to operations such as mutation, deep reinforcement learning, and objective function value evaluation in turn. The decision to exit is based on the result of the environment selection operation. If not, the individuals of the next generation population generated by the environment selection operation are subjected to operations such as mutation, deep reinforcement learning, and objective function value evaluation in turn until the preset conditions are met, thereby ending the optimization process.

[0144] In practical applications, the steps of the environmental selection operation can be as follows: Assume there is a population P consisting of a total of 16 individuals. First, population P is divided into three subpopulations: P1, P2, and P3. The environmental selection operation requires selecting 8 individuals from these 16 individuals as the next generation population. The selection starts with P1, which contains 6 individuals, so P1 is selected as a whole. Then, P2 is selected. P2 contains 6 individuals, so only 2 individuals can be selected from P2. These individuals are then merged with the 6 individuals in P1 to form the next generation population. The Euclidean distance between individuals in P2 is calculated and one of the two closest individuals is deleted until P2 has only two individuals remaining. P1 and P2 are then merged as the next generation population, thus completing the environmental selection process.

[0145] The flight decision generation method proposed in the embodiment of the present disclosure obtains mission requirement data; constructs a flight decision model based on the mission requirement data; wherein the flight decision model includes multiple decision tuple data for characterizing the original flight decision of the UAV, each decision tuple data includes an original hyperparameter group, and each original hyperparameter group is aggregated to form an original hyperparameter population; based on the flight decision model, a corresponding target learning function is constructed; the original hyperparameter population is updated and optimized according to the target learning function to obtain a target hyperparameter population; and the target flight decision of the UAV is obtained according to the target hyperparameter population. Based on the establishment of the flight decision model of the UAV, the embodiment of the present disclosure defines a target learning function for optimization. On the basis of enabling the UAV to complete the autonomous navigation task through the target flight decision, the original hyperparameter population is further updated and optimized through the optimization target of the target learning function to obtain the target flight decision, thereby improving the flexibility of the UAV's autonomous navigation task.

[0146] The present disclosure also provides a flight decision making device, such as Figure 10 As shown, the above-mentioned flight decision generation method can be implemented. The device includes: a data acquisition module 1010, a model construction module 1020, a function construction module 1030, a population optimization module 1040, and a decision acquisition module 1050. Among them, the data acquisition module 1010 is used to obtain task requirement data; the model construction module 1020 is used to construct a flight decision model based on the task requirement data; the flight decision model includes multiple decision tuple data used to characterize the original flight decision of the unmanned aerial vehicle, each decision tuple data includes an original hyperparameter group, and each original hyperparameter group is aggregated to form an original hyperparameter population; the function construction module 1030 is used to construct a corresponding target learning function based on the flight decision model; the population optimization module 1040 is used to update and optimize the original hyperparameter population according to the target learning function to obtain a target hyperparameter population; and the decision acquisition module 1050 is used to obtain the target flight decision of the unmanned aerial vehicle based on the target hyperparameter population.

[0147] It should be noted that the flight decision generation device of the embodiment of the present disclosure is used to execute the flight decision generation method in the above embodiment, and its specific processing process is the same as the flight decision generation method in the above embodiment, which will not be repeated here.

[0148] The flight decision generation device proposed in the embodiment of the present disclosure obtains mission requirement data; constructs a flight decision model based on the mission requirement data; wherein the flight decision model includes multiple decision tuple data for characterizing the original flight decision of the unmanned aerial vehicle, each decision tuple data includes an original hyperparameter group, and each original hyperparameter group is aggregated to form an original hyperparameter population; based on the flight decision model, a corresponding target learning function is constructed; the original hyperparameter population is updated and optimized according to the target learning function to obtain a target hyperparameter population; and the target flight decision of the unmanned aerial vehicle is obtained according to the target hyperparameter population. Based on the establishment of the flight decision model of the unmanned aerial vehicle, the embodiment of the present disclosure defines a target learning function for optimization. On the basis of enabling the unmanned aerial vehicle to complete the autonomous navigation task through the target flight decision, the original hyperparameter population is further updated and optimized through the optimization target of the target learning function to obtain the target flight decision, thereby improving the flexibility of the unmanned aerial vehicle's autonomous navigation task.

[0149] The present disclosure also provides a computer device, including:

[0150] at least one processor, and

[0151] a memory communicatively connected to at least one processor; wherein,

[0152] The memory stores instructions, which are executed by at least one processor so that the at least one processor implements a method as described in any one of the embodiments of the first aspect of the present application when executing the instructions.

[0153] The following combination Figure 11 The hardware structure of the computer device is described in detail. The computer device includes: a processor 1110 , a memory 1120 , an input / output interface 1130 , a communication interface 1140 , and a bus 1150 .

[0154] The processor 1110 may be implemented using a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of the present disclosure.

[0155] The memory 1120 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1120 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1120 and is called by the processor 1110 to execute the flight decision generation method of the embodiments of the present disclosure.

[0156] Input / output interface 1130, used for information input and output;

[0157] Communication interface 1140, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, Wi-Fi, Bluetooth, etc.); and

[0158] bus 1150 , which transmits information between various components of the device (e.g., processor 1110 , memory 1120 , input / output interface 1130 , and communication interface 1140 );

[0159] The processor 1110 , the memory 1120 , the input / output interface 1130 , and the communication interface 1140 are communicatively connected to each other within the device via a bus 1150 .

[0160] An embodiment of the present disclosure further provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the flight decision-making method of the embodiment of the present disclosure.

[0161] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0162] The flight decision generation method and apparatus, computer equipment, and storage medium proposed in the embodiments of the present disclosure obtain mission requirement data; construct a flight decision model based on the mission requirement data; wherein the flight decision model includes multiple decision tuple data for characterizing the original flight decision of the drone, each decision tuple data includes an original hyperparameter group, and each original hyperparameter group is aggregated to form an original hyperparameter population; based on the flight decision model, a corresponding target learning function is constructed; the original hyperparameter population is updated and optimized according to the target learning function to obtain a target hyperparameter population; and the target flight decision of the drone is obtained according to the target hyperparameter population. Based on the establishment of the flight decision model of the drone, the embodiments of the present disclosure define a target learning function for optimization. On the basis of enabling the drone to complete the autonomous navigation task through the target flight decision, the original hyperparameter population is further updated and optimized through the optimization objective of the target learning function to obtain the target flight decision, thereby improving the flexibility of the drone's autonomous navigation task.

[0163] The embodiments described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0164] It will be understood by those skilled in the art that Figure 1 、 Figure 4 、 Figure 5 、 Figure 6 、 Figure 7 and Figure 8 The technical solutions shown in the figures do not constitute a limitation on the embodiments of the present disclosure, and may include more or fewer steps than those shown in the figures, or a combination of certain steps, or different steps.

[0165] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0166] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0167] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0168] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0170] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0171] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0172] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0173] The preferred embodiments of the present disclosure are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present disclosure. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present disclosure should be within the scope of the present disclosure.

Claims

1. A method for generating flight decisions for an unmanned aerial vehicle, characterized in that: include: Obtain task requirement data; A flight decision model is constructed according to the mission requirement data; wherein the flight decision model includes a plurality of decision tuple data for characterizing the original flight decision of the UAV; each of the decision tuple data includes state data, action data, a state transfer function, a reward function and a discount factor; the state data includes the first distance between the UAV and the obstacle at a certain moment, the maximum detection range corresponding to the sensor of the UAV, the second distance and angle data between the current position of the UAV and the preset end point, the current flight speed and viewing direction of the UAV, and the maximum speed limit of the UAV; the action data includes the acceleration and deceleration data and the steering data of the UAV at a certain moment; the state transfer function is used to generate the state data of the UAV at the next moment; the reward function includes the original hyperparameter group, and the reward function is used to evaluate the preliminary quality of the UAV's action execution in a certain state: r(s t ,a t )=β d r d (s t ,a t )+β o r o (s t ,a t )+β s r s (s t ,a t )+β c r c (s t ,a t ), where r represents the reward function, β d ,β o ,β s ,β c There are 4 hyperparameters in [0,β max ] range, the four hyperparameters are combined to form an original hyperparameter group; a t Represents the action data, s t represents the state data, and the discount factor is used to combine with the reward function to calculate the quality of the drone's execution of an action in a certain state; Constructing a corresponding objective learning function according to the flight decision model, including: obtaining an optimization objective according to the flight decision model; wherein the optimization objective includes a flight time objective and a flight risk objective of the UAV, and the objective learning function includes a first objective function and a second objective function; constructing the first objective function according to the flight time objective, and constructing the second objective function according to the flight risk objective; Updating and optimizing the original hyperparameter population according to the target learning function to obtain a target hyperparameter population; A target flight decision of the UAV is obtained according to the target hyperparameter population.

2. The method according to claim 1, characterized in that The updating and optimizing the original hyperparameter population according to the target learning function to obtain a target hyperparameter population includes: Performing reinforcement learning training on each of the original hyperparameter groups in the original hyperparameter population through the flight decision model to obtain corresponding multiple original control strategies; Performing a mutation operation on each of the original hyperparameter groups to generate multiple descendant hyperparameter groups; Performing reinforcement learning training on each of the offspring hyperparameter groups through the flight decision model to obtain corresponding multiple offspring control strategies; The target hyperparameter population is obtained through the multiple original control strategies and the multiple descendant control strategies.

3. The method according to claim 2, characterized in that The step of obtaining the target hyperparameter population by using the multiple original control strategies and the multiple descendant control strategies includes: Using each of the original control strategies to control the UAV to interact with the environment, so as to calculate a first objective function value of each of the original hyperparameter groups according to the target learning function; Using each of the child control strategies to control the UAV to interact with the environment, so as to calculate a second objective function value of each of the child hyperparameter groups according to the objective learning function; Adding the multiple child hyperparameter groups to the original hyperparameter population to form an updated original hyperparameter population; According to the first objective function value and the second objective function value, an environment selection operation is performed on the updated original hyperparameter population to obtain the target hyperparameter population.

4. The method according to claim 3, characterized in that The step of performing an environment selection operation on the updated original hyperparameter population according to the first objective function value and the second objective function value to obtain the target hyperparameter population includes: Dividing the updated original hyperparameter population to obtain multiple preliminary hyperparameter populations; wherein each preliminary hyperparameter group is the original hyperparameter group or the descendant hyperparameter group, each preliminary hyperparameter population includes multiple preliminary hyperparameter groups, and each preliminary hyperparameter group in each preliminary hyperparameter population is not Pareto dominated by each other; performing non-dominated sorting on the plurality of preliminary hyperparameter populations according to numerical values ​​of the first objective function value and the second objective function value to obtain an environment selection order; An environment selection operation is performed on each of the preliminary hyperparameter populations according to the environment selection order to obtain the target hyperparameter population.

5. The method according to claim 4, characterized in that The step of performing an environment selection operation on each of the preliminary hyperparameter populations according to the environment selection order to obtain the target hyperparameter population includes: Get the preset individual quantity threshold; According to the environment selection order, a plurality of the preliminary hyperparameter groups that meet the individual quantity threshold are selected in sequence from the plurality of preliminary hyperparameter populations to form the target hyperparameter population.

6. A flight decision generating device for an unmanned aerial vehicle, characterized in that: include: Data acquisition module: used to obtain task requirement data; Model construction module: construct a flight decision model according to the task requirement data; wherein, the flight decision model includes a plurality of decision tuple data for characterizing the original flight decision of the UAV; each of the decision tuple data includes state data, action data, state transfer function, reward function and discount factor; the state data includes the first distance between the UAV and the obstacle at a certain moment, the maximum detection range corresponding to the sensor of the UAV, the second distance and angle data between the current position of the UAV and the preset end point, the current flight speed and viewing direction of the UAV, and the maximum speed limit of the UAV; the action data includes the acceleration and deceleration data and the steering data of the UAV at a certain moment; the state transfer function is used to generate the state data of the UAV at the next moment; the reward function includes the original hyperparameter group, and the reward function is used to evaluate the preliminary quality of the UAV's action execution in a certain state: r(s t ,a t )=β d r d (s t ,a t )+β o r o (s t ,a t )+β s r s (s t ,a t )+β c r c (s t ,a t ), where r represents the reward function, β d ,β o ,β s ,β c There are 4 hyperparameters in [0,β max ] range, and the four hyperparameters are combined to form an original hyperparameter group; a t Represents the action data, s t represents the state data, and the discount factor is used to combine with the reward function to calculate the quality of the drone's execution of an action in a certain state; A function construction module is configured to construct a corresponding target learning function based on the flight decision model, including: obtaining an optimization target based on the flight decision model; wherein the optimization target includes a flight time target and a flight risk target of the UAV; the target learning function includes a first target function and a second target function; constructing the first target function based on the flight time target, and constructing the second target function based on the flight risk target; Population optimization module: used to update and optimize the original hyperparameter population according to the target learning function to obtain a target hyperparameter population; Decision acquisition module: used to obtain the target flight decision of the UAV according to the target hyperparameter population.

7. A computer device, characterized in that: The computer device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is configured to execute the method according to any one of claims 1 to 5.

8. A storage medium, wherein the storage medium is a computer-readable storage medium, characterized in that: The computer readable storage device contains a computer program. When the computer program is executed by a computer, the computer is configured to perform the method according to any one of claims 1 to 5.

Images