An unmanned aerial vehicle motion consistency feedback optimization method and system, a storage medium and an electronic device

Abstract

Embodiments of the present application provide a kind of unmanned vehicle action consistency feedback optimization method, system, storage medium and electronic equipment, its method includes: collecting the real-time flight state information and environmental information in the process of unmanned vehicle executing first task;Establish historical action database;First task candidate action trajectory is generated based on the real-time flight state information and environmental information of first task;Based on the preset action consistency evaluation model, the consistency evaluation value between candidate action trajectory and high-quality action trajectory in historical action database is calculated;Reward signal is generated based on consistency evaluation value;According to reward signal, first control strategy is optimized, and the target action trajectory after optimization is generated;And indicate unmanned vehicle to execute first task according to target action trajectory.Through the present application, it solves the problem that unmanned vehicle is difficult to generate high-quality action trajectory in real time under complex dynamic environment.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of unmanned aerial vehicle control, and in particular, to an unmanned aerial vehicle action consistency feedback optimization method and system, a storage medium and an electronic device. BACKGROUND

[0002] In recent years, with the rapid development of unmanned aerial vehicle technology, unmanned aerial vehicles have been widely used in many fields. In actual application, unmanned aerial vehicles often need to efficiently and stably complete various tasks in complex and variable dynamic environments.

[0003] The prior art gradually adopts reinforcement learning algorithms to enable unmanned aerial vehicles to optimize control strategies based on real-time feedback of historical experience and environmental interaction. However, the existing reinforcement learning methods usually have the following shortcomings: on the one hand, due to the lack of an effective historical action trajectory utilization mechanism, it is difficult for unmanned aerial vehicles to quickly generate and optimize high-quality action trajectories in new complex environments; on the other hand, there is a lack of effective quantification of the relationship between action trajectory consistency and environmental complexity, which cannot dynamically adjust the optimization goal of the unmanned aerial vehicle action strategy in environments with varying complexity. SUMMARY

[0004] Embodiments of the present application provide an unmanned aerial vehicle action consistency feedback optimization method, system, storage medium and electronic device to at least solve the problem that unmanned aerial vehicles in related technologies are difficult to generate high-quality action trajectories in real time in complex dynamic environments.

[0005] According to an embodiment of the present application, an unmanned aerial vehicle action consistency feedback optimization method is provided, comprising:

[0006] Collecting real-time flight state information and environmental information during execution of a first task by an unmanned aerial vehicle; wherein the flight state information includes at least any one of position, attitude, speed and battery level information of the unmanned aerial vehicle; and the environmental information includes at least any one of obstacle position, motion state, weather information and meteorological disturbance information;

[0007] Establishing a historical action database, wherein the historical action database stores action trajectories in historical tasks of the unmanned aerial vehicle;

[0008] Generating a candidate action trajectory for the first task based on the real-time flight state information and the environmental information of the first task;

[0009] Calculating a consistency evaluation value between the candidate action trajectory and a high-quality action trajectory in the historical action database based on a preset action consistency evaluation model; wherein the calculation of the consistency evaluation value between the candidate action trajectory and the high-quality action trajectory in the historical action database based on the preset action consistency evaluation model comprises: selecting a high-quality action trajectory from the historical action database; and calculating the consistency evaluation value between the candidate action trajectory and the selected high-quality action trajectory based on the preset action consistency evaluation model. high-quality historical action trajectories as a reference trajectory set; calculating the Euclidean distance between the current task candidate action trajectory and each reference action trajectory; determining an overall consistency distance according to the Euclidean distance; converting the overall consistency distance into a consistency score through an exponential function; and generating a reward signal according to the consistency score and an environmental complexity index, wherein the environmental complexity index is obtained based on the influence weights of the number of obstacles, the average movement speed of the obstacles, the weather, and the meteorological disturbance in the current environment;

[0010] generating a reward signal based on the consistency evaluation value;

[0011] optimizing the first control strategy according to the reward signal to generate an optimized target action trajectory; and instructing the UAV to perform the first task according to the target action trajectory, wherein the first control strategy comprises a candidate action trajectory.

[0012] In one example embodiment, the generating of the candidate action trajectory of the first task based on the real-time flight state information and the environmental information of the first task comprises:

[0013] obtaining real-time flight state information of the UAV and environmental information , and generating the candidate action trajectory of the first task by an action consistency generation module:

[0014] ;

[0015] wherein, represents a Transformer or diffusion model generation function, represents a model parameter.

[0016] In one example embodiment, after the generating of the candidate action trajectory of the first task based on the real-time flight state information and the environmental information of the first task, the method further comprises:

[0017] calculating an action continuity loss and a task completion loss of the candidate action sequence, and determining a total loss according to the action continuity loss and the task completion loss;

[0018] performing a back propagation process based on the total loss to update the model parameter;

[0019] repeating the foregoing steps until the generated candidate action trajectory reaches a preset loss threshold or completes a maximum number of iterations.

[0020] In one example embodiment, the generating of the reward signal according to the consistency score comprises:

[0021] a real-time computing environment complexity index;

[0022] determine a weight coefficient of the action consistency reward and the task performance reward according to the environment complexity index;

[0023] calculate the reward signal according to the weight coefficient.

[0024] In one exemplary embodiment, the optimizing the first control strategy according to the reward signal comprises:

[0025] determine a first action based on current UAV state information and a preset policy function, and execute the first action;

[0026] After executing the first action, the UAV state is transferred to a next state, and a reward signal is obtained;

[0027] update the value function parameters according to the reward signal and the state transition;

[0028] update the policy parameters according to the value function parameters;

[0029] repeat the foregoing steps until the policy parameters converge or a preset training number is reached, and generate an optimized target action trajectory.

[0030] According to another embodiment of the present application, there is provided a UAV action consistency feedback optimization system, comprising:

[0031] a state information acquisition module for acquiring real-time flight state information and environment information during execution of a task by a UAV;

[0032] a historical action database module for storing action trajectories in historical tasks of the UAV;

[0033] an action consistency generation module for generating a candidate action trajectory for a first task based on the real-time flight state information and the environment information;

[0034] an action consistency evaluation module for calculating a consistency evaluation value between the candidate action trajectory and a high-quality action trajectory in the historical action database; wherein the calculation of the consistency evaluation value between the candidate action trajectory and the high-quality action trajectory in the historical action database based on a preset action consistency evaluation model comprises: selecting a plurality of high-quality action trajectories from the historical action database, and calculating the consistency evaluation value between the candidate action trajectory and each of the plurality of high-quality action trajectories based on the preset action consistency evaluation model. high-quality historical action trajectories as a reference trajectory set; calculating the Euclidean distance between each reference action trajectory and the current task candidate action trajectory; determining an overall consistency distance according to the Euclidean distance; converting the overall consistency distance into a consistency score through an exponential function; and generating the reward signal according to the consistency score and an environmental complexity index, wherein the environmental complexity index is obtained based on the influence weights of the number of obstacles, the average movement speed of the obstacles, the weather, and the meteorological disturbance in the current environment;

[0035] an adaptive reward module configured to generate a reward signal based on the consistency evaluation value;

[0036] a policy optimization module configured to optimize the current policy according to the reward signal and output an optimized action trajectory;

[0037] a flight control execution module configured to control the UAV to perform the first task based on the optimized action trajectory.

[0038] In one example embodiment, the adaptive reward module comprises:

[0039] an environmental complexity evaluation unit configured to monitor and calculate an environmental complexity index in real time;

[0040] a weight adjustment unit configured to dynamically adjust the weight proportion of the consistency reward and the task performance reward according to the environmental complexity index;

[0041] a reward fusion unit configured to fuse the consistency reward and the task performance reward to generate the reward signal.

[0042] According to still another embodiment of the present application, a computer readable storage medium is also provided, which stores a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0043] According to still another embodiment of the present application, an electronic device is also provided, which comprises a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above method embodiments.

[0044] By the present application, since historical action data is taken into account in the calculation and dynamic regulation of the UAV action in a complex environment is achieved through action consistency evaluation, the problem that the UAV is difficult to generate a high-quality action trajectory in real time in a complex dynamic environment can be solved, and the effect that the UAV can quickly generate and optimize an action trajectory with high consistency and high adaptability in a complex dynamic environment is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1A flowchart of steps of the unmanned aerial vehicle action consistency feedback optimization method according to an embodiment of the present application is shown in FIG. 1.

[0046] Figure 2 A structural schematic block diagram of the unmanned aerial vehicle action consistency feedback optimization system according to an embodiment of the present application is shown in FIG. 2. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application.

[0048] Hereinafter, the terms "first", "second", and the like are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second", and the like can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0049] In addition, in the present application, the orientation terms such as "up", "down", "left", "right", and the like can include but not limited to the orientation defined by the relative position of the components in the drawings. It should be understood that these directional terms can be relative concepts, which are used for relative description and clarification, and can be changed accordingly according to the change of the position of the components in the drawings.

[0050] In the present application, unless otherwise specified and limited, the term "connection" should be understood broadly, for example, "connection" can be fixed connection, or detachable connection, or integral; can be directly connected, or indirectly connected through intermediate medium. In addition, the term "coupling" can be an electrically connected manner for signal transmission.

[0051] As used herein, "about", "approximately", or "approximately" includes the stated value and the average value within an acceptable deviation range of the specific value, wherein the acceptable deviation range is determined by the ordinary skill in the art considering the measurement being discussed and the error related to the measurement of the specific quantity (i.e. the limitation of the measurement system).

[0052] An unmanned aerial vehicle action consistency feedback optimization method is provided in the present embodiment, Figure 1 A flowchart of the unmanned aerial vehicle action consistency feedback optimization method according to an embodiment of the present application is shown in FIG. 1, which includes the following steps: Figure 1

[0053] ​Step S11, collecting real-time flight state information and environment information of the UAV during execution of a first task; wherein the flight state information includes at least one of position, attitude, speed and battery capacity information of the UAV; the environment information includes at least one of obstacle position, motion state, weather information and meteorological disturbance information; the first task can be a task such as inspection in a complex environment, logistics distribution, aerial photography, etc.

[0054] Step S12, establishing a historical action database, wherein the historical action database stores action trajectories in historical tasks of the UAV;

[0055] Step S13, generating a candidate action trajectory of the first task based on the real-time flight state information and the environment information of the first task;

[0056] In the embodiment, the action consistency generation module can be implemented by using a Transformer or a diffusion model. The specific action generation algorithm operation steps include:

[0057] S131, obtaining real-time flight state information and environment information of the UAV , and generating a candidate action trajectory of the first task by using the action consistency generation module:

[0058] ;

[0059] wherein, represents a Transformer or diffusion model generation function, represents a model parameter;

[0060] S132, calculating an action continuity loss of the candidate action sequence and a task completion loss , and obtaining a total loss :

[0061] The action continuity loss is:

[0062] ;

[0063] wherein, represents an action vector at the t-th moment, represents an action change rate vector at the t-th moment; The task completion loss is calculated in the following manner:

[0064]

[0065] ;

[0066] wherein, ​​​represents the final state after the candidate action trajectory is executed, represents the target state expected to be reached by the current task;

[0067] The overall loss function of the action consistency generation module is:

[0068] ;

[0069] wherein, is a hyperparameter for balancing action continuity and task completion performance, and the value range is ;

[0070] S133, updating the model parameters by back propagation :

[0071] ;

[0072] wherein, is a learning rate, is the gradient of the loss function with respect to the model parameters;

[0073] S134, repeating steps S131 to S133 until the generated candidate action trajectory reaches a preset loss threshold or completes a maximum number of iterations, to obtain the final optimized action trajectory output to the flight control execution module.

[0074] Step S14, based on the preset action consistency evaluation model, calculating the consistency evaluation value between the candidate action trajectory and the high-quality action trajectory in the historical action database;

[0075] In this embodiment, the evaluation process of the action consistency evaluation model includes:

[0076] S141, selecting high-quality historical action trajectories from the historical action database as a reference trajectory set:

[0077] ;

[0078] S142, calculating the Euclidean distance between the current task candidate action trajectory and each reference action trajectory:

[0079] ;

[0080] S143, calculating the average value of the Euclidean distance to obtain the overall consistency distance:

[0081] ;

[0082] S144, converting the overall consistency distance into a consistency score by an exponential function :

[0083] ;

[0084] S145, using the calculated consistency score for the generation of subsequent reward signals to guide the reinforcement learning algorithm to optimize the current strategy of the UAV.

[0085] Step S15, generating a reward signal based on the consistency evaluation value, the reward signal being used to guide the reinforcement learning algorithm to optimize the first control strategy of the first task, the first control strategy including a candidate action trajectory;

[0086] In this embodiment, the generation of the reward signal includes:

[0087] S151, calculating the environmental complexity index in real time, defining the environmental complexity index as :

[0088] ;

[0089] wherein, represents the number of obstacles in the current environment; represents the average movement speed of the obstacles; represents the influence weight of weather and meteorological disturbance; , , is the weight coefficient of the environmental complexity evaluation; for example, in the urban inspection scenario, γ1=0.4, γ2=0.3, γ3=0.3, and in the open logistics scenario, γ1=0.2, γ2=0.2, γ3=0.6; the weather influence weight The quantitative standard is: sunny =0.1, cloudy =0.3, light rain / light wind =0.5, heavy rain / strong wind =0.8, and heavy rain / strong wind =1.0

[0090] S152, dynamically determining the weight coefficients of the action consistency reward and the task performance reward according to the environmental complexity index :

[0091] ;

[0092] ;

[0093] S153, calculating the comprehensive reward signal :

[0094] ​ ;

[0095] wherein, is the action consistency evaluation score, represents the current task performance score, such as task progress, energy consumption performance, or closeness to the target state;

[0096] The above adaptive reward signal is used for a reinforcement learning algorithm to optimize the action control strategy of the UAV in real time.

[0097] Step S16, the first control strategy is optimized according to the reward signal through the reinforcement learning algorithm, and an optimized target action trajectory is generated; and the UAV is instructed to perform the first task according to the target action trajectory.

[0098] In the embodiment, the reinforcement learning algorithm is one of proximal policy optimization (PPO), deep deterministic policy gradient (DDPG), or soft actor critic (SAC) algorithm, and the process of optimizing the current strategy by the reinforcement learning algorithm includes:

[0099] S161, based on the current UAV state information , the reinforcement learning algorithm adopts a policy function to select an action and execute:

[0100] ;

[0101] S162, after the action is executed, the state of the UAV is transferred to the next state , and an adaptive reward signal is obtained;

[0102] S163, using the action , the reward , and the state transition , the value function (Critic) parameters are updated:

[0103] For the PPO algorithm, the advantage function estimate is used, and the optimization target is:

[0104] ;

[0105] For the DDPG algorithm, the value function of the Critic is updated, and the update target is:

[0106] ;

[0107] For the SAC algorithm, the Critic function Objective is:

[0108] ;

[0109] wherein, is the clip range hyperparameter of PPO algorithm; is the discount factor;

[0110] S164, updating the actor policy parameter according to the feedback of the value function parameter :

[0111] PPO policy parameter update:

[0112] ;

[0113] DDPG policy parameter update:

[0114] ;

[0115] ;

[0116] SAC policy parameter update:

[0117] ;

[0118] ;

[0119] wherein, is the learning rate of the policy network, is the entropy adjustment coefficient in the SAC algorithm.

[0120] S165, repeating steps S161 to S164 until the policy parameter converges or reaches the preset training number, and generating an optimized action trajectory .

[0121] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the method described in each embodiment of the present application.

[0122] A UAV action consistency feedback optimization system is also provided in the embodiments, which is used to implement the above embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware is also possible and contemplated.

[0123] Figure 2 A structure block diagram of a UAV action consistency feedback optimization system according to an embodiment of the present application is shown in FIG. 1, which comprises: Figure 2

[0124] A state information collection module 21 is configured to collect real-time flight state information and environmental information during the execution of a task by the UAV;

[0125] A historical action database module 22 is configured to store action trajectories in historical tasks of the UAV;

[0126] An action consistency generation module 23 is configured to generate a candidate action trajectory for a first task based on the real-time flight state information and the environmental information;

[0127] An action consistency evaluation module 24 is configured to calculate a consistency evaluation value between the candidate action trajectory and a high-quality action trajectory in the historical action database;

[0128] An adaptive reward module 25 is configured to generate a reward signal based on the consistency evaluation value;

[0129] A policy optimization module 26 is configured to optimize a current policy according to the reward signal and output an optimized action trajectory;

[0130] A flight control execution module 27 is configured to control the UAV to execute the first task based on the optimized action trajectory.

[0131] In an optional embodiment, the adaptive reward module comprises:

[0132] An environmental complexity evaluation unit is configured to monitor and calculate an environmental complexity index in real time;

[0133] A weight adjustment unit is configured to dynamically adjust a weight ratio of the consistency reward and the task performance reward according to the environmental complexity index;

[0134] A reward fusion unit is configured to fuse the consistency reward and the task performance reward to generate the reward signal.

[0135] ​It should be noted that the above various modules can be implemented by software or hardware, and for the latter, the implementation can be achieved in the following ways, but is not limited thereto: the above modules are located in the same processor; or the above various modules are located in different processors in any combination.

[0136] Embodiments of the present application also provide a computer readable storage medium, which stores a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0137] In an example embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0138] Embodiments of the present application also provide an electronic device, which comprises a memory storing a computer program and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.

[0139] In an example embodiment, the above electronic device can further comprise a transmission device connected to the processor and an input and output device connected to the processor.

[0140] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0141] In the several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented by other ways. For example, the device embodiments described above are only schematic, and the division of the modules or units is only a logical function division, and there can be another division way in actual implementation, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0142] The units described as separate components may or may not be physically separate, and the components displayed as units may be a physical unit or multiple physical units, that is, may be located in one place, or also can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0143] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0144] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or the part of the prior art that contributes to the technical solutions or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0145] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for optimizing the consistency of drone actions through feedback, characterized in that, The method comprises: collecting real-time flight state information and environmental information of the UAV during execution of a first task, wherein the flight state information comprises at least one of position, attitude, speed and battery power information of the UAV, and the environmental information comprises at least one of obstacle position, motion state, weather information and meteorological disturbance information; establishing a historical action database, wherein the historical action database stores action trajectories in historical tasks of the UAV; generating a candidate action trajectory of the first task based on the real-time flight state information and the environmental information of the first task; Based on the preset action consistency evaluation model, the consistency evaluation value between the candidate action trajectory and the high-quality action trajectory in the historical action database is calculated; wherein, the consistency evaluation value between the candidate action trajectory and the high-quality action trajectory in the historical action database is calculated based on the preset action consistency evaluation model, which comprises: selecting a plurality of high-quality historical action trajectories as a reference trajectory set from the historical action database; calculating the Euclidean distance between the current task candidate action trajectory and each reference action trajectory; determining the overall consistency distance according to the Euclidean distance; converting the overall consistency distance into a consistency score through an exponential function; generating a reward signal according to the consistency score and an environmental complexity index, wherein the environmental complexity index is obtained based on the number of obstacles in the current environment, the average movement speed of the obstacles, the influence weight of the weather and the meteorological disturbance. generating a reward signal based on the consistency evaluation value; optimizing a first control strategy according to the reward signal to generate an optimized target action trajectory, and instructing the UAV to execute the first task according to the target action trajectory, wherein the first control strategy comprises a candidate action trajectory.

2. The method of claim 1, wherein, The method further comprises: Acquiring real-time flight state information of a drone and environmental information and generating a candidate action trajectory of the first task through an action consistency generation module: ； wherein, denotes a Transformer or diffusion model generating function, denotes model parameters.

3. The method of claim 1, wherein, calculating an action continuity loss and a task completion loss of the candidate action sequence, and determining a total loss according to the action continuity loss and the task completion loss; performing a back propagation process based on the total loss to update model parameters; repeating the foregoing steps until the generated candidate action trajectory reaches a preset loss threshold or a maximum number of iterations is completed. The method further comprises:

4. The method of claim 1, wherein, calculating an environmental complexity index in real time; dynamically determining weight coefficients of action consistency rewards and task performance rewards according to the environmental complexity index; calculating the reward signal according to the weight coefficients. The method further comprises:

5. The method of claim 1, wherein, determining a first action based on current UAV state information and a preset policy function, and executing the first action; after executing the first action, the UAV state is transferred to a next state, and a reward signal is obtained; updating a value function parameter according to the reward signal and the state transition; updating a policy parameter according to the value function parameter; repeating the foregoing steps until the policy parameter converges or a preset number of training times is reached, and generating an optimized target action trajectory. The system comprises:

6. A drone motion consistency feedback optimization system, comprising: a state information collection module for collecting real-time flight state information and environmental information of the UAV during execution of a task; a historical action database module for storing action trajectories in historical tasks of the UAV; an action consistency generation module for generating a candidate action trajectory of a first task based on real-time flight state information and environmental information; an adaptive reward module for generating a reward signal based on a consistency evaluation value; The motion consistency evaluation module is used to calculate the consistency evaluation value between candidate motion trajectories and high-quality motion trajectories in the historical motion database; wherein, the calculation of the consistency evaluation value between the candidate motion trajectory and the high-quality motion trajectories in the historical motion database based on a preset motion consistency evaluation model includes: selecting from the historical motion database... A set of high-quality historical action trajectories is used as a reference trajectory set; the Euclidean distance between the current task candidate action trajectory and each reference action trajectory is calculated; the overall consistency distance is determined based on the Euclidean distance; the overall consistency distance is converted into a consistency score through an exponential function; a reward signal is generated based on the consistency score and an environmental complexity index, wherein the environmental complexity index is obtained based on the number of obstacles in the current environment, the average speed of the obstacles, and the influence weight of weather and meteorological disturbances; a policy optimization module for optimizing a current policy according to the reward signal and outputting an optimized action trajectory; a flight control execution module for controlling the UAV to execute the first task based on the optimized action trajectory. The adaptive reward module comprises:

7. The system of claim 6, wherein, an environmental complexity evaluation unit for monitoring and calculating an environmental complexity index in real time; ​ a weight adjustment unit, configured to dynamically adjust a weight ratio of the consistency reward and the task performance reward according to the environment complexity index; a reward fusion unit, configured to fuse the consistency reward and the task performance reward to generate the reward signal.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is configured to execute the method in any one of claims 1 to 5 when running.

9. An electronic device comprising a memory and a processor, characterized in that The memory stores a computer program, and the processor is configured to execute the computer program to execute the method in any one of claims 1 to 5.

Images