Unmanned aerial vehicle linear flight control method and device
By constructing a linear flight dynamics model and simulation environment, and combining reinforcement learning algorithms and multi-objective fusion reward functions, the robustness and real-time performance issues of UAVs in complex environments were solved, achieving high-precision, high-stability, and energy-efficient autonomous linear flight control.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INST OF AUTOMATION CHINESE ACAD OF SCI
- Filing Date
- 2026-01-30
- Publication Date
- 2026-05-19
AI Technical Summary
Existing UAV straight flight control methods have poor robustness in complex environments and are prone to excessive side offset and heading deviation. Nonlinear control algorithms have high computational complexity and slow real-time response speed, making it difficult to meet the real-time requirements of long-distance straight flight.
By collecting flight status and environmental disturbance parameters of UAVs, a straight flight dynamics model and simulation environment are constructed. Combined with reinforcement learning algorithms and multi-objective fusion reward functions, interactive training is conducted to determine the straight flight control strategy, reducing the dependence on accurate models and improving robustness and adaptability.
It achieves high-precision, high-stability, and energy-efficient autonomous straight-line flight control for UAVs in complex environments, significantly reducing reliance on precise models and effectively coping with interference from complex environments.
Smart Images
Figure CN122064103A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of unmanned aerial vehicle (UAV) control technology, and in particular to a method and apparatus for controlling the linear flight of a UAV. Background Technology
[0002] Straight-line flight is a core and fundamental element of many flight missions, and its flight accuracy, stability, and anti-interference capabilities directly affect mission efficiency and safety. In civilian fields (such as long-distance logistics delivery and large-scale mapping) and military fields (such as formation flight and fixed-point reconnaissance), drones often need to maintain precise straight-line flight in complex environments (such as strong crosswind interference, dynamic load changes, positioning signal noise, and airflow disturbances caused by terrain undulations), which places extremely high demands on the robustness, adaptability, and real-time performance of control technology.
[0003] In related technologies, existing UAV straight flight control methods are mainly divided into two categories: traditional linear control and nonlinear control. Traditional linear control ignores the nonlinear characteristics of UAVs during straight flight, and the control parameters mostly rely on manual tuning. It has poor robustness in complex scenarios such as crosswind disturbances and sudden load changes, and is prone to problems such as excessive side offset and heading deviation. Nonlinear control has high algorithm computational complexity and slow real-time response speed, which limits its application in scenarios with high real-time requirements such as long-distance straight flight.
[0004] Application content This application provides a method and apparatus for controlling the straight flight of an unmanned aerial vehicle (UAV), which addresses the shortcomings of traditional linear control in the prior art. Traditional linear control ignores the nonlinear characteristics of the UAV during straight flight, resulting in poor robustness in complex scenarios such as crosswind disturbances and sudden load changes, and is prone to problems such as excessive side offset and heading deviation. Nonlinear control algorithms have high computational complexity and slow real-time response speed, limiting their application in scenarios with high real-time requirements such as long-distance straight flight. This application achieves high-precision, high-stability, and energy-saving autonomous straight flight control in actual flight, significantly reducing dependence on accurate models and effectively coping with complex environmental interference.
[0005] This application provides a method for controlling the straight flight of an unmanned aerial vehicle (UAV), including the following steps: Collect flight status parameters and environmental interference parameters of the UAV, wherein the flight status parameters include at least one of three-dimensional position, flight speed, heading angle, side offset and attitude angle; Based on the flight state parameters and the environmental disturbance parameters, a dynamic model of UAV straight flight, straight flight constraints, and a fusion reward function are constructed. A linear flight simulation environment constructed based on the aforementioned UAV linear flight dynamics model; Based on reinforcement learning algorithms, the straight flight constraints, and the fusion reward function, the UAV is interactively trained with the straight flight simulation environment to determine the straight flight control strategy. The drone is controlled to fly in a straight line according to the described straight-line flight control strategy.
[0006] According to the unmanned aerial vehicle (UAV) linear flight control method provided in this application, the step of constructing a UAV linear flight dynamics model based on the flight state parameters and the environmental disturbance parameters includes: The flight state parameters and the environmental disturbance parameters are normalized. The normalized flight state parameters and the environmental disturbance parameters are fused to obtain the initial state vector; Based on the initial state vector, the linear flight dynamics model is constructed, wherein the linear flight dynamics model is used to describe the mapping relationship between the initial state vector and the initial control strategy.
[0007] According to the UAV linear flight control method provided in this application, the fusion reward function is: Reward value = w1 × accuracy reward + w2 × stability reward + w3 × energy consumption reward; Wherein, w1, w2, and w3 are preset weighting coefficients. The accuracy reward is negatively correlated with the absolute value of the side offset and the absolute value of the heading angle deviation, respectively. The stability reward is negatively correlated with the absolute values of the roll angle and pitch angle, and the energy consumption reward is negatively correlated with the magnitude change of the control strategy.
[0008] According to the linear flight control method for a UAV provided in this application, the step of controlling the UAV to interactively train with the linear flight simulation environment to determine the linear flight control strategy includes: Based on the first state vector at the current moment, the current control strategy is selected and applied to the simulation environment to determine the current reward value and the second state vector at the next moment. The experience data is stored in the experience replay buffer, wherein the experience data includes: the first state vector, the current control strategy, the current reward value, and the second state vector; The empirical data from the empirical replay buffer determines the batch training size; The policy network parameters of the UAV are updated based on the empirical data of the batch training scale. The initial exploration rate is dynamically adjusted so that it gradually decreases to the preset exploration rate as the number of training steps increases; When the preset convergence condition is met, the updated policy network parameters are used as the straight-line flight control policy.
[0009] According to the UAV linear flight control method provided in this application, the preset convergence condition is any one or a combination of the following: The number of training steps reaches the threshold. The average fluctuation value among multiple reward values determined by N consecutive training rounds is less than a preset fluctuation threshold, where N is a positive integer.
[0010] According to the UAV linear flight control method provided in this application, the linear flight constraint conditions include at least one of the following: The attitude angle is less than or equal to the attitude angle threshold; The rate of change of the attitude angle is less than or equal to the rate of change threshold; The side offset is less than or equal to the side offset threshold; The flight speed is less than or equal to the speed threshold.
[0011] This application also provides a drone linear flight control device, including the following modules: The acquisition module is used to acquire flight status parameters and environmental interference parameters of the UAV, wherein the flight status parameters include at least one of three-dimensional position, flight speed, heading angle, side offset, and attitude angle; The first construction module is used to construct a UAV linear flight dynamics model, linear flight constraints, and fusion reward function based on the flight state parameters and the environmental interference parameters. The second construction module is used to construct a straight flight simulation environment based on the UAV's straight flight dynamics model; The training module is used to interactively train the UAV and the straight flight simulation environment based on the reinforcement learning algorithm, the straight flight constraints, and the fusion reward function, so as to determine the straight flight control strategy. The control module is used to control the UAV to fly in a straight line according to the straight-line flight control strategy.
[0012] According to the unmanned aerial vehicle (UAV) linear flight control device provided in this application, the first building module is specifically used for: The flight state parameters and the environmental disturbance parameters are normalized. The normalized flight state parameters and the environmental disturbance parameters are fused to obtain the initial state vector; Based on the initial state vector, the linear flight dynamics model is constructed, wherein the linear flight dynamics model is used to describe the mapping relationship between the initial state vector and the initial control strategy.
[0013] According to the unmanned aerial vehicle (UAV) linear flight control device provided in this application, the fusion reward function is: Reward value = w1 × Accuracy reward + w2 × Stability reward + w3 × Energy consumption reward; Wherein, w1, w2, and w3 are preset weighting coefficients. The accuracy reward is negatively correlated with the absolute value of the side offset and the absolute value of the heading angle deviation, respectively. The stability reward is negatively correlated with the absolute values of the roll angle and pitch angle, and the energy consumption reward is negatively correlated with the magnitude change of the control strategy.
[0014] According to the unmanned aerial vehicle (UAV) linear flight control device provided in this application, the training module is specifically used for: Based on the first state vector at the current moment, the current control strategy is selected and applied to the simulation environment to determine the current reward value and the second state vector at the next moment. The experience data is stored in the experience replay buffer, wherein the experience data includes: the first state vector, the current control strategy, the current reward value, and the second state vector; The empirical data from the empirical replay buffer determines the batch training size; The policy network parameters of the UAV are updated based on the empirical data of the batch training scale. The initial exploration rate is dynamically adjusted so that it gradually decreases to the preset exploration rate as the number of training steps increases; When the preset convergence condition is met, the updated policy network parameters are used as the straight-line flight control policy.
[0015] According to the unmanned aerial vehicle (UAV) linear flight control device provided in this application, the preset convergence condition is any one or a combination of the following: The number of training steps reaches the threshold. The average fluctuation value among multiple reward values determined by N consecutive training rounds is less than a preset fluctuation threshold, where N is a positive integer.
[0016] According to the unmanned aerial vehicle (UAV) linear flight control device provided in this application, the linear flight constraint conditions include at least one of the following: The attitude angle is less than or equal to the attitude angle threshold; The rate of change of the attitude angle is less than or equal to the rate of change threshold; The side offset is less than or equal to the side offset threshold; The flight speed is less than or equal to the speed threshold.
[0017] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the UAV linear flight control method described above.
[0018] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the UAV linear flight control method as described above.
[0019] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the UAV linear flight control method described above.
[0020] This application provides a method and apparatus for controlling the straight flight of an unmanned aerial vehicle (UAV). By collecting multi-dimensional flight state and environmental disturbance parameters, a straight flight dynamic model and simulation environment that can accurately describe the motion characteristics and environmental impact of the UAV are constructed. Combining multi-objective fusion reward functions and safety constraints such as path accuracy, flight stability and energy efficiency, a reinforcement learning algorithm is used to conduct efficient and safe autonomous interactive training in the simulation environment. This enables the UAV to autonomously learn a robust and adaptable straight flight control strategy through data-driven methods. Ultimately, in actual flight, high-precision, high-stability and energy-saving autonomous straight flight control is achieved, significantly reducing dependence on accurate models and effectively coping with complex environmental disturbances. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0022] Figure 1 This is a flowchart illustrating the UAV linear flight control method provided in this application.
[0023] Figure 2 This is a schematic diagram of the structure of the UAV linear flight control device provided in this application.
[0024] Figure 3 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0026] The following is combined Figure 1 This application describes the method for controlling the straight flight of an unmanned aerial vehicle (UAV).
[0027] Figure 1 This is one of the flowcharts illustrating the UAV linear flight control method provided in this application, such as... Figure 1 As shown, the method includes the following: Step 101: Collect the flight status parameters and environmental interference parameters of the UAV. The flight status parameters include at least one of the following: three-dimensional position, flight speed, heading angle, side offset, and attitude angle.
[0028] Among them, flight status parameters can be used to describe the flight status of the UAV. Flight status parameters include at least one of three-dimensional position, flight speed, heading angle, side offset, and attitude angle.
[0029] The three-dimensional position refers to the coordinates (x, y, z) of the UAV in three-dimensional space, usually with reference to the Earth coordinate system or a local coordinate system.
[0030] The flight speed is the velocity components (vx, vy, vz) or the resultant velocity of the UAV along the three coordinate axes.
[0031] The heading angle is the horizontal angle between the direction of the UAV's nose and a reference direction (such as due north), used to describe the flight direction.
[0032] The side offset is the vertical distance between the actual position of the UAV and the preset straight path, which can be used to reflect the path tracking accuracy.
[0033] The attitude angles include roll, pitch, and yaw, which describe the rotational state of the UAV itself.
[0034] Among them, environmental interference parameters refer to external factors that affect the flight of UAVs, such as wind speed, airflow disturbance, temperature change, and air pressure fluctuation.
[0035] In this embodiment of the application, the flight status parameters and environmental interference parameters of the UAV can be collected in real time using airborne sensors (such as GPS, IMU, barometer, optical flow sensor) and environmental interference parameters can be obtained through environmental sensors (such as anemometer) or meteorological data interface.
[0036] Step 102: Based on the flight state parameters and environmental disturbance parameters, construct the UAV linear flight dynamics model, linear flight constraints, and fusion reward function.
[0037] Among them, the straight flight dynamics model is a mathematical model built on the kinematics and dynamics equations of the UAV, used to describe the dynamic relationship between state vectors (such as position, velocity, attitude) and control inputs (such as throttle, control surface deflection).
[0038] Among them, straight-line flight constraints are physical or safety limitations that must be met during flight, such as maximum tilt angle limits, speed ranges, and position boundaries.
[0039] Among them, the fusion reward function is a comprehensive reward calculation function designed by combining multiple objectives (such as accuracy, stability, and energy consumption), which is used to guide reinforcement learning agents to optimize control strategies.
[0040] In some embodiments, a linear flight dynamics model of the UAV is constructed based on flight state parameters and environmental disturbance parameters. This can be achieved by normalizing the collected flight state parameters and environmental disturbance parameters, scaling them to the range of [-1, 1] or [0, 1] to accelerate model convergence, and then fusing the normalized parameters into an initial state vector. For example: ,in For heading angle, Lateral offset, For roll angle, Let the pitch angle be the equation. Then, the dynamic equations of the UAV can be established based on the Newton-Euler equations and discretized into a state-space form: ; in, For the initial control strategy, These are environmental interference parameters.
[0041] Optionally, in some embodiments, the straight-line flight constraint includes at least one of the following: Attitude angle is less than or equal to attitude angle threshold; The rate of change of the attitude angle is less than or equal to the rate of change threshold; Lateral offset is less than or equal to the lateral offset threshold; The flight speed is less than or equal to the speed threshold.
[0042] In this embodiment of the application, to ensure the stability and safety of straight-line flight and to prevent the UAV from losing control or rolling due to excessively large attitude angles, it is necessary to set an upper limit for the absolute value of the attitude angles (roll angle, pitch angle). For example, in a straight-line tracking task, the roll angle threshold is set to 15° and the pitch angle threshold is set to 10°, without any restrictions.
[0043] In this embodiment of the application, to prevent the UAV from experiencing severe shaking or excessive structural stress due to sudden changes in control commands, it is necessary to limit the rate of attitude change. For example, the roll angle change rate threshold is set to 30° / second, and no further restriction is imposed.
[0044] In this embodiment, the side offset is a key indicator for measuring the degree to which the UAV deviates from the predetermined straight path. To ensure the effectiveness and safety of the flight mission (such as avoiding collisions with obstacles outside the path), a maximum permissible deviation needs to be set. For example, in a power line inspection mission, a side offset threshold of 1.5 meters is set, and no further restriction is imposed.
[0045] In this embodiment of the application, the maximum flight speed of the UAV needs to be limited due to constraints on the structural strength of the UAV, the performance of its power system, or the safety regulations of the mission area. For example, when conducting straight-line mapping in a densely populated area, a maximum horizontal speed threshold of 8 meters per second is set, and no further restrictions are imposed. Optionally, in some embodiments, constructing a linear flight dynamics model of the UAV based on flight state parameters and environmental disturbance parameters may involve normalizing the flight state parameters and environmental disturbance parameters; fusing the normalized flight state parameters and environmental disturbance parameters to obtain an initial state vector; and constructing a linear flight dynamics model based on the initial state vector, wherein the linear flight dynamics model is used to describe the mapping relationship between the initial state vector and the initial control strategy.
[0046] In other words, in this embodiment of the application, the collected flight state parameters and environmental interference parameters are normalized, and the values of each parameter are uniformly scaled to a set range to eliminate the differences between different physical dimensions and facilitate subsequent model calculations. Then, all the normalized parameters are fused into a multi-dimensional vector in a preset order as an initial state vector describing the current state of the UAV, which may include flight states such as position, speed, heading angle, and side offset, as well as normalized interference factors.
[0047] Then, based on the initial state vector, a dynamic model of UAV linear flight can be constructed. The model describes the dynamic behavior of UAV in linear flight missions through kinematic and dynamic equations. The model establishes the mapping relationship between the initial state vector and the initial control strategy (such as attitude command) and considers the influence of environmental disturbances on state transition, providing a benchmark mathematical model for subsequent simulation environment construction and reinforcement learning training.
[0048] Optionally, in some embodiments, the fusion reward function is: Reward value = w1 × Accuracy reward + w2 × Stability reward + w3 × Energy consumption reward; Among them, w1, w2, and w3 are preset weighting coefficients. The accuracy bonus is negatively correlated with the absolute value of the side offset and the absolute value of the heading angle deviation, respectively. The stability bonus is negatively correlated with the absolute values of the roll angle and pitch angle, and the energy consumption bonus is negatively correlated with the magnitude change of the control strategy.
[0049] In this embodiment, the accuracy reward is negatively correlated with the absolute value of the lateral deviation of the UAV from the predetermined path and the absolute value of the heading angle deviation from the desired heading. That is, the greater the deviation, the more reward points are deducted, so as to guide the UAV to accurately track the straight path. The stability reward is negatively correlated with the absolute values of the roll angle and pitch angle that affect flight stability, encouraging the UAV to maintain a level flight attitude. The energy consumption reward is negatively correlated with the magnitude of the control command change, penalizing overly frequent or large control actions, so as to optimize energy efficiency.
[0050] Therefore, in this embodiment, the three reward components can be weighted and summed according to the importance of the task requirements: the influence of each component is adjusted by preset weight coefficients w1, w2, w3 (e.g., w1=0.5, w2=0.3, w3=0.2) to obtain the final reward value, which serves as the feedback signal for the reinforcement learning algorithm in each training step; the design of this fusion function enables the UAV to simultaneously optimize tracking accuracy, flight stability, and energy consumption during the learning process, thereby achieving multi-target collaborative control.
[0051] Step 103: Construct a linear flight simulation environment based on the UAV linear flight dynamics model.
[0052] Among them, the straight-line flight simulation environment can simulate the physical state changes of a UAV under a given control command (such as the desired pitch angle) and calculate the corresponding flight results.
[0053] For example, you can build a simulation environment using Python and a physics engine. In this environment, you can define a quadcopter drone model, input a control command to "increase the pitch angle by 5 degrees," and the simulation engine will calculate how the drone will accelerate forward based on the dynamics model and output the new position, speed, and other states.
[0054] Step 104: Based on reinforcement learning algorithms, straight flight constraints, and fusion reward functions, conduct interactive training between the UAV and the straight flight simulation environment to determine the straight flight control strategy.
[0055] Interactive training refers to the process of repeated data exchange between the intelligent agent (drone) and the simulation environment.
[0056] Optionally, in some embodiments, controlling the UAV to interactively train with a straight-line flight simulation environment to determine a straight-line flight control strategy can involve selecting the current control strategy based on the first state vector at the current moment, applying the current control strategy to the simulation environment to determine the current reward value and the second state vector at the next moment; storing empirical data in an experience replay buffer, wherein the empirical data includes: the first state vector, the current control strategy, the current reward value, and the second state vector; determining the empirical data for batch training scale from the experience replay buffer; updating the UAV's strategy network parameters based on the empirical data for batch training scale; dynamically adjusting the initial exploration rate so that it gradually decays to a preset exploration rate as the number of training steps increases; and when a preset convergence condition is met, using the updated strategy network parameters as the straight-line flight control strategy.
[0057] The first state vector is the state at the current time t (such as position, velocity, attitude, etc.); the second state vector is the new state obtained at the next time t+1 after the control strategy is applied to the environment.
[0058] The current control strategy refers to the control action (such as the desired roll and pitch angle commands) output by the policy network (or exploration strategy) based on the first state vector at the current moment.
[0059] The current reward value refers to the immediate reward calculated based on the fusion reward function after the control action is applied to the environment. It quantifies the comprehensive performance of the action in terms of accuracy, stability, and energy consumption.
[0060] In this context, experience data refers to a data tuple generated from a single interaction. It records a complete "state-action-feedback-result" process and serves as the foundational sample for training the policy network.
[0061] The experience replay buffer is a first-in-first-out storage pool used to accumulate and store large amounts of historical experience data.
[0062] The batch training size refers to the number of empirical data points randomly drawn from the empirical replay buffer each time to update the network (Batch Size), such as 128 or 256 data points.
[0063] Here, "variables" refers to the learnable variables that constitute the policy neural network (usually an Actor network), including the weights and biases of each layer. Updating these parameters is the process of optimizing the control policy itself.
[0064] The exploration rate is a probability value (often denoted by ε) used to control whether the agent "explores" (randomly tries new actions) or "utilizes" (executes actions currently considered optimal) during training. It is usually high initially and decreases as training progresses.
[0065] The preset convergence condition is an objective standard for determining when training can be terminated, such as reaching the maximum number of training steps, or the average reward value of the most recent N rounds has stabilized (fluctuation is less than a certain threshold).
[0066] Among them, the straight-line flight control strategy is the final and solidified strategy network after training, which can map any flight state into optimized control commands.
[0067] Optionally, in some embodiments, the preset convergence condition is any one or a combination of the following: the number of training steps reaches a training step threshold; the average fluctuation value among multiple reward values determined by N consecutive rounds of training is less than a preset fluctuation threshold, where N is a positive integer.
[0068] In this embodiment, at each training step, the algorithm selects and executes a current control policy based on the first state vector at the current moment, combined with a dynamically adjusted exploration rate (e.g., random exploration with probability ε, or using the policy network with probability 1-ε). This action is input into a straight-line flight simulation environment, which deduces the second state vector at the next moment based on the dynamics model and calculates the current reward value based on the fusion reward function. The resulting complete empirical data is immediately stored in the experience replay buffer to accumulate data samples for subsequent learning.
[0069] In this embodiment, the algorithm periodically (e.g., after every 4 interaction steps) randomly samples a small batch of empirical data (e.g., 256 data points) from the experience replay buffer. Using this batch of samples, the loss is calculated and the UAV's policy network parameters are updated via backpropagation and gradient descent. This random sampling from historical experience breaks the temporal correlation between data points, significantly improving learning stability and data efficiency. Simultaneously, the exploration rate gradually decreases according to a predetermined plan (e.g., linear decay) as the number of training steps increases, eventually approaching a low, preset exploration rate, achieving a smooth transition from extensive exploration to refined utilization.
[0070] In this embodiment, preset convergence conditions are continuously monitored during training. For example, it is determined whether the total number of training steps has reached the upper limit, or the average reward of the most recent N rounds (e.g., 100 rounds) of training is calculated. If the fluctuation is less than a very small preset threshold, it indicates that the strategy performance has stabilized. Once any convergence condition is met, the training cycle terminates. At this point, the strategy network parameters, after training and iterative optimization with massive amounts of data, are finally saved and solidified. This set of parameters becomes the final deployable straight-line flight control strategy capable of achieving high-precision and stable straight-line flight.
[0071] Step 105: Control the UAV to fly in a straight line according to the straight flight control strategy.
[0072] In this embodiment, after obtaining the trained straight-line flight control strategy, it is first deployed to the UAV's flight control system or onboard computing unit. During the execution of the straight-line flight mission, the flight control system collects the UAV's flight state parameters (such as three-dimensional position, velocity, attitude angle, and side offset) in real time, and calculates the optimal control commands (such as the desired roll and pitch angle corrections) based on the neural network model embedded in the control strategy, using the current state vector as input. These commands are then sent to the underlying attitude and speed controller (such as a PID controller) and converted into actual execution signals such as motor speed or control surface deflection, thereby accurately and stably driving the UAV to fly along the preset straight path and automatically performing robust compensation when encountering environmental disturbances. This application provides a method for controlling the straight flight of an unmanned aerial vehicle (UAV). By collecting multi-dimensional flight state and environmental disturbance parameters, a straight flight dynamic model and simulation environment that can accurately describe the UAV's motion characteristics and environmental impact are constructed. The method combines a multi-objective fusion reward function and safety constraints, such as path accuracy, flight stability, and energy efficiency, and uses reinforcement learning algorithms to conduct efficient and safe autonomous interactive training in the simulation environment. This enables the UAV to autonomously learn a robust and adaptable straight flight control strategy through data-driven methods. Ultimately, in actual flight, this achieves high-precision, high-stability, and energy-saving autonomous straight flight control, significantly reducing dependence on precise models and effectively coping with complex environmental disturbances.
[0073] The following describes the UAV linear flight control device provided in this application. The UAV linear flight control device described below can be referred to in correspondence with the UAV linear flight control method described above.
[0074] Figure 2 This is a schematic diagram of the structure of the UAV linear flight control device provided in this application, as shown below. Figure 2 As shown, the device includes the following: The acquisition module 201 is used to acquire flight status parameters and environmental interference parameters of the UAV. The flight status parameters include at least one of three-dimensional position, flight speed, heading angle, side offset, and attitude angle. The first construction module 202 is used to construct the UAV's linear flight dynamics model, linear flight constraints, and fusion reward function based on flight state parameters and environmental disturbance parameters. The second building module 203 is used to build a straight flight simulation environment based on the UAV straight flight dynamics model; Training module 204 is used to interactively train the UAV and the straight flight simulation environment based on reinforcement learning algorithm, straight flight constraints and fusion reward function, so as to determine the straight flight control strategy. The control module 205 is used to control the drone to fly in a straight line according to the straight-line flight control strategy.
[0075] According to the unmanned aerial vehicle (UAV) linear flight control device provided in this application, the first building module 202 is specifically used for: Normalize the flight status parameters and environmental disturbance parameters; The normalized flight state parameters and environmental disturbance parameters are fused to obtain the initial state vector; Based on the initial state vector, a linear flight dynamics model is constructed, which describes the mapping relationship between the initial state vector and the initial control strategy.
[0076] According to the UAV linear flight control device provided in this application, the fusion reward function is as follows: Reward value = w1 × Accuracy reward + w2 × Stability reward + w3 × Energy consumption reward; Among them, w1, w2, and w3 are preset weighting coefficients. The accuracy bonus is negatively correlated with the absolute value of the side offset and the absolute value of the heading angle deviation, respectively. The stability bonus is negatively correlated with the absolute values of the roll angle and pitch angle, and the energy consumption bonus is negatively correlated with the magnitude change of the control strategy.
[0077] According to the unmanned aerial vehicle (UAV) linear flight control device provided in this application, the training module 204 is specifically used for: The current control strategy is selected based on the first state vector at the current moment, and the current control strategy is applied to the simulation environment to determine the current reward value and the second state vector at the next moment. The experience data is stored in the experience replay buffer, which includes: the first state vector, the current control policy, the current reward value, and the second state vector. Empirical data used to determine batch training size from the experience replay buffer; The strategy network parameters of the UAV are updated based on empirical data from the batch training scale. The initial exploration rate is dynamically adjusted so that it gradually decreases to the preset exploration rate as the number of training steps increases; When the preset convergence conditions are met, the updated policy network parameters are used as the straight-line flight control policy.
[0078] According to the unmanned aerial vehicle (UAV) linear flight control device provided in this application, the preset convergence condition is any one or a combination of the following: The number of training steps has reached the training step threshold; The average fluctuation value among multiple reward values determined by N consecutive training rounds is less than a preset fluctuation threshold, where N is a positive integer.
[0079] According to the unmanned aerial vehicle (UAV) linear flight control device provided in this application, the linear flight constraint conditions include at least one of the following: Attitude angle is less than or equal to attitude angle threshold; The rate of change of the attitude angle is less than or equal to the rate of change threshold; Lateral offset is less than or equal to the lateral offset threshold; The flight speed is less than or equal to the speed threshold.
[0080] This application provides a UAV linear flight control device that, by collecting multi-dimensional flight state and environmental disturbance parameters, constructs a linear flight dynamics model and simulation environment that accurately describes the UAV's motion characteristics and environmental impact. It combines a multi-objective fusion reward function with safety constraints, including path accuracy, flight stability, and energy efficiency, and employs reinforcement learning algorithms for efficient and safe autonomous interactive training within the simulation environment. This enables the UAV to autonomously learn robust and adaptable linear flight control strategies through a data-driven approach. Ultimately, in actual flight, it achieves high-precision, high-stability, and energy-efficient autonomous linear flight control, significantly reducing reliance on precise models and effectively coping with complex environmental disturbances. Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a UAV linear flight control method, which includes: Collect flight status parameters and environmental interference parameters of the UAV. The flight status parameters include at least one of three-dimensional position, flight speed, heading angle, side offset and attitude angle. Based on flight state parameters and environmental disturbance parameters, a dynamic model of UAV straight flight, straight flight constraints, and fusion reward function are constructed. A linear flight simulation environment based on a UAV linear flight dynamics model; Based on reinforcement learning algorithms, straight flight constraints, and fusion reward functions, the UAV is interactively trained with a straight flight simulation environment to determine the straight flight control strategy. The drone is controlled to fly in a straight line according to the straight-line flight control strategy.
[0081] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0082] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the UAV linear flight control method provided by the above methods, the method including: Collect flight status parameters and environmental interference parameters of the UAV. The flight status parameters include at least one of three-dimensional position, flight speed, heading angle, side offset and attitude angle. Based on flight state parameters and environmental disturbance parameters, a dynamic model of UAV straight flight, straight flight constraints, and fusion reward function are constructed. A linear flight simulation environment based on a UAV linear flight dynamics model; Based on reinforcement learning algorithms, straight flight constraints, and fusion reward functions, the UAV is interactively trained with a straight flight simulation environment to determine the straight flight control strategy. The drone is controlled to fly in a straight line according to the straight-line flight control strategy.
[0083] Furthermore, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the UAV linear flight control method provided by the methods described above, the method comprising: Collect flight status parameters and environmental interference parameters of the UAV. The flight status parameters include at least one of three-dimensional position, flight speed, heading angle, side offset and attitude angle. Based on flight state parameters and environmental disturbance parameters, a dynamic model of UAV straight flight, straight flight constraints, and fusion reward function are constructed. A linear flight simulation environment based on a UAV linear flight dynamics model; Based on reinforcement learning algorithms, straight flight constraints, and fusion reward functions, the UAV is interactively trained with a straight flight simulation environment to determine the straight flight control strategy. The drone is controlled to fly in a straight line according to the straight-line flight control strategy.
[0084] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0085] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A method for controlling the linear flight of an unmanned aerial vehicle (UAV), characterized in that, include: Collect flight status parameters and environmental interference parameters of the UAV, wherein the flight status parameters include at least one of three-dimensional position, flight speed, heading angle, side offset and attitude angle; Based on the flight state parameters and the environmental disturbance parameters, a dynamic model of UAV straight flight, straight flight constraints, and a fusion reward function are constructed. A linear flight simulation environment constructed based on the aforementioned UAV linear flight dynamics model; Based on reinforcement learning algorithms, the straight flight constraints, and the fusion reward function, the UAV is interactively trained with the straight flight simulation environment to determine the straight flight control strategy. The drone is controlled to fly in a straight line according to the described straight-line flight control strategy.
2. The method according to claim 1, characterized in that, The step of constructing a linear flight dynamics model for the UAV based on the flight state parameters and the environmental disturbance parameters includes: The flight state parameters and the environmental disturbance parameters are normalized. The normalized flight state parameters and the environmental disturbance parameters are fused to obtain the initial state vector; Based on the initial state vector, the linear flight dynamics model is constructed, wherein the linear flight dynamics model is used to describe the mapping relationship between the initial state vector and the initial control strategy.
3. The method according to claim 1, characterized in that, The fusion reward function is: Reward value = w1 × accuracy reward + w2 × stability reward + w3 × energy consumption reward; Wherein, w1, w2, and w3 are preset weighting coefficients. The accuracy reward is negatively correlated with the absolute value of the side offset and the absolute value of the heading angle deviation, respectively. The stability reward is negatively correlated with the absolute values of the roll angle and pitch angle, and the energy consumption reward is negatively correlated with the magnitude change of the control strategy.
4. The method according to claim 1, characterized in that, The process of controlling the UAV to interact with the linear flight simulation environment for training, in order to determine the linear flight control strategy, includes: Based on the first state vector at the current moment, the current control strategy is selected and applied to the simulation environment to determine the current reward value and the second state vector at the next moment. The experience data is stored in the experience replay buffer, wherein the experience data includes: the first state vector, the current control strategy, the current reward value, and the second state vector; The empirical data from the empirical replay buffer determines the batch training size; The policy network parameters of the UAV are updated based on the empirical data of the batch training scale. The initial exploration rate is dynamically adjusted so that it gradually decreases to the preset exploration rate as the number of training steps increases; When the preset convergence condition is met, the updated policy network parameters are used as the straight-line flight control policy.
5. The method according to claim 4, characterized in that, The preset convergence condition is any one or a combination of the following: The number of training steps reaches the threshold. The average fluctuation value among multiple reward values determined by N consecutive training rounds is less than a preset fluctuation threshold, where N is a positive integer.
6. The method according to claim 1, characterized in that, The straight-line flight constraint includes at least one of the following: The attitude angle is less than or equal to the attitude angle threshold; The rate of change of the attitude angle is less than or equal to the rate of change threshold; The side offset is less than or equal to the side offset threshold; The flight speed is less than or equal to the speed threshold.
7. A linear flight control device for unmanned aerial vehicles (UAVs), characterized in that, include: The acquisition module is used to acquire flight status parameters and environmental interference parameters of the UAV, wherein the flight status parameters include at least one of three-dimensional position, flight speed, heading angle, side offset, and attitude angle; The first construction module is used to construct a UAV linear flight dynamics model, linear flight constraints, and fusion reward function based on the flight state parameters and the environmental interference parameters. The second construction module is used to construct a straight-flight simulation environment based on the UAV's straight-flight dynamics model; The training module is used to interactively train the UAV and the straight flight simulation environment based on the reinforcement learning algorithm, the straight flight constraints, and the fusion reward function, so as to determine the straight flight control strategy. The control module is used to control the UAV to fly in a straight line according to the straight-line flight control strategy.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the UAV linear flight control method as described in any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the UAV linear flight control method as described in any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the UAV linear flight control method as described in any one of claims 1 to 6.