Flight control method, aircraft, and computer-readable storage medium

Abstract

The application relates to the technical field of aircrafts, and provides a flight control method, an aircraft and a computer readable storage medium, which can realize end-to-end output from multi-modal sensor original data to flight control instructions, does not need to manually design intermediate modules such as perception features, pose estimation, map construction and trajectory planning, avoids serial conduction and amplification of errors at all levels in a traditional modular pipeline, can improve the robustness and precision of flight in a complex dynamic environment, simultaneously, comprehensively utilizes multi-modal sensor data through a cross-modal fusion mechanism, can realize redundancy complementation of the multi-modal sensors, and through a unified world model framework and task-independent latent space representation learning, a single model can execute multiple flight tasks such as hovering keeping, waypoint navigation, dynamic obstacle avoidance, high-speed crossing and autonomous landing, without the need to respectively design and train special models for different tasks, and the generalization and adaptation ability to diversified task scenes is significantly improved.

Description

Technical Field

[0001] This application relates to the field of aircraft technology, and in particular to a flight control method, an aircraft, and a computer-readable storage medium. Background Technology

[0002] An aircraft is an aerial vehicle capable of autonomous or remote-controlled flight, widely used in logistics, urban air transport, emergency rescue, and aerial surveying. With increasingly complex and diverse application scenarios, higher demands are placed on the autonomous flight capabilities of aircraft. They must not only achieve safe flight in open, unknown, and dynamic environments but also possess the ability to quickly adapt to various sudden disturbances. However, current technologies for autonomous flight control primarily rely on modular pipeline architectures, with information exchange between modules through manually defined intermediate representations. This can easily lead to the accumulation of errors at each level, making it difficult to achieve robust end-to-end autonomous flight in complex dynamic environments. Summary of the Invention

[0003] This application provides a flight control method, an aircraft, and a computer-readable storage medium that can improve the robustness and accuracy of autonomous flight of an aircraft in complex dynamic environments.

[0004] Firstly, the flight control method provided in this application includes: The aircraft's multimodal sensor data is acquired, and the multimodal sensor data is feature-encoded and fused across modes using a multimodal encoder to obtain a latent space representation. Obtain historical deterministic states and historical stochastic states, and based on the historical deterministic states, historical stochastic states, and latent space representations, obtain the current deterministic state through the deterministic state branch of the world model; Based on the current deterministic state, the current stochastic state is obtained through the stochastic state branch of the world model; Based on the current deterministic state and the current stochastic state, a sequence of future potential states is obtained through a world model; Based on the future potential state sequence, the flight control commands of the aircraft's actuators are obtained through the policy network; The actuators are driven to move according to flight control commands in order to control the aircraft to fly along the trajectory indicated by the future potential state sequence; The multimodal encoder, world model, and policy network are pre-trained jointly.

[0005] Secondly, the aircraft provided in this application includes: ontology; The drive mechanism is used to propel the aircraft into flight. Memory, used to store computer programs; A processor is used to execute computer programs to implement the flight control method provided in this application.

[0006] Thirdly, the computer-readable storage medium provided in this application stores a computer program, which, when executed by a processor, implements the flight control method provided in this application.

[0007] The flight control scheme provided in this application acquires multimodal sensor data from the aircraft and performs feature encoding and cross-modal fusion on the multimodal sensor data using a multimodal encoder to obtain a latent space representation. It then acquires historical deterministic and historical random states, and based on these historical deterministic and random states and the latent space representation, obtains the current deterministic state through the deterministic state branch of the world model, and then obtains the current random state through the random state branch of the world model, thereby constructing an environmental dynamic representation in the latent space that considers both deterministic and random dynamics. Furthermore, based on the current deterministic and random states, it obtains a future latent state sequence through the world model, and obtains flight control commands for the aircraft actuators through a policy network based on the future latent state sequence. This commands drive the actuators to fly along the trajectory indicated by the future latent state sequence. The multimodal encoder, world model, and policy network are pre-trained end-to-end. Therefore, this application achieves end-to-end output from raw multimodal sensor data to flight control commands, eliminating the need for manual design of intermediate modules such as sensing features, pose estimation, map building, and trajectory planning. This avoids the serial propagation and amplification of errors at each stage in traditional modular pipelines, improving the robustness and accuracy of flight in complex dynamic environments. Simultaneously, by comprehensively utilizing multimodal sensor data through a cross-modal fusion mechanism, redundant complementarity of multimodal sensors can be achieved. Furthermore, through a unified world model framework and task-independent latent space representation learning, a single model can perform various flight tasks such as hovering, waypoint navigation, dynamic obstacle avoidance, high-speed crossing, and autonomous landing, without the need to design and train dedicated models for different tasks, significantly improving the generalization and adaptability to diverse task scenarios. Attached Figure Description

[0008] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a schematic flowchart of the flight control method provided in the embodiments of this application; Figure 2This is a schematic diagram of the unified end-to-end world model framework provided in the embodiments of this application; Figure 3 yes Figure 1 Detailed process diagram of S120; Figure 4 yes Figure 1 Detailed process diagram of S130; Figure 5 yes Figure 1 Detailed process diagram of S140; Figure 6 This is a schematic diagram of the flight control device provided in the embodiments of this application; Figure 7 This is a schematic diagram of the structure of the aircraft provided in the embodiments of this application. Detailed Implementation

[0010] To make the technical problems, technical solutions, and beneficial effects solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0011] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0012] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0013] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0014] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0015] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in still other embodiments" appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0016] This application provides a flight control method, a flight control device, an aircraft, a computer-readable storage medium, and a computer program product, wherein the flight control method can be executed by the flight control device or by an aircraft integrating the flight control device.

[0017] The flight control method includes: acquiring multimodal sensor data of the aircraft, and performing feature encoding and cross-modal fusion on the multimodal sensor data through a multimodal encoder to obtain a latent space representation; acquiring historical deterministic states and historical random states, and obtaining the current deterministic state through the deterministic state branch of the world model based on the historical deterministic state, historical random state, and latent space representation; obtaining the current random state through the random state branch of the world model based on the current deterministic state; obtaining a future latent state sequence through the world model based on the current deterministic state and the current random state; obtaining flight control commands for the aircraft's actuators through a policy network based on the future latent state sequence; and driving the actuators to act according to the flight control commands to control the aircraft to fly according to the trajectory indicated by the future latent state sequence; wherein the multimodal encoder, the world model, and the policy network are jointly trained in advance.

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a flight control method provided in an embodiment of this application, as shown below. Figure 1 As shown, the flow of this flight control method can be as follows: In S110, multimodal sensor data of the aircraft is acquired, and feature encoding and cross-modal fusion of the multimodal sensor data are performed by a multimodal encoder to obtain a latent space characterization.

[0020] Please refer to Figure 2 In this application embodiment, a unified end-to-end world model framework consisting of a pre-trained multimodal encoder, a world model, and a policy network is provided. This enables the aircraft to extract a compact latent space representation from the raw sensor data of multimodal sensors through the multimodal encoder. Based on this latent space representation, the world model predicts the dynamic evolution of the environment, providing the policy network with high-quality decision-making basis—a sequence of future latent states. The policy network then generates flight control commands, achieving an end-to-end closed loop from perception to decision-making. This directly drives the actuators to complete complex maneuvering tasks, encompassing general flight capabilities such as autonomous hovering, high-speed crossing, dynamic obstacle avoidance, and autonomous landing. The flight control method provided in this application will be described in detail below.

[0021] It should be noted that the aircraft is equipped with sensors of various modes, including but not limited to: RGB cameras for acquiring visual images, inertial measurement units (IMUs) for acquiring inertial measurement data, and navigation and positioning units for acquiring the aircraft's position, velocity, and attitude information. The navigation and positioning unit may include a Global Positioning System (GPS) or Real-Time Kinematic (RTK) receiver, a magnetometer, and a barometric altimeter.

[0022] In this embodiment of the application, after the aircraft acquires multimodal sensor data, it performs feature encoding on each modal sensor data through each modal encoding branch of the multimodal encoder to obtain the initial feature representation corresponding to each mode.

[0023] For example, for RGB images, the aircraft inputs RGB image frames with a resolution of H×W×3 into an image modality coding branch based on a cascaded Convolutional Neural Network (CNN) and a multi-layer Vision Transformer (ViT). First, multi-scale spatial feature maps are extracted through the CNN backbone network (such as EfficientNetV2 or ConvNeXt). Then, the spatial feature maps are flattened into a sequence, and the global spatial dependencies are captured through the multi-layer Vision Transformer. Finally, the initial feature representation of the fixed-dimensional image modality is output.

[0024] For IMU inertial data, the aircraft inputs IMU measurements (such as triaxial angular velocity and triaxial acceleration) into the inertial mode coding branch based on residual multilayer perceptron (MLP) for feature encoding to obtain an initial feature representation of the inertial modes in fixed dimensions.

[0025] For navigation and positioning data, the aircraft inputs navigation and positioning data (such as GPS / RTK three-dimensional position, three-dimensional velocity, magnetometer heading angle, and barometric altimeter altitude) into the MLP-based navigation mode coding branch for feature encoding to obtain an initial feature representation of the navigation mode with fixed dimensions.

[0026] As described above, after encoding the initial feature representations of each modality, the aircraft further fuses these initial feature representations through the cross-modal fusion module of the multimodal encoder to generate a unified latent space representation. It should be noted that the latent space refers to the low-dimensional, compact representation space obtained by compressing the high-dimensional raw observation data through the encoder; the vectors in the latent space are latent states or latent space representations. By uniformly encoding the high-dimensional raw data of multiple modalities into a compact latent space, the computational complexity of subsequent world modeling and policy planning can be effectively reduced, while redundant information and noise in the raw data are filtered out.

[0027] In the process of cross-modal fusion, the aircraft can map the initial feature representations of each modality to a unified dimension through the learnable modal embedding matrix in the cross-modal fusion module, capture the interaction relationship between modalities through the self-attention mechanism, adaptively assign attention weights to each modality, and output a unified latent space representation after fusion.

[0028] Through the above cross-modal fusion mechanism, the multimodal encoder can learn to adjust the confidence weights when different modalities degrade. For example, when visual features are blurred (such as in low light or motion blur), it automatically increases the weight of IMU features to maintain the accuracy of pose estimation. Thus, by comprehensively utilizing the rich semantic information of RGB vision, the high-frequency inertial information of the IMU, and the global positioning information of the navigation unit, multi-sensor redundancy complementarity is achieved, ensuring the consistency and reliability of latent space representation when different modalities degrade.

[0029] In S120, historical deterministic states and historical stochastic states are obtained, and the current deterministic state is obtained through the deterministic state branch of the world model based on the historical deterministic states, historical stochastic states and latent space representations.

[0030] It should be noted that the historical deterministic state and the historical random state are two components of the latent state obtained by the spacecraft through the world model at the previous time step. They respectively carry the deterministic evolution and the random uncertainty of the spacecraft's dynamics at the previous moment. After obtaining the latent space representation at the current moment, the spacecraft uses the historical deterministic state, the historical random state, and the latent space representation at the current moment as inputs to the deterministic state branch of the world model. It selectively forgets and retains information from the historical deterministic state, retaining long-term dependency information useful for predicting the current and future states (such as the spacecraft's position, velocity, attitude, etc., which evolve continuously over time), and forgetting outdated or irrelevant information, thereby updating and obtaining the current deterministic state.

[0031] In S130, the current stochastic state is obtained from the stochastic state branch of the world model based on the current deterministic state.

[0032] It should be noted that the current deterministic state characterizes the deterministic and predictable aspects of the aircraft's dynamics. However, the real flight environment also includes uncertainties such as gust disturbances, aerodynamic mode switching, and sensor noise. These stochastic dynamics cannot be fully expressed by the deterministic state alone. Therefore, based on the current deterministic state, the aircraft also models the uncertainties in the environment through the stochastic state branch of the world model to obtain the current stochastic state, in order to supplement the characterization of unpredictable stochastic disturbances and mode switching uncertainties in the real flight environment.

[0033] In S140, the sequence of future potential states is obtained through the world model based on the current deterministic state and the current stochastic state.

[0034] In this embodiment, the aircraft performs a multi-step forward imagination rollout in the latent space of the world model based on the current deterministic state and the current random state. That is, starting from the current latent state (i.e., the current deterministic state and the current random state), it simulates the future state evolution in the latent space to obtain the sequence of latent states over a period of time in the future, without actually interacting with the environment, thereby avoiding the huge computational overhead caused by direct planning in the original high-dimensional observation space.

[0035] In S150, flight control commands for the aircraft's actuators are obtained through a policy network based on the future potential state sequence.

[0036] Actuators are components on an aircraft that translate flight control commands into physical actions, such as rotor motors and servos.

[0037] In this embodiment, the aircraft inputs the future potential state sequence into the policy network, and the policy network parses the potential trajectory intention and maps it into specific flight control commands. The flight control commands may include continuous control quantities such as thrust commands and three-axis angular rate commands to drive the rotor motors and servo motors to move.

[0038] It should be noted that the policy network takes the future potential state sequence as input, rather than just the current potential state. It can make forward-looking decisions based on the prediction of environmental evolution over a period of time in the future, and generate flight control commands that match the future trajectory, thus avoiding the lag caused by purely reactive control.

[0039] In S160, actuators are driven to move according to flight control commands in order to control the aircraft to fly along the trajectory indicated by the future potential state sequence.

[0040] As mentioned above, after receiving flight control commands, the aircraft sends them to actuators such as rotor motors and servos, driving the rotor motors and servos to generate corresponding thrust and torque, enabling the aircraft to accurately track the trajectory indicated by the future potential state sequence, and generate new observation data and state information, forming a closed-loop feedback, providing a basis for state estimation and planning in the next cycle, and realizing an end-to-end closed loop of perception, prediction and decision-making.

[0041] As can be seen from the above, the flight control scheme provided in this application obtains multimodal sensor data of the aircraft and performs feature encoding and cross-modal fusion on the multimodal sensor data through a multimodal encoder to obtain a latent space representation; then it obtains historical deterministic states and historical random states, and based on the historical deterministic states, historical random states and latent space representation, it obtains the current deterministic state through the deterministic state branch of the world model, and then obtains the current random state through the random state branch of the world model based on the current deterministic state, so as to construct an environmental dynamic representation in the latent space that takes into account both deterministic dynamics and random dynamics; then based on the current deterministic state and the current random state, it obtains the future latent state sequence through the world model, and obtains the flight control commands of the aircraft actuators through the policy network based on the future latent state sequence, thereby driving the actuators to act and control the aircraft to fly according to the trajectory indicated by the future latent state sequence. The multimodal encoder, world model and policy network are obtained through pre-end end-to-end joint training. Therefore, this application achieves end-to-end output from raw multimodal sensor data to flight control commands, eliminating the need for manual design of intermediate modules such as sensing features, pose estimation, map building, and trajectory planning. This avoids the serial propagation and amplification of errors at each stage in traditional modular pipelines, improving the robustness and accuracy of flight in complex dynamic environments. Simultaneously, by comprehensively utilizing multimodal sensor data through a cross-modal fusion mechanism, redundant complementarity of multimodal sensors can be achieved. Furthermore, through a unified world model framework and task-independent latent space representation learning, a single model can perform various flight tasks such as hovering, waypoint navigation, dynamic obstacle avoidance, high-speed crossing, and autonomous landing, without the need to design and train dedicated models for different tasks, significantly improving the generalization and adaptability to diverse task scenarios.

[0042] Alternatively, in one embodiment, please refer to Figure 3 Based on historical deterministic states, historical stochastic states, and latent space representations, the current deterministic state is obtained through the deterministic state branch of the world model, including: In S1210, based on the historical random state and latent space representation, the candidate latent state is obtained by selectively forgetting and retaining information on the historical deterministic state through deterministic state branches. In S1220, the current deterministic state is obtained by fusing the candidate hidden state and the historical deterministic state through the deterministic state branch.

[0043] In this embodiment, the deterministic state branch is implemented using a gated loop unit, which contains two gating mechanisms: a reset gate and an update gate. Accordingly, when obtaining the current deterministic state through the deterministic state branch of the world model based on historical deterministic states, historical random states, and latent space representations, the aircraft inputs the historical deterministic states, historical random states, and latent space representations into the deterministic state branch. The reset gate determines which information in the historical deterministic states needs to be ignored, and the update gate determines which information in the historical deterministic states needs to be retained and transmitted to the current moment. This achieves selective forgetting and retention of information from the historical deterministic states, resulting in candidate latent states. Subsequently, the aircraft uses the update gate to perform a weighted fusion of the candidate latent states and the historical deterministic states, thereby obtaining the current deterministic state that reflects the dynamics of the environment at the current moment.

[0044] Understandably, since aircraft dynamics have strong continuity and determinism in a short period of time, selectively forgetting and retaining information about historical deterministic states through gated cyclic units can avoid interference from irrelevant historical information on the estimation of the current state, while retaining long-term dependency information that is useful for predicting future states, thereby improving the accuracy of dynamic predictions by the world model.

[0045] Alternatively, in one embodiment, please refer to Figure 4 Based on the current deterministic state, the current stochastic state is obtained through the stochastic state branch of the world model, including: In S1310, based on the current deterministic state, the first probability distribution of the discrete latent variable and the second probability distribution of the continuous latent variable are estimated through stochastic state branches. In S1320, discrete latent variable samples are obtained by sampling the first probability distribution through random state branch, and continuous latent variable samples are obtained by sampling the second probability distribution. In S1330, the current random state is obtained by fusing discrete latent variable samples and continuous latent variable samples through random state branching.

[0046] It should be noted that, in this embodiment, the stochastic state branch of the world model is implemented by a combination of a stochastic encoder and a stochastic decoder, while simultaneously using discrete and continuous latent variables to perform hybrid modeling of the stochastic dynamics of the environment. The discrete latent variables are used to capture the multimodal characteristics of the environment, following a categorical distribution (i.e., the first probability distribution); the continuous latent variables are used to capture the continuous random changes in the environment, following a Gaussian distribution (i.e., the second probability distribution).

[0047] The process involves the aircraft first estimating a first probability distribution of discrete latent variables based on its current deterministic state using a random encoder. This first distribution characterizes the multimodal properties of the environment. Simultaneously, a second probability distribution of continuous latent variables is estimated to characterize the continuous random changes in the environment. Then, discrete latent variable samples are sampled from the first probability distribution, and continuous latent variable samples are sampled from the second probability distribution. Finally, the aircraft fuses the discrete and continuous latent variable samples using a random encoder to construct a hybrid representation, thereby obtaining the current random state that simultaneously reflects the multimodal switching and continuous random disturbances of the environment. The method for fusing the discrete and continuous latent variable samples is not specifically limited here; splicing, weighted summation, or other feature fusion methods can be flexibly selected according to the actual application scenario.

[0048] The introduction of discrete latent variables enables the world model to naturally capture the multimodal characteristics of the environment, such as the different aerodynamic modes faced by the aircraft and the switching between the ground effect region and the free flight region. When the aircraft is flying close to the ground, it is in the ground effect region, and its aerodynamic characteristics are significantly different from those in the free flight region. This mode switching can be characterized by different values ​​of discrete latent variables; while the continuous random changes experienced by the aircraft, such as gust disturbances and sensor noise, are characterized by continuous latent variables.

[0049] It should be noted that, in the embodiments of this application, the random state branch estimates the probability distribution based on the current deterministic state; that is, the distribution parameters of both discrete and continuous latent variables are conditioned on the current deterministic state. Therefore, the random state branch can fully utilize the deterministic dynamic information output by the deterministic state branch, enabling the random state to maintain consistency with deterministic dynamics while characterizing environmental uncertainty.

[0050] Alternatively, in one embodiment, please refer to Figure 5 Based on the current deterministic state and the current stochastic state, a sequence of future potential states is obtained through a world model, including: In S1410, based on the current random state, the action distribution is obtained through the policy network, and actions are sampled from the action distribution; In S1420, based on the action, the current deterministic state, and the current stochastic state, the predicted deterministic state and the predicted stochastic state for the next time step are obtained through the world model. In S1430, based on the predicted deterministic state and the predicted stochastic state of the next time step, the simulation continues iteratively through the policy network and the world model until the preset number of simulation steps is reached. In S1440, the future latent state sequence is obtained based on the predicted deterministic state and predicted random state at each time step during the iterative deduction process.

[0051] In this embodiment of the application, the policy network includes an Actor-Critic architecture.

[0052] When obtaining the future potential state sequence through the world model based on the current deterministic state and the current random state, the aircraft starts from the current potential state and performs a pre-set number of imaginary deduction steps H steps in the potential space of the world model in combination with the policy network.

[0053] In the first imagining step, the aircraft, based on its current random state, outputs an action distribution through the Actor network in the policy network and samples the action to be executed. Then, based on this action and the current latent state (including the current deterministic state and the current random state), the aircraft predicts the latent state for the next time step using a world model; that is, it predicts the deterministic state and the random state. In the second imagining step, the aircraft uses the predicted deterministic state and the predicted random state obtained in the previous step as the current input and follows the same process as in the first imagining step to predict the latent state for the next time step. This iterative process yields a sequence of latent states for the next H steps. Furthermore, the Critic network in the policy network is responsible for evaluating the value function of the latent states.

[0054] It should also be noted that while predicting the next potential state, the world model also outputs three prediction results in parallel: observation reconstruction, reward prediction, and termination prediction. Observation reconstruction is used to decode RGB images, IMU predictions, and navigation data predictions through a multimodal decoder, which is used to measure the world model's ability to reconstruct the original observations. Reward prediction is used to provide reward signals for strategy optimization. Termination prediction is used to determine whether the flight mission has been successfully completed or encountered a dangerous state (such as collision or instability), so as to promptly detect and avoid dangerous trajectories during the imaginative simulation process.

[0055] Understandably, by performing imaginative simulations in latent space, the enormous computational overhead of direct planning in the original high-dimensional observation space is avoided, significantly improving planning efficiency. For example, when performing dynamic obstacle avoidance tasks, multi-rotor aircraft can predict the evolution of multiple candidate trajectories in the next few seconds through imaginative simulations in latent space, and select the flight strategy most conducive to safe obstacle avoidance and mission completion accordingly.

[0056] Optionally, in one embodiment, the flight control method provided in this application further includes: Multiple candidate action trajectories are sampled from the action distribution obtained from the policy network; The cumulative reward for each candidate action trajectory is obtained by forward extrapolating the world model. The weighting coefficient of each candidate action trajectory is determined based on the cumulative reward of each candidate action trajectory, and the candidate action trajectories are weighted and fused based on the weighting coefficient of each candidate action trajectory to obtain a reference action sequence. The parameters of the policy network are updated based on the reference action sequence.

[0057] Based on the above embodiments, the embodiments of this application further reuse the action distribution generated during the deduction process to optimize the planning capability of the policy network.

[0058] The aircraft first samples N candidate action trajectories from the action distribution obtained from the policy network. Each trajectory contains an action sequence of H time steps. Then, it performs a rapid extrapolation of each candidate trajectory in the world model to calculate its cumulative reward. Then, it calculates the expected value of the optimal action sequence through an exponential weighting method, that is, it determines the weighting coefficient of each trajectory based on the cumulative reward of each candidate trajectory. The candidate trajectory with the higher the cumulative reward is assigned a higher weight. Finally, it performs a weighted fusion of all candidate trajectories to obtain a reference action sequence.

[0059] The weighting coefficient can be calculated as: w_i = exp((1 / λ)·J_i) / Σ_j exp((1 / λ)·J_j), where J_i is the cumulative reward of the i-th candidate trajectory, λ is the temperature coefficient used to control the selection pressure. The smaller λ is, the more inclined to select candidate trajectories with high cumulative rewards. The larger λ is, the less selection pressure there is for each candidate trajectory. The cumulative reward J_i is obtained by discounting and summing the rewards of each future time step through the discount factor γ.

[0060] After obtaining the reference action sequence, the aircraft uses it to guide the policy network in policy optimization, that is, updating the parameters of the policy network based on the reference action sequence. For example, it can minimize the difference between the action distribution output by the policy network and the reference action sequence, thereby gradually improving the quality of the actions directly output by the policy network.

[0061] By combining trajectory optimization with policy network output, the optimality and robustness of the planned trajectory are further improved while ensuring high-frequency real-time control.

[0062] Optionally, in one embodiment, before acquiring the multimodal sensor data of the aircraft, the method further includes: Obtain a simulation sample set and jointly train the world model, multimodal encoder, and policy network based on the simulation sample set; Obtain a real sample set and fine-tune the jointly trained world model and multimodal encoder based on the real sample set.

[0063] Based on the above embodiments, this application provides an optional end-to-end joint training scheme for the world model, multimodal encoder, and policy network.

[0064] End-to-end joint training refers to constructing a joint loss function that unifies the multimodal observation reconstruction loss, dynamic prediction KL divergence loss, reward prediction loss, and policy optimization loss within an end-to-end framework for joint optimization. This training consists of two phases: the first phase involves large-scale pre-training in diverse simulation environments, utilizing domain randomization strategies to enhance the model's robustness; the second phase uses a small amount of real flight data for latent space adaptive fine-tuning, achieving efficient simulation-to-real (Sim-to-Real) transfer by minimizing the distribution differences between simulation and real data in the latent space.

[0065] The joint loss function includes the following loss terms: observation reconstruction loss, which measures the accuracy of the world model decoder in reconstructing observations, including L2 loss for image reconstruction, mean squared error (MSE) loss for IMU prediction, and MSE loss for navigation data prediction; KL divergence loss, which measures the difference between the prior and posterior distributions of the random state, driving the world model to learn accurate dynamic predictions; reward prediction loss, which measures the accuracy of the world model in predicting rewards; and policy optimization loss, which is optimized in latent space imagination using an Actor-Critic framework. The weights of each loss term are dynamically adjusted through an adaptive multi-task weight balancing mechanism (such as uncertainty weighting or GradNorm) to ensure that each loss term converges at a balanced rate.

[0066] In the simulation pre-training phase, diverse training scenarios can be constructed within a high-fidelity aircraft simulation environment. This environment can be any one or a combination of Isaac Sim, Gazebo+Rotors, or AirSim. Training scenarios include various environmental types (e.g., forest, city, indoor, mountainous), diverse lighting and weather conditions (e.g., sunny, cloudy, dusk, foggy), and various flight missions (e.g., hovering, waypoint navigation, dynamic obstacle avoidance, high-speed passage through narrow passages, mobile platform landing). Simultaneously, a domain randomization strategy is employed to randomly sample physical and environmental parameters. Domain randomization refers to randomizing physical parameters and environmental conditions during simulation training, enabling the model to learn robust representations insensitive to parameter changes, thereby improving its ability to transfer simulation to reality. For example, the aircraft mass is randomized to ±15% of the nominal value, the moment of inertia to ±20% of the nominal value, the aerodynamic drag coefficient to ±30% of the nominal value, the motor thrust coefficient to ±10% of the nominal value, wind speed is randomly sampled within the range of 0 to 8 m / s, wind direction is randomly selected, and the noise standard deviation of the gyroscope and accelerometer, GPS positioning noise, and the exposure, white balance, and motion blur parameters of the RGB camera are all randomized, thereby collecting a large-scale simulation sample set in the simulation environment. At the beginning of each training round, each parameter is independently sampled from the above-mentioned randomized distribution, forcing the world model to learn a robust representation that is insensitive to physical parameters and environmental conditions.

[0067] In addition, the aircraft can acquire real sample sets, such as images, inertial measurement and navigation data recorded during real flight through airborne sensors, and construct real sample sets containing multimodal sensor data through time synchronization and spatial alignment processing.

[0068] After the simulation pre-training converges, the aircraft uses a real sample set consisting of a small amount of real flight data to fine-tune the jointly trained world model and multimodal encoder. The real data is used to correct the distribution offset between the simulation and reality, so that the model can adapt to the dynamic characteristics and sensor noise distribution of the real environment, thereby improving the transfer performance from simulation to reality.

[0069] In other embodiments, the training of the world model, multimodal encoder, and policy network described above can also be performed by a computing cluster or cloud server, with the aircraft only responsible for collecting and uploading data. After training is completed, the model parameters are distributed to the aircraft for deployment.

[0070] In addition, in other embodiments, the world model, multimodal encoder, and policy network can be optimized by operator fusion and quantization to reduce the inference latency and memory usage of the world model, multimodal encoder, and policy network on the aircraft's onboard embedded platform.

[0071] Through the aforementioned end-to-end joint training, the main computational burden of policy optimization is shifted from real-world interaction to latent space simulation, significantly improving sample efficiency. Simultaneously, the self-supervised learning mechanism of the world model enables it to extract effective environmental representations from unlabeled raw sensor data, reducing reliance on large-scale manually labeled data.

[0072] Optionally, in one embodiment, fine-tuning the jointly trained world model and multimodal encoder based on a real sample set includes: Based on the real sample set, the parameters of the random state branches in the jointly trained world model, the parameters of the last layer in the multimodal encoder, and the parameters for cross-modal fusion are updated until the preset stopping condition is met.

[0073] This application further illustrates how to fine-tune the jointly trained world model and multimodal encoder based on a real sample set.

[0074] The aircraft can freeze most of the parameters of the deterministic state branches and policy networks in the world model, and only fine-tune the stochastic state branches and specific layers of the multimodal encoder. By minimizing the maximum mean discrepancy (MMD) between the simulation data and the real data in the latent space, the latent space distribution of the real data is aligned to the distribution that the simulation pre-trained model has adapted to, thereby achieving efficient Sim-to-Real knowledge transfer and avoiding high-risk large-scale training from scratch on a real aircraft.

[0075] Since the deterministic state branch is responsible for propagating deterministic long-range temporal information, while the stochastic state branch is responsible for characterizing the uncertainty of the environment, by freezing the deterministic state branch and only fine-tuning the relevant parameters of the stochastic state branch and the multimodal encoder, it is possible to retain the deterministic dynamic knowledge obtained from simulation pre-training while enabling the stochastic state branch to quickly adapt to the stochastic characteristics of the real environment. Preset stopping conditions can be, for example, that the MMD distance is less than a preset threshold, or that the number of fine-tuning iterations reaches a preset number.

[0076] It should be noted that the fine-tuning mechanism in this application does not rely on precise system identification or exhaustive enumeration of domain randomization strategies. Instead, it achieves efficient knowledge transfer through latent space distribution alignment, requiring very little real flight data. This avoids high-risk, large-scale training from scratch on real aircraft, thus improving transfer efficiency and safety.

[0077] Optionally, in one embodiment, the flight control method provided in this application further includes: Obtain the distribution characteristics of random states within the time window in which the current random state is located; If the difference between the distribution characteristics and the reference distribution reaches a preset threshold, an abnormal sample set is constructed based on the random state within the time window. The parameters of the random state branch are updated based on the set of anomalous samples.

[0078] It should be noted that, in the embodiments of this application, the aircraft maintains a sliding window (the length of which can be a preset number of steps) to statistically analyze the distribution characteristics of random states in the latent space in real time. These distribution characteristics may include the mean and covariance of the random states.

[0079] When the difference between the distribution characteristics of the random states within the window and the reference distribution exceeds a preset threshold, it indicates a significant shift in the latent space state distribution, meaning that the environmental conditions of the aircraft have changed significantly. At this point, fast adaptive fine-tuning is triggered. The difference between the distribution characteristics and the reference distribution can be measured using Kullback-Leibler divergence.

[0080] Specifically, when the difference between the detected distribution characteristics and the reference distribution exceeds a preset threshold, the aircraft can construct a small batch of anomalous sample sets using the state-observation-reward tuples in the current sliding window during the forward inference interval. This anomalous sample set is then used to perform a small number (e.g., 1 to 5 steps) of gradient updates on the stochastic state branch parameters of the world model, enabling the latent state distribution of the world model to quickly adapt to new environmental conditions, such as sudden strong winds, load changes, or aerodynamic aging. It should be noted that this online adaptive mechanism has extremely low computational overhead; the computation time for a single update is less than 2 milliseconds, and it does not affect the real-time performance of the main control loop.

[0081] Through the above online adaptive mechanism, the aircraft can automatically adapt to environmental changes during flight without retraining or manual parameter adjustment, thereby improving the long-term operational reliability and safety of the aircraft in real complex environments.

[0082] To facilitate better implementation of the above flight control methods, this application also provides a corresponding flight control device. The meanings of the terms used are the same as in the flight control methods described above; for specific implementation details, please refer to the descriptions in the above method embodiments.

[0083] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the flight control device provided in an embodiment of this application, as shown below. Figure 6 As shown, the flight control device may include a data representation module 210, a determination branch module 220, a random branch module 230, a future prediction module 240, a command generation module 250, and a command execution module 260, wherein... The data characterization module 210 is used to acquire multimodal sensor data of the aircraft and perform feature encoding and cross-modal fusion on the multimodal sensor data through a multimodal encoder to obtain latent space characterization; The branching module 220 is used to obtain historical deterministic states and historical random states, and to obtain the current deterministic state through the deterministic state branch of the world model based on the historical deterministic states, historical random states and latent space representations. The random branch module 230 is used to obtain the current random state based on the current deterministic state through the random state branch of the world model; Future prediction module 240 is used to obtain a sequence of future potential states based on the current deterministic state and the current stochastic state through a world model; The instruction generation module 250 obtains the flight control instructions of the aircraft's actuators through a policy network based on the future potential state sequence. The command execution module 260 drives the actuators to move according to the flight control commands, so as to control the aircraft to fly according to the trajectory indicated by the future potential state sequence; The multimodal encoder, world model, and policy network are pre-trained jointly.

[0084] Optionally, in one embodiment, when the current deterministic state is obtained through the deterministic state branch of the world model based on historical deterministic states, historical stochastic states, and latent space representations, the branch determination module 220 is used to: Based on the historical random state and latent space representation, candidate latent states are obtained by selectively forgetting and retaining information about the historical deterministic state through deterministic state branches. The current deterministic state is obtained by fusing candidate hidden states and historical deterministic states through deterministic state branches.

[0085] Optionally, in one embodiment, when obtaining the current stochastic state through the stochastic state branch of the world model based on the current deterministic state, the stochastic branch module 230 is used to: Based on the current deterministic state, estimate the first probability distribution of discrete latent variables and the second probability distribution of continuous latent variables through stochastic state branching; Discrete latent variable samples are obtained by sampling the first probability distribution through random state branches, and continuous latent variable samples are obtained by sampling the second probability distribution. The current random state is obtained by fusing discrete latent variable samples and continuous latent variable samples through random state branching.

[0086] Optionally, in one embodiment, when obtaining a sequence of future potential states using a world model based on the current deterministic state and the current random state, the future prediction module 240 is used to: Based on the current random state, the action distribution is obtained through the policy network, and actions are sampled from the action distribution; Based on the action, the current deterministic state, and the current stochastic state, the predicted deterministic state and the predicted stochastic state for the next time step are obtained through the world model. Based on the predicted deterministic state and predicted stochastic state of the next time step, the simulation continues iteratively through the policy network and world model until the preset number of simulation steps is reached. Based on the predicted deterministic state and predicted random state at each time step during the iterative deduction process, the future potential state sequence is obtained.

[0087] Optionally, in one embodiment, the flight control device provided in this application further includes a strategy optimization module, used for: Multiple candidate action trajectories are sampled from the action distribution obtained from the policy network; The cumulative reward for each candidate action trajectory is obtained by forward extrapolating the world model. The weighting coefficient of each candidate action trajectory is determined based on the cumulative reward of each candidate action trajectory, and the candidate action trajectories are weighted and fused based on the weighting coefficient of each candidate action trajectory to obtain a reference action sequence. The parameters of the policy network are updated based on the reference action sequence.

[0088] Optionally, in one embodiment, the flight control device provided in this application further includes a model training module, used for: Obtain a simulation sample set and jointly train the world model, multimodal encoder, and policy network based on the simulation sample set; Obtain a real sample set and fine-tune the jointly trained world model and multimodal encoder based on the real sample set.

[0089] Optionally, in one embodiment, when fine-tuning the jointly trained world model and multimodal encoder based on a real sample set, the model training module is used to: Based on the real sample set, the parameters of the random state branches in the jointly trained world model, the parameters of the last layer in the multimodal encoder, and the parameters for cross-modal fusion are updated until the preset stopping condition is met.

[0090] Optionally, in one embodiment, the flight control device provided in this application further includes an online optimization module, used for: Obtain the distribution characteristics of random states within the time window in which the current random state is located; If the difference between the distribution characteristics and the reference distribution reaches a preset threshold, an abnormal sample set is constructed based on the random state within the time window. The parameters of the random state branch are updated based on the set of anomalous samples.

[0091] As can be seen from the above, the flight control device provided in this application realizes end-to-end output from raw data of multimodal sensors to flight control commands, without the need for manual design of intermediate modules such as perception features, pose estimation, map building, and trajectory planning. This avoids the serial propagation and amplification of errors at each level in traditional modular pipelines, and can improve the robustness and accuracy of flight in complex dynamic environments. At the same time, by comprehensively utilizing multimodal sensor data through a cross-modal fusion mechanism, it can achieve redundancy and complementarity of multimodal sensors. Furthermore, through a unified world model framework and task-independent latent space representation learning, a single model can perform a variety of flight tasks such as hovering, waypoint navigation, dynamic obstacle avoidance, high-speed crossing, and autonomous landing, without the need to design and train dedicated models for different tasks, which significantly improves the generalization and adaptability to diverse task scenarios.

[0092] It should be noted that specific limitations regarding the flight control device can be found in the limitations regarding the flight control method described above, and will not be repeated here. Each module in the aforementioned flight control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the aircraft's processor in hardware form or independently of it, or stored in the aircraft's memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0093] In one embodiment, an aircraft is provided whose internal structure can be as follows: Figure 7 As shown, the aircraft includes a main body and a memory 310, a processor 320, a power supply 330, a sensor 340, a communication module 350, a positioning module 360, a drive mechanism 370, and a bus 380, all mounted on the main body. The processor 320 is coupled to the memory 310, power supply 330, sensor 340, communication module 350, positioning module 360, and drive mechanism 370 via the bus 380.

[0094] Memory 310 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM). The RAM can be directly read and written by the processor 320 and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data. The RAM may include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DRAM), etc.

[0095] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 320. Non-volatile memory can include disk storage devices and flash memory.

[0096] The processor 320 may include at least one secure core that runs flight safety-related services and at least one non-secure core that runs non-flight safety-related services.

[0097] The memory 310 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 320. The one or more computer programs include multiple instructions, which, when executed by the processor 320, can implement the flight control method provided in this application, such as: The aircraft's multimodal sensor data is acquired, and the multimodal sensor data is feature-encoded and fused across modes using a multimodal encoder to obtain a latent space representation. Obtain historical deterministic states and historical stochastic states, and based on the historical deterministic states, historical stochastic states, and latent space representations, obtain the current deterministic state through the deterministic state branch of the world model; Based on the current deterministic state, the current stochastic state is obtained through the stochastic state branch of the world model; Based on the current deterministic state and the current stochastic state, a sequence of future potential states is obtained through a world model; Based on the future potential state sequence, the flight control commands of the aircraft's actuators are obtained through the policy network; The actuators are driven to move according to flight control commands in order to control the aircraft to fly along the trajectory indicated by the future potential state sequence; The multimodal encoder, world model, and policy network are pre-trained jointly.

[0098] In other embodiments, the aircraft may also include an external memory interface for connecting to an external memory to expand the aircraft's storage capacity.

[0099] Power source 330 is used to supply power to the aircraft. In one embodiment of this application, power source 330 may include any one or more power supply devices of the type of battery, fuel generator, solar power generation module, wind power generation module, etc.

[0100] Sensor 340 is used to acquire information for the aircraft, such as environmental information and aircraft movement information. In one embodiment of this application, sensor 340 may include a binocular camera, and may also include one or more sensors of the type such as lidar and infrared sensors.

[0101] The communication module 350 is used to enable communication between the aircraft and other devices. In one embodiment of this application, the communication module 350 can interact with other devices via wired and / or wireless communication. The aforementioned wireless communication may include one or more combinations of communication methods such as Bluetooth communication, Wi-Fi communication, and Near Field Communication (NFC).

[0102] The positioning module 360 ​​is used to determine the position of the aircraft. In some embodiments of this application, the positioning module 360 ​​may include one or more of the following types of positioning modules: Global Navigation Satellite System (GNSS), Inertial Navigation System, Real-time Kinematic (RTK) Carrier Phase Differential System, etc.

[0103] The drive mechanism 370 is used to drive the aircraft to fly and may include a rotor and a rotor motor. In some embodiments of this application, the drive mechanism 370 can realize the flight function of the aircraft under the control of the processor 320.

[0104] Bus 380 is used at least to provide a channel for communication between the memory 310, processor 320, power supply 330, sensor 340, communication module 350, positioning module 360, and drive mechanism 370 in the aircraft.

[0105] The main body of the aircraft can be a cargo hold or a crew cabin, used to carry goods or personnel.

[0106] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the aircraft. In other embodiments of this application, the aircraft may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0107] For example, the actual product form of the aircraft can be an electric multi-rotor aircraft, with a compound rotor as its drive mechanism and a crew cabin for carrying personnel as its main body.

[0108] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0109] This application also provides a computer program product, which includes a computer program that, when executed by a processor, causes the processor to implement the steps in the flight control method provided in this application.

[0110] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0111] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

[0112] It should be noted that when the above embodiments of this application are applied to specific products or technologies, user-related data is involved, and user permission or consent must be obtained. Furthermore, the collection, use, and processing of such data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

Claims

1. A flight control method, characterized in that, include: The multimodal sensor data of the aircraft is acquired, and the multimodal sensor data is feature-encoded and fused across modes by a multimodal encoder to obtain a latent space characterization. Obtain historical deterministic states and historical random states, and based on the historical deterministic states, the historical random states, and the latent space representation, obtain the current deterministic state through the deterministic state branch of the world model; Based on the current deterministic state, the current random state is obtained through the random state branch of the world model; Based on the current deterministic state and the current random state, a sequence of future potential states is obtained through the world model; Based on the future potential state sequence, flight control commands for the aircraft's actuators are obtained through a policy network; The actuators are driven to move according to the flight control commands to control the aircraft to fly along the trajectory indicated by the future potential state sequence; The multimodal encoder, the world model, and the policy network are jointly trained in advance.

2. The flight control method according to claim 1, characterized in that, The step of obtaining the current deterministic state through the deterministic state branch of the world model based on the historical deterministic state, the historical stochastic state, and the latent space representation includes: Based on the historical random state and the latent space representation, the candidate hidden state is obtained by selectively forgetting and retaining information about the historical deterministic state through the deterministic state branch. The current deterministic state is obtained by fusing the candidate hidden state and the historical deterministic state through the deterministic state branch.

3. The flight control method according to claim 1, characterized in that, The step of obtaining the current stochastic state through the stochastic state branch of the world model based on the current deterministic state includes: Based on the current deterministic state, the first probability distribution of the discrete latent variable and the second probability distribution of the continuous latent variable are estimated through the random state branch; Discrete latent variable samples are obtained by sampling the first probability distribution through the random state branch, and continuous latent variable samples are obtained by sampling the second probability distribution; The current random state is obtained by fusing the discrete latent variable samples and the continuous latent variable samples through the random state branch.

4. The flight control method according to claim 1, characterized in that, The step of obtaining the future potential state sequence through the world model based on the current deterministic state and the current random state includes: Based on the current random state, the action distribution is obtained through the policy network, and actions are sampled from the action distribution; Based on the action, the current deterministic state, and the current random state, the predicted deterministic state and the predicted random state for the next time step are obtained through the world model. Based on the predicted deterministic state and the predicted random state at the next time step, the simulation continues iteratively through the policy network and the world model until the preset number of simulation steps is reached. The future potential state sequence is obtained based on the predicted deterministic state and the predicted random state at each time step during the iterative deduction process.

5. The flight control method according to claim 4, characterized in that, Also includes: Multiple candidate action trajectories are sampled from the action distribution obtained from the policy network; The cumulative reward of each candidate action trajectory is obtained by forward extrapolating the world model. The weighting coefficient of each candidate action trajectory is determined based on the cumulative reward of each candidate action trajectory, and the candidate action trajectories are weighted and fused based on the weighting coefficient of each candidate action trajectory to obtain a reference action sequence. The parameters of the policy network are updated based on the reference action sequence.

6. The flight control method according to claim 1, characterized in that, Before acquiring the multimodal sensor data of the aircraft, the process also includes: Obtain a simulation sample set, and jointly train the world model, the multimodal encoder, and the policy network based on the simulation sample set; Obtain a real sample set, and fine-tune the jointly trained world model and the multimodal encoder based on the real sample set.

7. The flight control method according to claim 6, characterized in that, The fine-tuning of the jointly trained world model and the multimodal encoder based on the real sample set includes: Based on the real sample set, the parameters of the random state branches in the jointly trained world model, the parameters of the last layer in the multimodal encoder, and the parameters for cross-modal fusion are updated until a preset stopping condition is met.

8. The flight control method according to claim 1, characterized in that, Also includes: Obtain the distribution characteristics of the random states within the time window in which the current random state is located; If the difference between the distribution characteristics and the reference distribution reaches a preset threshold, then an abnormal sample set is constructed based on the random state within the time window; The parameters of the random state branch are updated based on the set of abnormal samples.

9. An aircraft, characterized in that, include: ontology; A drive mechanism is used to drive the body to fly; Memory, used to store computer programs; A processor for executing the computer program to implement the flight control method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the flight control method according to any one of claims 1 to 8.

Images