Hypersonic aircraft simulation flight test system based on multi-modal large model

Abstract

The present invention belongs to the technical field of hypersonic aircraft testing, and relates to a hypersonic aircraft simulation flight test system based on a multimodal large model, including a data acquisition and fusion system and a control system. Through the data acquisition and fusion system and the cross-modal attention mechanism, efficient alignment and feature representation of time series data, environmental data and historical case libraries collected by the ground test system are achieved, significantly improving the accuracy of data acquisition and analysis. Combined with the hardware system, the validity of the test parameters is ensured, shortening the aircraft design and verification cycle. The control system dynamically adjusts the test parameters through natural language instructions, and the model generates structured control parameter vectors through semantic understanding, thereby improving the performance and stability of the aircraft control system, reducing the complexity of test operations, and enhancing the user experience.

Description

Technical Field

[0001] The present invention belongs to the technical field of hypersonic aircraft testing, and relates to a hypersonic aircraft simulation flight test system based on a multi-modal large model. Background Art

[0002] Hypersonic vehicle testing typically relies on a combination of wind tunnel testing and numerical simulation, with dynamic aerodynamic force measurement and flight attitude simulation being key verification steps. Traditional wind tunnel testing relies on a six-degree-of-freedom platform to simulate the vehicle's motion and attitude, collecting aerodynamic and attitude signals through a variety of sensors. However, this approach neglects the ability to simultaneously process heterogeneous data such as mechanical vibration, flow field changes, and control commands. This makes it difficult to achieve both high fidelity and real-time performance in dynamic testing.

[0003] The six-degree-of-freedom platform simulation system proposed in the patent "Method and Apparatus for Processing Aerodynamic Signals of Balances in Wind Tunnel Dynamic Tests of Variant Aircraft" (CN117909659A) can simulate aircraft attitude changes, but it does not address the dynamic matching of sensor time series data (such as the balance dynamic signal) with wind tunnel environmental parameters (such as flow velocity and pressure field). During experiments, mechanical vibrations and inertial interference contained in the balance signal are coupled in the time domain with changes in the wind tunnel flow field. Relying solely on single-mode filtering (such as traditional low-pass filtering) will result in residual interference or loss of effective signal, and errors will accumulate over the test time.

[0004] Although the patent "A Semi-physical Simulation System for Hypersonic Aircraft Navigation and Control Systems" (CN113658340B) can realize flight parameter collection and multi-subsystem collaboration (such as VR scene generation and dynamic pressure simulation), it lacks an embedded real-time analysis engine. The system obtains flight attitude angle, altitude and other parameters through the data acquisition module, and uses the VR virtual scene generation system for image processing and three-dimensional modeling. However, the simulation data needs to be exported to offline tools for post-processing, and it is impossible to generate real-time visual reports (such as aerodynamic time domain curves and control command tracking error heat maps) during the test.

[0005] The patent "A Simulation Test Platform and Control Method for Hypersonic Aircraft Assessment" (CN104182272B) proposes a simulation test platform and control method for hypersonic aircraft assessment, enabling comparison of control methods for hypersonic aircraft. However, as a software assessment and verification platform for control algorithms, it only covers simulation verification of known aircraft dynamic and kinematic models and related aerodynamic parameters, making it less useful for aircraft application testing. Summary of the Invention

[0006] This paper provides a hypersonic aircraft simulation flight test system based on a large multimodal model. This system aims to address the challenges of trajectory tracking and control under the complex multi-physics coupling effects of aircraft using advanced data processing techniques. The core of this system lies in the fusion and processing of multimodal data, combined with reinforcement learning control strategies, to enable intelligent debugging and analysis of hypersonic aircraft ground test systems. It primarily includes the following components:

[0007] (1) The system acquires multi-source data from aerodynamics, real-time multimodal perception, control strategy, etc. through the data acquisition unit. The multimodal fusion unit is responsible for preprocessing and feature extraction of this data to ensure data accuracy and consistency.

[0008] (2) In the multimodal fusion unit, advanced algorithms (such as the Transformer architecture) are used to align and fuse data from different sources. This module uses a large model architecture to integrate multimodal information to achieve comprehensive perception, prediction, and analysis of the aircraft's status.

[0009] (3) The system uses the fused multimodal data to reinforce the learning algorithm, optimize the control strategy, and input the optimized results into the control system. This improves the control performance and response speed during the aircraft test process and adapts to complex flight environments.

[0010] The technical solutions of the present invention are as follows:

[0011] A hypersonic aircraft simulation flight test system based on a multi-modal large model includes a data acquisition and fusion system and a control system, specifically as follows:

[0012] (1) Data acquisition and fusion system

[0013] The data acquisition and fusion system is a core component of hypersonic vehicle simulation testing. It collects, processes, and fuses multi-source data to support the implementation of dynamic control and optimization strategies. The system consists of a state acquisition unit and a multimodal fusion unit, each of which plays a unique role in data processing and system interaction.

[0014] Among them, the state acquisition unit is responsible for collecting the status information and environmental data of the aircraft in real time, including aerodynamic data ( 、 、 、 、 and ), the motion state of the aircraft ( 、 、 、 、 、 、 、 and ) and external environmental conditions. The collected data enters the multimodal fusion unit through a high-speed transmission interface, providing a basis for subsequent data processing and analysis.

[0015] The multimodal fusion unit is the core of the system, responsible for integrating data from various sources to support real-time control and optimization of hypersonic vehicles. This unit consists of three modules: a time series data encoding module, an image / video data encoding module, and a natural language and knowledge base data encoding module. The core algorithms and mathematical models of each module are as follows:

[0016] Time series data encoding module:

[0017] Based on the feature extraction requirements of the time series data collected by the aircraft data acquisition unit, a coding module based on the time convolutional network is designed. According to the physical characteristics of the hypersonic aircraft ground test, the time series sensor data is defined as a multi-parameter fusion representation of the flight motion state and aerodynamics, including aerodynamic data ( 、 、 、 、 and ), the motion state of the aircraft ( 、 、 、 、 、 、 、 and ), etc., assuming that the aircraft status time series data is , where Indicates from 1 to The sampling timing during this period, Indicates the time, is the dimension of the matrix, represents the set of real numbers. The temporal convolutional network is used to extract local features. The Rectified Linear Unit (ReLU) used in its core operation is an activation function widely used in deep learning. It is mathematically defined as , where Indicates input, Indicates selecting the maximum value. The encoding formula for time series data is:

[0018]

[0019] Where, Indicates the Layer output features, Expressed as The layer can learn the convolution kernel weights, represents the expansion coefficient, represents the width of the convolution kernel, Indicates the By using ReLU to perform element-by-element nonlinear transformations on the convolution results, the encoding module can extract the temporal dependency between the aircraft's motion state and aerodynamic parameters layer by layer, providing highly discriminative feature representations for subsequent control strategy generation.

[0020] Image / video data encoding module:

[0021] For the dynamic images and video data collected in the wind tunnel flow field test, local flow field features are extracted through block operation. Specifically, the input image tensor is evenly divided into non-overlapping image blocks, each of size pixels, represents the horizontal pixels of the input image, represents the vertical pixels of the input image, Indicates the horizontal and vertical pixel size after block division. The block division process is performed by the operator accomplish:

[0022]

[0023] Where, Indicates the total number of blocks, Denotes the set of real numbers. Define the matrix , where each image block The linear projection matrix Mapping to other dimensions , and superimpose position coding To preserve the spatial temporal relationship, the input vector of the flow field image encoder is formed:

[0024]

[0025] in, represents the projected vector, Represents position encoding. The core of the encoder is composed of a stack of multi-head self-attention (MHSA) and multi-layer perceptron (MLP) modules. First, the input Split into independent attention heads, and fuse global features by scaling dot product attention weights:

[0026]

[0027] right Process again and perform nonlinear transformation through two layers of fully connected networks to further extract features

[0028]

[0029] Where, represents the output of the MHSA attention mechanism, represents the output features of the image, Indicates normalization. Finally, the encoding output Cross-modal fusion with other data.

[0030] Natural language and knowledge base data encoding module:

[0031] This module is the semantic understanding and knowledge fusion unit of the aircraft ground simulation system. It is responsible for encoding natural language instructions and structured knowledge base data (such as aerodynamic parameter library, test case library) into a unified semantic vector space to achieve cross-modal semantic alignment. This module uses the BERT (Bidirectional Encoder Representations from Transformers) model to achieve semantic encoding of natural language instructions and structured knowledge. The input natural language instruction is defined as ,in Represents the i-th word in the text instruction. The embedding mechanism adds position information to each word and maps it into a vector of dimension D:

[0032]

[0033] Where, Represents the embedding matrix, which is input to the Transformer encoder to extract context-aware semantic features:

[0034]

[0035] in, Represents the output of the attention mechanism, aggregating the semantic information features of natural language instructions.

[0036] After completing the independent encoding of the three modalities of time series, image / video, and natural language, deep feature fusion is achieved through cross-modal interaction modeling. The fusion process is divided into two stages: feature splicing and attention alignment. The output features of each encoding module are dimensionally aligned and spliced ​​according to physical meaning to construct a joint feature matrix:

[0037] Realizing feature space through multi-head attention mechanism Alignment:

[0038] Where, , is a learnable parameter, is the hidden layer length.

[0039] The outputs of each attention head are concatenated and linearly transformed to generate the final fusion feature:

[0040]

[0041] in, is a learnable parameter, For fusion features.

[0042] (2) Control system

[0043] In the ground simulation test of hypersonic aircraft, the control system serves as the core hub, responsible for achieving high-precision dynamic simulation and real-time optimization of flight attitude.

[0044] Specifically, the control system uses servo motors to drive the active components of the ground test system (such as the moving belt, side panels, servos, and nozzles) to simulate the attitude changes of the aircraft in a near-ground state. This mechanism allows real-time adjustment of the aircraft's attitude, ensuring the dynamic consistency of test conditions. Furthermore, based on the multimodal data from the data acquisition and fusion system, the control system constructs a state-action reward function and utilizes a reinforcement learning algorithm to dynamically adjust the servo motor control strategy. For example, if flow separation is detected, causing increased model vibration, the algorithm automatically reduces the moving belt speed or adjusts the angle of attack. Furthermore, the system supports control parameter adjustment via natural language commands (such as "increase wind speed to 50 m / s"), combined with keyword extraction (such as "bad word" filtering) to ensure the reliability and safety of the commands.

[0045] In the high-fidelity flight control verification unit, the control system uses a general dynamics model to support attitude simulation and analysis under complex flight conditions. The multi-body dynamics model uses Lagrange equations to describe the motion behavior of the aircraft, taking into account the coupling effects of multiple forces such as aerodynamic forces, gravity, and inertia. This model can simulate the dynamic response of the aircraft under different flight conditions and provide theoretical support for attitude control strategies. The dynamics model used is as follows:

[0046]

[0047]

[0048]

[0049]

[0050] In the formula, the superscript “ " represents the time derivative of the variable, 、 and are speed, track inclination angle and track yaw angle respectively; 、 and are the displacements of the aircraft in the ground coordinate system; 、 and The aircraft orbits 、 and Angular velocity of the axis; 、 and are the pitch angle, roll angle, and yaw angle respectively. 、 and are the angle of attack, sideslip angle, and roll angle respectively; 、 、 are the gravitational constant, distance from the center of the earth, and mass, respectively. 、 and They are 、 and The moment of inertia of the shaft. 、 、 、 、 and They are lift, drag, side force, rolling moment, yaw moment and pitching moment respectively; all of the above state quantities can be obtained through the data acquisition system. Specifically, a six-component force balance system is used to synchronously measure the aerodynamic force and moment components of the aircraft, including lift ,resistance , lateral force and rolling moment , yaw moment and pitching moment , attitude angle and attitude angular velocity are measured by the inertial measurement unit, while the angle of attack and sideslip angle This measurement, combined with a five-hole probe or miniature wind vane sensor mounted on the aircraft's nose, senses the relative relationship between the incoming airflow direction and the aircraft's axis system and directly outputs instantaneous values. For highly dynamic conditions, dynamic compensation is also required, combining inertial navigation data with aerodynamic model predictions. For example, using angular velocity to correct for delay errors in the angle of attack sensor allows the model to reflect the aircraft's true behavior in real time under different flight conditions.

[0051] Through high-fidelity flight control verification, the control system can verify the effectiveness and robustness of different control strategies without conducting actual tests. This synergy between simulation and experimental control enables the aircraft attitude control system to be fully optimized and adjusted during the design phase, reducing the risks and costs of actual testing.

[0052] The control system builds a state-action reward function based on the reinforcement learning algorithm to dynamically optimize the control strategy of the servo motor. Specifically, the state space is composed of the fusion features Definition, and the action space corresponds to the control instructions , including the steering gear deflection and nozzle thrust adjustment. The reward function is designed to optimize tracking accuracy, energy efficiency and stability.

[0053] The double-delayed deep deterministic policy gradient (TD3) network is the core of the reinforcement learning strategy, accepting Input and output control instructions, the mathematical expression is:

[0054]

[0055] Where, Indicates TD3 network, Represents the output control instruction. The fusion feature Input TD3 strategy network and update control instructions , which can adjust the servo and nozzle in a short time. This instruction drives the servo and nozzle to perform dynamic adjustment through a real-time closed-loop mechanism. At the same time, the reward function works in conjunction with the TD3 network to evaluate the control instruction. The long-term impact on the system state (such as reducing attitude oscillation or suppressing torque overshoot) is achieved by iteratively optimizing the parameters of the strategy network, thereby realizing adaptive correction of the servo motor control strategy.

[0056] Beneficial effects of the present invention:

[0057] 1. Through a data acquisition and fusion system and a cross-modal attention mechanism, we achieve efficient alignment and feature representation of time series data, environmental data, and historical case libraries collected by the ground test system, significantly improving the accuracy of data acquisition and analysis. Combined with the hardware system, this ensures the validity of test parameters and shortens the aircraft design and verification cycle.

[0058] 2. The control system dynamically adjusts test parameters through natural language commands, and the model generates structured control parameter vectors through semantic understanding, improving the performance and stability of the aircraft control system, reducing the complexity of test operations, and enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1It is a technical block diagram of a hypersonic aircraft simulation flight test system based on a multi-modal large model of the present invention;

[0060] Figure 2 is the attitude angle change curve;

[0061] Figure 3 is the angular velocity variation curve;

[0062] Figure 4 is the aerodynamic force variation curve;

[0063] Figure 5 is the aerodynamic torque variation curve;

[0064] Figure 6 is the servo change curve;

[0065] Figure 7 is the nozzle angle change curve. DETAILED DESCRIPTION

[0066] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0067] like Figure 1 As shown, a hypersonic aircraft simulation flight test system based on a multi-modal large model of the present invention includes a data acquisition and fusion system and a control system. In this embodiment, high-precision sensors are used to collect multi-modal data such as aircraft attitude, acceleration, wind tunnel parameters, etc., and high-precision state information is generated in combination with a data fusion algorithm, and the controller performance is verified through a hardware-in-the-loop simulation platform. Among them, some parameters of the time series data encoding module are configured as follows: the time convolution network adopts a 4-layer structure, the width of each layer is 5, and the initial learning rate is set to 0.0001. Some parameters of the image / video data encoding module are set as follows: the input image resolution is , number of blocks It is 1024 blocks. Each image block is mapped to a 512-dimensional vector through a linear projection matrix, and a sinusoidal position code is superimposed. The MHSA contains 12 layers of Transformer layers, 8 attention heads in each layer, a hidden layer dimension of 512, and an MLP expansion ratio of 4. Some parameter configurations of the natural language and knowledge base data encoding module are as follows: the pre-trained BERT-base model (12 layers, 768-dimensional hidden layers, 12 attention heads) is used, the learning rate is set to 0.00005 during fine-tuning, and the maximum sequence length is 512. The TD3 algorithm parameters are set to 3-layer full connection (256-128-64) for both the Actor network and the Critic network, the learning rate is 0.0003, and the discount factor is 0.99. The above encoding module inputs the proposed fusion features into the TD3 policy gradient network to generate a servo deflection command to ensure that the aircraft returns to its initial state. In this embodiment, at the beginning of the experiment, the initial flight parameters are set: speed At Mach 5, the angle of attack The initial sideslip angle is 5 degrees. 0 degrees, initial roll angle The aircraft's sensors measured lift of 152.3 kN, drag of 98.7 kN, side force of 0 kN, roll moment of approximately 0.2 kN / m, pitch velocity of 0.1 degrees per second, and yaw velocity of 0 degrees per second, with a local velocity of 1700 meters per second. Severe wind conditions were simulated using the natural language command, "Start crosswind disturbance, last 5 seconds." A side gust caused the sideslip angle to decrease to 3.2 degrees and gradually decrease to 0, the side force to increase to 28.5 kN and gradually decrease to 0, and the yaw moment to decrease to 15.3 kN / m and gradually decrease to 0. High-speed cameras captured the local velocity dropping to 1650 meters per second. After the side gust, a multi-head attention mechanism aligned the time series, image, and language features to generate a joint feature matrix, which was then input into the reinforcement learning policy network. The time series data includes attitude angles and angular velocities, encoded as a 256-dimensional feature vector. The image data is encoded into a 512-dimensional feature vector, and the natural language instruction "start crosswind disturbance, last for 5 seconds" is encoded into a 768-dimensional semantic vector. Figure 2 shown.

[0068] from Figure 2 It can be seen that when wind measurement occurs, the pitch angle, yaw angle and roll angle of the aircraft all change to cope with the wind measurement interference, and return to a stable state after 100 seconds. Figure 3 is the change of attitude angular velocity. It can be seen that when disturbed by the side wind, the attitude angular velocity changes according to the control command and quickly returns to a stable state. Figure 4 and Figure 5 The corresponding aerodynamic force and aerodynamic moment change curves are shown in Figure 2. From this, it can be seen that the aircraft can quickly adjust to a stable state within 100 seconds after being disturbed by the crosswind. Figure 6 and Figure 7 These are the changing curves of the aircraft's servo and nozzle, respectively. It can be seen that when disturbed by the crosswind, the servo and nozzle adjust quickly, allowing the aircraft to return to a stable state at high-speed flight.

Claims

1. A hypersonic aircraft simulation flight test system based on a multi-modal large model, including a data acquisition and fusion system and a control system, characterized in that: The details are as follows: (1) Data acquisition and fusion system It consists of a state acquisition unit and a multimodal fusion unit; Among them, the state acquisition unit is responsible for collecting the status information and environmental data of the aircraft in real time, including aerodynamic data 、 、 、 、 and , the motion state of the aircraft 、 、 、 、 、 、 、 and and external environmental conditions; the collected data enters the multimodal fusion unit through a high-speed transmission interface; among them, 、 、 、 、 and They are lift, drag, side force, rolling moment, yaw moment and pitching moment respectively; 、 and are speed, track inclination angle and track yaw angle respectively; 、 and The aircraft orbits 、 and Angular velocity of the axis; 、 and are pitch angle, roll angle and yaw angle respectively; The multimodal fusion unit consists of the following three modules: time series data encoding module, image / video data encoding module, and natural language and knowledge base data encoding module. The mathematical models of each module are as follows: Time series data encoding module: The time series sensor data is defined as a multi-parameter fusion representation of flight motion state and aerodynamics, which includes aerodynamic data and time series data of aircraft motion state. Assume that the aircraft state time series data is , where Indicates from 1 to The sampling timing during this period, Indicates the time, is the dimension of the matrix, represents the set of real numbers; A temporal convolutional network is used to extract local features. The linear rectifier unit ReLU used in the operation is an activation function widely used in deep learning. It is mathematically defined as , where Indicates input, Indicates selecting the maximum value; the encoding formula for time series data is: ， Where, Indicates the Layer output features, Expressed as The layer can learn the convolution kernel weights, represents the expansion coefficient, represents the convolution kernel width, Indicates the The bias term of the layer; through the ReLU element-by-element nonlinear transformation of the convolution result, the encoding module extracts the temporal dependency between the aircraft motion state and aerodynamic parameters layer by layer, providing a highly discriminative feature representation for subsequent control strategy generation; Image / video data encoding module: The dynamic images and video data collected in the wind tunnel flow field test are used to extract local flow field features through block operation; specifically, the input image tensor is evenly divided into non-overlapping image blocks, each of size pixels, represents the horizontal pixels of the input image, represents the vertical pixels of the input image, Indicates the horizontal and vertical pixel size after block division; the block division process is performed by the operator accomplish: ， Where, Indicates the total number of blocks, Represents the set of real numbers; defines the matrix , where each image block The linear projection matrix Mapping to other dimensions , and superimpose position coding To preserve the spatial temporal relationship, the input vector of the flow field image encoder is formed: ， in, represents the projected vector, Represents position encoding; the encoder is composed of a stack of multi-head self-attention MHSA and multi-layer perceptron MLP modules; first, the input Split into independent attention heads, and fuse global features by scaling dot product attention weights: ， right After further processing, nonlinear transformation is performed through a two-layer fully connected network to further extract features: ， Where, represents the output of the MHSA attention mechanism, represents the output features of the image, represents normalization; finally, the encoding output Cross-modal fusion with other data; Natural language and knowledge base data encoding module: Encode natural language instructions and structured knowledge base data into a unified semantic vector space to achieve cross-modal semantic alignment; use the BERT model to achieve semantic encoding of natural language instructions and structured knowledge; define the input natural language instructions as ,in Represents the i-th word element of the text instruction; the embedding mechanism adds position information to each word element and maps it into a vector of dimension D: ， Where, Represents the embedding matrix, which is input to the Transformer encoder to extract context-aware semantic features: ， in, Represents the output of the attention mechanism, aggregating the semantic information features of natural language instructions; After completing the independent encoding of the three modalities of time series, image / video, and natural language, deep feature fusion is achieved through cross-modal interaction modeling. The fusion process is divided into two stages: feature splicing and attention alignment. The output features of each encoding module are dimensionally aligned and spliced ​​according to physical meaning to construct a joint feature matrix: , Realizing feature space through multi-head attention mechanism Alignment: , Where, , is a learnable parameter, is the length of the hidden layer; The outputs of each attention head are concatenated and linearly transformed to generate the final fusion feature: ， in, is a learnable parameter, It is a fusion feature; (2) Control system Based on the multimodal data from the data acquisition and fusion system, the control system constructs a state-action reward function and uses a reinforcement learning algorithm to dynamically adjust the control strategy of the servo motor; The dynamic model used in the control system is as follows: ， ， ， ， In the formula, the superscript " " represents the time derivative of the variable, 、 and are the displacements of the aircraft in the ground coordinate system; 、 and are the angle of attack, sideslip angle, and roll angle respectively; 、 、 are the gravitational constant, distance from the center of the earth, and mass, respectively. 、 and They are 、 and The moment of inertia of the shaft; The control strategy of dynamic optimization servo motor is as follows: the state space is composed of fusion features Definition, and the action space corresponds to the control instructions , including servo deflection and nozzle thrust adjustment; the reward function is designed with tracking accuracy, energy efficiency and stability as optimization goals; The double-delayed deep deterministic policy gradient TD3 network is the core of the reinforcement learning strategy, accepting Input and output control instructions, the mathematical expression is: ， Where, Indicates TD3 network, Represents the output control instruction; the fusion feature Input TD3 strategy network and update control instructions , which can make the servo and nozzle adjust in a short time; this instruction drives the servo and nozzle to perform dynamic adjustment through a real-time closed-loop mechanism; at the same time, the reward function works in conjunction with the TD3 network to evaluate the control instruction The influence on the system state is calculated and the parameters of the strategy network are iteratively optimized to achieve adaptive correction of the servo motor control strategy.

Images