Hypersonic flight vehicle simulation flight test system based on multi-modal large model

Abstract

The invention belongs to the technical field of hypersonic flight vehicle test, and relates to a hypersonic flight vehicle simulation flight test system based on a multi-modal large model, which comprises a data acquisition and fusion system and a control system. Through a data acquisition and fusion system and a cross-modal attention mechanism, efficient alignment and feature representation of time sequence data, environmental data and a historical case library acquired by a ground test system are realized, and the data acquisition and analysis precision is remarkably improved. In combination with a hardware system, effectiveness of test parameters is ensured, and the aircraft design and verification period is shortened. The control system dynamically adjusts test parameters through a natural language instruction, the model generates a structured control parameter vector through semantic understanding, the performance and stability of the aircraft control system are improved, the test operation complexity is reduced, and the user experience is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of hypersonic vehicle test technology, and relates to a hypersonic vehicle simulation flight test system based on a multimodal large model. Background Art

[0002] In the technical field of hypersonic vehicle test, the method of combining wind tunnel test and numerical simulation is usually adopted, in which dynamic aerodynamic force measurement and flight attitude simulation are the core verification links. Traditional wind tunnel tests rely on a six-degree-of-freedom platform to simulate the motion attitude of the vehicle, and collect aerodynamic force signals, attitude signals, etc. through a variety of sensors, but ignore the ability to synchronously process three types of heterogeneous data: mechanical vibration, flow field change, and control command, resulting in it being difficult to achieve both the fidelity and real-time performance of dynamic tests.

[0003] The six-degree-of-freedom platform simulation system proposed in the patent "Method and Device for Processing Aerodynamic Force Signals of a Balance in a Wind Tunnel Dynamic Test of a Variant Aircraft" (CN117909659A) can simulate the attitude change of the aircraft, but does not solve the dynamic matching problem between sensor timing data (such as balance dynamic signals) and wind tunnel environmental parameters (such as flow velocity, pressure field). During the test, there is a time-domain coupling between the mechanical vibration and inertial force interference contained in the balance signal and the change of the wind tunnel flow field. If only relying on a single-mode filter (such as traditional low-pass filtering), it will lead to the residue of interference or the loss of effective signals, and the error accumulates with the test time.

[0004] The patent "A Hardware-in-the-Loop Simulation System for the Navigation and Control System of a Hypersonic Vehicle" (CN113658340B) can achieve flight parameter acquisition and multi-subsystem coordination (such as VR scene generation, dynamic pressure simulation), but lacks an embedded real-time analysis engine. The system obtains flight attitude angles, altitude and other parameters through a data acquisition module, and uses a VR virtual scene generation system for image processing and three-dimensional modeling. However, the simulation data needs to be exported to an offline tool for post-processing, and it is impossible to generate visual reports (such as aerodynamic force time-domain curves, control command tracking error heat maps) in real time during the test.

[0005] The patent "A Simulation Test Platform and Control Method for Assessing Hypersonic Vehicles" (CN104182272B) proposes a simulation test platform and control method for assessing hypersonic vehicles, which can compare the advantages and disadvantages of control methods for hypersonic vehicles. However, as a software assessment and verification platform for control algorithms, it only covers the simulation verification means of the known dynamic model, kinematic model and related aerodynamic parameters of the vehicle, and has weak reference for the application test of the vehicle. Summary of the Invention

[0006] The present invention provides a hypersonic vehicle simulation flight test system based on a multimodal large model, aiming to solve the problems of ballistic tracking and control of the vehicle under complex multi-physical field coupling effects through advanced data processing technologies. The core lies in the fusion and processing of multimodal data, combined with a control strategy of reinforcement learning, to achieve intelligent debugging and analysis of the hypersonic vehicle ground test system. It mainly includes the following parts:

[0007] (1) The system obtains multi-source data from aerodynamics, real-time multimodal perception, control strategies, etc. through a data acquisition unit. The multimodal fusion unit is responsible for preprocessing and feature extraction of these data to ensure the accuracy and consistency of the data.

[0008] (2) In the multimodal fusion unit, advanced algorithms (such as the Transformer architecture) are used to align and fuse data from different sources. This module utilizes the large model architecture to integrate multimodal information and achieve comprehensive perception, prediction, and analysis of the vehicle state.

[0009] (3) The system applies the reinforcement learning algorithm to the fused multimodal data to optimize the control strategy and inputs the optimized results into the control system. It improves the control performance and response speed during the vehicle test process and adapts to complex flight environments.

[0010] The technical solution of the present invention is as follows:

[0011] A hypersonic vehicle simulation flight test system based on a multimodal large model, including a data acquisition and fusion system and a control system, is specifically as follows:

[0012] (1) Data acquisition and fusion system

[0013] The data acquisition and fusion system is one of the core components of the hypersonic vehicle simulation test, responsible for collecting, processing, and fusing multi-source data to support the implementation of dynamic control and optimization strategies. This system consists of a state acquisition unit and a multimodal fusion unit, and each unit plays a unique role in data processing and system interaction.

[0014] Among them, the state acquisition unit is responsible for collecting the state information and environmental data of the vehicle in real time, including aerodynamic data ( , , , , , and ), the motion state of the vehicle ( , , , , , , , and ) and external environmental conditions. The collected data enters the multimodal fusion unit through a high-speed transmission interface, providing a basis for subsequent data processing and analysis.

[0015] The multimodal fusion unit is the core unit of the system, responsible for integrating data from different sources to support the real-time control and optimization of hypersonic vehicles. This unit includes the following three modules: the time-series data encoding module, the image / video data encoding module, and the natural language and knowledge base data encoding module. The core algorithms and mathematical models of each module are as follows:

[0016] Time-series data encoding module:

[0017] Based on the feature extraction requirements of the time-series data collected by the vehicle data acquisition unit, an encoding module based on a temporal convolutional network is designed. Considering the physical characteristics of hypersonic vehicle ground tests, the time-series sensor data is defined as a multi-parameter fusion representation of flight motion states and aerodynamics, including aerodynamic data ( , , , , and ), and the motion states of the vehicle ( , , , , , , , and ), etc. Let the time-series data of the vehicle state be , where represents the sampling time series from 1 to , represents the th moment, is the dimension of the matrix, and represents the set of real numbers. A temporal convolutional network is used to extract local features. The rectified linear unit (ReLU) used in its core operation is a widely used activation function in deep learning, and its mathematical definition is , where represents the input, and represents selecting the maximum value. The encoding formula for time-series data is:

[0018]

[0019] where represents the Layer output features, Denoted as the layer learnable convolution kernel weights, Denoted as the dilation coefficient, Denoted as the convolution kernel width, Denoted as the layer bias term. Through the element-wise non-linear transformation of the convolution result by ReLU, the encoding module can extract the temporal dependence relationship between the aircraft motion state and aerodynamic parameters layer by layer, providing a highly discriminative feature representation for the subsequent generation of control strategies.

[0020] Image / Video Data Encoding Module:

[0021] For the dynamic images and video data collected in the wind tunnel flow field experiment, local flow field features are extracted through block operations. Specifically, the input image tensor is evenly divided into non-overlapping image blocks, each block with a size of pixels, Denoted as the horizontal pixels of the input image, Denoted as the vertical pixels of the input image, Denoted as the horizontal and vertical pixel sizes after block division. The block division process is implemented through the operator as follows:

[0022]

[0023] In the formula, Denoted as the total number of blocks, Denoted as the set of real numbers. Define the matrix , where each image block can be mapped to other dimensions through the linear projection matrix , and the position encoding is superimposed to retain the spatial temporal relationship, thus forming the input vector of the flow field image encoder:

[0024]

[0025] Among them, Denoted as the projected vector, Denoted as the position encoding. The core of the encoder consists of a stack of multi-head self-attention (MHSA) and multi-layer perceptron (MLP) modules. First, the input is divided into independent attention heads, and the global features are weighted and fused through scaled dot-product attention:

[0026]

[0027] For Re - process, perform non - linear transformation through two - layer fully - connected networks to further extract features

[0028]

[0029] In the formula, represents the output of the MHSA attention mechanism, represents the output features of the image, represents normalization. Finally, the encoded output is cross - modality fused with other data.

[0030] Natural language and knowledge base data encoding module:

[0031] This module is the semantic understanding and knowledge fusion unit of the aircraft ground simulation system, responsible for encoding natural language instructions and structured knowledge base data (such as aerodynamic parameter library, test case library) into a unified semantic vector space to achieve cross - modality semantic alignment. This module uses the BERT (Bidirectional Encoder Representations from Transformers) model to implement the semantic encoding of natural language instructions and structured knowledge. Define the input natural language instruction as where represents the i - th token of the text instruction. Add position information to each token through the Embedding mechanism and map it into a vector of dimension D:

[0032]

[0033] In the formula, represents the embedding matrix, which is input into the Transformer encoder to extract context - aware semantic features:

[0034]

[0035] Among them, represents the output of the attention mechanism, aggregating the semantic information features of the natural language instruction.

[0036] After completing the independent encoding of the three modalities of time series, image / video, and natural language, achieve deep feature fusion through cross - modality interaction modeling. The fusion process is divided into two stages: feature splicing and attention alignment. The output features of each encoding module are dimension - aligned and spliced according to physical meaning to construct a joint feature matrix:

[0037] Align the feature space through the multi - head attention mechanism:

[0038] In the formula, , are learnable parameters, is the length of the hidden layer.

[0039] After concatenating the outputs of each attention head and performing a linear transformation, the final fused feature is generated:

[0040]

[0041] Among them, are learnable parameters, is the fused feature.

[0042] (2) Control System

[0043] In the ground simulation test of hypersonic vehicles, the control system, as the core center, is responsible for realizing the high-precision dynamic simulation and real-time optimization of flight attitudes.

[0044] Specifically, the control system drives the moving parts of the ground test system (such as the moving belt, side plates, servo motors, and nozzles) through servo motors to simulate the attitude changes of the vehicle in the near-earth state. This mechanism allows for real-time adjustment of the vehicle's attitude to ensure the dynamic consistency of the test conditions. In addition, based on the multi-modal data of the data acquisition and fusion system, the control system constructs a state-action reward function and uses reinforcement learning algorithms to dynamically adjust the control strategy of the servo motors. For example, when it detects that the flow field separation causes the model vibration to intensify, the algorithm will automatically reduce the speed of the moving belt or adjust the angle of attack. In addition, the system supports the adjustment of control parameters through natural language instructions (such as "increase the wind speed to 50 m / s"), combined with keyword extraction (such as "bad word" filtering) to ensure the reliability and safety of the instructions.

[0045] In the high-fidelity flight control verification unit, the control system adopts a general dynamic model to support the attitude simulation and analysis under complex flight conditions. The multi-body dynamic model uses Lagrange's equations to describe the motion behavior of the vehicle, considering the coupling effects of various acting forces such as aerodynamic forces, gravity, and inertial forces. This model can simulate the dynamic responses of the vehicle under different flight states, providing theoretical support for the attitude control strategy. The specific dynamic model adopted is as follows:

[0046]

[0047]

[0048]

[0049]

[0050] In the formula, the superscript " ” represents the derivative of the variable with respect to time, , and are the velocity, flight path inclination angle, and flight path yaw angle, respectively; , and are the displacements of the aircraft in the ground coordinate system, respectively; , and are the angular velocities of the aircraft about the , and axes, respectively; , and are the pitch angle, roll angle, and yaw angle, respectively. , and are the angle of attack, sideslip angle, and bank angle, respectively; , , are the gravitational constant, distance from the center of the earth, and mass, respectively, , and are the moments of inertia about the , and axes, respectively. , , , , and are the lift force, drag force, side force, roll moment, yaw moment, and pitch moment, respectively; the above state variables can all be obtained through a data acquisition system. Specifically, a six-component balance system is used to synchronously measure the aerodynamic forces and moment components acting on the aircraft, including the lift force , drag force , side force and roll moment , yaw moment and pitch moment . The attitude angle and attitude angular velocity are measured by an inertial measurement unit, while the angle of attack and sideslip angle are measured by combining a five-hole probe or a miniature wind vane sensor installed at the head of the aircraft, and directly output instantaneous values by sensing the relative relationship between the oncoming flow direction and the body axis system. For high-dynamic conditions, dynamic compensation is also required by combining inertial navigation data with aerodynamic model prediction values. For example, the delay error of the angle-of-attack sensor is corrected using the angular velocity. This enables the model to real-time reflect the true behavior of the aircraft under different flight conditions.

[0051] Through high-fidelity flight control verification, the control system can verify the effectiveness and robustness of different control strategies without actual tests. This synergy between simulation and experimental control enables the aircraft attitude control system to be fully optimized and adjusted during the design phase, reducing risks and costs in actual tests.

[0052] The control system constructs a state-action reward function based on the reinforcement learning algorithm to dynamically optimize the control strategy of the servo motor. Specifically, the state space is defined by the fusion features while the action space corresponds to the control instructions , including the rudder deflection and the nozzle thrust adjustment amount. The reward function is designed with the optimization objectives of tracking accuracy, energy consumption efficiency, and stability.

[0053] The Twin Delayed Deep Deterministic Policy Gradient (TD3) network, as the core of the reinforcement learning strategy, accepts the input and outputs control instructions, and its mathematical expression is:

[0054]

[0055] In the formula, represents the TD3 network, represents the output control instructions. Inputting the fusion features into the TD3 policy network to update the control instructions can cause the rudder and nozzle to adjust within a short time. This instruction drives the rudder and nozzle to perform dynamic adjustments through a real-time closed-loop mechanism. At the same time, the collaborative work of the reward function and the TD3 network iteratively optimizes the parameters of the policy network by evaluating the long-term impact of the control instructions on the system state (such as reducing attitude oscillation or suppressing torque overshoot), thereby achieving adaptive correction of the servo motor control strategy.

[0056] Advantages of the present invention:

[0057] 1. Through the data acquisition and fusion system and the cross-modal attention mechanism, efficient alignment and feature representation of the time-series data, environmental data, and historical case library collected by the ground test system are realized, significantly improving the accuracy of data acquisition and analysis. Combining with the hardware system ensures the effectiveness of test parameters and shortens the aircraft design and verification cycle.

[0058] 2. The control system dynamically adjusts test parameters through natural language instructions, and the model generates a structured control parameter vector through semantic understanding, improving the performance and stability of the aircraft control system, reducing the complexity of test operations, and enhancing the user experience. Brief Description of the Drawings

[0059] Figure 1It is the technical block diagram of the hypersonic vehicle simulation flight test system based on the multimodal large model of the present invention;

[0060] Figure 2 is the attitude angle change curve;

[0061] Figure 3 is the angular velocity change curve;

[0062] Figure 4 is the aerodynamic force change curve;

[0063] Figure 5 is the aerodynamic moment change curve;

[0064] Figure 6 is the servo change curve;

[0065] Figure 7 is the nozzle angle change curve. Detailed implementation manners

[0066] The following further illustrates the detailed implementation manners of the present invention in conjunction with the accompanying drawings and technical solutions.

[0067] As Figure 1 shown, a hypersonic vehicle simulation flight test system based on the multimodal large model of the present invention includes a data acquisition and fusion system and a control system. In this embodiment, multi-modal data such as the attitude, acceleration, and wind tunnel parameters of the vehicle are collected by high-precision sensors, high-precision state information is generated by combining data fusion algorithms, and the performance of the controller is verified through a hardware-in-the-loop simulation platform. Among them, the partial parameter configuration of the time series data encoding module is as follows: The temporal convolutional network adopts a 4-layer structure, the width of each layer is 5, and the initial learning rate is set to 0.0001. The partial parameter settings of the image / video data encoding module are as follows: The input image resolution is , the number of blocks is 1024 blocks. Each image block is mapped to a 512-dimensional vector through a linear projection matrix, and sinusoidal position encoding is superimposed. The MHSA contains 12 Transformer layers, 8 attention heads in each layer, the hidden layer dimension is 512, and the MLP expansion ratio is 4. The partial parameter configuration of the natural language and knowledge base data encoding module is as follows: The pre-trained BERT-base model (12 layers, 768-dimensional hidden layer, 12 attention heads) is adopted. When fine-tuning, the learning rate is set to 0.00005, and the maximum sequence length is 512. The TD3 algorithm parameters are set as follows: Both the Actor network and the Critic network are 3-layer fully connected (256-128-64), the learning rate is 0.0003, and the discount factor is 0.99. The above encoding module inputs the proposed fusion features into the TD3 policy gradient network to generate servo deflection commands to ensure that the vehicle returns to the initial state. In this embodiment, at the beginning of the test, the initial flight parameters are set: speed is 5 Mach, angle of attack is 5 degrees, initial sideslip angle is 0 degrees, initial bank angle is 0 degrees. When there is no crosswind, the sensor measures the lift force to be 152.3 kN, the drag force to be 98.7 kN, the side force to be 0 kN, the roll moment to be approximately 0.2 kN·m, the pitch angular velocity to be 0.1 deg / s, and the yaw angular velocity to also be 0 deg / s. The local flow velocity in the flow field is 1700 m / s. Through a natural language instruction, "Start crosswind perturbation for 5 seconds", to simulate harsh wind field conditions. The lateral gust causes the sideslip angle to be 3.2 degrees and gradually decreases to 0, the side force becomes 28.5 kN and gradually decreases to 0, the yaw moment is 15.3 kN·m and gradually decreases to 0, and the high-speed camera captures that the local flow velocity drops to 1650 m / s. After being affected by the lateral gust, the temporal, image, and language features are aligned through a multi-head attention mechanism to generate a joint feature matrix, which is input into the reinforcement learning policy network. Among them, the temporal data includes attitude angles and angular velocities, encoded as 256-dimensional feature vectors. The image data is encoded as a 512-dimensional feature vector, and the natural language instruction "Start crosswind perturbation for 5 seconds" is encoded as a 768-dimensional semantic vector. The changes in attitude angles during flight are as Figure 2 shown.

[0068] From Figure 2 it can be seen that when the crosswind appears, the pitch angle, yaw angle, and roll angle of the aircraft all change to cope with the crosswind interference and return to a stable state after 100 seconds. Figure 3 For the changes in attitude angular velocity, it can be seen that when affected by the crosswind perturbation, the attitude angular velocity changes according to the control instruction and quickly returns to a stable state. Figure 4 and Figure 5 are the corresponding curves of aerodynamic forces and aerodynamic moments. From this, it can be seen that the aircraft can quickly adjust to a stable state within 100 s after being affected by the crosswind perturbation. Figure 6 and Figure 7 are the change curves of the aircraft's servo and nozzle respectively. It can be seen that when affected by the crosswind perturbation, the servo and nozzle quickly adjust to make the aircraft return to a stable state during high-speed flight.

Claims

1. A hypersonic vehicle simulation flight test system based on a multimodal large model, comprising a data acquisition and fusion system and a control system, characterized in that The details are as follows: (1) Data acquisition and fusion system It consists of a status acquisition unit and a multi-modal fusion unit; Among them, the state acquisition unit is responsible for collecting the state information and environmental data of the aircraft in real time, including aerodynamic data , , , , and , the motion state of the aircraft , , , , , , , and as well as external environmental conditions; the collected data enters the multi-modal fusion unit through a high-speed transmission interface; among them, , , , , and are lift, drag, side force, roll moment, yaw moment and pitch moment respectively; , and are speed, track inclination angle and track yaw angle respectively; , and are the angular velocities of the aircraft around , and axes respectively; , and are pitch angle, roll angle and yaw angle respectively; The multi-modal fusion unit includes the following three modules: time-series data encoding module, image / video data encoding module, natural language and knowledge base data encoding module; the mathematical models of each module are as follows: Time-series data encoding module: Define the time-series sensor data as a multi-parameter fusion representation of the flight motion state and aerodynamics, including aerodynamic data and the time-series data of the motion state of the aircraft. Let the time-series data of the aircraft state be , where represents the sampling time series from 1 to during this period, represents the moment, is the dimension of the matrix, represents the set of real numbers; Use a time convolutional network to extract local features. The rectified linear unit ReLU used in the operation is an activation function widely used in deep learning, and its mathematical definition is , where represents the input, represents choosing the maximum value; the encoding formula for the time-series data is: wherein, represents the output feature of the th layer, is expressed as the learnable convolution kernel weight of the th layer, represents the dilation coefficient, represents the convolution kernel width, represents the bias term of the th layer; through the element-wise non-linear transformation of the ReLU on the convolution result, the encoding module extracts the temporal dependence relationship between the aircraft motion state and the aerodynamic parameters layer by layer, providing a highly discriminative feature representation for the subsequent generation of control strategies; Image / video data encoding module: The dynamic images and video data collected in the wind tunnel flow field test are used to extract local flow field features through block operation. Specifically, the input image tensor is evenly divided into non-overlapping image blocks, each with a size of pixels, represents the horizontal pixels of the input image, represents the vertical pixels of the input image, represents the horizontal and vertical pixel sizes after blocking; the blocking process is implemented by the operator as follows: Wherein, represents the total number of sub - blocks, represents the set of real numbers; define the matrix , where each image block can be mapped to other dimensions through the linear projection matrix , and the position encoding is superimposed to retain the spatio - temporal relationship, thereby forming the input vector of the flow - field image encoder: ​ Among them, represents the projected vector, represents the positional encoding; the encoder is composed of a stack of multi-head self-attention (MHSA) and multi-layer perceptron (MLP) modules; first, the input is divided into independent attention heads, and the global features are weighted and fused through scaled dot-product attention: Pairwise Processed again, and a non-linear transformation is performed through two fully connected networks to further extract features: In the formula, represents the output of the MHSA attention mechanism, represents the output features of the image, represents normalization; finally, the encoded output is cross - modality fused with other data; Natural language and knowledge base data encoding module: Encode natural language instructions and structured knowledge base data into a unified semantic vector space to achieve cross-modal semantic alignment; use the BERT model to implement the semantic encoding of natural language instructions and structured knowledge; define the input natural language instructions as , where represents the i-th token of the text instruction; add position information to each token through the Embedding mechanism and map it into a vector with dimension D: Wherein, represents an embedding matrix, which is input into the Transformer encoder to extract context-aware semantic features: Among them, represents the output of the attention mechanism, aggregating the semantic information features of natural language instructions; After completing the independent encoding of the three modalities of timing, image / video, and natural language, deep feature fusion is achieved through cross-modal interaction modeling; the fusion process is divided into two stages: feature concatenation and attention alignment; the output features of each encoding module are dimensionally aligned and concatenated according to their physical meanings to construct a joint feature matrix: Feature space alignment is achieved through the multi-head attention mechanism : In the formula, , are learnable parameters, is the length of the hidden layer; After concatenating the outputs of each attention head and performing a linear transformation, the final fusion feature is generated: Among them, is a learnable parameter, is the fused feature; (2) Control system Based on the multi-modal data of the data acquisition and fusion system, the control system constructs a state-action reward function and dynamically adjusts the control strategy of the servo motor using a reinforcement learning algorithm; The specific dynamic model adopted by the control system is as follows: In the formula, the superscript " " represents the derivative of the variable with respect to time, , and are the displacements of the aircraft in the ground coordinate system, respectively; , and are the angle of attack, sideslip angle and bank angle, respectively; , , are the gravitational constant, distance from the center of the earth, and mass, respectively, , and are , and the moments of inertia about the The control strategy for dynamically optimizing the servo motor is as follows: The state space is defined by the fusion features while the action space corresponds to the control commands , including the servo deflection and the nozzle thrust adjustment; The reward function is designed with the tracking accuracy, energy consumption efficiency, and stability as the optimization objectives; As the core of the reinforcement learning strategy, the Twin Delayed Deep Deterministic Policy Gradient (TD3) network accepts inputs and outputs control instructions, and its mathematical expression is: In the formula, represents the TD3 network, represents the output control instruction; the fused feature is input into the TD3 policy network to update the control instruction , enabling the servo and nozzle to be adjusted within a short time; this instruction drives the servo and nozzle to perform dynamic adjustment through a real-time closed-loop mechanism; meanwhile, the reward function and the TD3 network work together to iteratively optimize the parameters of the policy network by evaluating the influence of the control instruction on the system state, thereby achieving adaptive correction of the servo motor control strategy.

Images