Cross-domain scenario for data-real fusion testing of aerospace equipment - mission generation and evaluation methods
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TIANMUSHAN LABORATORY
- Filing Date
- 2026-03-03
- Publication Date
- 2026-05-26
Smart Images
Figure CN121766417B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of aerospace equipment testing and evaluation technology, and in particular to a cross-domain scenario-task generation and evaluation method for aerospace equipment data-real fusion testing. Background Technology
[0002] As the intelligence level of aerospace equipment (such as drones, satellites, missiles, and new types of aircraft) continues to increase, the need to test their adaptability in complex environments and their cross-domain collaborative capabilities is becoming increasingly urgent. Traditional testing methods mainly include field physical tests and pure digital simulation tests. Field tests are costly, time-consuming, and pose safety risks, and are difficult to cover extreme operating conditions; while pure digital simulation tests are low-cost and repeatable, the scenario construction relies on manual pre-setting, lacks randomness and diversity, and there are often significant differences between the simulation model and the physical entity, resulting in insufficient credibility of the simulation results.
[0003] In recent years, generative artificial intelligence (AIGC) technology has made breakthroughs in the fields of large language model (LLM) and image generation. However, existing AIGC technology is mainly applied to media content creation and has not yet been effectively applied to the professional testing field of aerospace equipment. The main difficulties are: (1) general generative models are difficult to generate professional scenarios that meet the strict physical constraints of aerospace equipment; (2) there is a lack of an effective mechanism to transform the generated static scenarios into dynamic testing environments with physical attributes; and (3) there is a lack of a closed-loop consistency verification and quantitative evaluation system for the specific cross-domain scenario of "data-real fusion".
[0004] Therefore, there is an urgent need for a method that can automatically generate diverse test scenarios that conform to physical laws using large models and accurately evaluate cross-domain tasks in a data-real integration environment. Summary of the Invention
[0005] This invention aims to solve the aforementioned technical problems by providing a cross-domain scenario-task generation and evaluation method for data-real fusion testing of aerospace equipment. This method constructs a cross-domain test definition model, utilizes a large model and a visual language model to generate high-quality, varied test scenarios, and achieves automated, highly reliable testing of aerospace equipment through rigorous data-real consistency verification and a multi-dimensional performance evaluation model.
[0006] To achieve the above objectives, the present invention adopts the following technical solution:
[0007] In a first aspect, the present invention provides a method for cross-domain scenario-task generation and evaluation of aerospace equipment data-real fusion testing, comprising the following steps:
[0008] Step S1: Construct a cross-domain test definition model: Construct a cross-domain test definition model oriented towards data-real integration. The model adopts a layered architecture, including a cross-domain task layer for defining multi-equipment collaboration and virtual-real interaction rules, a variable scenario layer for defining dynamic environmental change parameters and entity object attributes, and an evaluation logic layer for defining the evaluation index system.
[0009] Step S2: Generate cross-domain tasks and scenario instances: Receive natural language test requirements and information on the aerospace equipment under test, construct instructions based on preset prompt engineering templates and input them into the large language model; use the large language model to retrieve the aerospace domain knowledge base, perform cross-domain decoupling and parameterized parsing on the test requirements, and generate cross-domain task flow instances that conform to the cross-domain test definition model specifications, as well as variable scenario description instances containing random perturbation parameters.
[0010] Step S3, Scene Asset Generation and Closed-Loop Verification: Analyze the entity object prompts in the variable scene description instance, generate multiple candidate images using the fine-tuned generative model, and introduce a visual language model to construct a closed-loop verification mechanism to verify the semantic consistency and structural integrity of the generated images, and select qualified images that conform to physical laws.
[0011] Step S4: Generate a dynamic and variable simulation scene: Perform 3D reconstruction processing on the selected qualified images to generate an object-level 3D model; fuse the 3D model with scene environment parameters and associate it with physical property mapping, instantiate it as a static scene in the simulation engine, and superimpose time-varying environmental factors to form a dynamic and variable scene.
[0012] Step S5: Perform automated evaluation: Convert the cross-domain task flow instance into an executable simulation script to drive the data-real fusion test system to run; establish a data-real synchronization verification mechanism, collect the operating data of the equipment under test in the dynamic and changing scenario in real time, calculate multi-dimensional performance indicators based on the evaluation logic layer, and complete the automated evaluation of aerospace equipment.
[0013] Secondly, the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method for cross-domain scenario-task generation and evaluation of aerospace equipment data-real fusion testing.
[0014] Thirdly, the present invention provides a computer-readable storage medium storing executable instructions thereon, which, when executed by a processor, enable the processor to implement the aforementioned method for cross-domain scenario-task generation and evaluation of aerospace equipment data-real fusion testing.
[0015] Fourthly, the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the aforementioned method for cross-domain scenario-task generation and evaluation of aerospace equipment data-real fusion testing.
[0016] The beneficial effects of this invention are as follows:
[0017] Guide the generation of a hierarchical professional test scenario description for a cross-domain test definition model that conforms to the physical constraints of aerospace equipment;
[0018] It innovatively introduces a closed-loop mechanism of "generation-verification-correction" and uses a visual language model to ensure that the generated scene conforms to the physical constraints of the aerospace field;
[0019] A mathematical model for verifying consistency between data and reality and evaluating multi-dimensional performance was constructed, transforming qualitative scenario generation into quantitative scientific evaluation, which significantly improved the credibility and practical value of the test results. Attached Figure Description
[0020] Figure 1 This is a schematic diagram of the process for cross-domain scenario-task generation and evaluation method of aerospace equipment data-real fusion testing according to the present invention. Detailed Implementation
[0021] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
[0022] like Figure 1 As shown, the present invention provides a cross-domain scenario-task generation and evaluation method for data-real fusion testing of aerospace equipment, which mainly includes the following steps:
[0023] Step S1: Construct a cross-domain testing definition model: Construct a cross-domain testing definition model (CTD-Model) oriented towards data-real integration; the model adopts a layered architecture, including a cross-domain task layer for defining multi-equipment collaboration and virtual-real interaction rules, a variable scenario layer for defining dynamic environmental parameters and entity object attributes, and an evaluation logic layer for defining the evaluation index system.
[0024] This model serves as a bridge connecting natural language requirements with digital testing systems.
[0025] The cross-domain mission layer includes a mission sequence diagram, a virtual-real data mapping table, and a cooperative action library. The virtual-real data mapping table defines the state synchronization mechanism between the digital twin and the physical entity. The cooperative action library defines the communication and control interface between cross-space equipment. The cross-domain mission layer mainly defines the mission sequence diagram (such as the state machine of "takeoff-cruise-reconnaissance-return").
[0026] The variable scenario layer includes a basic environment library (desert, ocean, city, etc.), dynamic disturbance factors (such as sudden gusts of wind and electromagnetic suppression), and an adversarial sample library. The dynamic disturbance factors cover meteorological changes, electromagnetic interference, and non-steady-state changes in illumination, while the adversarial sample library defines the extreme behavior patterns of non-cooperative targets.
[0027] The evaluation logic layer defines the specific calculation logic for evaluation indicators such as task achievement, robustness, and consistency between data and reality.
[0028] Step S2: Generate cross-domain task and scenario instances: Receive natural language test requirements and information on the aerospace equipment under test, construct instructions based on preset prompt engineering templates and input them into the large language model; use the large language model to retrieve the aerospace domain knowledge base, perform cross-domain decoupling and parameterized parsing on the test requirements, and generate cross-domain task flow instances that conform to the cross-domain test definition model specifications, as well as variable scenario description instances containing random perturbation parameters.
[0029] For example: Receive a natural language testing request from a user, such as: "Test the low-altitude penetration capability of drones in urban areas under strong electromagnetic interference, requiring at least three types of enemy radar facilities." Input this request into a large language model (LLM, such as one based on Llama 3 or Qwen). The LLM outputs structured JSON data, including:
[0030] Cross-domain task flow example: Defines the drone's flight path points, communication frequency switching logic, and virtual-real interaction nodes.
[0031] Example of a variable scenario description: It describes in detail the density and height distribution of urban buildings, the location of electromagnetic interference sources, and the intensity variation curves.
[0032] Step S3: Scene Asset Generation and Closed-Loop Verification: Parse the entity object prompts in the variable scene description instance, generate multiple candidate images using a fine-tuned generative model, and introduce a visual language model to construct a closed-loop verification mechanism. Verify the semantic consistency and structural integrity of the generated images, and select qualified images that conform to physical laws.
[0033] For example, based on the entity objects in the scene description (such as "enemy radar vehicle" or "high-rise building"), generate corresponding visual assets.
[0034] The specific steps for generating multi-view candidate images using a fine-tuned generative model include: constructing a dedicated dataset containing aerospace equipment and related environments; using a pre-trained diffusion model as a foundation, fine-tuning the model using the dedicated dataset with low-rank adaptation techniques; during the fine-tuning process, freezing the encoder and decoder weights of the variational autoencoder, performing low-rank adaptation fine-tuning only on the attention module of the diffusion model and the connection layer of the variational autoencoder, and introducing skip connections composed of zero convolutions to directly pass the multi-scale global features from the encoding stage to the decoding stage. Specifically:
[0035] Generative models can employ pre-trained diffusion models, such as Stable Diffusion XL. To enable the general diffusion model to accurately generate images with features of aerospace equipment and related scenarios (such as specific aerodynamic shapes, special coating materials, and various types of terrain data), this embodiment uses low-rank adaptation (LoRA) technology to update the weights of the pre-trained diffusion model.
[0036] Based on Stable Diffusion XL, a professional dataset of 10,000-15,000 "clean background - single object" images was built.
[0037] Specifically, for any weight matrix in the pre-trained model Where d is the number of rows in the weight matrix and k is the number of columns in the weight matrix, its parameters are frozen and not updated; instead, two low-rank matrices are introduced. (Initialized to a Gaussian distribution) and (Initialized to 0) to represent incremental updates of weights, where rank (Take 8 or 16). During the forward propagation process, the output h of the input feature x is calculated using the following formula:
[0038] ,
[0039] in, The scaling factor is used to adjust the influence weight of the adapter on the original model, and is set to 1.0~2.0 in this embodiment.
[0040] Furthermore, to ensure that the generated image possesses both high fidelity and conforms to physical perception characteristics, this embodiment designs a composite loss function for model optimization. The composite loss function... It is composed of a weighted average of pixel-level reconstruction loss and perceptual loss, and its mathematical expression is:
[0041] ,
[0042] in, The original image is the input. and These are represented as the encoder and decoder of a variational autoencoder, respectively. For the generated image, It is the L2 norm. The perceptual feature extraction network is represented by the first... Feature layer of the layer, For the first Layer feature weights, and These are the corresponding weighting coefficients.
[0043] To address potential issues such as distortion and language inconsistencies in the generated images, the system utilizes a Visual Language Model (VLM) for multi-dimensional constraint closed-loop verification. This verification includes background interference verification, instruction compliance verification, and single-object independence verification. If a candidate image fails any verification dimension, the current image and the reason for the failure are input into the VLM, which generates targeted corrective prompts. The corrected prompts are then re-input into a fine-tuned generative model for the next round of image generation, continuing until the generated result satisfies all constraints or reaches the maximum number of iterations.
[0044] For example, the system checks whether the background of multiple generated candidate images, such as an image of a radar vehicle, is clean, whether there is the correct number of tires, and whether the image contains only a single object. If the verification fails, the VLM generates correction prompts (such as "correct antenna angle") and feeds them back to the generative model for regeneration until the final verification passes.
[0045] Step S4: Generate a dynamic and variable simulation scene: Perform 3D reconstruction processing on the selected qualified images to generate an object-level 3D model; fuse the 3D model with scene environment parameters and associate it with physical property mapping, instantiate it as a static scene in the simulation engine, and simultaneously superimpose time-varying environmental factors to form a dynamic and variable scene.
[0046] Step S4 may further include: using a super-resolution reconstruction network to enhance the resolution of qualified images; using a saliency detection algorithm to extract the foreground target mask of the image and remove residual background noise; using image inpainting technology to complete the texture of the occluded viewpoints required for 3D reconstruction, generating a consistent image set from multiple viewpoints, and then generating an object-level 3D model through a 3D reconstruction algorithm.
[0047] The qualified images after screening are subjected to three-dimensional reconstruction processing to generate object-level 3D models, which may include generating object-level 3D models with specular and reflective properties through algorithms based on Neural Radiation Field (NeRF) or 3D Gaussian Splatting.
[0048] In simulation engines (such as Unity3D or Unreal Engine), the key step in instantiating these 3D models is to fuse the 3D models with scene environment parameters and associate them with physical property mappings (such as assigning rigid body mass, aerodynamic coefficients, and collision volume), and then instantiate them in the simulation engine as a static scene with physical interaction capabilities.
[0049] Subsequently, dynamic factors of the time dimension (such as moving clouds and changing interference signals) are superimposed to construct a dynamic and ever-changing scene.
[0050] Step S5: Perform automated evaluation: Convert cross-domain task flow instances into executable simulation scripts to drive the data-real fusion test system to run; establish a data-real synchronization verification mechanism to collect the operating data of the equipment under test in the dynamic and changing scenario in real time, calculate multi-dimensional performance indicators based on the evaluation logic layer, and complete the automated evaluation of aerospace equipment.
[0051] Step S5 may include: connecting the system to physical testing equipment (such as a physical drone in a darkroom) and the digital simulation environment, and performing the following parallel operations:
[0052] S5-1, Real-Data Synchronization Verification Mechanism: When generating executable simulation scripts, a unified time base is established between the physical test equipment and the digital simulation environment through a timestamp alignment algorithm; during testing, a real-data consistency metric between the physical entity and the digital model is calculated based on a sliding time window. .
[0053] In the process of data-real fusion testing, ensuring the synchronization of the states of physical entities and digital models is crucial for the reliability of cross-domain tasks. In one implementation, a state residual calculation method based on a sliding time window is proposed for real-time monitoring of virtual-real consistency.
[0054] Define the state vector of the physical entity at time t as (representing position, velocity, and attitude respectively), the corresponding digital model state vector is Considering communication link and computational latency, the system's data-real consistency metric C(t) is defined as:
[0055] ,
[0056] Where N is the number of sampling points within the sliding time window (50~100). The sampling interval is... and These are the state vectors of the physical entity and the digital model, respectively. This is the inherent delay compensation amount of the system. Let T be the state weight vector. T is the transpose of the matrix, and i is the cumulative index.
[0057] If the above If the preset safety threshold is exceeded, and the data-real fusion environment is determined to be divergent, the test circuit breaker mechanism will be triggered.
[0058] S5-2, Automated Evaluation: During test execution, the simulation engine's data probes record the pose information, sensor status, and control commands of the equipment under test in real time. When a critical event is detected, the corresponding atomic evaluation function in the evaluation logic layer is called for local scoring. After the test, a dynamically weighted comprehensive performance evaluation model is used to calculate the comprehensive evaluation score. .
[0059] Suppose a cross-domain test task contains M atomic evaluation items (such as target recognition rate, maneuver response time, and anti-interference success rate), and the normalized score of the j-th evaluation item is... Then the overall evaluation score for this task is... The calculation formula is as follows:
[0060] ,
[0061] This represents the scene complexity coefficient. For example, the scene complexity at a wind speed of 20 m / s is less than that at a wind speed of 20 m / s and an electromagnetic interference intensity of 80 dB. This is the weight of the j-th assessment item. It is the penalty coefficient. This is a violation and penalty item.
[0062] The task score is the sum of the combined weighted scores of various task indicators.
[0063] The penalty item is the punishment component imposed when a task fails or a constraint is violated.
[0064] The above implementation methods demonstrate that, through the detailed implementation steps described above, the present invention not only achieves automated generation of test scenarios, but also ensures the physical credibility of the generated content and the scientific nature of the evaluation results through a rigorous mathematical model. It effectively solves the problems of single test scenario construction, asynchronous data and reality, and subjective evaluation in the prior art, and has significant engineering application value.
[0065] The present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method for cross-domain scenario-task generation and evaluation of aerospace equipment data-real fusion testing.
[0066] The present invention provides a computer-readable storage medium storing executable instructions thereon, which, when executed by a processor, enable the processor to implement the aforementioned method for cross-domain scenario-task generation and evaluation of aerospace equipment data-real fusion testing.
[0067] This invention provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method for cross-domain scenario-task generation and evaluation of aerospace equipment data-real fusion testing.
[0068] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for cross-domain scenario-task generation and evaluation of aerospace equipment data-real fusion testing, characterized in that, The method includes: Step S1: Construct a cross-domain test definition model: Construct a cross-domain test definition model oriented towards data-real integration. The model adopts a layered architecture, including a cross-domain task layer that defines the rules for multi-equipment collaboration and virtual-real interaction, a variable scenario layer that defines the dynamic parameters of the environment and the attributes of objects, and an evaluation logic layer that defines the evaluation index system. Step S2: Generate cross-domain tasks and scenario instances: Receive the user's input of natural language test requirements and the information of the aerospace equipment to be tested, construct instructions based on the preset prompt engineering template and input them into the large language model; use the large language model to call up the aerospace domain knowledge base, perform cross-domain decoupling and parameterized parsing on the test requirements, and generate cross-domain task flow instances that conform to the cross-domain test definition model specifications, as well as variable scenario description instances containing random perturbation parameters; Step S3, Scene Asset Generation and Closed-Loop Verification: Analyze the entity object prompts in the variable scene description instance, generate multiple candidate images using the fine-tuned generative model, and introduce a visual language model to construct a closed-loop verification mechanism to verify the semantic consistency and structural integrity of the generated images, and select qualified images that conform to physical laws. Step S4: Generate a dynamic and variable simulation scene: Perform 3D reconstruction processing on the selected qualified images to generate an object-level 3D model; fuse the object-level 3D model with scene environment parameters and associate it with physical property mapping, instantiate it as a static scene in the simulation engine, and superimpose time-varying environmental factors to form a dynamic and variable scene. Step S5: Perform automated evaluation: Convert the cross-domain task flow instance into an executable simulation script to drive the data-real fusion test system to run; establish a data-real synchronization verification mechanism to collect the running data of the equipment under test in the dynamic and changing scenario in real time; calculate multi-dimensional performance indicators based on the evaluation logic layer in the cross-domain test definition model to complete the automated evaluation of aerospace equipment. The specific workflow of the closed-loop verification mechanism described in step S3 is as follows: The generated candidate image is subjected to multi-dimensional constraint verification using a visual language model. This multi-dimensional constraint verification includes background interference verification, instruction compliance verification, and single object independence verification. If the candidate image fails any verification dimension, the current image and the reason for the verification failure are input into the visual language model, which then generates targeted prompt words for correction. The corrected prompt words are re-input into the fine-tuned generative model for the next round of image generation, until the generated result satisfies all constraints or reaches the maximum number of iterations.
2. The method for cross-domain scenario-task generation and evaluation of aerospace equipment data-real fusion testing according to claim 1, characterized in that, In step S1, the cross-domain task layer includes a task sequence diagram, a virtual-real data mapping table, and a cooperative action library. The virtual-real data mapping table defines the state synchronization mechanism between the digital twin and the physical entity, and the cooperative action library defines the communication and control interface between cross-domain equipment. The variable scenario layer includes a basic environment library, dynamic disturbance factors, and an adversarial sample library. The dynamic disturbance factors cover meteorological changes, electromagnetic interference, and unsteady changes in illumination, and the adversarial sample library defines the extreme behavior patterns of non-cooperative targets. The evaluation logic layer defines the calculation logic for task achievement indicators, system robustness indicators, and data-real consistency indicators.
3. The method for cross-domain scenario-task generation and evaluation of aerospace equipment data-real fusion testing according to claim 1, characterized in that, In step S3, generating multiple candidate images using the fine-tuned generative model includes: constructing a dedicated dataset containing aerospace equipment and related environments; using the pre-trained diffusion model as a foundation, fine-tuning the model using the dedicated dataset with low-rank adaptation technology; during the fine-tuning process, freezing the encoder and decoder weights of the variational autoencoder, performing low-rank adaptation fine-tuning only on the attention module of the diffusion model and the connection layer of the variational autoencoder, and introducing skip connections composed of zero convolutions to directly pass the multi-scale global features of the encoding stage to the decoding stage; and optimizing the model using a composite loss function, wherein the composite loss function satisfies: ， in, The original image is the input. and These are represented as the encoder and decoder of a variational autoencoder, respectively. For the generated image, It is the L2 norm. The perceptual feature extraction network is represented by the first... Feature layer of the layer, For the first Layer feature weights, and These are the corresponding weighting coefficients.
4. The method for cross-domain scenario-task generation and evaluation of aerospace equipment data-real fusion testing according to claim 1, characterized in that, The 3D reconstruction process described in step S4 includes: using a super-resolution reconstruction network to enhance the resolution of qualified images; using a saliency detection algorithm to extract the foreground target mask of the image and remove residual background noise; using image inpainting technology to complete the texture of the occluded viewpoints required for 3D reconstruction, generating a consistent image set from multiple viewpoints, and then generating an object-level 3D model through a 3D reconstruction algorithm.
5. The method for cross-domain scenario-task generation and evaluation of aerospace equipment data-real fusion testing according to claim 1, characterized in that, The specific implementation of the automated evaluation described in step S5 is as follows: During the test execution, the simulation engine's data probes record the pose information, sensor status, and control commands of the equipment under test in real time; when a critical event is detected, the corresponding atomic evaluation function in the evaluation logic layer is called to perform local scoring; after the test, a dynamically weighted comprehensive performance evaluation model is used to calculate the comprehensive evaluation score. The comprehensive evaluation score The calculation formula satisfies: ， in, Here, M represents the scene complexity coefficient, and M represents the total number of atomic assessment items. Let the weight of the j-th assessment item be . For the j-th assessment item, the normalized score is... For violations and penalties, This is the penalty coefficient.
6. The method for cross-domain scenario-task generation and evaluation of aerospace equipment data-real fusion testing according to claim 1, characterized in that, The specific implementation of the data-real synchronization verification mechanism described in step S5 is as follows: when generating the executable simulation script, a unified time base is established between the physical test equipment and the digital simulation environment through a timestamp alignment algorithm; during the test, a data-real consistency metric between the physical entity and the digital model is calculated based on a sliding time window. The Specifically, it is stated as follows: ， Where N is the number of sampling points within the sliding time window, and t is a certain moment in the operation. The sampling interval is... and These are the state vectors of the physical entity and the digital model, respectively. This is the inherent delay compensation amount of the system. Here, T represents the state weight vector; T is the matrix transpose operation, and i is the accumulation index. If the above If the preset safety threshold is exceeded, the test circuit breaker mechanism will be triggered.
7. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When one or more programs are executed by the one or more processors, the one or more processors implement the cross-domain scenario-task generation and evaluation method for data-real fusion testing of aerospace equipment as described in any one of claims 1-6.
8. A computer-readable storage medium, characterized in that, It stores executable instructions, which, when executed by a processor, enable the processor to implement the cross-domain scenario-task generation and evaluation method for data-real fusion testing of aerospace equipment as described in any one of claims 1-6.
9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the cross-domain scenario-task generation and evaluation method for data-real fusion testing of aerospace equipment as described in any one of claims 1-6.