Agent-based aviation simulation test case generation and coverage verification method

Abstract

The application discloses an agent-based aviation simulation test case generation and coverage verification method, comprising: after acquiring system requirements, interface control files, simulation model descriptions, flight working conditions, fault modes and test criteria, performing semantic analysis to build an aviation simulation test knowledge base and a multi-dimensional coverage model, generating candidate test cases through a large language model agent, and using an aviation constraint rule base to verify the legality of the candidate test cases in terms of flight envelope, interface range, state machine migration and safety constraints, calculating the coverage of each dimension after obtaining effective test cases, identifying coverage gaps and driving the agent to generate targeted supplements until the coverage meets the preset threshold, and finally outputting a test case set, a coverage report, a requirement traceability matrix and a simulation script template. The application realizes intelligent generation of aviation simulation test cases, closed-loop verification of coverage and requirement traceability management, and improves test design efficiency and coverage integrity.

Description

Technical Field

[0001] This invention relates to the field of aerospace engineering simulation and testing technology, and in particular to a method for generating and verifying aerospace simulation test cases based on intelligent agents. Background Technology

[0002] In the process of aerospace engineering research and development, simulation testing is integrated throughout the stages of system design, software development, control law verification, equipment integration and testing, hardware-in-the-loop simulation, and system integration verification. It is a crucial link in ensuring the safety and reliability of aerospace systems. Testers need to design simulation test cases covering various normal operating conditions, boundary conditions, abnormal states, and fault injection scenarios based on a large amount of heterogeneous data, including system requirements specifications, interface control documents, simulation model documentation, flight envelope and mission profile data, failure mode and effects analysis documents, test criteria, and acceptance criteria. With the continuous increase in the complexity of aerospace systems, manually designing test cases faces increasingly severe challenges in terms of efficiency and completeness.

[0003] Currently, the design of test cases for aviation simulation mainly adopts a manual approach. Testers first read through the requirements document, interface document, and model description, manually extract the test objects, input and output signals, operating parameters, and judgment rules, and then enumerate the excitation combinations and expected responses under different flight states based on their personal experience, and write test steps and expected results. For fault injection testing, testers need to manually design fault triggering conditions and recovery criteria based on the fault mode analysis document. After completing the test case writing, testers use spreadsheets or test management tools to record the traceability relationship between test cases and requirements and interfaces, and manually calculate the pass rate of each functional module and operating condition after the test is executed.

[0004] The existing technology has the following drawbacks: First, test requirements are highly fragmented, lacking a unified semantic association between requirement documents, interface documents, model parameters, fault analysis documents, and test procedures. Manually maintaining the traceability relationships between test cases and requirements, interfaces, operating conditions, and faults is extremely costly for testers. After requirement changes, it is difficult to promptly identify affected test cases and locate coverage gaps, leading to wasted test resources and insufficient regression testing. Second, aerospace simulation operating conditions have numerous dimensions, including altitude, speed, attitude, load, environmental conditions, equipment modes, and fault states. Manually enumerating test scenarios easily leads to the omission of boundary extreme value combinations and complex overlapping operating conditions. Furthermore, test coverage is usually manually calculated only after test execution, failing to guide test case generation during the test design phase, resulting in both redundant testing and insufficient coverage. Summary of the Invention

[0005] To overcome the shortcomings and deficiencies of existing technologies, this invention provides a method for generating and verifying aviation simulation test cases based on intelligent agents.

[0006] The technical solution adopted in this invention is an agent-based method for generating and verifying aviation simulation test cases, comprising the following steps: Step S1, acquiring aviation simulation test input data, including system requirements, interface control files, simulation model specifications, flight conditions, fault modes, and test criteria; Step S2, performing semantic dependency parsing and entity relation extraction on the input data to construct a heterogeneous information graph with test objects, interface signals, flight condition variables, constraints, fault types, and judgment rules as nodes; Step S3, constructing an aviation simulation test knowledge base based on the heterogeneous information graph, and establishing a high-order relationship between requirements, interface signals, model parameters, flight conditions, fault modes, and test criteria through a hypergraph structure. Relationship; Step S4: Construct a multi-dimensional coverage model based on the coverage target set in the knowledge base. The multi-dimensional coverage model includes at least one or more of the following: requirement coverage, interface coverage, operating condition coverage, boundary coverage, fault coverage, and criterion coverage; Step S5: Execute test intent parsing and scenario decomposition based on the aviation simulation test knowledge base and the multi-dimensional coverage model using a large language model intelligent agent to generate candidate test cases; Step S6: Perform legality verification on the candidate test cases using the aviation constraint rule base, eliminating or correcting candidate test cases that do not meet the constraints to obtain valid test cases; Step S7: Calculate the coverage based on the valid test cases, and generate a coverage report and a requirement traceability matrix based on the coverage results.

[0007] Furthermore, in step S4, for any coverage dimension D, the coverage value is determined by the formula... Calculate, where, This represents the coverage ratio of dimension D. This represents the number of target items in dimension D that are covered by at least one valid test case. This represents the total number of target items that need to be covered in dimension D. The target items include requirement items, interface signal value ranges, flight condition grid nodes, boundary value combinations, failure mode instances, and criterion branches.

[0008] Furthermore, in step S6, when the aviation constraint rule base performs legality verification on candidate test cases, it calculates a comprehensive constraint satisfaction index for any candidate test case. ,in, This represents the comprehensive constraint verification result of candidate test case c. This indicates the total number of constraint rule categories participating in the validation. Let represent the validation input feature vector of candidate test case c under the k-th type of constraint rule. This represents the set of legal value ranges specified by the k-th type of constraint rule. The indicator function takes a value of 1 when the input feature is within the valid range, and a value of 0 otherwise. The candidate test case c is considered valid only when the indicator function of all categories takes a value of 1.

[0009] Further, in step S5, the large language model agent transforms the requirement items into test intentions, and decomposes the test intentions into normal scenarios, boundary scenarios, abnormal scenarios, and fault scenarios. When generating candidate test cases, it uses the formula... Calculate the generation probability of each test scenario, where, This indicates that, given a requirement item q and a knowledge base... and coverage model Generate the j-th test scenario under the given conditions. The probability, This indicates the total number of candidate scenarios. Representing a scene semantic embedding vector, This represents the generated condition vector jointly driven by requirement items, knowledge base, and coverage model. This represents a semantic similarity function between vectors. It is an exponential function.

[0010] Furthermore, in step 56, when performing legality verification on the candidate test cases, for the flight condition parameter combinations in the candidate test cases, the envelope margin index is calculated based on the flight envelope constraints. ,in, This represents the flight condition parameter vector in the candidate test cases. This represents the total number of dimensions of the flight condition parameters. This represents the value of the nth flight condition parameter. and These represent the minimum and maximum permissible values ​​of the nth flight condition parameter under flight envelope constraints, respectively. This represents the nearest normalized distance from the combination of flight condition parameters to the envelope boundary, when... If the value is less than a preset threshold, the candidate test case is determined to not meet the flight envelope constraint.

[0011] Furthermore, in step S6, when performing legality verification on the candidate test cases, for the system state transition sequence in the candidate test cases, a state transition legality index is calculated based on the system state machine constraints. ,in, This represents a system state transition sequence of length T in the candidate test cases. This represents the system state identifier at time t. This indicates a transition from state to state under system state machine constraints. The set of next states that are allowed to be transitioned to from the starting point. For state transition verification functions, when belong The value is 1 if the check function is true and 0 otherwise. The check function is only valid when all adjacent state transitions in the sequence have a value of 1. A value of 1 indicates that the state transition sequence satisfies the system state machine constraints.

[0012] Further, step S2 includes the following steps: Step S21, performing semantic annotation and dependency parsing based on aerospace ontology on the system requirements and the test criteria to extract requirement items and their associated judgment conditions; Step S22, performing field-level parsing and data type inference on the interface control file to extract the names, data types, and value constraints of input and output signals; Step S23, performing parameter extraction and time-series relationship modeling on the simulation model description and the flight conditions to extract model parameters, flight condition variables, and their coupling relationships; Step S24, performing fault tree logic parsing on the fault modes to extract fault type, fault injection location, fault duration, and recovery conditions.

[0013] Further, step S3 includes the following steps: Step S31, instantiating the parsed test objects, interface signals, flight condition variables, constraints, fault types, and judgment rules into entity nodes in the knowledge graph; Step S32, establishing requirement-interface association edges based on the reference relationship between requirement items and interface signals, and establishing interface-parameter association edges based on the mapping relationship between interface signals and model parameters; Step S33, establishing condition-requirement association edges based on the applicability relationship between flight condition variables and requirement items, and establishing fault-object association edges based on the causal relationship between fault modes and affected test objects; Step S34, establishing criterion-requirement association edges based on the verification relationship between test criteria and requirement items, forming an aviation simulation test knowledge base with a directed hyperedge structure.

[0014] Further, step S4 includes the following steps: Step S41, extracting a set of requirement items as requirement coverage targets, a set of interface signal value ranges as interface coverage targets, and a grid of flight condition parameter combinations as condition coverage targets from the aviation simulation test knowledge base; Step S42, generating a set of boundary coverage targets based on the boundary extreme value combinations of each flight condition parameter, generating a set of fault coverage targets based on the set of fault mode instances, and generating a set of criterion coverage targets based on the set of test criterion branch conditions; Step S43, constructing a coverage target dictionary according to the six dimensions of requirement, interface, condition, boundary, fault, and criterion, and assigning a unique identifier and associated tracing path to each coverage target; Step S44, establishing a cross-coverage matrix between each coverage dimension, recording the association strength between any two coverage targets of different dimensions, as the priority basis for subsequent supplementary test case generation.

[0015] Further, step S5 includes the following steps: Step S51, vectorizing the current coverage state of each coverage dimension in the multi-dimensional coverage model to generate a coverage state vector; Step S52, retrieving requirement items and test criteria related to the current coverage state vector from the aviation simulation test knowledge base, and constructing a test intent generation prompt context; Step S53, reasoning on the prompt context through a large language model agent, transforming the functional descriptions in the requirement items into a test intent set including normal input ranges, boundary input values, and abnormal input conditions; Step S54, for each test intent in the test intent set, automatically matching preconditions, input stimuli, expected outputs, and judgment criteria according to the association relationships in the knowledge base, and combining them into structured candidate test cases.

[0016] Beneficial Effects: This invention proposes an agent-based method for generating and verifying test cases for aviation simulation. By constructing an aviation simulation test knowledge base, it establishes a unified high-order semantic association between requirements, interface signals, model parameters, flight conditions, fault modes, and test criteria. This overcomes the shortcomings of traditional manual methods, such as scattered test requirement sources and high maintenance costs for tracing relationships. When requirements or interfaces change, the system can quickly identify affected test cases and locate coverage gaps. By constructing a multi-dimensional coverage model and automatically triggering a large language model agent to supplement and generate test cases based on coverage gaps until the coverage meets a preset threshold, this method solves the problem of easily overlooking boundaries when manually enumerating test scenarios. The system addresses the challenges of extreme value combinations and complex operating conditions, creating a closed loop between coverage assessment and test design to avoid both redundant testing and insufficient coverage. By employing an aviation constraint rule base, the system performs deterministic verification of flight envelopes, interface values, state machine transitions, and safety constraints on candidate test cases, resolving the potential for invalid or dangerous test cases when large language models are directly applied to the aviation field. Simultaneously, the system automatically outputs test case sets, coverage reports, requirement traceability matrices, and simulation script templates, improving the efficiency, coverage completeness, and result traceability of aviation simulation test case design, while reducing the workload of manual design and the risk of missed regression tests due to requirement changes. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention. Figure 2 This is a flowchart of method step S2 of the present invention; Figure 3 This is a flowchart of method step S3 of the present invention; Figure 4 This is a flowchart of method step S4 of the present invention; Figure 5 This is a flowchart of step S5 of the method of the present invention. Detailed Implementation

[0018] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] like Figure 1As shown, the method for generating and verifying aviation simulation test cases based on intelligent agents includes the following steps: Step S1: Obtain aviation simulation test input data, including system requirements, interface control files, simulation model descriptions, flight conditions, fault modes, and test criteria; Step S2: Perform semantic dependency parsing and entity relation extraction on the input data to construct a heterogeneous information graph with test objects, interface signals, flight condition variables, constraints, fault types, and judgment rules as nodes; Step S3: Construct an aviation simulation test knowledge base based on the heterogeneous information graph, and establish high-order associations between requirements, interface signals, model parameters, flight conditions, fault modes, and test criteria through a hypergraph structure; Step S4: Construct a multi-dimensional coverage model based on the coverage target set in the knowledge base. The multi-dimensional coverage model includes at least one or more of the following: requirement coverage, interface coverage, operating condition coverage, boundary coverage, fault coverage, and criterion coverage. Step S5: Execute test intent parsing and scenario decomposition based on the aviation simulation test knowledge base and the multi-dimensional coverage model using a large language model intelligent agent to generate candidate test cases. Step S6: Perform legality verification on the candidate test cases using an aviation constraint rule base, eliminating or correcting candidate test cases that do not meet the constraints to obtain valid test cases. Step S7: Calculate the coverage based on the valid test cases, and generate a coverage report and a requirement traceability matrix based on the coverage results.

[0020] Step S1: Obtain input data for aviation simulation testing: Collect all input data required for simulation testing from each stage of the entire life cycle of aviation system development. This includes system requirements specifications described in natural language, interface control documents presented in the form of tables and field definitions, model description documents describing the mathematical relationships and state logic of the simulation model, flight condition data defining continuous variable ranges such as flight altitude from sea level to 15,000 meters, speed from 0 to Mach 2, and angle of attack from -5 degrees to +45 degrees, various fault types and their injection conditions and duration parameters provided by the Fault Mode and Effects Analysis (FMEA) document, and test criteria data including quantitative indicators such as response time of each functional module not exceeding 100 milliseconds, steady-state error not exceeding 2%, and overshoot not exceeding 10%. Integrate all data into the system in the original document format.

[0021] Step S2: Perform structured parsing of the input data: Perform semantic annotation based on an aerospace ontology dictionary on the system requirements specification, identifying the functional subject, operational action, and constraint values ​​in each requirement item, transforming the unstructured natural language requirements into structured records including requirement numbers, preconditions, and acceptance criteria; perform field-level parsing on the interface control file, extracting the name, data type, physical value range, and enumeration list of each input / output signal; record the lower and upper limits of the range for analog signals, and record the effective state combinations for discrete signals; perform... The system extracts model parameters such as aerodynamic derivatives, time constants, and gain coefficients, along with their units. It also discretizes flight condition data, dividing the altitude range into 500-meter intervals and the velocity range into 0.1 Mach intervals, forming a flight condition parameter combination matrix. Finally, it performs logic tree parsing on the fault mode document, extracting fault codes, fault injection methods, configurable fault durations ranging from 50 to 5000 milliseconds, and recovery condition statements. Finally, it extracts branch conditions from the test criteria, extracting the upper and lower thresholds and logical operation relationships for each criterion.

[0022] Step S3: Construct an aviation simulation test knowledge base: Instantiate and store all structured elements parsed in Step S2 as nodes and edges in a graph database. Assign a globally unique identifier to each requirement item and record its text description and acceptance criteria. Assign a signal identifier to each interface signal and record its data type and value range. Assign a parameter identifier to each model parameter and record its physical meaning and unit. Assign a variable identifier to each flight condition variable and record its discretized grid node set. Assign a fault identifier to each fault mode and record its injection location and duration interval. Assign a criterion identifier to each judgment rule and record its threshold interval. Establish the associated edges between the above nodes based on the explicit reference relationships and domain logic rules in the input data, including the reference association between requirement items and interface signals, the mapping association between interface signals and model parameters, the applicable association between flight condition variables and requirement items, the influence association between fault modes and test objects, and the verification association between test criteria and requirement items. At the same time, establish temporal adjacency relationships between adjacent grid nodes to form a knowledge network covering all test elements and their associated paths.

[0023] Step S4: Construct a multi-dimensional coverage model: Based on the set of test element nodes stored in the knowledge base constructed in Step S3, establish a list of coverage targets from six dimensions and initialize them to an uncovered state. The requirement coverage dimension uses each requirement item as an independent coverage target. The interface coverage dimension uses the normal value range and boundary value points of each interface signal as coverage targets. The operating condition coverage dimension uses each node in the two-dimensional grid of altitude and speed as a coverage target, including each boundary extreme point. The boundary coverage dimension uses the combination of the lower limit, upper limit, and inflection point values ​​of each flight parameter as a coverage target. The fault coverage dimension uses each fault mode instance and its injection combination in different working stages as a coverage target. The criterion coverage dimension uses each branch condition of each test criterion as a coverage target. Assign a binary status flag bit to each coverage target. A value of 0 indicates that it is not covered by any test case, and a value of 1 indicates that it has been covered by at least one valid test case. Record the identifiers and current states of all coverage targets in the coverage status table. Store the associated requirement identifier and traceability path string for each coverage target.

[0024] Step S5: Generate candidate test cases using the large language model agent: Read the status flag bits of all currently covered targets from the coverage status table in Step S4, filter out the set of uncovered targets with status flag bits of 0, encode the identifiers and associated requirement descriptions of the uncovered targets into structured prompt words after sorting them by priority, and simultaneously retrieve the interface signal value range, flight condition parameter grid node coordinates, and test criterion threshold intervals associated with the uncovered targets from the knowledge base in Step S3, fill the prompt word template with the retrieval results, call the large language model to perform generation inference on the filled prompt words, requiring the model to transform each uncovered requirement into a test intent and further decompose it into functional verification sub-intents under normal input conditions, robustness verification sub-intents under boundary value conditions, and fault tolerance verification sub-intents under abnormal input conditions, generate corresponding precondition descriptions, input stimulus sequences, state hold time parameters, expected response descriptions, and judgment criterion expressions for each sub-intent, map the generated results into fields and format them according to the preset test case data structure, and obtain a set of candidate test cases including complete test steps and expected results.

[0025] Step S6: Verify the legality of candidate test cases using the aviation constraint rule base: Input each candidate test case generated in step S5 into the aviation constraint rule base in sequence. First, verify whether the altitude value in the flight condition parameters is within the range of sea level to 15,000 meters, whether the speed value is within the range of Mach 0 to 2, and whether the angle of attack value is within the range of -5 degrees to +45 degrees. Candidate test cases that exceed any of the range boundaries are marked as envelope violations and rejected. Second, verify whether the input signal values ​​in the interface control file are within the recorded lower and upper limits of the range, and whether the output signal values ​​are within the specified lower and upper limits of the range. Within the specified range, candidate test cases that exceed the range are marked as interface violations and rejected; each state transition in the system state transition sequence is re-verified to ensure it is within the allowed transition set defined by the state machine constraints, and candidate test cases that do not meet the transition conditions are marked as state violations and rejected; finally, it is verified whether there are mutually exclusive faults injected simultaneously in the fault injection combination and whether the fault duration is within the configurable range of 50 milliseconds to 5000 milliseconds, and candidate test cases that violate the mutual exclusion rules or exceed the duration limit are marked as security violations and rejected, and candidate test cases that pass all verification items are recorded as valid test cases and a verification pass log is attached.

[0026] Step S7: Calculate coverage and generate a report based on valid test cases: Extract the requirement identifier, interface signal identifier, working condition grid node coordinates, boundary extreme point identifier, fault mode identifier, and criterion branch identifier covered by each test case from all valid test cases obtained in Step S6. Update the target status flag bit corresponding to the coverage status table in Step S4 from 0 to 1. Calculate the ratio of the number of targets with the current status flag bit of 1 to the total number of covered targets in each dimension, and record the current coverage value of each dimension as a percentage. When the current coverage value of a certain dimension is lower than the preset 95% threshold, construct a supplementary generation request based on the set of uncovered targets and trigger the iterative execution of Steps S5 to S6 until the coverage values ​​of all six dimensions reach more than 95% or the number of iterations reaches the preset upper limit of 10. Associate each valid test case with all the target identifiers it covers to generate a requirement traceability matrix. Record the change trend of the coverage value of each dimension over time as a coverage report. Output the traceability matrix and coverage report in the specified format.

[0027] Preferably, in step S4, for any coverage dimension D, the coverage value is determined by the formula... Calculate, where, This represents the coverage ratio of dimension D. This represents the number of target items in dimension D that are covered by at least one valid test case. This represents the total number of target items that need to be covered in dimension D. The target items include requirement items, interface signal value ranges, flight condition grid nodes, boundary value combinations, failure mode instances, and criterion branches.

[0028] Specifically, coverage calculation is based on the element counting principle in set theory. All target items under any coverage dimension are considered a finite set, and target items touched by effective test cases are considered subsets of this set. Coverage is the ratio of the number of elements in the subset to the number of elements in the entire set. This form can intuitively quantify the test sufficiency of each dimension. The total number of target items in the denominator is pre-determined when constructing the coverage model in step S4. For example, the total number of requirement items under the requirement coverage dimension is a certain number; the total number of discretized segments of the interface signal value range under the interface coverage dimension is a certain number; the total number of two-dimensional grid nodes for height and speed under the operating condition coverage dimension is a certain number; the total number of extreme value combinations of each parameter under the boundary coverage dimension is a certain number; the total number of fault mode instances under the fault coverage dimension is a certain number; and the total number of criterion branch conditions under the criterion coverage dimension is a certain number. The number of covered target items in the numerator is obtained by accumulating the count of target items whose status flag is updated from 0 to 1 under each dimension after traversing all effective test cases in step S7. The ratio result of this formula is between 0 and 1. After multiplying by 100, it is presented as a percentage and used for comparison with a preset 95% threshold. When the ratio is lower than the threshold, a supplementary generation process is triggered. This formula does not require any weighting coefficients or smoothing factors, thus maintaining the objectivity and interpretability of the coverage assessment.

[0029] Preferably, in step S6, when the aviation constraint rule base performs legality verification on candidate test cases, it calculates a comprehensive constraint satisfaction index for any candidate test case. ,in, This represents the comprehensive constraint verification result of candidate test case c. This indicates the total number of constraint rule categories participating in the validation. Let represent the validation input feature vector of candidate test case c under the k-th type of constraint rule. This represents the set of legal value ranges specified by the k-th type of constraint rule. The indicator function takes a value of 1 when the input feature is within the valid range, and a value of 0 otherwise. The candidate test case c is considered valid only when the indicator function of all categories takes a value of 1.

[0030] Specifically, the comprehensive constraint satisfaction index is based on multi-rule serial verification logic, treating each type of constraint rule as an independent verification channel. Candidate test cases must pass all channels simultaneously to be deemed valid. This design stems from the extremely high safety and correctness requirements in aerospace simulation testing, where any violation in a single dimension can lead to simulation failure or dangerous conditions. First, the verification input feature vectors of candidate test cases are grouped according to constraint rule categories. Each rule category corresponds to a feature sub-vector; for example, the flight envelope rule corresponds to a three-dimensional feature vector composed of altitude, speed, and angle of attack; the interface rule corresponds to a multi-dimensional feature vector composed of various signal values; and the state machine rule corresponds to a two-dimensional feature vector composed of state transition pairs. Then, for the k-th rule category, its feature sub-vector is compared with the set of legal value ranges specified by that rule category to determine inclusion relationships. Each dimension in the set of legal value ranges is stored as a lower and upper limit of the interval, such as the flight envelope. The valid range for altitude is 0 to 15000, the valid range for speed is 0 to 2, and the valid range for angle of attack is -5 to +45. During verification, the indicator function compares each component in the feature vector one by one to see if it falls within the valid range of the corresponding dimension. When all components fall within the valid range, the indicator function is set to 1, and when any component exceeds the limit, it is set to 0. Finally, the values ​​of the indicator functions of all categories are multiplied together. The comprehensive product result is 1 only when all indicator functions are 1. If any one of them is 0, the product result is 0. The verification output of this formula is a binary result rather than a continuous value, ensuring that there is no ambiguity or compromise space.

[0031] Preferably, in step S5, the large language model agent transforms requirement items into test intentions, and decomposes the test intentions into normal scenarios, boundary scenarios, abnormal scenarios, and fault scenarios. When generating candidate test cases, it uses the formula... Calculate the generation probability of each test scenario, where, This indicates that, given a requirement item q and a knowledge base... and coverage model Generate the j-th test scenario under the given conditions. The probability, This indicates the total number of candidate scenarios. Representing a scene semantic embedding vector, This represents the generated condition vector jointly driven by requirement items, knowledge base, and coverage model. This represents a semantic similarity function between vectors. It is an exponential function.

[0032] Specifically, the probability calculation of test scene generation is based on the soft maximization normalization principle in semantic space. The generation preference of the large language model for candidate scenes is modeled as a conditional probability distribution. This design is because the evaluation value of each candidate output by the large language model under given context conditions usually exists in the form of non-normalized semantic similarity scores, which need to be converted into a probability distribution through exponential normalization in order to sample or rank. First, the encoder of the large language model jointly maps the natural language text of the demand item, the description of related entities in the knowledge base, and the target state encoding in the coverage model into a fixed-dimensional condition vector. The dimension of this vector is usually 768 or 1024, depending on the architecture of the large language model used. Then, the L candidate test scenarios retrieved from the knowledge base are input into the encoder one by one to obtain their respective semantic embedding vectors, with the vector dimension consistent with the dimension of the condition vector. Next, the semantic similarity between the condition vector and the embedding vector of each candidate scenario is calculated. The similarity function adopts the cosine similarity form, that is, the dot product of the two vectors divided by the product of their respective magnitudes, and the resulting similarity value is between -1 and +1. Then, an exponential operation is performed on each similarity value with the natural constant e as the base, and the sum of all L exponential values ​​is used as the normalized denominator. Finally, the exponential value of each candidate scenario is divided by the denominator to obtain the conditional generation probability of the scenario. The sum of all L probabilities is 1. The large language model samples candidate scenarios based on this probability distribution or selects the scenario with the highest probability as the output.

[0033] Preferably, in step 56, when performing legality verification on the candidate test cases, for the flight condition parameter combinations in the candidate test cases, the envelope margin index is calculated based on the flight envelope constraints. ,in, This represents the flight condition parameter vector in the candidate test cases. This represents the total number of dimensions of the flight condition parameters. This represents the value of the nth flight condition parameter. and These represent the minimum and maximum permissible values ​​of the nth flight condition parameter under flight envelope constraints, respectively. This represents the nearest normalized distance from the combination of flight condition parameters to the envelope boundary, when... If the value is less than a preset threshold, the candidate test case is determined to not meet the flight envelope constraint.

[0034] Specifically, the envelope margin index is based on the principle of normalized distance measurement. It treats the flight condition parameter vector of the candidate test case as a point in a multi-dimensional space, and the flight envelope constraint as a hyperrectangular feasible region in this space. The margin index reflects the shortest normalized distance from the point to the boundary of the hyperrectangle. This design stems from the fact that in aerospace engineering, it is not only necessary to determine whether the test case point is located inside the envelope, but also to quantify its distance from the boundary in order to evaluate the safety margin and boundary approximation of the test. In the derivation of the formula, for each dimension of the N-dimensional flight condition parameter vector, the lower limit and upper limit of the envelope value of that dimension are first read from the knowledge base. Then, the two differences between the current value and the lower and upper limits are calculated respectively. These two differences reflect the absolute distance of the current value above the lower limit and below the upper limit. Next, these two differences are divided by the full range width of that dimension to obtain two normalized distance values ​​between 0 and 1, representing the normalized distance of the current value in the lower limit direction and the normalized distance in the upper limit direction, respectively. Then, the smaller of these two normalized distance values ​​is taken as the tightest boundary distance in that dimension. Finally, the tightest boundary distances are calculated for all N dimensions, and the minimum value is taken as the closest normalized distance from the entire flight condition parameter vector to the envelope boundary. This value is between 0 and 1. The smaller the value, the closer it is to the envelope boundary. When this value is less than the preset envelope safety threshold, such as 0.05, it indicates that the candidate test case is too close to or exceeds the envelope boundary, and the system rejects the test case to ensure the safety margin of the simulation operation.

[0035] Preferably, in step S6, when performing legality verification on the candidate test cases, for the system state transition sequence in the candidate test cases, a state transition legality index is calculated based on the system state machine constraints. ,in, This represents a system state transition sequence of length T in the candidate test cases. This represents the system state identifier at time t. This indicates a transition from state to state under system state machine constraints. The set of next states that are allowed to be transitioned to from the starting point. For state transition verification functions, when belong The value is 1 if the check function is true and 0 otherwise. The check function is only valid when all adjacent state transitions in the sequence have a value of 1. A value of 1 indicates that the state transition sequence satisfies the system state machine constraints.

[0036] Specifically, the state transition legality index is based on the principle of serial logic AND operation. It treats the system state transition sequence in candidate test cases as a set of ordered state transition events, and the system state machine constraints as an adjacency list of the state transition graph. Legality verification requires checking each pair of adjacent states in the sequence to verify whether their transition relationship exists in the adjacency list. This design stems from the fact that state transitions in aviation systems must strictly adhere to the designed state machine topology; any undefined state jump may lead to system logic disorder or safety mechanism failure. In the formula derivation process, a system state sequence of length T is first parsed from the candidate test cases, and each state in the sequence is identified... The values ​​are discrete enumerations predefined in the knowledge base, such as initialization state coded as 0, standby state coded as 1, active state coded as 2, fault state coded as 3, and shutdown state coded as 4; then, for the state at time t in the sequence... Query from the state machine constraint library The set of next states that are allowed to be transitioned to from the starting point. This set stores all the target state codes that are allowed by the design in list form; then, a state transition verification function is executed to compare... Does the encoded value belong to For any code in the list, if it belongs to the list, the check function takes a value of 1; otherwise, it takes a value of 0. Perform the above check on all T-minus-1 pairs of adjacent states in the sequence one by one. The check result for each pair of states is a binary constant. Finally, multiply the values ​​of all T-minus-1 check functions together. Due to the nature of the multiplication operation, if there is any pair of adjacent states in the sequence that has no defined transition (i.e., the check value for that pair is 0), the entire product result is 0, indicating that the sequence is invalid. Only when all adjacent state transitions are allowed by the state machine constraints (i.e., all check values ​​are 1) is the product result 1, indicating that the sequence is valid. The output of this formula is a deterministic binary result, without introducing any probability or weight factors.

[0037] Preferred, such as Figure 2 As shown, step S2 includes the following steps: Step S21, performing semantic annotation and dependency parsing based on aerospace ontology on the system requirements and the test criteria to extract requirement items and their associated judgment conditions; Step S22, performing field-level parsing and data type inference on the interface control file to extract the names, data types, and value constraints of input and output signals; Step S23, performing parameter extraction and time-series relationship modeling on the simulation model description and the flight conditions to extract model parameters, flight condition variables, and their coupling relationships; Step S24, performing fault tree logic parsing on the fault modes to extract fault type, fault injection location, fault duration, and recovery conditions.

[0038] Specifically, step S21 performs semantic annotation and dependency parsing based on the aerospace domain ontology on system requirements and test criteria. The aerospace domain ontology pre-constructs a professional terminology dictionary and semantic relationship templates including subdomains such as flight control, navigation, propulsion, and electromechanical systems. The semantic annotation process matches and maps each word in the requirement text with the concept nodes in the ontology. The dependency parsing identifies the subject-verb-object structure and modification relationship, extracting the requirement items and their associated judgment conditions. For example, it extracts the three elements of functional subject, operation action, and numerical constraint from statements with threshold descriptions. Step S22 performs field-level parsing and data type inference on the interface control file. Field-level parsing reads the signal definition lines field by field according to delimiters or fixed widths, and the data... Type inference determines whether the signal is floating-point, integer, Boolean, or enumerated based on keywords in the field, and extracts the lower and upper limits of the value range or the enumerated value list for each signal; Step S23 extracts parameters and models the timing relationship for the simulation model description and flight conditions. Parameter extraction identifies the parameter names, values, and units in the document. Timing relationship modeling establishes the calculation dependency order and execution cycle between parameters based on the state transition table or differential equation order in the model description; Step S24 performs fault tree logic parsing for the fault mode, identifies the top, middle, and bottom event nodes in the fault tree, and extracts the injection location identifier, the default value of the duration parameter (500 milliseconds), and the state variable threshold in the recovery condition for each fault type.

[0039] Preferred, such as Figure 3 As shown, step S3 includes the following steps: Step S31: Instantiate the parsed test objects, interface signals, flight condition variables, constraints, fault types, and judgment rules into entity nodes in the knowledge graph; Step S32: Establish requirement-interface association edges based on the reference relationship between requirement items and interface signals, and establish interface-parameter association edges based on the mapping relationship between interface signals and model parameters; Step S33: Establish condition-requirement association edges based on the applicability relationship between flight condition variables and requirement items, and establish fault-object association edges based on the causal relationship between fault modes and affected test objects; Step S34: Establish criterion-requirement association edges based on the verification relationship between test criteria and requirement items, forming an aviation simulation test knowledge base with a directed hyperedge structure.

[0040] Specifically, step S31 instantiates the parsed test objects, interface signals, flight condition variables, constraints, fault types, and judgment rules into entity nodes in the knowledge graph. Each entity node occupies one record in the database and is assigned a unique 64-bit integer identifier. The node attributes store its name, type label, and original description text. Step S32 performs exact string matching between the signal names referenced in the requirement items and the signal names in the interface signal definitions. If a match is successful, a directed association edge is established from the requirement entity to the interface signal entity. Then, a directed association edge is established from the interface signal entity to the model parameter entity based on the model parameter names mapped by the interface signals. Each association edge stores its confidence value, which is 1.0 by default. Step S33 Matching is performed based on the mission profile category to which the flight condition variables belong and the applicable profile category indicated in the requirement items. After successful matching, a directed association edge is established from the condition variable entity to the requirement entity. Then, matching is performed based on the affected function name recorded in the failure mode analysis document and the name of the test object. After successful matching, a directed association edge is established from the failure mode entity to the test object entity. Step S34 performs precise matching based on the requirement number referenced in the test criteria and the requirement item number. After successful matching, a directed association edge is established from the criterion entity to the requirement entity. All association edges are organized according to the directed hypergraph structure, allowing a hyperedge to connect multiple head nodes and multiple tail nodes at the same time, ultimately forming an aviation simulation test knowledge base covering all test elements and their associated paths.

[0041] Preferred, such as Figure 4 As shown, step S4 includes the following steps: Step S41: Extract the set of requirement items as requirement coverage targets, the set of interface signal value intervals as interface coverage targets, and the combination grid of flight condition parameters as condition coverage targets from the aviation simulation test knowledge base; Step S42: Generate a set of boundary coverage targets based on the boundary extreme value combination of each flight condition parameter, generate a set of fault coverage targets based on the set of fault mode instances, and generate a set of criterion coverage targets based on the set of test criterion branch conditions; Step S43: Construct a coverage target dictionary according to the six dimensions of requirement, interface, condition, boundary, fault, and criterion, and assign a unique identifier and associated tracing path to each coverage target; Step S44: Establish a cross-coverage matrix between each coverage dimension, record the association strength between any two coverage targets of different dimensions, and use it as the priority basis for subsequent supplementary test case generation.

[0042] Specifically, step S41 executes a structured query statement from the aviation simulation test knowledge base to extract a list of all requirement item numbers as the requirement coverage target set, extracts all interface signal identifiers and their value range segments as the interface coverage target set, divides the altitude range from 0 to 15000 meters into 30 segments at 500-meter intervals, and divides the speed range from 0 to Mach 2 into 20 segments at 0.1 Mach intervals. The segments in the two dimensions are combined by Cartesian product to generate 600 working condition grid nodes as the working condition coverage target set; step S42 generates all extreme value combinations based on the lower and upper limits of each flight working condition parameter as the boundary coverage target set, queries a list of all fault mode instance numbers from the knowledge base as the fault coverage target set, and queries the knowledge base for the logical expression numbers corresponding to all test criterion branch conditions. The list serves as the set of criteria coverage targets. Step S43 constructs a coverage target dictionary according to six dimensions: requirements, interfaces, operating conditions, boundaries, faults, and criteria. Each coverage target occupies a key-value pair entry in the dictionary. The key is the unique identifier string of the target, and the value is the description information and the associated path array of the target. The associated path array records the identifiers of all other test elements reachable from the target through the associated edges in the knowledge base. Step S44 constructs a six-row, six-column cross coverage matrix. The rows and columns of the matrix correspond to the six coverage dimensions. The element in the i-th row and j-th column of the matrix is ​​the total number of associated edges between all targets in the i-th dimension and all targets in the j-th dimension. This matrix is ​​used to determine the priority when generating supplementary test cases. The larger the total number of associations in a dimension combination, the tighter the coupling between the two. When generating supplementary test cases, this cross area is covered first.

[0043] Preferred, such as Figure 5 As shown, step S5 includes the following steps: Step S51: Vectorize the current coverage state of each coverage dimension in the multi-dimensional coverage model to generate a coverage state vector; Step S52: Retrieve requirement items and test criteria related to the current coverage state vector from the aviation simulation test knowledge base to construct a test intent generation prompt context; Step S53: Reason about the prompt context through a large language model agent to transform the functional descriptions in the requirement items into a test intent set including normal input ranges, boundary input values, and abnormal input conditions; Step S54: For each test intent in the test intent set, automatically match the preconditions, input stimuli, expected outputs, and judgment criteria according to the association relationships in the knowledge base to combine them into structured candidate test cases.

[0044] Specifically, step S51 involves vectorizing the current coverage status of each coverage dimension in the multidimensional coverage model. The ratio of the number of covered targets to the total number of targets for each of the six dimensions is read from the coverage status table. These six ratios are then concatenated into a six-dimensional floating-point vector in a fixed order of requirement, interface, operating condition, boundary, fault, and criterion. Each component ranges from 0.0 to 1.0, and this vector serves as the overall numerical representation of the current coverage status. Step S52 involves performing a search query from the aerospace simulation test knowledge base. The coverage target identifiers corresponding to dimensions with components below 0.95 in the current coverage status vector are used as search conditions. Requirement entries and test criterion nodes directly connected to these uncovered targets via association edges are retrieved from the knowledge base. The description texts of all retrieved nodes are sorted from highest to lowest association strength, and the top 20 are extracted, along with the coverage status vector. Together, they constitute the context for generating test intent prompts; step S53 uses a large language model agent to reason and generate the prompt context, parsing the functional description statements in the requirement items into a sequence of test actions, generating test intents to verify basic functions for normal input ranges, generating test intents to verify robustness for boundary input values, and generating test intents to verify fault tolerance mechanisms for abnormal input conditions. Each test intent includes two attribute fields: input stimulus type and expected response type; step S54 retrieves matching initial values ​​of precondition state variables, specific numerical sequences of input stimuli, expected output signal names and numerical ranges, and threshold conditions of judgment criteria from the knowledge base for each test intent, combines these retrieval results with the attribute fields of the test intent, and formats them according to the field structure of standard test cases, finally outputting a complete candidate test case record.

[0045] This invention presents an agent-based method for generating and verifying aviation simulation test cases and coverage. It constructs an aviation simulation test knowledge base integrating requirements, interface signals, model parameters, flight conditions, fault modes, and test criteria. A hypergraph structure is used to establish high-order semantic relationships between various test elements, providing a unified data foundation for test case generation and traceability. A multi-dimensional coverage model is used to quantitatively evaluate six dimensions: requirements, interfaces, conditions, boundaries, faults, and criteria. Based on coverage gaps, a large language model agent is automatically driven to perform targeted supplementation generation, enabling the test design process to have closed-loop self-feedback capabilities. Simultaneously, an aviation constraint rule base is introduced to perform deterministic verification of flight envelopes, interface ranges, state machine transitions, and safety rules for candidate test cases during the generation phase, ensuring the executability and security of the generated results on the actual simulation platform. Finally, a structured test case set, coverage report, traceability matrix, and simulation script template are automatically output, significantly improving test design efficiency and automation.

[0046] This invention establishes a unified association between requirement items, interface signals, operating condition variables, and failure modes through a knowledge base. This allows the system to automatically maintain the traceability relationship between test cases and various test elements, enabling rapid location of affected test cases after requirement changes, significantly reducing the risk of omissions and management overhead in regression testing. Addressing the issues in the prior art where manual enumeration of operating conditions easily misses boundary combinations and coverage cannot be fed back during the design phase, this invention uses a multi-dimensional coverage model to quantify the coverage status of each dimension in real time. It identifies uncovered requirement branches, boundary extremes, operating condition grid nodes, and failure combinations, and directly uses coverage gaps as supplementary generation instructions for the intelligent agent. This transforms coverage evaluation from passive statistics after test completion to proactive guidance during the test design phase, fundamentally solving the problems of insufficient coverage in manual design and unreasonable allocation of test resources.

[0047] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," "link," and "fix" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0048] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various equivalent changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for generating and verifying test cases for aviation simulation based on intelligent agents, characterized in that, Includes the following steps: Step S1: Obtain aviation simulation test input data, which includes system requirements, interface control files, simulation model description, flight conditions, failure modes, and test criteria. Step S2: Perform semantic dependency parsing and entity relation extraction on the input data to construct a heterogeneous information graph with test objects, interface signals, flight condition variables, constraints, fault types and judgment rules as nodes; Step S3: Construct an aviation simulation test knowledge base based on the heterogeneous information graph, and establish high-order correlations between requirements, interface signals, model parameters, flight conditions, failure modes and test criteria through a hypergraph structure; Step S4: Construct a multi-dimensional coverage model based on the set of coverage targets in the knowledge base. The multi-dimensional coverage model includes at least one or more of the following: requirement coverage, interface coverage, operating condition coverage, boundary coverage, fault coverage, and criterion coverage. Step S5: Based on the aviation simulation test knowledge base and the multidimensional coverage model, the large language model intelligent agent performs test intent parsing and scenario decomposition to generate candidate test cases. Step S6: Perform a legality check on the candidate test cases using the aviation constraint rule base, eliminate or correct candidate test cases that do not meet the constraints, and obtain valid test cases; Step S7: Calculate coverage based on the valid test cases, and generate a coverage report and a requirement traceability matrix based on the coverage results.

2. The method for generating and verifying aviation simulation test cases based on intelligent agents according to claim 1, characterized in that, In step S4, for any coverage dimension D, the coverage value is determined by the formula... Calculate, where, This represents the coverage ratio of dimension D. This represents the number of target items in dimension D that are covered by at least one valid test case. This represents the total number of target items that need to be covered in dimension D. The target items include requirement items, interface signal value ranges, flight condition grid nodes, boundary value combinations, failure mode instances, and criterion branches.

3. The method for generating and verifying aviation simulation test cases based on intelligent agents according to claim 1, characterized in that, In step S6, when the aviation constraint rule base performs legality verification on candidate test cases, it calculates a comprehensive constraint satisfaction index for any candidate test case. ,in, This represents the comprehensive constraint verification result of candidate test case c. This indicates the total number of constraint rule categories participating in the validation. Let represent the validation input feature vector of candidate test case c under the k-th type of constraint rule. This represents the set of legal value ranges specified by the k-th type of constraint rule. The indicator function takes a value of 1 when the input feature is within the valid range, and a value of 0 otherwise. Only when the indicator function of all categories takes a value of 1 is the candidate test case c determined to be valid.

4. The method for generating and verifying aviation simulation test cases based on intelligent agents according to claim 1, characterized in that, In step S5, the large language model agent transforms requirement items into test intentions, and decomposes the test intentions into normal scenarios, boundary scenarios, abnormal scenarios, and fault scenarios. When generating candidate test cases, it follows the formula... Calculate the generation probability of each test scenario, where, This indicates that given a requirement item q and a knowledge base and coverage model Generate the j-th test scenario under the given conditions. The probability, This indicates the total number of candidate scenarios. Representing a scene semantic embedding vector, This represents the generated condition vector jointly driven by requirement items, knowledge base, and coverage model. This represents a semantic similarity function between vectors. It is an exponential function.

5. The method for generating and verifying aviation simulation test cases based on intelligent agents according to claim 1, characterized in that, In step 56, when performing legality verification on the candidate test cases, for the flight condition parameter combinations in the candidate test cases, the envelope margin index is calculated based on the flight envelope constraints. ,in, This represents the flight condition parameter vector in the candidate test cases. This represents the total number of dimensions of the flight condition parameters. This represents the value of the nth flight condition parameter. and These represent the minimum and maximum permissible values ​​of the nth flight condition parameter under flight envelope constraints, respectively. This represents the nearest normalized distance from the combination of flight condition parameters to the envelope boundary, when... If the value is less than a preset threshold, the candidate test case is determined to not meet the flight envelope constraint.

6. The method for generating and verifying aviation simulation test cases based on intelligent agents according to claim 1, characterized in that, In step S6, when performing legality verification on the candidate test cases, for the system state transition sequence in the candidate test cases, a state transition legality index is calculated based on the system state machine constraints. ,in, This represents a system state transition sequence of length T in the candidate test cases. This represents the system state identifier at time t. This indicates a transition from state to state under system state machine constraints. The set of next states that are allowed to be transitioned to from the starting point. For state transition verification functions, when belong The value is 1 if the check function is true and 0 otherwise. The check function is only valid when all adjacent state transitions in the sequence have a value of 1. A value of 1 indicates that the state transition sequence satisfies the system state machine constraints.

7. The method for generating and verifying aviation simulation test cases based on intelligent agents according to claim 1, characterized in that, Step S2 includes: The system requirements and test criteria are subjected to semantic annotation and dependency parsing based on aerospace ontology to extract requirement items and their associated judgment conditions; the interface control file is subjected to field-level parsing and data type inference to extract the names, data types and value constraints of input and output signals. The simulation model is described and the flight condition execution parameters are extracted and the timing relationship is modeled. The model parameters, flight condition variables and their coupling relationships are extracted. Fault tree logic is parsed for the fault mode to extract the fault type, fault injection location, fault duration and recovery conditions.

8. The method for generating and verifying aviation simulation test cases based on intelligent agents according to claim 1, characterized in that, Step S3 includes: The parsed test objects, interface signals, flight condition variables, constraints, fault types, and judgment rules are instantiated as entity nodes in the knowledge graph; the requirement-interface association is established based on the reference relationship between requirement items and interface signals, and the interface-parameter association is established based on the mapping relationship between interface signals and model parameters; the condition-requirement association is established based on the applicability relationship between flight condition variables and requirement items, and the fault-object association is established based on the causal relationship between fault modes and affected test objects. Based on the verification relationship between test criteria and requirement items, establish the association edge between criteria and requirements to form an aerospace simulation test knowledge base with a directed hyperedge structure.

9. The method for generating and verifying aviation simulation test cases based on intelligent agents according to claim 1, characterized in that, Step S4 includes: The set of requirement items is extracted from the aviation simulation test knowledge base as the requirement coverage target, the set of interface signal value ranges is extracted as the interface coverage target, and the flight condition parameter combination grid is extracted as the condition coverage target. A boundary coverage target set is generated based on the boundary extreme value combination of each flight condition parameter, a fault coverage target set is generated based on the fault mode instance set, and a criterion coverage target set is generated based on the test criterion branch condition set. A coverage target dictionary is constructed according to six dimensions: requirements, interface, operating condition, boundary, fault, and criterion, and a unique identifier and associated tracing path are assigned to each coverage target. A cross-coverage matrix is ​​established between each coverage dimension to record the association strength between any two coverage targets of different dimensions, which serves as the priority basis for the generation of subsequent supplementary test cases.

10. The method for generating and verifying aviation simulation test cases based on intelligent agents according to claim 1, characterized in that, Step S5 includes the following steps: The current coverage state of each coverage dimension in the multidimensional coverage model is vectorized and encoded to generate a coverage state vector; the requirement items and test criteria related to the current coverage state vector are retrieved from the aviation simulation test knowledge base to construct a test intent generation prompt context; the prompt context is reasoned through a large language model agent to transform the functional descriptions in the requirement items into a test intent set including normal input range, boundary input values ​​and abnormal input conditions; For each test intent in the set of test intents, the preconditions, input stimuli, expected outputs and judgment criteria are automatically matched according to the associations in the knowledge base, and combined into structured candidate test cases.

Images