Test evaluation method and system based on large language model

Abstract

The invention relates to the technical field of flight tests, in particular to a test evaluation method and system based on a large language model, and the method comprises the steps: constructing a flight test knowledge base; receiving a flight test task request of a natural language and analyzing the flight test task request to generate a structured task description; based on domain knowledge retrieval constraint conditions, a flight test task plan meeting the conditions is generated through a large language model; analyzing a scene, an instruction and an evaluation demand from the flight test task plan; constructing a three-dimensional test scene model; generating and verifying a flight instruction sequence; executing flight simulation and collecting test data; and calculating an evaluation index based on the evaluation demand and the test data and generating an evaluation report. According to the method, the full-process automation from natural language input to evaluation output is realized, the test efficiency is remarkably improved, the safety compliance of the scheme is ensured, and the test effect is comprehensively reflected through multi-dimensional quantitative evaluation.

Description

Technical Field

[0001] This application relates to the field of flight test technology, specifically to a test evaluation method and system based on a large language model. Background Technology

[0002] Flight testing is a crucial step in verifying aircraft performance and system reliability. As aviation technology develops towards intelligence and systematization, higher demands are placed on the efficiency, coverage, and accuracy of test evaluations.

[0003] In existing technologies, flight test planning and evaluation methods mainly assist the test process by constructing digital simulation environments: some systems use parametric modeling tools to predefine test scenarios, drive flight simulators to execute predetermined routes through scripts, and collect simulation data for post-test analysis; other solutions attempt to introduce expert systems, encoding some flight rules into judgment logic to verify the rationality of mission plans. These methods have improved the automation level of test preparation to a certain extent.

[0004] However, existing flight test planning and evaluation methods still have significant limitations: First, the flight test design process heavily relies on the manual participation of domain experts. From mission conception and scenario construction to command generation, everything must be done manually, and the system lacks the ability to directly understand and plan complex test missions from natural language. Second, the generated mission plans lack safety-related constraints with domain knowledge, making it difficult to ensure that they comply with the normative requirements of authoritative documents such as flight manuals and airworthiness regulations from beginning to end. Third, the scheme verification stage is relatively independent, and the physical feasibility of the command sequence is often only revealed after simulation execution, lacking a rapid verification and closed-loop correction mechanism before or during the process. Fourth, the evaluation dimensions are usually limited to basic flight parameters, failing to conduct a comprehensive quantitative evaluation from multiple perspectives such as mission completion quality and command execution fidelity. Summary of the Invention

[0005] To address the technical problems of existing flight test planning and evaluation methods that rely on manual design and lack natural language understanding capabilities, are disconnected from domain knowledge constraints in mission planning, suffer from delayed command verification, and have a single evaluation dimension, this application provides a test evaluation method and system based on a large language model. By constructing a flight test knowledge base, realizing natural language task parsing, generating mission plans by combining a large language model with domain constraints, introducing flight mechanics simulation verification, and establishing a multi-dimensional evaluation system, the entire process from mission reception to evaluation output is made intelligent, significantly improving test efficiency, safety, and the comprehensiveness of evaluation.

[0006] Firstly, this application provides an experimental evaluation method based on a large language model, comprising the following steps: S1. Construct a flight test knowledge base, which stores flight test knowledge data, including structured and unstructured data; S2. Receive flight test mission requests in natural language form and perform semantic parsing to generate a structured mission description containing mission instructions, constraint elements, and mission objectives; S3. Based on the structured task description, retrieve relevant flight constraints from the flight test knowledge base; S4. Based on the structured mission description and flight constraints, a large language model is used to generate a flight test mission plan that includes mission phase division, resource allocation and time nodes; S5. Extract the 3D scene configuration requirements, flight command sequence requirements, and test evaluation requirements from the flight test mission plan; The test evaluation requirements include a list of monitoring parameters, definitions of evaluation indicators, and evaluation standards. S6. Construct a 3D test scenario model based on 3D scene configuration requirements; S7. Generate an initial flight command sequence based on the flight command sequence requirements and perform dynamic feasibility verification. If the verification fails, iteratively optimize the flight command sequence until it passes the verification to obtain an executable flight command sequence. S8. Based on the executable flight command sequence, perform flight simulation in a three-dimensional test scenario model, and collect test data of corresponding parameters during the flight simulation based on the monitoring parameter list; S9. Based on the test evaluation requirements and test data, calculate the corresponding evaluation indicators and generate an evaluation report containing the evaluation indicators.

[0007] It should be further noted that in step S1, the sources of flight test knowledge data include flight manuals, test reports, failure modes, airworthiness standards, performance parameters and technical specifications.

[0008] It should be further noted that in step S1, the structured data is stored by constructing a knowledge graph based on an attribute graph model, wherein: The nodes of the knowledge graph represent entities in the field of flight testing, including aircraft entities containing model and performance parameters, test subject entities containing test types and risk levels, and constraint entities containing safety restrictions and regulatory requirements. Edges represent relationships between entities, including relationship type and relationship strength attributes.

[0009] It should be further noted that in step S1, the unstructured data is stored by constructing a vector database, specifically including: Segment unstructured documents into semantically coherent text fragments; Use an embedding model to convert text fragments into vector representations; Store vectors in a vector database and create a semantic retrieval index.

[0010] It should be further noted that each text segment is 800-1200 characters in size, and there is an overlap of 150-250 characters between adjacent text segments.

[0011] It should be further noted that the flight test knowledge base supports hybrid retrieval methods, including keyword-based retrieval, semantic similarity-based retrieval, and graph path-based reasoning retrieval. Among them, graph path-based reasoning retrieval involves traversing and querying along entity relationship paths in the knowledge graph to discover implicit constraint relationships.

[0012] It should be further noted that the flight test knowledge base also supports a dynamic update mechanism, which can receive new flight test data and update the knowledge graph and vector database.

[0013] It should be further explained that in step S2, the flight test mission request is semantically parsed to generate a structured mission description containing mission instructions, constraint elements, and mission objectives, specifically including: Key entities in flight test mission requests are identified using named entity recognition technology; Based on the identified key entities, the natural language description in the flight test mission request is converted into a standardized sequence of flight operation instructions to form the mission instructions; Based on key entities and task instructions, identify the constraint elements to be queried; Based on the identified key entities, the descriptions related to the mission objectives in the flight test mission request are parsed into quantifiable evaluation indicators, which constitute the mission objectives. The consistency of task instructions, constraint elements, and task objectives is verified. Once the consistency verification is successful, a structured task description is obtained.

[0014] It should be further noted that the constraint elements include one or more of physical limitations, safety boundaries, and regulatory requirements, used for precise retrieval from the flight test knowledge base.

[0015] It should be further noted that key entities include multiple aspects such as aircraft model, test subjects, environmental conditions, and performance parameters.

[0016] It should be further noted that resolving descriptions related to mission objectives into quantifiable evaluation metrics includes: resolving regional search tasks into specific search coverage metrics, and resolving performance testing tasks into speed, altitude, or maneuverability parameters.

[0017] It should be further explained that consistency verification includes checking the initial compliance of the task instruction sequence with the constraint elements, as well as the matching degree between the task instruction sequence and the task objective.

[0018] It should be further noted that, in step S3, retrieving relevant flight constraints from the flight test knowledge base includes: Based on constraint elements, graph queries are performed in the knowledge graph to extract structured flight constraints, including discovering implicit constraint relationships related to the current test mission through graph path traversal. Implicit constraint relationships include one or more of the following when a specified type of aircraft performs a specific maneuver: load limit, altitude limit, and speed limit. Based on constraint features, a semantic search is performed in the vector database to extract unstructured flight constraints. The semantic search in the vector database includes: Generate search keywords based on constraint elements; Calculate the semantic similarity between the search keywords and text fragments in the knowledge base; Return the text fragment with the highest similarity as the search result.

[0019] It should be further noted that structured flight constraints include physical limitations and safety boundaries extracted from the knowledge graph, while unstructured flight constraints include regulatory requirements extracted from documents through semantic search.

[0020] It should be further noted that the process also includes integrating all retrieved flight constraints to obtain a complete set of flight constraints. The integration process includes deduplication, merging, and priority sorting.

[0021] It should be further noted that the mission phases include the takeoff phase, cruise phase, test maneuver phase, and landing phase; Resource allocation includes aircraft type selection, payload configuration, and fuel calculation; The time nodes include the start time, duration, and end time of each task phase.

[0022] It should be further noted that step S4, which involves generating the flight test mission plan using a large language model, includes: The structured task description and flight constraints are combined into prompt words and input into the large language model. The large language model generates a preliminary task plan that includes task phase division, resource allocation and time nodes. The preliminary mission plan undergoes rule checks and model verification. Once both rule checks and model verifications are passed, the flight test mission plan is obtained.

[0023] It should be further noted that the rule check includes logical consistency checks and constraint compliance verification to ensure that the flight test mission plan meets all constraints.

[0024] It should be further noted that model validation includes multiple rounds of review of the initial task plan using a large language model, identifying potential conflicts and iteratively optimizing it.

[0025] It should be further noted that in step S5, the requirements for 3D scene configuration include geographical environment parameters, airspace feature elements, ground facility configuration, and meteorological conditions.

[0026] It should be further noted that meteorological conditions include wind speed, visibility, cloud cover, and special weather conditions.

[0027] It should be further noted that in step S5, the flight command sequence requirements include flight phase division, operation timing arrangement, and flight parameter settings, among which: Flight phases are divided into takeoff, cruise, test maneuver, and landing phases. The operational sequence includes the time nodes and duration of each flight phase; Flight parameter settings include speed, altitude, and heading parameters.

[0028] It should be further noted that in step S5, the list of monitoring parameters includes aircraft position data, attitude data, speed data, and engine status data; The evaluation indicator definition includes the names of all evaluation indicators involved in the evaluation and their corresponding calculation rules; The criteria for success include the threshold range and judgment rules for all evaluation indicators involved in the assessment.

[0029] It should be further noted that the evaluation metrics included in the assessment are at least one of regional search coverage, instruction execution completeness, and maximum speed completion rate.

[0030] It should be further explained that step S6, which involves constructing the three-dimensional test site model, specifically includes: Based on the geographic environment parameters in the 3D scene configuration requirements, construct a basic geographic environment including terrain and landforms. Based on the airspace feature elements in the configuration requirements of the 3D scene, no-fly zones and detection ranges are configured in the basic geographic environment; Based on the configuration requirements of ground facilities in the three-dimensional scene, a ground facility model is established in the basic geographic environment; Meteorological environment fields are generated based on the meteorological conditions in the 3D scene configuration requirements and integrated into the basic geographic environment to form a complete 3D test scene model.

[0031] It should be further explained that constructing the basic geographic environment includes: using a digital elevation model to construct the terrain surface and overlaying vector map data to form a complete basic geographic environment.

[0032] It should be further noted that the ground facility model includes the airport runway model and the navigation facility model.

[0033] It should be further noted that step S7, generating the initial flight command sequence based on the flight command sequence requirements, includes the following steps: Analyze the flight phase divisions in the flight instruction sequence requirements to determine the sequence and connection relationships of each mission phase; Based on the operational sequence arrangement in the flight instruction sequence requirements, allocate time nodes and durations for each flight phase; Based on the flight parameter settings in the flight command sequence requirements, generate command parameters including waypoint coordinates, speed values, altitude values, and heading angles; The sequence of flight phases, the allocated time nodes and durations, and the generated command parameters are arranged and integrated according to time sequence and logical relationship to form a complete initial flight command sequence.

[0034] It should be further noted that step S7, which verifies the dynamic feasibility of the initial flight command sequence, includes: Input the initial flight command sequence into the flight mechanics simulator to perform a 6-DOF dynamics simulation. During the simulation, the angle of attack, thrust-to-weight ratio, and trajectory parameters are monitored. The verification fails when the angle of attack exceeds the stall threshold, the thrust-to-weight ratio does not meet the climb requirements, or the flight path crosses a no-fly zone.

[0035] It should be further noted that the dynamic feasibility verification also includes checking whether the command sequence violates flight safety boundaries and airworthiness standards.

[0036] It should be further noted that flight dynamics models using MATLAB, Simulink, or X-Plane were used for simulation verification.

[0037] It should be further noted that in step S7, the iterative optimization of the flight command sequence includes: Generate natural language error descriptions based on the monitoring results of failed verification; The natural language error descriptions are used as feedback information input into the large language model; The optimized flight instruction sequence is regenerated using a self-refine strategy based on a large language model.

[0038] It should be further noted that the natural language error description includes specific abnormal parameter values ​​and the location where the error occurred.

[0039] It should be further noted that step S8, performing flight simulation, includes: Use a flight simulator to load executable flight command sequences and 3D test scenario models; The motion state and system response of an aircraft are simulated based on a six-degree-of-freedom flight dynamics model.

[0040] It should be further noted that the flight simulator uses flight dynamics models from MATLAB, Simulink, or X-Plane for simulation calculations.

[0041] It should be further noted that in step S8, the test data includes the aircraft's position coordinates, attitude angles, velocity vectors, acceleration values, and engine state parameters.

[0042] It should be further noted that in step S9, an evaluation report is generated using a large language model, including: Construct evaluation report prompts that include task description elements, execution process elements, evaluation result elements, and structured requirement elements, where: The mission description elements include the mission objectives, constraints, and performance requirements in the flight test mission plan; The execution process elements include the instruction steps of the executable flight command sequence and the key flight data recorded in the test data; The evaluation results include the calculation results of the evaluation indicators; The structured requirements specify the elements to generate an evaluation report that includes four parts: an execution summary, detailed analysis, problem identification, and improvement recommendations. An evaluation report is generated based on prompts from a large language model, including an execution summary, detailed analysis, problem identification, and improvement suggestions. The execution summary includes a summary and evaluation of the overall task completion status; Detailed analysis includes performance analysis of various parameters during flight; Problem identification includes analyzing the discrepancy between actual results and expected goals; The improvement suggestions include specific optimization schemes for identifying problems.

[0043] Secondly, this application provides an experiment evaluation system based on a large language model to implement the above-mentioned experiment evaluation method, including: The Flight Knowledge Base Construction Module is used to build a flight test knowledge base; The mission description generation module is used to receive flight test mission requests in natural language form and perform semantic parsing to generate a structured mission description that includes mission instructions, constraint elements, and mission objectives. The constraint retrieval module is used to retrieve relevant flight constraints from the flight test knowledge base based on the structured task description; The mission plan generation module is used to generate flight test mission plans that include mission phase divisions, resource allocation, and time nodes based on structured mission descriptions and flight constraints using a large language model. The mission plan parsing module is used to parse the three-dimensional scene configuration requirements, flight command sequence requirements, and test evaluation requirements from the flight test mission plan; The 3D scene model building module is used to build 3D test scene models based on 3D scene configuration requirements. The flight command sequence generation module is used to generate an initial flight command sequence based on the flight command sequence requirements and perform dynamic feasibility verification. If the verification fails, the flight command sequence is iteratively optimized until the verification is passed, and an executable flight command sequence is obtained. The flight simulation module is used to perform flight simulation in a three-dimensional test scenario model based on an executable flight command sequence, and to collect test data of corresponding parameters during the flight simulation based on a list of monitoring parameters. The evaluation index and evaluation report generation module is used to calculate the corresponding evaluation indexes based on the test evaluation requirements and test data, and generate an evaluation report containing the evaluation indexes.

[0044] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described test evaluation method.

[0045] Fourthly, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described test evaluation method.

[0046] As can be seen from the above technical solutions, this application has the following advantages: 1. This application constructs a flight test knowledge base that integrates structured and unstructured knowledge, providing a unified and authoritative knowledge source for test missions. Relevant constraints can be automatically acquired and embedded during the mission analysis and planning stages, thereby improving the standardization and security of the generated solutions from the outset.

[0047] 2. This application significantly lowers the barrier to entry for the system by receiving natural language task requests and automatically parsing and generating structured task descriptions. Even non-professional users can intuitively describe complex experimental intentions, reducing the reliance on human experts for task planning.

[0048] 3. This application achieves rapid and automatic generation of experimental plans by introducing a large language model to generate task plans and combining them with knowledge base constraints for verification. At the same time, it ensures that the plans comply with physical laws and safety standards, and significantly shortens the experimental design cycle.

[0049] 4. This application introduces a flight mechanics simulator to verify the dynamic feasibility of the command generation process and establishes an iterative optimization mechanism based on natural language feedback, thereby realizing the pre-physical verification and automatic correction of the command sequence and effectively preventing infeasible solutions from entering the execution process.

[0050] 5. This application achieves a leap from single result verification to multi-dimensional task completion quality assessment by automatically calculating evaluation indicators and generating evaluation reports based on multi-dimensional experimental data, providing a more comprehensive and quantitative analysis of experimental results. Attached Figure Description

[0051] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart of an experimental evaluation method based on a large language model in one embodiment of this application.

[0053] Figure 2 This is a schematic block diagram of an experiment evaluation system based on a large language model in one embodiment of this application.

[0054] Figure 3 This is a schematic diagram of the hardware structure of an electronic device in one embodiment of this application. Detailed Implementation

[0055] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] The experimental evaluation methods involved in this application will be described in detail below. Specific details such as particular system structures and technologies are presented for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details.

[0057] In the test evaluation methods involved in this application, the term "comprising" indicates the presence of the described feature, whole, step, operation, element, and / or component, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or collections thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0058] To facilitate a clear description of the technical solutions of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0059] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0060] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0061] The test evaluation method provided in this application embodiment is executed by a computer device, and correspondingly, the test evaluation system based on a large language model runs on the computer device.

[0062] Figure 1 This is a flowchart of an embodiment of an experimental evaluation method based on a large language model according to this application. Wherein, Figure 1 The implementing entity can be a test evaluation system. Depending on different needs, the order of the steps in this flowchart can be changed, and some can be omitted.

[0063] like Figure 1 As shown, the experimental evaluation method based on the large language model includes: Step S1: Construct a flight test knowledge base, which stores flight test knowledge data, including structured and unstructured data.

[0064] By constructing a flight test knowledge base that includes both structured and unstructured data, unified storage and systematic management of heterogeneous data from multiple sources, such as flight manuals and test reports, have been achieved, providing comprehensive and reliable domain knowledge support for subsequent mission planning and safety verification.

[0065] In some specific embodiments, the sources of flight test knowledge data include flight manuals, test reports, failure modes, airworthiness standards, performance parameters, and technical specifications.

[0066] By clearly defining the sources of flight test knowledge data, including flight manuals, test reports, failure modes, airworthiness standards, performance parameters, and technical specifications, the comprehensiveness and authority of the knowledge base content are ensured, providing a rich and accurate knowledge foundation for subsequent intelligent planning and safety verification.

[0067] In some specific embodiments, structured data is stored by constructing a knowledge graph based on an attribute graph model, wherein: The nodes of the knowledge graph represent entities in the field of flight testing, including aircraft entities containing model and performance parameters, test subject entities containing test types and risk levels, and constraint entities containing safety restrictions and regulatory requirements. Edges represent relationships between entities, including relationship type and relationship strength attributes.

[0068] By using a knowledge graph based on an attribute graph model to store structured data and establish a relationship network between entities, the systematic organization and semantic association of knowledge in the field of flight testing are realized, supporting complex graph path queries and reasoning retrieval.

[0069] In some specific embodiments, unstructured data is stored by constructing a vector database, specifically including: Segment unstructured documents into semantically coherent text fragments; Use an embedding model to convert text fragments into vector representations; Store vectors in a vector database and create a semantic retrieval index.

[0070] By constructing a vector database to store unstructured data and converting text fragments into vector representations, semantic storage and efficient retrieval of unstructured documents are achieved, improving the utilization efficiency of textual materials such as flight manuals.

[0071] In some specific embodiments, each text segment is 800-1200 characters in size, and adjacent text segments maintain an overlap of 150-250 characters.

[0072] By controlling the size of text segments to 800-1200 characters and maintaining an overlap of 150-250 characters between adjacent segments, the semantic integrity and continuity of text chunks are ensured, improving the accuracy of subsequent vectorization processing and semantic retrieval.

[0073] In some specific embodiments, the flight test knowledge base supports hybrid retrieval methods, including keyword-based retrieval, semantic similarity-based retrieval, and graph path-based reasoning retrieval; Among them, graph path-based reasoning retrieval involves traversing and querying along entity relationship paths in the knowledge graph to discover implicit constraint relationships.

[0074] By supporting a hybrid retrieval method based on keywords, semantic similarity, and graph path reasoning, flexible queries of the flight test knowledge base from different dimensions and levels are realized, meeting the needs for multi-faceted knowledge acquisition of complex technical issues.

[0075] In some specific embodiments, the flight test knowledge base also supports a dynamic update mechanism, which can receive new flight test data and update the knowledge graph and vector database.

[0076] By establishing a dynamic update mechanism, the knowledge base can receive new flight test data and update the knowledge graph and vector database, ensuring the timeliness and scalability of the knowledge base content and adapting to the continuous development needs of flight test technology.

[0077] Step S2: Receive flight test mission requests in natural language form and perform semantic parsing to generate a structured mission description containing mission instructions, constraint elements, and mission objectives.

[0078] By receiving flight test mission requests in natural language and performing semantic parsing to generate structured mission descriptions, non-professional users can describe complex test requirements in an intuitive way, significantly reducing the barrier to entry for the system and improving mission input efficiency.

[0079] In some specific embodiments, semantic parsing is performed on the flight test mission request to generate a structured mission description containing mission instructions, constraint elements, and mission objectives, specifically including: Key entities in flight test mission requests are identified using named entity recognition technology; Based on the identified key entities, the natural language description in the flight test mission request is converted into a standardized sequence of flight operation instructions to form the mission instructions; Based on key entities and task instructions, identify the constraint elements to be queried; Based on the identified key entities, the descriptions related to the mission objectives in the flight test mission request are parsed into quantifiable evaluation indicators, which constitute the mission objectives. The consistency of task instructions, constraint elements, and task objectives is verified. Once the consistency verification is successful, a structured task description is obtained.

[0080] By employing a multi-step semantic parsing method that includes named entity recognition, instruction conversion, constraint recognition, and target parsing, accurate conversion from natural language to structured task descriptions is achieved, providing precise input conditions for subsequent intelligent planning.

[0081] In some specific embodiments, constraint elements include one or more of physical limitations, safety boundaries, and regulatory requirements, used for precise retrieval from the flight test knowledge base.

[0082] By clearly defining the constraint elements, including categories such as physical limitations, security boundaries, and regulatory requirements, a clear query guide is provided for accurately retrieving relevant constraints from the knowledge base, ensuring the relevance and completeness of constraint retrieval.

[0083] In some specific embodiments, key entities include multiple of the following: aircraft model, test subject, environmental conditions, and performance parameters.

[0084] By identifying multiple key entities, including aircraft type, test subjects, environmental conditions, and performance parameters, the core elements of the flight test mission were comprehensively captured, laying the foundation for accurately understanding the mission intent and generating corresponding solutions.

[0085] In some specific embodiments, resolving the description related to the mission objective into quantifiable evaluation metrics includes: resolving a region search task into specific search coverage metrics, and resolving a performance test task into speed, altitude, or maneuverability parameters.

[0086] By parsing the task objective description into quantifiable evaluation indicators, a precise expression of vague task requirements is achieved, providing clear measurement standards and judgment criteria for subsequent scheme evaluation.

[0087] In some specific embodiments, consistency verification includes checking the preliminary compliance of the task instruction sequence with the constraint elements, as well as the matching degree between the task instruction sequence and the task objective.

[0088] By verifying the consistency of task instructions, constraints, and task objectives, the generated structured task description is ensured to be logically consistent and conflict-free, thus creating a prerequisite for reliable task planning in the future.

[0089] Step S3: Based on the structured task description, retrieve relevant flight constraints from the flight test knowledge base.

[0090] By receiving flight test mission requests in natural language and performing semantic parsing to generate structured mission descriptions, non-professional users can describe complex test requirements in an intuitive way, significantly reducing the barrier to entry for the system and improving mission input efficiency.

[0091] In some specific embodiments, retrieving relevant flight constraints from the flight test knowledge base includes: Based on constraint elements, graph queries are performed in the knowledge graph to extract structured flight constraints, including discovering implicit constraint relationships related to the current test mission through graph path traversal. Implicit constraint relationships include one or more of the following when a specified type of aircraft performs a specific maneuver: load limit, altitude limit, and speed limit. Based on constraint features, a semantic search is performed in the vector database to extract unstructured flight constraints. The semantic search in the vector database includes: Generate search keywords based on constraint elements; Calculate the semantic similarity between the search keywords and text fragments in the knowledge base; Return the text fragment with the highest similarity as the search result.

[0092] By employing a dual retrieval mechanism—combining graph queries in a knowledge graph and semantic searches in a vector database based on constraint elements—comprehensive acquisition of both structured and unstructured flight constraints is achieved, providing complete constraint information for mission planning.

[0093] In some specific embodiments, structured flight constraints include physical limitations and security boundaries extracted from knowledge graphs, while unstructured flight constraints include regulatory requirements extracted from documents through semantic search.

[0094] By extracting physical constraints and security boundaries from knowledge graphs and regulatory requirements from documents, specialized processing of different types of constraints is achieved, ensuring the accuracy and relevance of the extraction of various constraints.

[0095] In some specific embodiments, the process also includes integrating all retrieved flight constraints to obtain a complete set of flight constraints. The integration process includes deduplication, merging, and priority sorting.

[0096] By clearly defining the mission phases, including takeoff, cruise, test maneuver, and landing, and the resource allocation, including aircraft type selection, payload configuration, and fuel calculation, and the time nodes, including the start time, duration, and end time of each phase, a complete content framework and clear planning elements are provided for the flight test mission plan.

[0097] Step S4: Based on the structured mission description and flight constraints, a large language model is used to generate a flight test mission plan that includes mission phase division, resource allocation, and time nodes.

[0098] By combining structured task descriptions and flight constraints and using a large language model to generate flight test mission plans, rapid and automatic planning of test schemes was achieved, while ensuring that the schemes met both mission objectives and domain constraints, significantly improving planning efficiency and quality.

[0099] In some specific embodiments, the mission phases are divided into takeoff phase, cruise phase, test maneuver phase, and landing phase; Resource allocation includes aircraft type selection, payload configuration, and fuel calculation; The time nodes include the start time, duration, and end time of each task phase.

[0100] By clearly defining the mission phases, including takeoff, cruise, test maneuver, and landing, and the resource allocation, including aircraft type selection, payload configuration, and fuel calculation, and the time nodes, including the start time, duration, and end time of each phase, a complete content framework and clear planning elements are provided for the flight test mission plan.

[0101] In some specific embodiments, generating flight test mission plans using large language models includes: The structured task description and flight constraints are combined into prompt words and input into the large language model. The large language model generates a preliminary task plan that includes task phase division, resource allocation and time nodes. The preliminary mission plan undergoes rule checks and model verification. Once both rule checks and model verifications are passed, the flight test mission plan is obtained.

[0102] By combining structured task descriptions and flight constraints into prompt words and inputting them into a large language model to generate preliminary task plans, and then performing rule checks and model verification, intelligent generation and quality assurance of task plans are achieved, balancing generation efficiency and technical reliability.

[0103] In some specific embodiments, rule checks include logical consistency checks and constraint compliance verification to ensure that the flight test mission plan meets all constraints.

[0104] By conducting logical consistency checks and constraint compliance verifications on the preliminary mission plan, it was ensured that the generated flight test mission plan was logically self-consistent and complied with all technical constraints, thus guaranteeing the rationality of the plan from a rule-based perspective.

[0105] In some specific embodiments, model validation includes multiple rounds of review of the initial task plan using a large language model, identifying potential conflicts, and iteratively optimizing it.

[0106] By employing a large language model to conduct multiple rounds of review of the initial task plan and identifying potential conflicts for iterative optimization, the quality of the task plan was further improved by leveraging the model's reasoning capabilities, thus achieving continuous improvement and refinement of the solution.

[0107] Step S5: Extract the three-dimensional scene configuration requirements, flight command sequence requirements, and test evaluation requirements from the flight test mission plan; The test evaluation requirements include a list of monitoring parameters, definitions of evaluation indicators, and evaluation standards.

[0108] By extracting the 3D scene configuration requirements, flight command sequence requirements, and test evaluation requirements from the flight test mission plan, a structured decomposition of complex test missions was achieved, providing clear technical inputs for subsequent scene construction, command generation, and effect evaluation.

[0109] In some specific embodiments, the requirements for 3D scene configuration include geographical environment parameters, airspace feature elements, ground facility configuration, and meteorological conditions.

[0110] By clearly defining the configuration requirements for 3D scenarios, including geographical environment parameters, airspace features, ground facility configuration, and meteorological conditions, comprehensive and detailed technical requirements are provided for the construction of 3D experimental scenarios, ensuring the integrity and practicality of the scenario model.

[0111] In some specific embodiments, meteorological conditions include wind speed, visibility, cloud cover, and special weather conditions.

[0112] By specifically defining meteorological conditions, including wind speed, visibility, cloud distribution, and special weather conditions, clear parameter guidelines are provided for the generation of meteorological environmental fields, enhancing the realism and accuracy of experimental environment simulation.

[0113] In some specific embodiments, the flight command sequence requirements include flight phase division, operation timing arrangement, and flight parameter settings, wherein: Flight phases are divided into takeoff, cruise, test maneuver, and landing phases. The operational sequence includes the time nodes and duration of each flight phase; Flight parameter settings include speed, altitude, and heading parameters.

[0114] By defining a list of monitoring parameters including aircraft position, attitude, speed, and engine status data, defining evaluation indicators including indicator names and calculation rules, and defining success criteria including threshold ranges and judgment rules, a systematic indicator system and working standards have been established for test evaluation.

[0115] In some specific embodiments, the list of monitoring parameters includes aircraft position data, attitude data, speed data, and engine status data; The evaluation indicator definition includes the names of all evaluation indicators involved in the evaluation and their corresponding calculation rules; The criteria for success include the threshold range and judgment rules for all evaluation indicators involved in the assessment.

[0116] By defining a list of monitoring parameters including aircraft position, attitude, speed, and engine status data, defining evaluation indicators including indicator names and calculation rules, and defining success criteria including threshold ranges and judgment rules, a systematic indicator system and working standards have been established for test evaluation.

[0117] In some specific embodiments, the evaluation metrics involved in the evaluation include at least one of regional search coverage, instruction execution completeness, and maximum speed completion rate.

[0118] By clearly defining the evaluation metrics, including regional search coverage, instruction execution completeness, and maximum speed completion rate, the core dimensions of task completion quality are focused on, providing key performance measurement standards for evaluating the effectiveness of the experiment.

[0119] Step S6: Construct a three-dimensional test scenario model based on the three-dimensional scene configuration requirements.

[0120] By constructing a 3D test scenario model that includes geographical environment, airspace features and meteorological conditions based on the configuration requirements of the 3D scene, a realistic virtual test environment is provided for flight simulation, ensuring the credibility and practicality of the simulation results.

[0121] In some specific embodiments, constructing a three-dimensional test site model specifically includes: Based on the geographic environment parameters in the 3D scene configuration requirements, construct a basic geographic environment including terrain and landforms. Based on the airspace feature elements in the configuration requirements of the 3D scene, no-fly zones and detection ranges are configured in the basic geographic environment; Based on the configuration requirements of ground facilities in the three-dimensional scene, a ground facility model is established in the basic geographic environment; Meteorological environment fields are generated based on the meteorological conditions in the 3D scene configuration requirements and integrated into the basic geographic environment to form a complete 3D test scene model.

[0122] By constructing a basic geographic environment based on geographic environmental parameters, configuring no-fly zones and detection ranges based on airspace characteristic elements, establishing ground facility models based on ground facility configuration, and generating meteorological environmental fields based on meteorological conditions, a systematic construction and hierarchical assembly of three-dimensional experimental scenario models has been achieved.

[0123] In some specific embodiments, constructing the basic geographic environment includes: using a digital elevation model to construct a terrain surface and overlaying vector map data to form a complete basic geographic environment.

[0124] By constructing the terrain surface using a digital elevation model and overlaying vector map data to build the basic geographic environment, the accuracy of terrain representation and the integrity of geographic features are ensured, providing a realistic geographic basis for the experimental scenario.

[0125] In some specific embodiments, the ground facility model includes an airport runway model and a navigation facility model.

[0126] By establishing a ground facility model that includes an airport runway model and a navigation facility model, the configuration of ground elements in the test scenario was improved, providing the necessary environmental support for flight phases such as takeoff, landing, and navigation.

[0127] Step S7: Generate an initial flight command sequence based on the flight command sequence requirements and perform dynamic feasibility verification. If the verification fails, iteratively optimize the flight command sequence until the verification is passed, and obtain an executable flight command sequence.

[0128] By generating an initial flight command sequence based on the flight command sequence requirements and performing dynamic feasibility verification and iterative optimization, the generated flight commands were ensured to meet both mission requirements and flight dynamic constraints, thus technically guaranteeing the executability of the test plan.

[0129] In some specific embodiments, generating an initial flight command sequence based on flight command sequence requirements includes the following steps: Analyze the flight phase divisions in the flight instruction sequence requirements to determine the sequence and connection relationships of each mission phase; Based on the operational sequence arrangement in the flight instruction sequence requirements, allocate time nodes and durations for each flight phase; Based on the flight parameter settings in the flight command sequence requirements, generate command parameters including waypoint coordinates, speed values, altitude values, and heading angles; The sequence of flight phases, the allocated time nodes and durations, and the generated command parameters are arranged and integrated according to time sequence and logical relationship to form a complete initial flight command sequence.

[0130] By analyzing the flight phase division to determine the sequence of mission phases, arranging and allocating time nodes according to the operation sequence, and generating command parameters based on flight parameters and integrating them to form an initial flight command sequence, the systematization and structured organization of flight commands are realized.

[0131] In some specific embodiments, verifying the dynamic feasibility of the initial flight command sequence includes: Input the initial flight command sequence into the flight mechanics simulator to perform a 6-DOF dynamics simulation. During the simulation, the angle of attack, thrust-to-weight ratio, and trajectory parameters are monitored. The verification fails when the angle of attack exceeds the stall threshold, the thrust-to-weight ratio does not meet the climb requirements, or the flight path crosses a no-fly zone.

[0132] By inputting the initial flight command sequence into the flight mechanics simulator for 6-DOF dynamic simulation and monitoring the angle of attack, thrust-to-weight ratio, and trajectory parameters, the physical feasibility of the flight commands was verified, ensuring flight safety from a dynamic perspective.

[0133] In some specific embodiments, dynamic feasibility verification also includes checking whether the command sequence violates flight safety boundaries and airworthiness standards.

[0134] By additionally checking whether the command sequence violates flight safety boundaries and airworthiness standards during the dynamic feasibility verification, the scope of verification was further expanded, ensuring that flight commands fully comply with aviation safety regulations.

[0135] In some specific embodiments, flight dynamics models using MATLAB, Simulink, or X-Plane are used for simulation verification.

[0136] Simulation verification was performed using flight dynamics models based on MATLAB, Simulink, or X-Plane, utilizing mature and reliable simulation tools to ensure the accuracy and authority of the dynamics verification results.

[0137] In some specific embodiments, iteratively optimizing the flight command sequence includes: Generate natural language error descriptions based on the monitoring results of failed verification; The natural language error descriptions are used as feedback information input into the large language model; The optimized flight instruction sequence is regenerated using a self-refine strategy based on a large language model.

[0138] By generating natural language error descriptions based on the monitoring results of failed verifications and using them as feedback input to a large language model to regenerate command sequences, an iterative optimization mechanism based on natural language interaction was established, enabling automatic improvement of flight commands.

[0139] In some specific embodiments, the natural language error description includes specific abnormal parameter values ​​and the location where the error occurred.

[0140] By clearly specifying the abnormal parameter values ​​and their locations in the natural language error descriptions, the system provides clear problem identification and improvement directions, enhancing the practicality and guidance of the feedback information.

[0141] Step S8: Based on the executable flight command sequence, perform flight simulation in the three-dimensional test scenario model, and collect test data of corresponding parameters during the flight simulation based on the monitoring parameter list.

[0142] By executing flight simulations and collecting test data in a three-dimensional test scenario model based on executable flight command sequences, the digital execution of the test process and comprehensive data recording were achieved, providing a complete and accurate data foundation for subsequent quantitative evaluation.

[0143] In some specific embodiments, performing flight simulation includes: Use a flight simulator to load executable flight command sequences and 3D test scenario models; The motion state and system response of an aircraft are simulated based on a six-degree-of-freedom flight dynamics model.

[0144] By using a flight simulator to load executable flight command sequences and a three-dimensional test scenario model, and simulating the aircraft's motion state based on a six-degree-of-freedom flight dynamics model, high-fidelity digital simulation of flight tests was achieved, providing a reliable platform for test data acquisition.

[0145] In some specific embodiments, the flight simulator uses flight dynamics models from MATLAB, Simulink, or X-Plane for simulation calculations.

[0146] By using flight dynamics models from MATLAB, Simulink, or X-Plane for simulation calculations, the authenticity and accuracy of the flight simulation results are ensured, providing a reliable data source for subsequent evaluation and analysis.

[0147] In some specific embodiments, the test data includes aircraft position coordinates, attitude angles, velocity vectors, acceleration values, and engine state parameters.

[0148] By collecting test data including aircraft position coordinates, attitude angles, velocity vectors, acceleration values, and engine status parameters, key status information during flight was comprehensively recorded, providing sufficient data support for multidimensional evaluation.

[0149] Step S9: Based on the test evaluation requirements and test data, calculate the corresponding evaluation indicators and generate an evaluation report.

[0150] By executing flight simulations and collecting test data in a three-dimensional test scenario model based on executable flight command sequences, the digital execution of the test process and comprehensive data recording were achieved, providing a complete and accurate data foundation for subsequent quantitative evaluation.

[0151] In some specific embodiments, a large language model is used to generate an evaluation report, including: Construct evaluation report prompts that include task description elements, execution process elements, evaluation result elements, and structured requirement elements, where: The mission description elements include the mission objectives, constraints, and performance requirements in the flight test mission plan; The execution process elements include the instruction steps of the executable flight command sequence and the key flight data recorded in the test data; The evaluation results include the calculation results of the evaluation indicators; The structured requirements specify the elements to generate an evaluation report that includes four parts: an execution summary, detailed analysis, problem identification, and improvement recommendations. An evaluation report is generated based on prompts from a large language model, including an execution summary, detailed analysis, problem identification, and improvement suggestions. The execution summary includes a summary and evaluation of the overall task completion status; Detailed analysis includes performance analysis of various parameters during flight; Problem identification includes analyzing the discrepancy between actual results and expected goals; The improvement suggestions include specific optimization schemes for identifying problems.

[0152] By constructing evaluation report prompts that include task description elements, execution process elements, evaluation result elements, and structured requirement elements, and using a large language model to generate evaluation reports that include an execution summary, detailed analysis, problem identification, and improvement suggestions, intelligent analysis and structured presentation of experimental evaluation results have been achieved.

[0153] The following are embodiments of the experimental evaluation system based on a large language model provided in this application. This experimental evaluation system based on a large language model belongs to the same inventive concept as the experimental evaluation methods in the above embodiments. For details not described in detail in the embodiments of the experimental evaluation system, please refer to the embodiments of the experimental evaluation methods based on a large language model described above.

[0154] like Figure 2 As shown, the experimental evaluation system based on the large language model includes: The Flight Knowledge Base Construction Module is used to build a flight test knowledge base; The mission description generation module is used to receive flight test mission requests in natural language form and perform semantic parsing to generate a structured mission description that includes mission instructions, constraint elements, and mission objectives. The constraint retrieval module is used to retrieve relevant flight constraints from the flight test knowledge base based on the structured task description; The mission plan generation module is used to generate flight test mission plans that include mission phase divisions, resource allocation, and time nodes based on structured mission descriptions and flight constraints using a large language model. The mission plan parsing module is used to parse the three-dimensional scene configuration requirements, flight command sequence requirements, and test evaluation requirements from the flight test mission plan; The 3D scene model building module is used to build 3D test scene models based on 3D scene configuration requirements. The flight command sequence generation module is used to generate an initial flight command sequence based on the flight command sequence requirements and perform dynamic feasibility verification. If the verification fails, the flight command sequence is iteratively optimized until the verification is passed, and an executable flight command sequence is obtained. The flight simulation module is used to perform flight simulation in a three-dimensional test scenario model based on an executable flight command sequence, and to collect test data of corresponding parameters during the flight simulation based on a list of monitoring parameters. The evaluation index and evaluation report generation module is used to calculate the corresponding evaluation indexes based on the test evaluation requirements and test data, and generate an evaluation report containing the evaluation indexes.

[0155] The test evaluation system in this embodiment is used to implement a test evaluation method based on a large language model.

[0156] This application also provides an electronic device for implementing the various embodiments of this application. Figure 3 To illustrate the hardware structure of an electronic device according to various embodiments of this application, as shown in the following diagram... Figure 3 As shown, the electronic device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor.

[0157] Those skilled in the art will understand that the electronic device structure involved in the embodiments of this application does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0158] In embodiments of this application, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0159] In this application embodiment, the processor can be implemented using at least one of an Application-Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a processor, a controller, a microcontroller, a microprocessor, or an electronic unit designed to perform the functions described herein. In some cases, such implementations can be implemented within a controller. For software implementations, implementations such as processes or functions can be implemented with separate software modules that allow the performance of at least one function or operation. The software code can be implemented by a software application (or program) written in any suitable programming language, and the software code can be stored in memory and executed by the controller.

[0160] In addition, the electronic device includes some functional modules not shown, which will not be described in detail here.

[0161] Those skilled in the art will understand that the various aspects of the electronic device provided in this application can be implemented as a system, method, or program product. Therefore, the various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0162] This application also provides a storage medium storing a program product capable of implementing a large language model-based experimental evaluation method. In some possible implementations, various aspects of this application can also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the foregoing "Exemplary Methods" section of this specification according to various exemplary embodiments of this application.

[0163] The storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0164] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for evaluating experiments based on a large language model, characterized in that, include: S1. Construct a flight test knowledge base, which stores flight test knowledge data, including structured and unstructured data; S2. Receive flight test mission requests in natural language form and perform semantic parsing to generate a structured mission description containing mission instructions, constraint elements, and mission objectives; S3. Based on the structured task description, retrieve relevant flight constraints from the flight test knowledge base; S4. Based on the structured mission description and flight constraints, a large language model is used to generate a flight test mission plan that includes mission phase division, resource allocation and time nodes; S5. Extract the 3D scene configuration requirements, flight command sequence requirements, and test evaluation requirements from the flight test mission plan; The test evaluation requirements include a list of monitoring parameters, definitions of evaluation indicators, and evaluation standards. S6. Construct a 3D test scenario model based on 3D scene configuration requirements; S7. Generate an initial flight command sequence based on the flight command sequence requirements and perform dynamic feasibility verification. If the verification fails, iteratively optimize the flight command sequence until it passes the verification to obtain an executable flight command sequence. S8. Based on the executable flight command sequence, perform flight simulation in a three-dimensional test scenario model, and collect test data of corresponding parameters during the flight simulation based on the monitoring parameter list; S9. Based on the test evaluation requirements and test data, calculate the corresponding evaluation indicators and generate an evaluation report.

2. The test evaluation method as described in claim 1, characterized in that, In step S1, the structured data is stored by constructing a knowledge graph based on an attribute graph model, wherein: The nodes of the knowledge graph represent entities in the field of flight testing, including aircraft entities containing model and performance parameters, test subject entities containing test types and risk levels, and constraint entities containing safety restrictions and regulatory requirements. Edges represent relationships between entities, including relationship type and relationship strength attributes.

3. The test evaluation method as described in claim 1, characterized in that, In step S1, unstructured data is stored by constructing a vector database, specifically including: Segment unstructured documents into semantically coherent text fragments; Use an embedding model to convert text fragments into vector representations; Store vectors in a vector database and create a semantic retrieval index.

4. The test evaluation method as described in claim 1, characterized in that, In step S2, the flight test mission request is semantically parsed to generate a structured mission description containing mission instructions, constraint elements, and mission objectives, specifically including: Key entities in flight test mission requests are identified using named entity recognition technology; Based on the identified key entities, the natural language description in the flight test mission request is converted into a standardized sequence of flight operation instructions to form the mission instructions; Based on key entities and task instructions, identify the constraint elements to be queried; Based on the identified key entities, the descriptions related to the mission objectives in the flight test mission request are parsed into quantifiable evaluation indicators, which constitute the mission objectives. The consistency of task instructions, constraint elements, and task objectives is verified. Once the consistency verification is successful, a structured task description is obtained.

5. The test evaluation method as described in claim 1, characterized in that, In step S3, the relevant flight constraints are retrieved from the flight test knowledge base, including: Based on constraint elements, graph queries are performed in the knowledge graph to extract structured flight constraints, including discovering implicit constraint relationships related to the current test mission through graph path traversal. Implicit constraint relationships include one or more of the following when a specified type of aircraft performs a specific maneuver: load limit, altitude limit, and speed limit. Based on constraint features, a semantic search is performed in the vector database to extract unstructured flight constraints. The semantic search in the vector database includes: Generate search keywords based on constraint elements; Calculate the semantic similarity between the search keywords and text fragments in the knowledge base; Return the text fragment with the highest similarity as the search result.

6. The test evaluation method as described in claim 1, characterized in that, In step S4, generating the flight test mission plan using a large language model includes: The structured task description and flight constraints are combined into prompt words and input into the large language model. The large language model generates a preliminary task plan that includes task phase division, resource allocation and time nodes. The preliminary mission plan undergoes rule checks and model verification. Once both rule checks and model verifications are passed, the flight test mission plan is obtained.

7. The test evaluation method as described in claim 1, characterized in that, In step S7, generating the initial flight command sequence based on the flight command sequence requirements includes the following steps: Analyze the flight phase divisions in the flight instruction sequence requirements to determine the sequence and connection relationships of each mission phase; Based on the operational sequence arrangement in the flight instruction sequence requirements, allocate time nodes and durations for each flight phase; Based on the flight parameter settings in the flight command sequence requirements, generate command parameters including waypoint coordinates, speed values, altitude values, and heading angles; The sequence of flight phases, the allocated time nodes and durations, and the generated command parameters are arranged and integrated according to time sequence and logical relationship to form a complete initial flight command sequence.

8. The test evaluation method as described in claim 1, characterized in that, Step S7, which verifies the dynamic feasibility of the initial flight command sequence, includes: Input the initial flight command sequence into the flight mechanics simulator to perform a 6-DOF dynamics simulation. During the simulation, the angle of attack, thrust-to-weight ratio, and trajectory parameters are monitored. The verification fails when the angle of attack exceeds the stall threshold, the thrust-to-weight ratio does not meet the climb requirements, or the flight path crosses a no-fly zone.

9. The test evaluation method as described in claim 1, characterized in that, In step S9, an evaluation report is generated using a large language model, including: Construct evaluation report prompts that include task description elements, execution process elements, evaluation result elements, and structured requirement elements, where: The mission description elements include the mission objectives, constraints, and performance requirements in the flight test mission plan; The execution process elements include the instruction steps of the executable flight command sequence and the key flight data recorded in the test data; The evaluation results include the calculation results of the evaluation indicators; The structured requirements specify the elements to generate an evaluation report that includes four parts: an execution summary, detailed analysis, problem identification, and improvement recommendations. An evaluation report is generated based on prompts from a large language model, including an implementation summary, detailed analysis, problem identification, and improvement suggestions.

10. An experiment evaluation system based on a large language model, characterized in that, To implement the test evaluation method as described in any one of claims 1-9, comprising: The Flight Knowledge Base Construction Module is used to build a flight test knowledge base; The mission description generation module is used to receive flight test mission requests in natural language form and perform semantic parsing to generate a structured mission description that includes mission instructions, constraint elements, and mission objectives. The constraint retrieval module is used to retrieve relevant flight constraints from the flight test knowledge base based on the structured task description; The mission plan generation module is used to generate flight test mission plans that include mission phase divisions, resource allocation, and time nodes based on structured mission descriptions and flight constraints using a large language model. The mission plan parsing module is used to parse the three-dimensional scene configuration requirements, flight command sequence requirements, and test evaluation requirements from the flight test mission plan; The 3D scene model building module is used to build 3D test scene models based on 3D scene configuration requirements. The flight command sequence generation module is used to generate an initial flight command sequence based on the flight command sequence requirements and perform dynamic feasibility verification. If the verification fails, the flight command sequence is iteratively optimized until the verification is passed, and an executable flight command sequence is obtained. The flight simulation module is used to perform flight simulation in a three-dimensional test scenario model based on an executable flight command sequence, and to collect test data of corresponding parameters during the flight simulation based on a list of monitoring parameters. The evaluation index and evaluation report generation module is used to calculate the corresponding evaluation indexes based on the test evaluation requirements and test data, and generate an evaluation report containing the evaluation indexes.

Images