A model-driven algorithm testing method, system, computer and storage medium

Abstract

The application provides a model-driven algorithm test method, system, computer and storage medium, which comprises the following steps: normalizing non-standard algorithm requirements to generate a structured algorithm requirement document; based on the normalized algorithm requirement document, using a large language model to generate test data information meeting the test sufficiency requirement; parsing the algorithm source code structure corresponding to the algorithm requirement to construct a meta-model of the algorithm under test; loading the meta-model of the algorithm under test and importing the test data information to complete the conversion from the meta-model to an instantiated test model, forming a test case set; based on an extensible stylesheet transformation language template, batch converting the test case set into an executable test script; and executing the test script in an algorithm test engine. Through the model-driven test technology and the algorithm test engine, the automatic verification of the test case is completed and the test result is collected, thereby effectively improving the intelligentization and automation level of the algorithm test.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a model-driven algorithm testing method, system, computer, and storage medium. Background Technology

[0002] As a core component of software systems, software algorithms are fundamental modules responsible for key functions such as data processing, logic control, mathematical operations, and complex decision-making. In various software applications, including artificial intelligence and industrial control software, the correctness and robustness of algorithms directly determine the performance and reliability of the entire system. With the continuous expansion of algorithmic structures and the increasing complexity of business logic, ensuring the quality and reliability of algorithms becomes increasingly difficult. How to conduct comprehensive and systematic testing of algorithms effectively has become a crucial process for ensuring the quality and reliability of software products. This requires testing to not only cover routine functional verification but also to conduct in-depth verification of the algorithm's functional logic, boundary conditions, and anomalies. Simultaneously, algorithm testing needs to be integrated into automated testing frameworks to achieve rapid iteration and continuous verification.

[0003] However, the requirements analysis and test case design for algorithm testing still heavily rely on manual analysis. Testers need to manually extract key functionalities and then transform them into test requirements. Based on these requirements, they manually design test data and write corresponding test scripts.

[0004] Due to the complexity and high degree of specialization of modern algorithms, their requirements specifications often involve numerous mathematical formulas, domain terminology, and complex logical constraints, resulting in lengthy documents with a high level of technical depth. For engineers lacking relevant domain background or testing experience, fully and accurately understanding the algorithm's requirements is quite difficult. Secondly, the design of test data for complex algorithms typically relies on manual derivation and analysis. However, manual methods are prone to overlooking certain critical test paths, such as boundary conditions and abnormal input scenarios, when dealing with high-dimensional input spaces, multiple condition combinations, or nonlinear logic. This leads to insufficient test coverage, undetected potential defects, and further increases the risk of test blind spots. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a model-driven algorithm testing method, system, computer and storage medium, which aims to solve the technical problem of poor algorithm testing results in the prior art.

[0006] To achieve the above objectives, in a first aspect, the present invention provides: a model-driven algorithm testing method, comprising the following steps:

[0007] Based on the large language model, non-standard algorithm requirements are standardized to generate structured algorithm requirement documents;

[0008] Based on the standardized algorithm requirements document, test data information that meets the test sufficiency requirements is generated using a large language model;

[0009] Analyze the algorithm source code structure corresponding to the algorithm requirements and construct the meta-model of the algorithm under test;

[0010] Load the meta-model of the algorithm under test and import the test data information to complete the transformation from the meta-model to the instantiated test model and form a test case set;

[0011] Based on the extensible stylesheet language conversion template, the test case set is batch converted into executable test scripts.

[0012] The test script is executed in the algorithm testing engine to automatically verify the test results and save the test data.

[0013] According to one aspect of the above technical solution, the steps for standardizing non-standard algorithm requirements based on a large language model specifically include:

[0014] Obtain non-standard algorithm requirements documents input by the user;

[0015] A customized prompt word framework is constructed based on the large language module and semantic parsing is performed to output a standardized algorithm requirement document that conforms to the norm based on the non-standard algorithm requirement document.

[0016] According to one aspect of the above technical solution, the steps of generating test data information that meets the test adequacy requirements using a large language model specifically include:

[0017] Import the normalization algorithm requirements document;

[0018] A customized prompt word framework is constructed based on the large language module and semantic parsing is performed to output a structured test data document based on the requirements document of the standardized algorithm.

[0019] According to one aspect of the above technical solution, the steps of analyzing the algorithm source code structure corresponding to the algorithm requirements and constructing the meta-model of the algorithm under test specifically include:

[0020] Import the source code corresponding to the requirements of the algorithm to be tested into the algorithm testing environment building module, and automatically match the programming language type;

[0021] The code parsing function is used to analyze the structure of the source code and obtain the structured data of the algorithm under test. The structured data includes global variables, functions, input and output parameters and function call information.

[0022] A graphical modeling method is used to define the test interface of the algorithm under test. The cosine similarity search algorithm is used to calculate the similarity between the requirement description parameters and the parsed variables in order to automatically match input parameters, output parameters, global variables and instrumentation functions.

[0023] Select the algorithm under test in the test scenario canvas, bind the test interface, and complete the meta-model construction;

[0024] Save the metamodel as an XML file for easy import and instantiation.

[0025] According to one aspect of the above technical solution, the steps for forming a test case set specifically include:

[0026] Load the XML file corresponding to the metamodel;

[0027] Import structured test data information and automatically match algorithm object names, input / output parameters and meta-model interfaces in the test data document;

[0028] After a successful match, a test list is created, and each test data is converted into an XML format test case file, which includes input data, expected output data, and status indicators.

[0029] You can save, view, and modify test case sets through the test case management module.

[0030] According to one aspect of the above technical solution, the step of batch converting the test case set into executable test scripts specifically includes:

[0031] Automatically selects the extensible stylesheet language conversion template based on the algorithm source code type;

[0032] This function converts XML format test case files into test scripts that are consistent with the programming language, enabling batch conversion.

[0033] According to one aspect of the above technical solution, the steps for automatically verifying test results specifically include:

[0034] Import test scripts into the algorithm testing engine and automatically select the compiler;

[0035] Create a test process and a control process. The test process loads the algorithm model, and the control process sends test stimuli and receives output.

[0036] The verification module compares the actual output with the expected results, updates the test case status, and saves the test data.

[0037] Secondly, the present invention provides a model-driven algorithm testing system, comprising:

[0038] The standardization module is used to standardize non-standard algorithm requirements based on a large language model and generate structured algorithm requirement documents.

[0039] The test data module is used to standardize non-standard algorithm requirements based on a large language model and generate structured algorithm requirement documents.

[0040] The building module is used to parse the algorithm source code structure corresponding to the algorithm requirements and build the meta-model of the algorithm under test;

[0041] The test case module is used to load the meta-model of the algorithm under test and import the test data information, complete the transformation from the meta-model to the instantiated test model, and form a test case set.

[0042] The conversion module is used to convert the test case set into executable test scripts in batches based on the extensible stylesheet language template conversion.

[0043] The execution module is used to execute the test script in the algorithm testing engine, automatically verify the test results, and save the test data.

[0044] Thirdly, the present invention also provides a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the model-driven algorithm testing method as described in the above technical solutions.

[0045] Fourthly, the present invention also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the model-driven algorithm testing method as described in the above technical solutions.

[0046] Compared with existing technologies, the advantages of this invention are as follows: It utilizes the natural language processing capabilities of a large language model to perform semantic parsing and standardization on unstructured software algorithm requirement documents, completing semantic understanding, entity recognition, and relation extraction of algorithm requirements, transforming fuzzy business descriptions into structured requirement representations; it establishes test data generation rules by designing a prompt word framework for test generation strategies, and based on the standardized algorithm requirements, uses the logical reasoning capabilities of the large language model to automatically extract key test elements and generate test data that meets test sufficiency requirements; it employs a model-driven automation method to parse and encapsulate algorithm code, and automatically generates algorithm model test scenarios and test cases by combining the test data generated by the large language model; and it completes automated verification of test cases and collects test results through model-driven testing technology and an algorithm testing engine, thereby effectively improving the intelligence and automation level of algorithm testing. Attached Figure Description

[0047] Figure 1This is a flowchart illustrating the model-driven algorithm testing method in the first embodiment of the present invention;

[0048] Figure 2 This is a flowchart illustrating the model-driven algorithm testing method in the first embodiment of the present invention.

[0049] Figure 3 This is a core flowchart of step S100 in the first embodiment of the present invention;

[0050] Figure 4 This is a core flowchart of step S200 in the first embodiment of the present invention;

[0051] Figure 5 This is a core flowchart of step S300 in the first embodiment of the present invention;

[0052] Figure 6 This is a core flowchart of step S400 in the first embodiment of the present invention;

[0053] Figure 7 This is a structural block diagram of the model-driven algorithm testing system in the second embodiment of the present invention;

[0054] Figure 8 This is a schematic diagram of the hardware structure of the computer in the third embodiment of the present invention;

[0055] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0056] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0057] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0059] Example 1

[0060] Please see Figures 1-2 The figures shown are flowcharts and schematic diagrams of the model-driven algorithm testing method in the first embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0061] Step S100: Based on the large language model, the non-standard algorithm requirements are standardized to generate a structured algorithm requirement document.

[0062] Specifically, the steps for normalizing non-standard algorithm requirements based on a large language model include:

[0063] Obtain non-standard algorithm requirements documents from user input.

[0064] A customized prompt word framework is constructed based on a large language module, and semantic parsing is performed to output a standardized algorithm requirement document that conforms to the standardization based on the non-standard algorithm requirement document. The prompt word framework consists of: customized definition + supplementary words + input information.

[0065] (1) Customized definition: Customize the role of the large language model in the standardization of requirements. This includes defining "role", "background", "overview", "skills" and "goals".

[0066] Role: Describe the role of the large language model in regulating requirements.

[0067] Background: Describe the working context of requirements standardization.

[0068] Overview: Describes the overall process of requirements standardization.

[0069] Skills: Describe the abilities a character needs to possess in the process of standardizing requirements.

[0070] Objective: Describe the desired effect of requirements standardization.

[0071] Constraints: Describe the rules and regulations that need to be followed during the standardization of requirements.

[0072] Output: Describes the output included in the requirements standardization.

[0073] Formatting requirements: Describe the formatting requirements that the output should conform to.

[0074] For example: Provide a case that meets the format requirements, and let the large language model refer to the case to output in a standardized manner.

[0075] (2) Supplementary words: Personalized prompt words added by users according to their specific requirements for standardization.

[0076] (3) Input information: Non-standard algorithm requirement text.

[0077] (4) Output information: The algorithm requirement text after standardization as required.

[0078] The input algorithm requirements document is used, and a pre-designed framework of prompts is used to output standardized algorithm requirement text. The generated content is then compiled into a document and saved, and the information is sent to the front end for display.

[0079] To facilitate understanding, in some application scenarios of this embodiment, firstly, a suitable large language model is selected; for example, DeepSeek-R1, DeepSeek-V3, Doubao, Kimi, Xinghuo Xunfei, ChatGPT-o1, etc. Designers can choose to use a large number of large language model APIs, or they can use locally deployed private large language models; the large language model engine module is accessed through the API interface. It should be noted that in cases where data privacy requirements are particularly high, locally deployed private large language models will be preferred. A private knowledge base can be designed to fine-tune the large language model, but this requires higher funding, and in cases of high software complexity, the computational effect of the local large language model may not be as good as that of calling a full-fledged large language model API. If funding is limited, time is insufficient to support the establishment of a private knowledge base, and the requirement for data privacy is not particularly high, then directly using a suitable model and API service, with pricing based on tokens, is more appropriate and often yields better results. Moreover, the large language models currently available on the market can also support fine-tuning of private knowledge bases, providing great convenience in deployment. The most important consideration is the issue of data privacy.

[0080] Furthermore, Figure 3 This is a detailed process of standardizing non-standard algorithm requirements based on the large language model prompt word project, such as... Figure 3 As shown, this step includes the following steps:

[0081] Establish communication between the C# and Python sides, ensure a connection is established between the two ends, and provide a prompt message when the connection is successful.

[0082] Import the non-standardized algorithm requirements specification into the C# client. If there are special requirements, you can enter supplementary prompts. Determine the generation path and select the large language model type and version. Organize this information into user requirements and non-standardized algorithm requirements, package it in JSON format, and transmit it to the Python large language model client.

[0083] The Python side receives the transmitted information, parses and breaks it down to obtain the initial algorithm requirements and the user's basic requirements (large language model and its version information, personalized requirements, etc.). This information is then combined with the designed standard and normalized requirement prompts to form the final prompts.

[0084] Based on the large language model information provided by the user, the corresponding large language model interface is selected, and the final prompt word is passed to the large language model engine to generate the result. The output result is converted into a file and saved on the Python large language model side to complete the final normalization work.

[0085] Furthermore, under normal circumstances, the algorithm requirements should include the algorithm requirement name, algorithm requirement description, algorithm input parameters, algorithm input parameter types, algorithm input parameter ranges, algorithm input parameter initial values, algorithm output parameter ranges, algorithm output parameter types, algorithm testing rules, etc. Typically, the software algorithm requirements specification to be standardized hides this core information in a large amount of redundant information or omits certain parts. This invention optimizes the prompts and improves the output effect by embedding standardized templates in the prompts. The standardized templates include the following parameters: algorithm name, algorithm description, input parameters (parameter type, initial value, range, description), output parameters (parameter type, initial value, range, description), and testing rules.

[0086] In some application scenarios of this embodiment, the above steps are performed to implement the operational command decision-making algorithm requirements for mission planning. Below is the operational command decision-making algorithm requirements section, which is visibly complex and contains some missing information. For example, the input parameters, initial values, and ranges are not clearly expressed. Testers find it difficult to quickly identify the key aspects of the algorithm and analyze it for subsequent test case writing. Therefore, this section has low standardization and poor structure, requiring the use of the powerful semantic understanding and extraction capabilities of large language models to standardize the operational command decision-making model algorithm requirements below.

[0087] Non-standard algorithm requirements: "The algorithm name is OperationalCommandDecision, and the model algorithm name is a mission planning-type combat command decision algorithm. Based on the current battlefield situation, enemy strength, friendly strength, terrain advantage, whether enemy activity has been detected, available resources, and distance to friendly forces, rationally select and plan the most suitable action plan. The following are the rules: If friendly strength is greater than 1.5 times that of the enemy, has terrain advantage, and available resources are greater than 50, then launch an attack. If enemy strength is greater than 1.5 times that of friendly strength, has no terrain advantage, and available resources are less than 50, then retreat. If enemy strength is less than 1.5 times that of friendly strength but greater than friendly strength, and has terrain advantage..." If your available resources are greater than 50, defend. If the enemy's strength is less than 1.5 times your strength but greater, you have no terrain advantage, and friendly forces can support you from within 10 units, maintain your current position. If your available resources are less than 50, enemy activity is detected, and friendly forces can support you from within 10 units, request reinforcements. If enemy activity is detected, your available resources are greater than 50, and friendly forces can support you from within 10 units, continue investigating. If your strength is greater than 1.5 times the enemy's strength, you have no terrain advantage, and no enemy activity is detected, await orders from higher command. If the enemy's strength is greater than 1.5 times your strength, you have a terrain advantage, and no enemy activity is detected, await orders from higher command. Enemy strength: Represents the enemy's overall combat power. friendlyStrength: Friendly strength: Represents the friendly's overall combat power. terrainAdvantage: Whether there is terrain advantage; 1 indicates terrain advantage, 0 indicates no. enemyActivity: Whether enemy activity has been detected; 1 indicates enemy activity has been detected, 0 indicates no detection. friendlySupportDist: Distance to friendly forces, representing the distance to the nearest friendly support. resourcesAvailable: Amount of available resources, representing the current level of available supplies and equipment. Function return value: int type decision instruction. Output: Return, initial value: 0, results include {0 = "Initiate attack", 1 = "Defend", 2 = "Retreat", 3 = "Maintain current position", 4 = "Request reinforcements", 5 = "Continue investigation", 6 = "Wait for superior instructions".

[0088] The standardized operational command decision-making model algorithm requirements, after processing by the large language model, are shown below. It can be observed that the format of the operational command decision-making model algorithm requirements after processing by the large language model is very clear and standardized. The large language model effectively utilizes designed prompts to understand the context of the model algorithm requirements and perform basic mathematical reasoning, thus adding missing parts appropriately and removing useless parts. The final standardized operational command decision-making model algorithm requirements mainly include the algorithm name, model algorithm name, algorithm description, input parameter information, output parameter information, and test rules. Testers can clearly see the entire operational command decision-making model algorithm and quickly understand the algorithm requirements, saving testers a lot of time and improving the efficiency of automated testing. Specifically, the structured algorithm requirements document is as follows:

[0089] Algorithm name: operationalCommandDecision;

[0090] Model algorithm name: Mission planning-type combat command decision algorithm;

[0091] Algorithm description: Based on the current battlefield situation, combined with information such as enemy troop strength, friendly troop strength, whether there is terrain advantage, whether enemy activity has been detected, available resources, and distance to friendly forces, the most suitable action plan is rationally selected and planned.

[0092] Input parameter information:

[0093] enemyStrength, initial value: none, value range: [0, +∞), description: the total combat strength of the enemy, int type;

[0094] friendlyStrength, initial value: none, range: [0, +∞), description: the overall combat strength of our side, int type;

[0095] terrainAdvantage, initial value: 0, value range: {0, 1}, description: whether there is terrain advantage, 1 indicates yes, 0 indicates no, int type;

[0096] enemyActivity, initial value: 0, value range: {0, 1}, description: whether enemy activity is detected, 1 indicates detected, 0 indicates not detected, int type;

[0097] friendlySupportDist, initial value: none, value range: [0, +∞), description: distance to the nearest friendly support, int type;

[0098] resourcesAvailable, initial value: none, value range: [0, +∞), description: the current level of available resources and equipment, int type;

[0099] Output parameter information:

[0100] Return, initial value: 0, value range: {0 = "Initiate attack", 1 = "Defend", 2 = "Retreat", 3 = "Maintain current position", 4 = "Request reinforcements", 5 = "Continue investigation", 6 = "Wait for superior instructions"}, description: Decision instruction output, int type;

[0101] Test Rules: If your forces are more than 1.5 times stronger than the enemy forces, have a terrain advantage, and have more than 50 available resources, launch an attack; if the enemy forces are more than 1.5 times stronger than your forces, do not have a terrain advantage, and have less than 50 available resources, retreat; if the enemy forces are less than 1.5 times stronger than your forces, have a terrain advantage, and have more than 50 available resources, defend; if the enemy forces are less than 1.5 times stronger than your forces, have no terrain advantage, and friendly forces can support you from within 1 kilometer of your base... If the available resources are less than 50, enemy activity is detected, and friendly forces can provide support from a distance greater than 10, then request reinforcements; if enemy activity is detected, available resources are greater than 50, and friendly forces can provide support from a distance less than 10, then continue the investigation; if friendly forces are more than 1.5 times stronger than enemy forces, have no terrain advantage, and no enemy activity is detected, then wait for orders from higher command; if enemy forces are more than 1.5 times stronger than friendly forces, have a terrain advantage, and no enemy activity is detected, then wait for orders from higher command.

[0102] Step S200: Based on the standardized algorithm requirement document, use a large language model to generate test data information that meets the test sufficiency requirements.

[0103] Specifically, the steps for generating test data that meets the test adequacy requirements using a large language model include:

[0104] Import the normalization algorithm requirements document;

[0105] A customized prompt word framework is constructed based on the large language module and semantic parsing is performed to output a structured test data document based on the requirements document of the standardized algorithm.

[0106] Furthermore, Figure 4 This is a flowchart detailing the core steps involved in generating test data documents that meet sufficiency requirements using a large language model, based on the requirements of the normalization algorithm. Figure 4 As shown, this step mainly includes the following steps:

[0107] Establish communication between the C# and Python sides, ensure a connection is established between the two ends, and provide a prompt message when the connection is successful.

[0108] Import the standardized algorithm requirements specification into the C# client. If there are special requirements, you can enter supplementary prompts. Determine the generation path and select the large language model type and version. Enter the number of test cases to be generated and the preferred test methods. Transmit the information to the Python client in JSON format.

[0109] The Python side receives the transmitted information, breaks it down, processes it, and combines it with the user's personalized requirements to compile the final prompt words.

[0110] The corresponding interface is called based on the user's selection of the large language model, and the standardized algorithm requirements and final prompt words are transmitted to the large language model. The returned test cases are then saved as documents.

[0111] Specifically, if the connection between the two ends has already been established in the above steps, this step can be skipped and does not need to be repeated. Users need to import the generated standardized algorithm requirements document and can enter special requirement prompts, such as: if two line segments are connected end-to-end, they are considered connected but not intersecting or overlapping. In addition, users need to enter the number of test cases to be generated and select a more preferred testing method (boundary testing, equivalence class testing, etc.), or they can leave it as is and let the subsequent large language model infer the appropriate testing method automatically.

[0112] Next, users can choose the corresponding large language model and version. This invention can access various mainstream large language models currently on the market, such as DeepSeek-R1, DeepSeek-V3, Doubao, Kimi, Xinghuo Xunfei, ChatGPT o1, etc. Designers can choose to access a large number of large language model APIs for use, or they can use locally deployed private large language models. This invention provides unified management for API large language models and locally deployed large language models, which greatly improves subsequent maintainability and scalability.

[0113] It's important to note that in situations with extremely high data privacy requirements, local deployment of a private large language model should be prioritized. A private knowledge base can be designed for fine-tuning the large language model, but this requires significant funding. Furthermore, with high software complexity, the computational performance of a local large language model may not be as good as calling a full-fledged large language model API. In cases with limited funding, insufficient time to build a private knowledge base, and where data privacy requirements are not particularly high, directly using a suitable model and API service on a token-based billing basis is more appropriate and often yields better results. Currently, most large language models on the market also support private knowledge base fine-tuning, providing significant convenience in deployment. The most crucial consideration here is data privacy.

[0114] Finally, after obtaining all user requirements on the Python side, the user's requirement prompts (number of test cases, preferred test methods, selection of large language model, etc.) and standard prompts are integrated to form the final prompts. The appropriate interface is selected for large language model processing based on the user's chosen large language model and version. The test cases output by the large language model are integrated into a standard document and saved. In addition to the standardized requirement document, this test case document also needs to include the core elements of test case number, test case description, test case input, and test case output, laying the groundwork for the subsequent construction of a visual test model.

[0115] This example uses a standardized combat command decision algorithm in mission planning. The standardized combat command decision algorithm requirements document is imported, and the large language model integrates the basic prompt words based on the user requirements from the C# client and the standardized combat command decision algorithm requirements to generate a test case set document. Below is a demonstration of the test case generation effect after importing the standardized aerospace model algorithm requirements into the large language model. The test cases generated by the large language model are very well-organized and complete, including test numbers, rule descriptions, input parameters, and output parameters. They can be successfully imported into this system for structured test case display and can be manually modified by testers for easy use.

[0116] Test cases:

[0117] R1:

[0118] Rule description: Your forces are more than 1.5 times stronger than the enemy forces, you have a terrain advantage, and you have more than 50 available resources;

[0119] Input parameter values: enemyStrength=100, friendlyStrength=200, terrainAdvantage=1, enemyActivity=0, friendlySupportDist=20, resourcesAvailable=60;

[0120] Output: 0;

[0121] R2:

[0122] Rule description: The enemy's troop strength is more than 1.5 times that of your own troops, you have no terrain advantage, and your available resources are less than 50;

[0123] Input parameter values: enemyStrength=200, friendlyStrength=100, terrainAdvantage=0, enemyActivity=0, friendlySupportDist=20, resourcesAvailable=40;

[0124] Output: 2;

[0125] R3:

[0126] Rule description: The enemy's troop strength is less than 1.5 times that of your own, but greater than your own, has a terrain advantage, and has more than 50 available resources;

[0127] Input parameter values: enemyStrength=150, friendlyStrength=100, terrainAdvantage=1, enemyActivity=1, friendlySupportDist=20, resourcesAvailable=60;

[0128] Output: 1.

[0129] Step S300: Analyze the algorithm source code structure corresponding to the algorithm requirements and construct the meta-model of the algorithm under test.

[0130] Specifically, in this embodiment, the steps of parsing the algorithm source code structure corresponding to the algorithm requirements and constructing the meta-model of the algorithm under test include:

[0131] Import the source code corresponding to the requirements of the algorithm to be tested into the algorithm testing environment building module, and automatically match the programming language type;

[0132] The code parsing function is used to analyze the structure of the source code and obtain the structured data of the algorithm under test. The structured data includes global variables, functions, input and output parameters and function call information.

[0133] A graphical modeling method is used to define the test interface of the algorithm under test. The cosine similarity search algorithm is used to calculate the similarity between the requirement description parameters and the parsed variables in order to automatically match input parameters, output parameters, global variables and instrumentation functions.

[0134] Select the algorithm under test in the test scenario canvas, bind the test interface, and complete the meta-model construction;

[0135] Save the metamodel as an XML file for easy import and instantiation.

[0136] Furthermore, Figure 5 This is a core flowchart illustrating the steps involved in analyzing the algorithm source code structure corresponding to the algorithm requirements and constructing the algorithm metamodel. Taking the task planning algorithm set and the combat command decision algorithm as an example, ... Figure 5 As shown, this step mainly includes:

[0137] First, a new model algorithm test environment needs to be created to prepare for the process. The development language of the corresponding algorithm should be selected. Since task planning algorithms mainly use C language, C language should be selected when setting up the environment.

[0138] Next, a visualization model environment needs to be established. First, a model set is established, which represents the storage location of an entire algorithm. The combat command decision algorithm is used, so an algorithm node is established. At this point, the initial visualization of the model algorithm testing and cross-linking environment is completed.

[0139] After establishing the model set and algorithm nodes, you need to import the algorithm code files into the model set. Here, we import the C language code file of the combat command decision algorithm. The software will parse and display the imported algorithm code. After that, you need to select the function interface that needs to be tested in the algorithm node. Here, we select the function interface of the combat command decision algorithm.

[0140] At this point, the cross-linking environment for testing the model algorithm has been established. Users need to save this environment, and the front end will also provide relevant prompts to users.

[0141] Specifically, the test cross-linking environment model embodies the static structure of the test, primarily highlighting the topological structure information of the software algorithm model or application test. Before defining model interaction behaviors in the test verification process model, a specific model interaction carrier needs to be defined. After completing the model interaction carrier, an algorithm carrier needs to be established for users to edit the algorithm's input and output interfaces, facilitating easier use by the subsequent test verification process model construction module.

[0142] After building the model scene, you can access the model interface by clicking on the module node. Adding the necessary files in the model interface allows you to directly parse those files; the parsing primarily involves analyzing the input and output of the code files.

[0143] Step S400: Load the meta-model of the algorithm under test and import the test data information to complete the transformation from the meta-model to the instantiated test model and form a test case set.

[0144] Specifically, in this embodiment, the steps for forming a test case set include:

[0145] Load the XML file corresponding to the metamodel;

[0146] Import structured test data information and automatically match algorithm object names, input / output parameters and meta-model interfaces in the test data document;

[0147] After a successful match, a test list is created, and each test data is converted into an XML format test case file, which includes input data, expected output data, and status indicators.

[0148] You can save, view, and modify test case sets through the test case management module.

[0149] Furthermore, Figure 6 This is the core flowchart of the steps to automatically import test requirement data documents, load the corresponding test data for the algorithm model, and generate test cases, as shown below. Figure 6 As shown, this step mainly includes:

[0150] First, a new algorithm application scenario needs to be created, which needs to be bound to the previously established test cross-linking environment.

[0151] After creating a new application scenario for the algorithm in the corresponding combat command decision-making algorithm test cross-linking environment, the software can automatically import the test case set that was previously automatically generated using the large language model, and parse the entire test case document.

[0152] After the software extracts the basic information of the corresponding test cases, it needs to be converted into an XML file. The software will automatically generate an algorithm test model and display all the corresponding test cases in a structured form.

[0153] After displaying all test cases, the software has completed the creation of a corresponding algorithm application scenario, and supports users to modify the test cases and add new test data. All test case content is also saved to the database.

[0154] Specifically, after setting up the environment, it is necessary to deploy application scenarios for the algorithm test case model. First, you need to select the cross-linking environment for binding the previous model algorithm test, and then enter the name of the algorithm model. Next, the most crucial step is to automatically import the standardized test case document. The backend will parse the entire test case document. The parsing method here uses regular expressions to operate on all document content and split its test cases, extracting each element and filling it into each variable.

[0155] The generation of test sequences can then be automated through a graphical configuration interface. After completing the above two steps, the software combines the sequence structure of the test cases with the test case data to form the test case structure, and then converts the structure into test document configuration using an XML conversion method. This facilitates testers in locally saving, viewing, modifying, and retrieving test case data, as well as using it for test script generation. The test data is then stored in a MySQL database to prepare for subsequent test execution and report generation.

[0156] Step S500: Based on the extensible stylesheet conversion language template, the test case set is converted into executable test scripts in batches.

[0157] Specifically, in this embodiment, the step of batch converting the test case set into executable test scripts includes:

[0158] Automatically selects the extensible stylesheet language conversion template based on the algorithm source code type;

[0159] This function converts XML format test case files into test scripts that are consistent with the programming language, enabling batch conversion.

[0160] Furthermore, in some application scenarios of this embodiment, the steps for batch generating executable test scripts based on standardized test case sets and template methods mainly include:

[0161] Design XSLT (Auto Select Extensible) stylesheets, and consider the universality of templates. Based on the common characteristics of scripts summarized in the text and the characteristics of the scripts themselves, formulate conversion templates for different scripts.

[0162] The XML files generated from test cases are converted into executable test scripts that meet the specified conditions using a template approach.

[0163] Specifically, the test case document has already been generated and contains standard formatted information. Here, regular expressions are used to strictly segment the document information, converting the test data from document format into a structured format for display. The original test case document format will be converted into the XML format of each test case required later.

[0164] Next, this invention designs a script conversion template, which is essentially a translator template that can translate test case files into a specific character format that the target script interpreter can understand. A keyword-driven method is used here to automatically generate test scripts. These keywords are divided into two categories: script execution keywords and script description keywords. Script execution keywords involve function names, parameter names, and execution logic, used to generate test code and ensure no duplicate naming. Script description keywords cover basic information, test conditions, expected output, and test descriptions, appearing in the script as comments. The test case script file is generated as an XML file based on graphical test cases. According to the designed template, the feature data of the formal model is extracted and translated into a specific file that the target script interpreter or compiler can understand. This invention uses XSLT style sheets as the template representation; generating executable test scripts utilizes XSLT templates to transform the test cases designed by testers into compileable and executable test scripts.

[0165] Step S600: Execute the test script in the algorithm test engine to automatically verify the test results and save the test data.

[0166] Specifically, in this embodiment...

[0167] The steps for automatically verifying test results specifically include:

[0168] Import test scripts into the algorithm testing engine and automatically select the compiler;

[0169] Create a test process and a control process. The test process loads the algorithm model, and the control process sends test stimuli and receives output.

[0170] The verification module compares the actual output with the expected results, updates the test case status, and saves the test data.

[0171] Furthermore, in some application scenarios of this embodiment, the steps of executing test scripts in the algorithm testing engine to automatically verify test results and save test data mainly include:

[0172] After converting test cases into executable scripts using template methods, the test panel needs to be built. A virtual test control panel can be constructed using C# general simulation controls, allowing for the binding of interactive test data to achieve visualized data observation. For combat command decision algorithms, bar charts can be designed to represent enemy and friendly forces, signal lights and text boxes can be used to express decision results, and simulation flow views can be used to represent the test process, etc.

[0173] The test panel is used to bind the combat command decision test model that needs to be tested, select the test cases that need to be tested, and complete the test configuration.

[0174] When the test is executed, the test panel will automatically bind the data of the combat command decision test case, execute the test script of the test case internally, and return the bound data results for visualization.

[0175] After the test case is completed, a report for the test case is automatically generated based on the template, and the data is saved to the database for subsequent construction of a visualized operational command decision-making algorithm analysis view.

[0176] Specifically, during the testing process, a simulation interface is used to visualize the test flow and verify results in real time. This test simulation is not based on a physical object, but rather on the dynamic operation and real-time interactive observation of the test flow model, thereby ensuring the accuracy and consistency of the test design. To achieve real-time simulation of the test flow model, test process data needs to be synchronized with the data bound to the simulation view. When test cases are executed, the simulation view interface loads synchronously, and test data is transmitted to the simulation view via pipeline communication. The simulation view maps real-time test data onto simulation controls. During the test, the test data undergoes logical judgment to determine whether the test steps pass, and the verification results are reflected in real time on the test flow controls.

[0177] Next, after building the simulation test panel, we need to bind the test model we need, as mentioned above. The test model has been bound to each test case in the previous process, and all test cases have generated executable test scripts. This completes the configuration work for test execution.

[0178] After configuration, you can execute selected test cases on the test panel. You can pause or terminate the process midway. Upon receiving an execution signal, the test panel internally executes the executable test script for that test case. This script contains the corresponding functions to be executed and the input data for the test case. After executing the test script, it returns the output result for that test case. The test panel then performs logical checks, comparing the result with the expected result in the test script to determine whether the test case passes for the algorithm code. If it passes, the test panel will display a success message; otherwise, it will display a failure message.

[0179] Finally, after the test cases are executed, a report for the test cases will be automatically generated based on the template. The test results of the algorithm can be analyzed based on the results of the test cases that have been executed. This analysis mainly involves information such as operation coverage, interface coverage, link coverage, and test case pass status.

[0180] In summary, the model-driven algorithm testing method in the above embodiments of the present invention utilizes the natural language processing capabilities of a large language model to perform semantic parsing and normalization on unstructured software algorithm requirement documents, completing semantic understanding of algorithm requirements, entity recognition, and relation extraction, transforming fuzzy business descriptions into structured requirement representations. By designing a prompt word framework for test generation strategies, test data generation rules are established. Based on the normalized algorithm requirements, the logical reasoning capabilities of the large language model are used to automatically extract key test elements and generate test data that meets test sufficiency requirements. A model-driven automation method is employed to parse and encapsulate algorithm code, and combined with the test data generated by the large language model, algorithm model test scenarios and test cases are automatically generated. Through model-driven testing technology and an algorithm testing engine, automated verification of test cases and collection of test results are completed, thereby effectively improving the intelligence and automation level of algorithm testing.

[0181] Example 2

[0182] The second embodiment of this application also provides a model-driven algorithm testing system for implementing the embodiments and preferred embodiments described herein, which will not be repeated hereafter. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0183] like Figure 7 As shown, the system includes: a standardization module 100, a test data module 200, a construction module 300, a test case module 400, a conversion module 500, and an execution module 600.

[0184] The standardization module 100 is used to standardize non-standard algorithm requirements based on a large language model and generate structured algorithm requirement documents.

[0185] Test data module 200 is used to standardize non-standard algorithm requirements based on a large language model and generate structured algorithm requirement documents.

[0186] Module 300 is used to parse the algorithm source code structure corresponding to the algorithm requirements and build the meta-model of the algorithm under test.

[0187] The test case module 400 is used to load the meta-model of the algorithm under test and import the test data information to complete the transformation from the meta-model to the instantiated test model and form a test case set.

[0188] The conversion module 500 is used to convert the test case set into executable test scripts in batches based on the extensible stylesheet language template conversion.

[0189] The execution module 600 is used to execute the test script in the algorithm testing engine, automatically verify the test results and save the test data.

[0190] It should be noted that the modules can be functional modules or program modules, and can be implemented in software or hardware. For modules implemented in hardware, the modules can reside in the same processor; or the modules can be located in different processors in any combination.

[0191] Example 3

[0192] A third embodiment of this application provides a computer that may include a processor 81 and a memory 82 storing computer program commands.

[0193] Specifically, the processor 81 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0194] The memory 82 may include a large-capacity storage device for data or commands. For example, and not limitingly, the memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 82 may include removable or non-removable (or fixed) media. Where appropriate, the memory 82 may be internal or external to a data processing device. In a particular embodiment, the memory 82 is non-volatile memory. In a particular embodiment, the memory 82 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0195] The memory 82 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program commands executed by the processor 81.

[0196] The processor 81 reads and executes computer program commands stored in the memory 82 to implement any of the model-driven algorithm testing methods in the above embodiments.

[0197] In some embodiments, the computer may further include a communication interface 83 and a bus 80. For example, Figure 8 As shown, the processor 81, memory 82, and communication interface 83 are connected through bus 80 and complete communication with each other.

[0198] The communication interface 83 is used to enable communication between the various modules, devices, units, and / or equipment in the embodiments of this application. The communication interface 83 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0199] Bus 80 includes hardware, software, or both, that couples computer components together. Bus 80 includes, but is not limited to, at least one of the following: data bus, address bus, control bus, expansion bus, and local bus. For example, and not as a limitation, bus 80 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 80 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0200] Example 4

[0201] The fourth embodiment of this application provides a readable storage medium. This readable storage medium stores computer program commands; when executed by a processor, these computer program commands implement any of the model-driven algorithm testing methods described in the above embodiments.

[0202] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0203] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A model-driven algorithm testing method, characterized in that, Includes the following steps: Based on the large language model, non-standard algorithm requirements are standardized to generate standardized algorithm requirement documents; Based on the standardized algorithm requirements document, test data information that meets the test sufficiency requirements is generated using a large language model; Analyze the algorithm source code structure corresponding to the algorithm requirements and construct the meta-model of the algorithm under test; Load the meta-model of the algorithm under test and import the test data information to complete the transformation from the meta-model to the instantiated test model and form a test case set; Based on the extensible stylesheet language conversion template, the test case set is batch converted into executable test scripts. The test script is executed in the algorithm testing engine to automatically verify the test results and save the test data. The specific steps for analyzing the algorithm source code structure corresponding to the algorithm requirements and constructing the meta-model of the algorithm under test include: Import the source code corresponding to the requirements of the algorithm to be tested into the algorithm testing environment building module, and automatically match the programming language type; The code parsing function is used to analyze the structure of the source code and obtain the structured data of the algorithm under test. The structured data includes global variables, functions, input and output parameters and function call information. A graphical modeling method is used to define the test interface of the algorithm under test. The cosine similarity search algorithm is used to calculate the similarity between the requirement description parameters and the parsed variables in order to automatically match input parameters, output parameters, global variables and instrumentation functions. Select the algorithm under test in the test scenario canvas, bind the test interface, and complete the meta-model construction; Save the metamodel as an XML file for easy import and instantiation.

2. The model-driven algorithm testing method according to claim 1, characterized in that, The specific steps for normalizing non-normalized algorithm requirements based on a large language model include: Obtain non-standard algorithm requirements documents input by the user; A customized prompt word framework is constructed based on the large language module and semantic parsing is performed to output a standardized algorithm requirement document that conforms to the norm based on the non-standard algorithm requirement document.

3. The model-driven algorithm testing method according to claim 1, characterized in that, The specific steps for generating test data that meets the test adequacy requirements using a large language model include: Import the normalization algorithm requirements document; A customized prompt word framework is constructed based on the large language module and semantic parsing is performed to output a structured test data document based on the requirements document of the standardized algorithm.

4. The model-driven algorithm testing method according to claim 1, characterized in that, The specific steps for creating a test case set include: Load the XML file corresponding to the metamodel; Import structured test data information and automatically match algorithm object names, input / output parameters and meta-model interfaces in the test data document; After a successful match, a test list is created, and each test data is converted into an XML format test case file, which includes input data, expected output data, and status indicators. You can save, view, and modify test case sets through the test case management module.

5. The model-driven algorithm testing method according to claim 1, characterized in that, The steps for converting the test case set into executable test scripts in batches specifically include: Automatically selects the extensible stylesheet language conversion template based on the algorithm source code type; This function converts XML format test case files into test scripts that are consistent with the programming language, enabling batch conversion.

6. The model-driven algorithm testing method according to claim 1, characterized in that, The steps for automatically verifying test results specifically include: Import test scripts into the algorithm testing engine and automatically select the compiler; Create a test process and a control process. The test process loads the algorithm model, and the control process sends test stimuli and receives output. The verification module compares the actual output with the expected results, updates the test case status, and saves the test data.

7. A model-driven algorithm testing system based on the model-driven algorithm testing method according to any one of claims 1-6, characterized in that, include: The normalization module is used to normalize non-normalized algorithm requirements based on a large language model and generate normalized algorithm requirement documents. The test data module is used to generate test data information that meets the test sufficiency requirements based on the standardized algorithm requirement document and using a large language model. The building module is used to parse the algorithm source code structure corresponding to the algorithm requirements and build the meta-model of the algorithm under test; The test case module is used to load the meta-model of the algorithm under test and import the test data information, complete the transformation from the meta-model to the instantiated test model, and form a test case set. The conversion module is used to convert the test case set into executable test scripts in batches based on the extensible stylesheet language template conversion. The execution module is used to execute the test script in the algorithm testing engine, automatically verify the test results, and save the test data. The building module is specifically used for: Import the source code corresponding to the requirements of the algorithm to be tested into the algorithm testing environment building module, and automatically match the programming language type; The code parsing function is used to analyze the structure of the source code and obtain the structured data of the algorithm under test. The structured data includes global variables, functions, input and output parameters and function call information. A graphical modeling method is used to define the test interface of the algorithm under test. The cosine similarity search algorithm is used to calculate the similarity between the requirement description parameters and the parsed variables in order to automatically match input parameters, output parameters, global variables and instrumentation functions. Select the algorithm under test in the test scenario canvas, bind the test interface, and complete the meta-model construction; Save the metamodel as an XML file for easy import and instantiation.

8. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the model-driven algorithm testing method as described in any one of claims 1-6.

9. A storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the model-driven algorithm testing method as described in any one of claims 1-6 above.

Images