An automated system testing method and apparatus based on a discrete event simulation engine
By using an automated testing method based on intelligent agents and historical data, combined with multiple testing modes, the problem of low efficiency in traditional testing is solved, enabling efficient and accurate testing of discrete event simulation engines and meeting their automated testing requirements.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INST OF SOFTWARE - CHINESE ACAD OF SCI
- Filing Date
- 2024-05-31
- Publication Date
- 2026-07-31
AI Technical Summary
Traditional software testing methods are inefficient for testing discrete event simulation engines, making it difficult to meet rapidly evolving testing needs, and are also costly in terms of manpower.
The method employs agent-based automatic testing, historical data-based automatic testing, and human-assisted semi-automatic testing, combined with automatic serial testing, automatic parallel testing, and periodic cyclic testing. It generates test instructions through agents and historical data, monitors the running status of the discrete event simulation engine, and generates test reports.
It enables efficient, fast, and accurate testing of discrete event simulation engines, with broad coverage, reduced manual costs, and meets the automated testing needs of discrete event simulation engines.
Smart Images
Figure CN118672903B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of software testing technology, specifically relating to an automatic system testing method and apparatus based on a discrete event simulation engine. Background Technology
[0002] Software testing is the process of evaluating software quality, aiming to discover defects, errors, or problems in software products. It is a crucial step in ensuring software quality. Traditional software testing mainly relies on manual testing, that is, verifying system functionality and performance through manual operation. However, this method requires a large amount of manpower and has disadvantages such as long testing cycles, strong dependence on professional backgrounds, and high error rates, making it difficult to adapt to rapidly evolving testing needs. With the development of computer technology, automated testing has become a convenient option. Automated testing refers to the process of using automated tools and technologies to execute testing activities, achieving a smooth, low-interaction testing process through a predefined series of test flows and interactions.
[0003] Discrete event simulation (DIS) is an event-driven simulation method where the behavior of a system is modeled as a series of discrete events that occur at specific times and cause changes in the system's state. Events can be defined as different types, such as message arrival events, task start or end events, etc. The discrete event simulation engine is the underlying support platform for discrete event simulation systems, possessing functions such as event management, time management, and entity management. Discrete event simulation systems are currently widely used in various industries, including transportation and logistics, industrial production, social relations, and urban planning.
[0004] Discrete event simulation engines store, process, and execute a large number of events during operation. Their core function is to ensure these events are executed according to a prescribed priority and time sequence. Conventional software testing methods struggle to focus on the various events and interactions occurring during the operation of a discrete event simulation engine, thus failing to meet its testing requirements. Against this backdrop, automated system testing methods and devices based on discrete event simulation engines have emerged.
[0005] Chinese patent: Performance testing method and storage medium for simulation engine (CN 107315889 B) provides a performance testing method for simulation engine. However, this patent only provides a manual testing method, which has high labor costs. Summary of the Invention
[0006] The purpose of this invention is to provide an automatic system testing method and apparatus based on a discrete event simulation engine. This method utilizes agent-based automatic testing, historical data-based automatic testing, and semi-automatic testing with human assistance to conduct system testing based on the discrete event simulation engine. By performing multi-dimensional analysis of indicators such as discrete event scheduling, time overhead, and anomaly handling during operation, it enables testers to efficiently and quickly perform stability testing on the discrete event simulation engine, providing a testing platform and high-precision testing assurance for the engine.
[0007] The technical solution adopted in this invention is as follows:
[0008] An automated system testing method based on a discrete event simulation engine includes the following steps:
[0009] Select the test mode of the discrete event simulation engine, including agent-based automatic testing, historical data-based automatic testing, and human-assisted semi-automatic testing;
[0010] Generate test instructions based on the selected test mode;
[0011] The discrete event simulation engine is tested according to the generated test instructions, and the running status of the discrete event simulation engine is monitored.
[0012] Generate test reports for the discrete event simulation engine.
[0013] Furthermore, the step of generating test instructions according to the selected test mode includes: automatic testing based on intelligent agents generates test instructions based on the intelligent agent training model; automatic testing based on historical data extracts and issues instructions based on instruction data in historical operation data; and semi-automatic testing based on manual assistance issues instructions manually.
[0014] Furthermore, the testing of the discrete event simulation engine according to the generated test instructions includes implementing automatic serial testing, automatic parallel testing, and periodic loop testing using different test modes. The automatic agent-based automatic testing and the automatic historical data-based automatic testing select between automatic serial testing and automatic parallel testing. Automatic serial testing involves conducting tests one by one based on the same discrete event simulation engine and simulation conditions. Automatic parallel testing involves deploying the same simulation conditions simultaneously in multiple computing environments and conducting tests separately. Periodic loop testing executes specific test scenarios and instructions according to a user-defined loop period and number of loops, covering multiple test cycles for continuous testing to discover potential problems and logical errors in the discrete event simulation engine and verify the engine's correctness and stability. If it is periodic loop testing, the test mode is selected and test parameters are configured to continue testing; otherwise, the test ends.
[0015] Furthermore, the agent-based automatic testing utilizes artificial intelligence technology to intelligently manage and interact with the testing process of the discrete event simulation engine. The agent generates corresponding instructions based on the testing strategy and issues them to the discrete event simulation engine. The agent-based automatic testing includes data processing, feature identification, feature model generation, and deep learning training. First, data processing is performed on the instruction dataset or real-time generated simulation data to provide a data foundation for the training of the deep neural network model. The deep neural network model is iteratively trained based on the feature model, updating the feature identifiers in the feature model in real time to create conditions for agent decision-making. After the feature model is updated, instructions are automatically generated based on the real-time simulation state, causing the system to generate a large number of discrete events and driving the system operation, thereby achieving a more comprehensive and efficient test of the discrete event simulation engine.
[0016] Furthermore, the automated testing based on historical data includes:
[0017] Acquire historical data, including historical configurations, commands, and situational data;
[0018] Historical data is preprocessed to meet the needs of subsequent testing;
[0019] Based on the vector space model, each historical instruction file is represented as a vector in a high-dimensional space. This vector consists of the occurrence count of all historical instructions in the historical instruction file, reflecting the frequency and importance of the historical instructions. By calculating the similarity between two vectors, the correlation analysis between historical instructions is realized, thereby extracting historical instructions and verifying their correctness and completeness.
[0020] The extracted historical instructions are sent to the discrete event simulation engine to trigger a specific event or interactive behavior at a specified time.
[0021] Furthermore, the monitoring of the discrete event simulation engine's operating status includes:
[0022] Select and configure the test mode;
[0023] Automatically configure test parameters, including simulation runtime, simulation speed, simulation data acquisition frequency, and data storage path;
[0024] Generate automated test scripts that meet the testing requirements of the discrete event simulation engine, and run the test scripts to start the automated testing process;
[0025] During operation, monitor whether the discrete event simulation engine generates any abnormal states. If no abnormal states are generated, proceed to the next step; if abnormal states are generated, execute the exception handling process.
[0026] Collect data generated during the real-time operation of the discrete event simulation engine to provide a basis for generating test reports;
[0027] Monitor the real-time status data generated during the operation of the discrete event simulation engine, and combine the real-time status data generated during operation with monitoring indicators to analyze the real-time operation status of the discrete event simulation engine.
[0028] An automated system testing device based on a discrete event simulation engine, comprising:
[0029] The test application module is used to select the test mode of the discrete event simulation engine, including agent-based automatic testing, historical data-based automatic testing, and human-assisted semi-automatic testing. It generates test instructions according to the selected test mode and tests the discrete event simulation engine according to the generated test instructions.
[0030] The runtime monitoring module is used to monitor the running status of the discrete event simulation engine;
[0031] The report generation module is used to generate test reports for the discrete event simulation engine.
[0032] This invention designs an automatic testing method and apparatus for discrete event simulation engines, providing automated and semi-automated testing methods. It can perform automatic testing based on intelligent agents, automatic testing based on historical data, and semi-automatic testing with human assistance. While improving testing efficiency, it can also test the scheduling process of discrete events, making the test scenario coverage wider and the test branches more comprehensive, thus meeting the automatic testing needs of systems based on discrete event simulation engines.
[0033] This invention provides three testing modes: automatic testing based on intelligent agents, automatic testing based on historical data, and semi-automatic testing based on human assistance. It also provides a complete testing process that includes test preparation, test monitoring, and result analysis, enabling rapid and accurate testing of discrete event simulation engines, effectively improving testing efficiency and reducing labor costs.
[0034] The automated system testing method based on a discrete event simulation engine provided by this invention has the following functions: ① It provides agent-based automated testing, enabling rapid and efficient command issuance and data interaction with the discrete event simulation engine, effectively improving the coverage of test branches; ② It provides historical data-based automated testing, combining the instruction set in existing test scenarios to quickly generate command interactions for test scenarios, effectively improving testing efficiency and flexibility; ③ It conducts testing for the discrete event simulation engine, automating the entire testing process of the discrete event simulation engine, including test preparation, test monitoring, and result analysis, and employs a multi-dimensional discrete event analysis method to specifically meet the testing needs of the discrete event simulation engine. In summary, the automated system testing method based on a discrete event simulation engine provided by this invention can meet the needs for automated, efficient, and accurate testing of discrete event simulation engines, providing a solid testing foundation for the optimization and iteration of discrete event simulation engines. Attached Figure Description
[0035] Figure 1 This is a functional composition diagram of the present invention.
[0036] Figure 2 This is a flowchart of the engine's automated testing process.
[0037] Figure 3 This is a flowchart of the test application module's operation.
[0038] Figure 4 This is a flowchart of the automatic testing process based on historical data.
[0039] Figure 5 This is the process flowchart for the operation monitoring module.
[0040] Figure 6 This is a flowchart of the report generation module's operation.
[0041] Figure 7 This is a comparison chart of simulation time versus real time cost. Detailed Implementation
[0042] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.
[0043] This invention provides an automated system testing device based on a discrete event simulation engine, mainly comprising three modules: a test application module, a operation monitoring module, and a report generation module. It realizes automated system testing functionality based on the discrete event simulation engine. The test application module defines three test modes and three instruction generation modes (instructions are user inputs that trigger specific events or interactive behaviors during engine operation), providing test users with diverse and convenient testing methods. The operation monitoring module primarily manages the testing process and monitors the test operation status, with functions including automatic configuration of test parameters, automatic script generation and execution, handling of operational anomalies, and real-time data acquisition. The report generation module is mainly used to generate test reports, with functions including automatic comparison of multi-dimensional events, visualization and report generation, diagnosis of operational anomalies, and iterative optimization of the system. Figure 1 The diagram shown is a functional composition diagram of the automatic system testing device based on a discrete event simulation engine according to the present invention. Figure 2 The diagram shows the workflow of an automated system testing method based on a discrete event simulation engine.
[0044] The specific contents covered by this invention include:
[0045] 1. Test application module:
[0046] The testing application module mainly includes three testing modes: agent-based automatic testing, historical data-based automatic testing, and human-assisted semi-automatic testing. It can realize various functions such as multi-cycle automatic cyclic testing, serial testing, and parallel testing.
[0047] like Figure 3 The diagram shows the execution flowchart of the test application module, and its steps are as follows:
[0048] Step 1: Select the test mode and configure the test parameters. Start the test run after ensuring that the system configuration and test requirements are accurate.
[0049] Step 2: Generate test instructions based on different test modes. Automatic testing based on intelligent agents can generate instructions from the trained intelligent agent model. Automatic testing based on historical data can extract and issue instructions from historical operational data. Manually assisted testing uses a manual testing method to issue instructions.
[0050] Step 3: In the automatic testing mode (automatic testing based on intelligent agents, automatic testing based on historical data), select whether to use automatic serial testing or automatic parallel testing. Serial testing involves conducting tests one by one based on the same discrete event simulation engine and simulation conditions. Parallel testing involves deploying the same simulation conditions simultaneously in multiple computing environments and conducting tests separately to improve the efficiency of simulation execution.
[0051] Step 4: Engine operation and status monitoring, which involves monitoring the engine's real-time status and data input / output.
[0052] Step 5: Generate a test report and analyze the performance of the discrete event simulation engine.
[0053] Step Six: Determine if a periodic loop test is required. Periodic loop testing executes specific test scenarios and instructions according to a user-defined loop period and number of iterations, covering multiple test cycles for continuous testing. This aims to uncover potential problems and logical errors in the discrete event simulation engine, and verify the engine's correctness and stability. If it is a periodic loop test, proceed to Step One; otherwise, end the process.
[0054] The three test modes of this invention are described in detail below:
[0055] 1) Agent-Based Automated Testing: Agent-based automated testing refers to the use of artificial intelligence technology to intelligently manage and interact with the testing process of a discrete event simulation engine, including the automatic generation and transmission of instructions. In automated testing, the agent generates corresponding instructions according to the testing strategy and issues them to the discrete event simulation engine. These instructions include operation instructions, event triggering instructions, or other instructions related to system interaction during engine operation. The generation method is not limited to rule-based, randomness-based, or machine learning-based technologies.
[0056] The automatic testing based on intelligent agents mainly consists of data processing, feature labeling, feature model generation, and deep learning training. First, data processing is performed on the instruction dataset or real-time generated simulation data to provide a data foundation for training the deep neural network model. The deep neural network model is iteratively trained based on the existing feature model, updating the feature labels in the model in real time to create conditions for the agent's decision-making. After the feature model is updated, instructions are automatically generated based on the real-time simulation state (the format of the generated instructions is shown in Table 1). Once the generated instructions enter the system, they can cause the system to generate a large number of discrete events, driving the system's operation, thereby achieving a more comprehensive and efficient test of the discrete event simulation engine.
[0057] Table 1 User Input Data Fields Table
[0058]
[0059] 2) Automated Testing Based on Historical Data: Automated testing based on historical data refers to the automatic extraction, filtering, and sending of commands by analyzing past test data (including historical scenarios, configurations, instructions, and situations). In automated testing based on historical data, historical operational data and historical instructions (historical instructions are instruction strings composed of fields such as issuance time, issuing personnel, instruction type, executor, target, parameters, priority, and duration) are used to simulate the interactive behavior and operational scenarios of the discrete event simulation engine, thereby achieving automated system testing.
[0060] Specifically, the process begins by collecting simulation scenarios, configurations, and command data constructed during testing and verification using a discrete event simulation engine, along with the resulting situational data. Since the command parameters in historical command data vary, the command strings need to be preprocessed to selectively identify commands suitable for the current situation and configure their parameters to ensure their effectiveness. Then, commands or command sequences are generated based on the current system state and patterns and regularities in historical data. Finally, the generated commands are sent to the simulation engine under test, and its operation is monitored. Through these steps, historical data can be used to generate and issue commands, enabling automated testing, improving testing efficiency and accuracy, and better simulating interactive behaviors and operational scenarios.
[0061] Figure 4 This is a flowchart of an automated testing process based on historical data, including the following steps:
[0062] Step 1: Obtain historical configuration, command, and situation data, including but not limited to extraction from databases, files, or other data sources.
[0063] Step 2: Perform preprocessing operations such as data cleaning, filtering, and transformation on historical data to meet the needs of subsequent testing.
[0064] Step 3: Historical Command Extraction. Based on the vector space model, each historical command file is represented as a vector in a high-dimensional space. This vector consists of the occurrence counts of all historical commands in the file (represented as weights), reflecting the frequency and importance of historical commands. By calculating the similarity (Euclidean distance) between two vectors, the correlation between historical commands is analyzed, thereby extracting historical commands and verifying their correctness and completeness.
[0065] Step 4: Engine operation and status monitoring. The extracted historical instructions are sent to the discrete event simulation engine to trigger a specific event or interactive behavior at a specified time.
[0066] Step 5: Test report generation.
[0067] 3) Semi-automatic testing with manual assistance: Operators and testers cooperate to manually generate and issue instructions to the simulation engine under test based on business knowledge and operational experience. This method offers relatively high flexibility, allowing for customized testing based on specific needs, resulting in greater testing flexibility, more specialized instructions, and broader test branch coverage.
[0068] 2. Operation monitoring module:
[0069] In this invention, the operation monitoring module covers the entire testing process, including test preparation, test monitoring, and result analysis. It mainly includes automatic configuration of test parameters, automatic script generation and execution, exception handling, and real-time data acquisition. For example... Figure 5 The diagram shows the processing flow of the operation monitoring module and its corresponding steps:
[0070] Step 1: Select and configure the test mode.
[0071] Step 2: Automatic configuration of test parameters, including automatic filling of configuration parameters such as simulation running time, simulation running speed, simulation data acquisition frequency, and data storage path.
[0072] Step 3: Automatic Generation and Execution of Test Scripts. This step generates automated test scripts that meet the testing requirements of the discrete event simulation engine. These scripts include execution steps such as starting the simulation engine, setting the simulation speed, monitoring simulation runtime and events, as well as data interaction processes and expected data processing functions. The entire testing process is managed and monitored. Running the test scripts begins the automated testing workflow.
[0073] Step 4: During the operation, monitor whether the discrete event simulation engine generates any abnormal states. If no abnormal states are generated, proceed to the next step; if abnormal states are generated, proceed to Step 7.
[0074] Step 5: Real-time Data Acquisition. Acquire data generated during the real-time operation of the discrete event simulation engine to provide a foundation for Module 3 (Test Report Generation).
[0075] Step Six: Data Monitoring. Monitor real-time status data such as event scheduling and event overhead generated during the operation of the discrete event simulation engine. Combine the real-time data generated during operation with monitoring indicators to analyze the real-time operation of the discrete event simulation engine. Proceed to Step Eight.
[0076] Step Seven: Execute Exception Handling and Exception Information Collection. When an exception occurs in the discrete event simulation engine, it should be promptly identified and a processing flow initiated to prevent engine crashes or other more serious situations. The exception should be logged to lay the foundation for subsequent fault diagnosis, defect repair, and optimization iterations.
[0077] Specifically, the anomaly handling described in this invention includes situations such as configuration errors in the discrete event simulation engine's runtime environment, operational anomalies caused by external engine interruptions, anomalies caused by internal engine logic errors, and program blocking caused by waiting for user interaction during operation. It primarily uses real-time collection of anomaly output information and pre-set keywords to match different handling measures, including but not limited to pre-set user input information, anomaly information recording and storage, and program exit.
[0078] Step 8: Test report generation.
[0079] The operation monitoring module can automatically conduct tests based on the discrete event simulation engine and automatically record the test results. This not only saves testing time, allowing engine testers to focus more on analyzing and solving problems, but also ensures the reliability and efficiency of the tests.
[0080] 3. Report Generation Module:
[0081] In automated system testing, the system's performance and functionality are analyzed based on the results of automated operations and monitored data, and a test report is generated. The report generation module includes functions such as multi-dimensional automatic event comparison, visualization and report generation, operational anomaly diagnosis, and system iterative optimization. Figure 6 The diagram shows the workflow of the report generation module, which includes the following steps:
[0082] Step 1: Select and configure the test mode, start the test run, and monitor whether the running status is normal.
[0083] Step 2: If the status monitoring is normal, proceed to the next step; if the status monitoring is abnormal, proceed to step 5.
[0084] Step 3: Conduct multi-dimensional automatic event comparison. A multi-dimensional comparison of the scheduling and execution of discrete events is conducted, considering factors such as the number of event scheduling attempts and average execution time overhead. This allows for in-depth analysis of the discrete event simulation engine's functional completeness, computational efficiency, and scheduling accuracy. The discrete event simulation engine invokes many simulation events during operation. These events have a fixed execution order and number of executions. Comparing the execution order allows analysis of the engine's functional completeness and scheduling accuracy. Furthermore, the time overhead of the same scheduling of the same simulation event in different test instances reflects computational efficiency and the engine's algorithm efficiency. This is also a key metric focused on in this invention's testing of the discrete event simulation engine.
[0085] Step Four: Visualize the results. Use statistical methods and data visualization techniques to process, compare, and analyze the test results data. Calculate the average, standard deviation, and other statistical quantities for each dimension, and represent the data differences using charts such as line graphs and bar charts. Proceed to Step Ten.
[0086] Step 5: Runtime Anomaly Diagnosis. During the analysis of test results, if any problems or anomalies are found in the system, it is necessary to record a detailed description of the problem, reproduction steps, and scope of impact in the logs, and to trace and diagnose the source of the anomaly. For anomalies occurring during operation, such as configuration errors in the discrete event simulation engine's runtime environment, external interrupt events causing runtime anomalies, internal logic errors causing anomalies, and program blocking due to waiting for user interaction, classification and processing are performed by matching keywords in the output information of different situations. This includes, but is not limited to, pre-setting user input information, recording and storing anomaly information, and program exit.
[0087] Step Six: Determine if there is a defect. If there is, proceed to the next step; if not, proceed to Step Eight.
[0088] Step Seven: Iterative Optimization and Defect Fixing. For diagnosed anomalies and defects, timely defect fixing is necessary. First, the defect needs to be located using dynamic code analysis techniques to pinpoint the possible defective code locations within the discrete event simulation engine. Then, patching techniques are used to fix the defective code, and the effectiveness of the fix is verified. Finally, relevant information regarding the defect fix is recorded promptly.
[0089] Step 8: Determine if there are any changes in requirements. If so, proceed to the next step; otherwise, proceed to Step 10.
[0090] Step Nine: Iterate on Test Requirements. Based on the above defect repair and test requirement change verification process, determine the rationality of the current test requirements and the necessity of adjustments. For engine function and performance test requirements that cannot be covered at present, iterative test requirements need to be implemented in a timely manner.
[0091] Step 10: Test Report Generation. Organize, summarize, and conclude the test comparison results of the discrete event simulation engine, as well as all necessary situations that occurred during the test, to form a test summary report.
[0092] Step 10: Determine if the test is performed periodically. If yes, proceed to Step 1; otherwise, end the test.
[0093] The multi-dimensional automatic comparison and visualization of events based on the discrete event simulation engine directly reflects the engine's test results. Table 2 statistically analyzes and compares the scheduling of discrete events generated by the discrete event simulation engine during the scheduling process, mainly comparing key indicators such as the number of scheduling events, simulation time, and physical event overhead. The numbers represent the order of the relevant information recorded periodically during operation. The simulation time starts at 12:30:00 on January 1, 2030, and the situation is summarized and saved every hour the simulation time advances. The physical time at the start of the simulation is 10:00:00 on June 1, 2023. For event 1, its cumulative scheduling counts are 2351, 2876, 3202, 3869, 4122, and 4387, showing a certain increasing trend. Events 2, 3, ..., and event n also show certain patterns.
[0094] Table 2 Comparison of Discrete Event Scheduling (Data Example)
[0095]
[0096] like Figure 7 The graph shown is a line graph comparing simulation time and actual time overhead, representing the time overhead of the discrete event simulation engine during operation, including both system initialization and the time overhead of scheduling and calculating discrete events.
[0097] By generating detailed test reports and analyzing the test results, a comprehensive and accurate understanding of the system's performance and functionality under different scenarios can be achieved. The report generation module is an indispensable part of automated system testing, helping testers better grasp the system's status and providing strong support for problem localization and resolution, iterative optimization, and more.
[0098] Key points of this invention:
[0099] This invention proposes an automated system testing method and apparatus based on a discrete event simulation engine. It achieves automated system testing by constructing functional modules such as test application, operation and monitoring, and report generation. The test application module primarily utilizes agent-based automated testing, historical data-based automated testing, and manual-assisted semi-automatic testing modes, employing methods such as automated serial testing, automated parallel testing, and multi-cycle cyclic testing to conduct system testing based on the discrete event simulation engine. The operation monitoring module covers the entire testing process, including test preparation, test monitoring, and result analysis, effectively managing the entire test operation and providing a data foundation and operational support for subsequent analysis. The report generation module analyzes indicators such as discrete event scheduling, time overhead, and anomaly handling during operation, employing multi-dimensional event analysis methods to enable testers to efficiently and quickly perform stability testing on the discrete event simulation engine, providing a testing platform and high-precision testing assurance for the engine.
[0100] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps of the method of the present invention.
[0101] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) storing a computer program that, when executed by a computer, implements the various steps of the method of the present invention.
[0102] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.
Claims
1. A method for automatically testing a system based on a discrete event simulation engine, characterized in that, Includes the following steps: Select the test mode of the discrete event simulation engine, including agent-based automatic testing, historical data-based automatic testing, and human-assisted semi-automatic testing; Generate test instructions based on the selected test mode; The discrete event simulation engine is tested according to the generated test instructions, and the running status of the discrete event simulation engine is monitored. Generate test reports for the discrete event simulation engine; The process of generating test instructions according to the selected test mode includes: automatic testing based on intelligent agent, which generates test instructions based on the intelligent agent training model; automatic testing based on historical data, which extracts and issues instructions based on instruction data in historical running data; and semi-automatic testing based on manual assistance, which issues instructions using a manual testing method. The discrete event simulation engine is tested according to the generated test instructions, including using different test modes to achieve automatic serial testing, automatic parallel testing, and periodic loop testing. In the automatic testing based on intelligent agents and the automatic testing based on historical data, automatic serial testing or automatic parallel testing can be selected. Automatic serial testing is to conduct tests one by one based on the same discrete event simulation engine and simulation conditions. Automatic parallel testing is to deploy the same simulation conditions in multiple computing environments at the same time and conduct tests separately. The cycle test executes specific test scenarios and instructions according to the user-defined cycle period and number of cycles, covering multiple test cycles for continuous testing, in order to discover potential problems and logical errors in the discrete event simulation engine, and verify the correctness and stability of the engine; If it is a periodic test, select the test mode and configure the test parameters to continue the test; otherwise, end the test. The agent-based automated testing utilizes artificial intelligence technology to intelligently manage and interact with the testing process of the discrete event simulation engine. The agent generates corresponding instructions based on the testing strategy and issues them to the discrete event simulation engine. The agent-based automated testing includes data processing, feature identification, feature model generation, and deep learning training. First, data processing is performed on the instruction dataset or real-time generated simulation data to provide a data foundation for the training of the deep neural network model. The deep neural network model is iteratively trained based on the feature model, and the feature identifiers in the feature model are updated in real time to create conditions for agent decision-making. After the feature model is updated, instructions are automatically generated based on the real-time simulation state, causing the system to generate a large number of discrete events and driving the system to run, thereby achieving a more comprehensive and efficient test of the discrete event simulation engine. The instructions automatically generated based on real-time simulation status include the following fields in their format: Command Type: The data type is a string, indicating the specific type of the command; Executor: The data type is string, representing the object or unit that executes the instruction; Target: The data type is string, representing the object to which the instruction will be applied; Parameters: The data type is either an integer or a string, depending on the specific instruction type; Priority: The data type is an integer or an enumeration, representing the priority of the instruction, used to determine the execution order of instructions; Duration: The data type is an integer or an enumeration, representing the duration for which the instruction needs to be executed. The automated testing based on historical data includes: Acquire historical data, including historical configurations, commands, and situational data; Historical data is preprocessed to meet the needs of subsequent testing; Based on the vector space model, each historical instruction file is represented as a vector in a high-dimensional space. This vector consists of the occurrence counts of all historical instructions in the file, reflecting the frequency and importance of the historical instructions. By calculating the similarity between two vectors, the correlation between historical instructions is analyzed, thereby extracting historical instructions and verifying their correctness and completeness. The historical instructions are instruction strings composed of the issuance time, the issuing personnel, the instruction type, the executor, the target, the parameters, the priority, and the duration. The extracted historical instructions are sent to the discrete event simulation engine to trigger a specific event or interactive behavior at a specified time. The test report for the discrete event simulation engine includes: Step 1: Select and configure the test mode, start the test run, and monitor whether the running status is normal; Step 2: If the status monitoring is normal, proceed to the next step; if the status monitoring is abnormal, proceed to step 5. Step 3: Conduct a multi-dimensional comparison of the scheduling and execution of discrete events, and conduct an in-depth analysis of the functional completeness, computational efficiency, and scheduling accuracy of the discrete event simulation engine; Step 4: Use statistical methods and data visualization techniques to process, compare and analyze the test results data, calculate the statistics of each dimension indicator, and represent the data differences in the form of charts. Then proceed to Step 10. Step 5: During the analysis of test results, if any problems or anomalies are found in the system, record a detailed description of the problem, the steps to reproduce it, and the scope of its impact. Then, trace the source and diagnose the anomalies. The anomalies include: configuration errors in the discrete event simulation engine's operating environment, external interruption events causing operational anomalies, internal logic errors causing anomalies, and program blocking caused by waiting for user interaction during operation. The anomalies are classified and processed by matching keywords in the output information of different situations, including pre-setting user input information, recording and storing anomaly information, and program exit. Step Six: Determine if any defects exist. If so, proceed to the next step; otherwise, proceed to Step Eight. Step Seven: Repair any diagnosed abnormalities and defects promptly. Step 8: Determine if there are any changes in requirements. If so, proceed to the next step; otherwise, proceed to Step 10. Step 9: Based on the defect repair and test requirement change verification process, determine the rationality of the current test requirements and the necessity of adjustment; for engine function and performance test requirements that cannot be covered at present, iterate the test requirements in a timely manner. Step 10: Organize, summarize, and conclude the test comparison of the discrete event simulation engine and all necessary situations that occurred during the test, and form a test report; Step 10: Determine if the test is performed periodically. If yes, proceed to Step 1; otherwise, end the test.
2. The method of claim 1, wherein, The monitoring of the discrete event simulation engine's operating status includes: Select and configure the test mode; Automatically configure test parameters, including simulation runtime, simulation speed, simulation data acquisition frequency, and data storage path; Generate automated test scripts that meet the testing requirements of the discrete event simulation engine, and run the test scripts to start the automated testing process; During operation, monitor whether the discrete event simulation engine generates any abnormal states. If no abnormal states are generated, proceed to the next step; if abnormal states are generated, execute the exception handling process. Collect data generated during the real-time operation of the discrete event simulation engine to provide a basis for generating test reports; Monitor the real-time status data generated during the operation of the discrete event simulation engine, and combine the real-time status data generated during operation with monitoring indicators to analyze the real-time operation status of the discrete event simulation engine.
3. A system for automatic testing of a discrete event simulation engine based system using the method of claim 1 or 2, characterized in that, include: The test application module is used to select the test mode of the discrete event simulation engine, including agent-based automatic testing, historical data-based automatic testing, and human-assisted semi-automatic testing. It generates test instructions according to the selected test mode and tests the discrete event simulation engine according to the generated test instructions. The runtime monitoring module is used to monitor the running status of the discrete event simulation engine; The report generation module is used to generate test reports for the discrete event simulation engine.
4. A computer device, comprising: It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of claim 1 or 2.
5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a computer, implements the method of claim 1 or 2.