Script test verification method and electronic device
By constructing a mapping relationship between server simulation configuration information and response data, the virtualization and automation of script testing and verification are realized, solving the problems of time-consuming and labor-intensive test environment construction and insufficient automation in existing technologies, and realizing high-frequency and low-cost full-process verification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INSPUR SUZHOU INTELLIGENT TECH CO LTD
- Filing Date
- 2026-05-20
- Publication Date
- 2026-06-16
AI Technical Summary
Existing technologies for script-based testing and verification suffer from incomplete verification dimensions, limited automation, and poor scenario adaptability, making it difficult to achieve full-process verification. Furthermore, manually setting up the test environment is time-consuming and labor-intensive, failing to meet the automated testing needs in the server testing field.
By acquiring the server's simulated configuration information and response data files, a mapping relationship between test commands and response data is established, a search engine is built, and the virtualization and automation of script testing and verification are realized, freeing it from dependence on real servers and completing the entire process verification.
The entire verification process of test scripts can be completed without the participation of a real server, shortening environment preparation time, reducing hardware costs, shielding network and hardware instability factors, and improving the reliability of test results and the efficiency of script defect location.
Smart Images
Figure CN122220199A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a script testing and verification method and an electronic device. Background Technology
[0002] With the widespread adoption of cloud computing, edge computing, and microservice architectures, server complexity, deployment scale, and iteration speed have increased, making traditional manual testing insufficient. Automated testing has gradually become a core testing method. Currently, the main approach combines static review of automated script code with manual verification by Automation for Development (ADE) engineers. This still requires building a real-world test environment based on the business scenario to perform complete test verification. Especially in server testing, setting up the test environment is essentially equivalent to building the entire server, which is time-consuming, labor-intensive, and has significant material requirements. This fails to meet the increasing demand for automated testing in the server testing field. Current related technologies include static code analysis tools, which can check basic code specifications but lack business logic verification, resulting in insufficient flexibility and difficulty in achieving full-process verification. Automated testing frameworks have built-in verification to ensure framework adaptation but lack universal standards, and their verification capabilities for test scenarios relying on hardware drivers and tools are insufficient. When automated verification is insufficient, it is necessary to build servers with corresponding configurations according to business requirements and manually execute script tests to verify the logic and functionality of the scripts themselves. Summary of the Invention
[0003] This application provides a script-based testing and verification method and an electronic device to at least solve the problems in related technologies, such as incomplete verification dimensions, limited automation, poor scenario adaptability, and difficulty in achieving full-process verification. Manual verification requires building an actual test environment based on the business scenario to fully execute the test verification. Building the test environment is time-consuming and labor-intensive, and has high material requirements, which cannot meet the increasing demand for automated testing in the server testing field.
[0004] This application provides a script-based testing and verification method, including: Obtain the response data file of the server processing test instructions, as well as the simulation configuration information of the server; parse the response data file according to the simulation configuration information to obtain the mapping relationship between test instructions and response data; The simulation configuration information and the mapping relationship between test commands and response data are stored, and a search engine is built. When the server test script executes a test, and when the server test script executes a test on the server under test, the target configuration information of the server under test is obtained, and the target instruction for the server test script is obtained based on the target configuration information. The search engine determines the response data corresponding to the target instruction in the target database based on the target instruction. Identify dynamically filled data in the response data corresponding to the target instruction, generate dynamically updated data based on the dynamically filled data, and replace the dynamically filled data with the dynamically updated data to obtain the target response data; The execution result of the server test script is determined based on the target response data, and a test verification report is generated based on the execution result.
[0005] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described script testing and verification methods when executing the computer program. Obtain the response data file of the server processing test instructions, as well as the simulation configuration information of the server; parse the response data file according to the simulation configuration information to obtain the mapping relationship between test instructions and response data; The simulation configuration information and the mapping relationship between test commands and response data are stored, and a search engine is built. When the server test script executes a test, and when the server test script executes a test on the server under test, the target configuration information of the server under test is obtained, and the target instruction for the server test script is obtained based on the target configuration information. The search engine determines the response data corresponding to the target instruction in the target database based on the target instruction. Identify dynamically filled data in the response data corresponding to the target instruction, generate dynamically updated data based on the dynamically filled data, and replace the dynamically filled data with the dynamically updated data to obtain the target response data; The execution result of the server test script is determined based on the target response data, and a test verification report is generated based on the execution result.
[0006] This application achieves virtualization and automation of the script testing and verification process by acquiring the server's simulated configuration information and its response data files for processing test commands, parsing them, establishing a mapping relationship between test commands and response data, storing the aforementioned data, and building a retrieval engine. This allows the verification of test scripts to be completely independent of specific models and configurations of real physical servers, enabling full-process verification of test scripts without the need for a real server. It eliminates the need to build and maintain complex physical test environments, significantly shortening environment preparation time, reducing hardware costs, and allowing for high-frequency, low-cost verification of test scripts. It also shields the test scripts from interference from unstable factors such as network and hardware in the real environment, ensuring that test results are entirely determined by the script logic, facilitating rapid location and repair of script defects. Attached Figure Description
[0007] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0008] Figure 1 This is a diagram illustrating the application environment of a script testing and verification method in one embodiment of this application. Figure 2 This is a flowchart illustrating a script testing and verification method in one embodiment of this application; Figure 3 This is a schematic diagram illustrating the process of obtaining a response data file of a server processing test instructions and the server's simulated configuration information in one embodiment of this application, and parsing the response data file based on the simulated configuration information to obtain the mapping relationship between test instructions and response data. Figure 4 This is a flowchart illustrating semantic analysis based on the instruction syntax tree in one embodiment of this application; Figure 5 This is a schematic diagram of a process in one embodiment of the present application to identify dynamically filled data in response data corresponding to the target instruction, generate dynamically updated data based on the dynamically filled data, and replace the dynamically filled data with the dynamically updated data to obtain target response data. Figure 6 This is a structural block diagram of a script testing and verification device in one embodiment of this application; Figure 7 This is an internal structural diagram of a computer device in one embodiment of this application. Detailed Implementation
[0009] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.
[0010] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.
[0011] In related technologies, script-based testing and verification lead to complex verification processes, long verification cycles, and unstable verification results. They are highly dependent on hardware environments: test scripts must run on servers with configurations exactly as intended when the scripts were written; otherwise, failures will occur due to configuration differences (such as operating system version, firmware version, component dependencies, etc.). Resource consumption and cost: Maintaining a large number of physical test servers with different configurations requires high costs and operational effort. Poor test stability: Real servers may experience network fluctuations, temporary service unavailability, etc., leading to unstable test results and making it difficult to distinguish between script logic errors and environmental issues.
[0012] The script testing and verification method provided in this application can be applied to, for example... Figure 1 The application environment shown is illustrated. The script testing and verification system introduces a Mock mechanism to construct a virtual command response environment, enabling automated test scripts to complete the entire verification process without the involvement of a real server. The script testing and verification system mainly involves four modules: a script execution module, a mock execution module, a command response service module, and an interface and configuration information database module.
[0013] The script execution module is responsible for scheduling and executing the written automated test scripts.
[0014] During script execution, when interaction with the server under test is required (such as executing Shell commands to query status, configuration information, or calling API interfaces), this module sends the corresponding command request to the simulation execution module. Shell commands are computer shell commands.
[0015] Receive simulated response data from the simulation execution module, and continue to execute the subsequent business logic judgment process of the script based on the response data.
[0016] The simulation execution module acts as an adapter between the script execution module and the command response service. It receives raw shell commands from the script execution module, standardizes and encapsulates them into a unified command request object, which includes at least the command content and the initiation time. The encapsulated command request is then sent to the command response service module, and the module awaits a response. Finally, the module receives the structured response data returned by the command response service, converts it into the format expected by the test script (such as standard output stream, return value, etc.), and feeds it back to the script execution module.
[0017] The instruction response service module is responsible for parsing instructions and generating intelligent responses.
[0018] The instruction response service module includes an instruction parsing unit, a data query unit, and an instruction assembly unit.
[0019] Command parsing unit: Performs in-depth parsing on received command requests, breaking them down into the command type (such as ipmitool, curl, systemctl), target operation object, input parameters, and expected response format.
[0020] Data Query Unit: Based on the parsing results, it initiates queries to the interface and configuration information database module to obtain the data required for the simulated response. For example, the command `systemctl status nginx` will query the default status of the "nginx" service in the simulation environment.
[0021] Response assembly unit: Based on the expected response format of the instruction (such as generating a file, outputting to the console, etc.), extract data from the query results and assemble it into simulated data that is completely consistent with the real server response.
[0022] The interface and configuration information base module serves as the system's data hub, storing the data required for all simulation test behaviors. This module includes a data import interface, a command mapping library, a configuration information base, and a search engine.
[0023] Data import interface: Supports importing API documentation, server configuration information, service status tables, etc. from various file formats (such as Excel, JSON, YAML). The system can automatically parse these files and build a structured internal database.
[0024] Command mapping library (API information base): Stores the mapping relationship between commands and response data. For example, the command ipmitool power status is defined to return "Chassis Power is on".
[0025] Configuration information repository: Using "machine identifier" (such as "Server-Type-A-v1.0") as the key, it stores the simulated configuration information of various servers, such as network configuration, firmware version, component configuration, driver version, BMC information, etc.
[0026] Search engine: Provides efficient search functionality, supporting quick matching of corresponding response data and configuration information through keywords such as command type, operation object, and machine identifier.
[0027] like Figure 2 As shown, an embodiment of this application provides a script testing and verification method, including the following steps: Step S1: Obtain the response data file of the server processing the test command, as well as the server's simulation configuration information. Based on the simulation configuration information, parse the response data file to obtain the mapping relationship between the test command and the response data. Step S2: Store the simulation configuration information and the mapping relationship between test commands and response data, and build a retrieval engine; Step S3: In response to the server test script executing a test on the server under test, obtain the target configuration information of the server under test, and obtain the target instructions to be executed by the server test script based on the target configuration information; Step S4: The retrieval engine determines the response data corresponding to the target instruction in the target database based on the target instruction; Step S5: Identify the dynamically filled data in the response data corresponding to the target instruction, generate dynamically updated data based on the dynamically filled data, and replace the dynamically filled data with the dynamically updated data to obtain the target response data; Step S6: Determine the execution result of the server test script based on the target response data, and generate a test verification report based on the execution result.
[0028] This system achieves virtualization and automation of the script testing and verification process by acquiring simulated server configuration information and its response data files for processing test commands. After parsing, a mapping relationship is established between test commands and response data. This data is then stored and a retrieval engine is built. This allows for the complete verification of test scripts, independent of specific models and configurations of real physical servers, enabling full-process verification without the need for a real server. The elimination of the need to build and maintain complex physical test environments significantly reduces environment preparation time and hardware costs, allowing for high-frequency, low-cost verification of test scripts. Furthermore, it shields the system from interference from unstable factors such as network and hardware in the real environment, ensuring that test results are entirely determined by script logic, facilitating rapid identification and repair of script defects.
[0029] In this embodiment, response data files from various servers processing various test commands are collected, and the mapping relationship between various test commands and response data is obtained by parsing the response data files, including: The response data files of the server for processing various test commands are obtained through the data import interface, and the format type of the response data files is identified. The response data file is in the format of an application programming interface specification file. The response data file is parsed to extract the interface endpoints (paths), request methods (get, post, etc.), request parameters, and the corresponding response body, status code, and response format (schema) as extracted information. The extracted information is then converted into instruction mapping entries. Since the response data file is in the format of a specification file, the response data file is parsed to generate a simulation data template that conforms to the scenario constraints; Since the response data file is in tabular format, parsing the response data file maps the rows or columns of the table to request-response pairs for the interface. The instruction mapping entries, simulated data templates, and request-response pairs are normalized to transform them into a normalized data model.
[0030] Specifically, by automatically identifying file types and performing targeted processing, heterogeneous response data is standardized into a unified data model. This standardization transformation not only ensures the consistency and scalability of the data structure but also enables the system to achieve universal verification under various server specifications, thereby significantly improving the method's scenario adaptability and data compatibility.
[0031] like Figure 3 As shown, the key to constructing simulation data is to quickly and accurately convert external API definitions into standardized data structures within the system. The system has built-in dedicated parsers for different file formats.
[0032] The first parser, using the OpenAPI parser, extracts all defined API endpoints (paths), request methods (GET, POST, etc.), request parameters, and corresponding response bodies, status codes, and schemas for YAML or JSON application programming interface specification files. This information is then converted into entries in the instruction mapping library.
[0033] The second parser uses a JSON Schema parser: for scenarios where data structures are defined using independent JSON Schema files (specifications, etc.), the parser will parse the Schema and generate a simulated data template that conforms to the constraints of that structure.
[0034] The third parser, employing an Excel / CSV parser, is used to process tabular data provided by ADE or upstream / downstream testing systems. The parser maps the rows or columns of the table to API request-response pairs. For example, the first column might be the API URL, the second the request parameters, and the third a predefined JSON response string.
[0035] Data Normalization and Storage: The parsed data is uniformly converted into a standardized data model within the system. This model abstracts common fields such as "instruction identifier," "request pattern," and "response pattern," and stores them in the API and configuration information repository. During storage, indexes (such as indexes based on API endpoint URLs and methods) are created to support efficient subsequent queries.
[0036] In this embodiment, the system stores simulation configuration information and the mapping relationship between test commands and response data, and constructs a retrieval engine, including: Configure the configuration information base and command mapping base in the target database; The configuration information base stores the simulated configuration information, and the command mapping base stores the mapping relationship between various test commands and response data. Based on the simulated configuration information of various servers and the correlation between various test commands and response data, a retrieval engine is built that supports matching corresponding response data and simulated configuration information by command type, operation object, and machine identifier.
[0037] Through hierarchical database management and index optimization, the mapping relationship between server configurations and command responses can be efficiently indexed and accurately located. The multi-condition matching mechanism of the search engine significantly improves the speed and accuracy of data queries, providing structured data support for subsequent automated verification. This organizational approach enables the system to quickly respond to and match different server configurations and command types, ensuring the real-time performance and scalability of testing and verification.
[0038] The simulated configuration information for various servers includes, but is not limited to: hardware configuration (CPU model / quantity, memory quantity / size, hard drive model / capacity / RAID level), firmware information (BIOS / BMC version), software configuration (operating system version, installed software packages and versions), and network configuration (IP address, MAC address).
[0039] Virtual Configuration Model Construction: The parsed configuration information is constructed into a hierarchical, addressable virtual configuration model. This model uses a "machine identifier" (e.g., Server-Type-A-v1.0) as its root node. Internally, data is organized using keys similar to file paths, for example: / Server-Type-A-v1.0 / hardware / cpu / model -> IntelXeon Gold 6330, / Server-Type-A-v1.0 / software / packages / nginx / version -> 1.18.0. This structure allows data query units to quickly retrieve any configuration information using a clearly defined "path".
[0040] like Figure 4 As shown, in this embodiment, in response to the server test script executing a test on the server under test, the target configuration information of the server under test is obtained, and the target instructions for the execution of the server test script are obtained according to the target configuration information, including: Obtain the parameter list of the server under test, extract the target machine identifier from the parameter list, and determine the target configuration information of the server under test based on the target machine identifier; The system retrieves the target instructions for the interaction between the server test script and the server under test, obtains the original instruction string of the target instructions, parses and serializes the original instruction string and concatenates it to generate a structured instruction request object. The instruction request object includes a unique identifier, command line string, parsed command body, parameter list, target machine identifier and request timestamp. Encapsulate the instruction request object into a standard format instruction; The standard format instructions are constructed into an instruction syntax tree, in which the main command, subcommands, options and parameters are marked; Semantic analysis is performed based on the instruction syntax tree to identify the instruction type, operation type, target operation object, and parameters as the parsing results.
[0041] By employing structured encapsulation and syntax tree construction, unstructured instructions are transformed into a unified standard format, facilitating subsequent semantic analysis and database retrieval. Standardized encapsulation reduces the parsing complexity caused by differences in different scripting languages and server commands, enabling unified processing of test scripts across platforms and scenarios. This improves the system's compatibility and testing accuracy in multi-server environments, laying the foundation for automated semantic recognition.
[0042] The system modifies the environment variables at runtime to redirect script calls to system tools (such as ipmitool and dmidecode) to a wrapper script provided by the Mock system. This wrapper does not actually execute the commands but captures them and their arguments.
[0043] Request object serialization: After capturing the raw command string (such as ipmitool -I lanplus -H192.168.1.1 -U admin -P password power status), the encapsulation module parses and serializes it to generate a structured command request object. This object is typically in JSON format and contains the following fields: command_id: Unique identifier (UUID).
[0044] command_line: The raw command line string.
[0045] parsed_command: The parsed command body (ipmitool).
[0046] arguments: List of arguments (['-I', 'lanplus', '-H', '192.168.1.1', ...]).
[0047] target_machine_id: Target machine identifier (extracted from parameters or obtained from context).
[0048] timestamp: Request timestamp.
[0049] In this embodiment, semantic analysis based on the instruction syntax tree includes: Semantic extraction is performed on the instruction syntax tree to obtain the command intent. Based on the pre-built instruction semantic rule base, the command intent is classified into instructions and semantic tags are generated. The semantic similarity is calculated by comparing semantic labels with historical execution samples, and the confidence of instruction classification is dynamically adjusted. When the confidence level is lower than a preset threshold, a secondary judgment is made on the instruction classification based on the context vector, and the semantic label is adjusted according to the judgment result.
[0050] The introduction of a semantic understanding mechanism enables the system to extract operational intent from command syntax and dynamically adjust classification confidence, thereby achieving accurate recognition of complex or ambiguous instructions. By comparing semantic tags with historical samples, the system possesses self-learning capabilities and can continuously optimize its semantic rule base. This intelligent semantic analysis significantly improves the accuracy of script parsing and the system's intelligence level, making testing and verification closer to real execution logic and reducing false positives and false negatives.
[0051] Instruction syntax tree analysis involves the following steps: For complex command-line tools (such as ipmitool), the system maintains its command syntax rules. The parser uses these rules to construct an instruction syntax tree from the encapsulated instruction request, clearly distinguishing the main command, subcommands, options, and parameters.
[0052] For example, the syntax tree will recognize the following when executing `ipmitool sel list -c`: Main command: ipmitool; Subcommand: sel list; Option: -c (clear-on-read); Semantic extraction: Based on syntactic analysis, further semantic information is extracted.
[0053] Operation type identification: Determine whether the command is a "query" (such as status, list), a "configuration" (such as set, config), or an "operation" (such as power on, reset).
[0054] In this embodiment, the retrieval engine determines the response data corresponding to the target instruction in the target database based on the target instruction, including: The query conditions are constructed from the target database based on the parsing results using the retrieval engine. The query conditions include the instruction type, the target operation object, and the target machine identifier (machine ID). First-level filtering is performed based on instruction type and target operation object; A second level of filtering is performed using the target machine identifier to obtain the configuration or status of the target machine; For commands with parameters, the parameters are used as filtering conditions for a third level of screening to obtain response data that matches the target command.
[0055] The system implements a hierarchical retrieval logic, enabling it to quickly locate the most matching response data from a large-scale dataset. Through multi-dimensional filtering, the system improves matching accuracy while maintaining query speed, effectively avoiding the problems of response data confusion or misuse. This precise matching mechanism provides a reliable foundation for subsequent simulated response generation, enhancing the accuracy of automatic verification and system response performance.
[0056] The retrieval engine utilizes multiple indexes established during the data import phase for efficient querying.
[0057] Dynamic data generator: For certain data that cannot be fully pre-defined (such as timestamps that are different for each query, or auto-incrementing IDs), dynamic functions are embedded in the pre-defined response template.
[0058] During a query, these functions are also triggered if the returned results contain placeholders or other identifiers. For example, if you define "timestamp": "${@timestamp}" in the response template, the placeholders will be replaced with the current real-time timestamp when the query is complete. Other functions include ${@randomInt(1,100)} (random number) and ${@sequentialId} (sequence number).
[0059] In this embodiment, determining the response data corresponding to the target instruction in the target database using a retrieval engine based on the target instruction further includes: Set an exit code for querying the target database based on the parsed results using the search engine; If the response data matching the target instruction is successfully obtained, the exit code is returned as the first value; If no response data matching the target instruction is received, the exit code is returned as the second value; If the server being tested does not exist, the exit code is returned as the third value. In response to insufficient permissions, the exit code is returned as the fourth value.
[0060] By establishing an exit code system, the system automates error identification and status feedback during script testing and verification. Different exit states allow the system to quickly pinpoint the problem type, avoiding manual troubleshooting and improving the traceability of test execution and the efficiency of anomaly diagnosis. This explicit status management mechanism enhances the system's robustness and fault tolerance, making the automated verification process safer and more controllable.
[0061] Console output (standard output / error stream) simulation: Renders structured response data into a specific text format based on the command type. For example, it renders JSON-formatted BMC information obtained from a query into a human-readable, table-lined text format, similar to the output of the ipmitool command in the terminal, using a preset template.
[0062] Exit code simulation: Set the correct exit code for the operation. For example, 0 for success, 1 for failure, 2 for service not found, and 3 for insufficient permissions.
[0063] File output simulation: For commands that generate files (such as ipmitool sel list>sel.log), the assembly module will generate a specified file according to the parameter definition, the contents of which completely simulate the output of the real command.
[0064] Exception and Delay Injection: To test script robustness, the assembly module supports injecting preset exceptions and delays into the response. Depending on the configuration, it can return an error message simulating command execution failure and the exit code at the time of the error. Alternatively, it can wait for a specified period of time before returning the response to simulate network latency or slow server processing scenarios.
[0065] like Figure 5 As shown, in this embodiment, dynamically filled data in the response data corresponding to the target instruction is identified, dynamically updated data is generated based on the dynamically filled data, and the dynamically filled data is replaced with the dynamically updated data to obtain the target response data; The execution result of the server test script is determined based on the target response data, and a test verification report is generated based on the execution result, including: Return the response data corresponding to the target instruction as preset static data; Determine whether to add dynamic data content to the preset static data. If yes, execute the dynamic function to update the value of the dynamically populated data in the preset static data to form the dynamically updated data. Replace the dynamically populated data with the dynamically updated data to obtain the query result. Otherwise, use the preset static data as the query result and return the query result. The query results are assembled according to the server response format, and the assembled query results are converted into a text format that the server test script can recognize and used as the simulated response data of the target instruction. Analyze the simulated response data to determine the test results of executing the target instruction.
[0066] The dynamically populated data includes timestamps and dynamic sequence numbers. By combining static data with dynamic variables, a simulated output consistent with the real server response is achieved. The system can automatically adjust timestamps, sequence numbers, and other content as needed, making the simulation results closer to actual operating scenarios. This dynamic response generation mechanism not only improves the realism of test results but also ensures the consistency of script verification across different times and environments, providing a reliable foundation for automated regression testing and continuous verification.
[0067] In this embodiment, the execution result of the server test script is determined based on the target response data, and a test verification report is generated based on the execution result, including: In response to the target response data obtained from the target instruction, determine whether the server test script has completed the test; If the server test script has completed the test, the execution result of the server test script is determined by comparing the target response data with the expected test result of the server test script, and a test report is generated based on the execution result; If the server test script has not completed the test, the next target instruction of the interaction between the server test script and the server under test is obtained. The next target instruction is standardized, encapsulated, and parsed to obtain the next parsing result. The target database is searched by the retrieval engine according to the next parsing result to obtain the next response data corresponding to the next target instruction. The next response data obtained by the query is dynamically updated, assembled, and formatted. The next response data is analyzed to determine the next test result, and then it is determined again whether the server test script has completed the test.
[0068] A self-driven testing and verification process was constructed, allowing the system to automatically control the testing pace based on the script execution status, cyclically executing verification instructions until the task is completed. After testing, a report is automatically generated, reducing manual intervention and improving the completeness and efficiency of the verification loop. This self-looping and report generation mechanism truly enables unattended and continuous automation of the testing and verification process, enhancing the overall intelligence level of the system.
[0069] In this embodiment, generating a test report includes: The simulated response data of the target instruction is compared with the expected response template at the field level, and the response field matching rate and content similarity are calculated. When the matching rate or similarity falls below a preset threshold, a difference analysis report is output. The difference analysis report includes a description of the field differences, the direction of the deviation, the inferred cause, and suggestions for improvement. The results of each test are stored in a time-series format to generate a test execution chain; Generate a test report based on the test execution chain, including execution statistics, failure distribution, response latency curves, and configuration correlations; Test reports can be filtered, exported, and compared through an interactive interface.
[0070] By comparing the simulated response data of the target instruction with the expected response template at the field level, the system achieves refined verification of test results. This accurately identifies structural or numerical differences in the response content, significantly improving the accuracy of the verification process. By calculating the field matching rate and content similarity, the system quantifies the execution consistency of the test script and automatically generates a difference analysis report when the matching rate or similarity falls below a threshold, enabling immediate identification and location of abnormal results. The difference analysis report includes explanations of field differences, deviation directions, hypothetical causes, and remediation suggestions, allowing developers to quickly pinpoint the source of the problem and optimize the script logic accordingly, shortening the defect investigation cycle.
[0071] Furthermore, by storing each test result in a time-series format and forming a test execution chain, the system can track the evolution of each round of testing, identify trending issues and performance degradation phenomena, and provide data support for test quality monitoring. The generated test reports not only include multi-dimensional indicators such as execution statistics, failure distribution, response latency curves, and configuration correlations, but also support filtering, exporting, and comparing through an interactive interface, achieving visualization of results and intelligent analysis. Thus, this solution significantly improves the systematic nature, traceability, and decision-making support value of test result analysis, expanding the script testing and verification process from a single detection to a continuously optimized closed-loop analysis system.
[0072] The aforementioned script testing and verification method acquires the server's simulated configuration information and its response data files for processing test commands. After parsing, a mapping relationship is established between test commands and response data. This data is then stored and a retrieval engine is built, achieving virtualization and automation of the script testing and verification process. This allows the verification of test scripts to be completely independent of specific models and configurations of real physical servers, enabling full-process verification of test scripts without the participation of a real server. The elimination of the need to build and maintain complex physical test environments significantly shortens environment preparation time, reduces hardware costs, and allows for high-frequency, low-cost verification of test scripts. Furthermore, it shields the script from interference from unstable factors such as network and hardware in the real environment, ensuring that test results are entirely determined by script logic, facilitating rapid identification and repair of script defects.
[0073] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.
[0074] In one embodiment, such as Figure 6 As shown, a script testing and verification device 10 is provided, including: a data acquisition and mapping module 1, a database construction and retrieval module 2, an instruction standardization and parsing module 3, a data query and matching module 4, a response assembly and result analysis module 5, and a report generation module 6.
[0075] The data acquisition and mapping module 1 is used to acquire the response data file of the server processing test commands, as well as the server's simulation configuration information. Based on the simulation configuration information, it parses the response data file to obtain the mapping relationship between test commands and response data.
[0076] The database construction and retrieval module 2 is used to store simulation configuration information and the mapping relationship between test commands and response data, and to build a retrieval engine.
[0077] The instruction standardization and parsing module 3 is used to obtain the target configuration information of the server under test when the server test script executes the test on the server under test, and obtain the target instructions to be executed by the server test script based on the target configuration information.
[0078] The data query and matching module 4 is used to determine the response data corresponding to the target instruction in the target database based on the target instruction through the retrieval engine.
[0079] The response assembly and result analysis module 5 is used to identify dynamically filled data in the response data corresponding to the target instruction, generate dynamically updated data based on the dynamically filled data, and replace the dynamically filled data with the dynamically updated data to obtain the target response data. The report generation module 6 is used to determine the execution result of the server test script based on the target response data, and generate a test verification report based on the execution result.
[0080] In this embodiment, response data files from various servers processing various test commands are collected, and the mapping relationship between various test commands and response data is obtained by parsing the response data files, including: The response data files of the server for processing various test commands are obtained through the data import interface, and the format type of the response data files is identified. The response data file is in the format of an application programming interface specification file. The response data file is parsed to extract the interface endpoints (paths), request methods (get, post, etc.), request parameters, and the corresponding response body, status code, and response format (schema) as extracted information. The extracted information is then converted into instruction mapping entries. Since the response data file is in the format of a specification file, the response data file is parsed to generate a simulation data template that conforms to the scenario constraints; Since the response data file is in tabular format, parsing the response data file maps the rows or columns of the table to request-response pairs for the interface. The instruction mapping entries, simulated data templates, and request-response pairs are normalized to transform them into a normalized data model.
[0081] In this embodiment, the system stores simulation configuration information and the mapping relationship between test commands and response data, and constructs a retrieval engine, including: Configure the configuration information base and command mapping base in the target database; The configuration information base stores the simulated configuration information, and the command mapping base stores the mapping relationship between various test commands and response data. Based on the simulated configuration information of various servers and the correlation between various test commands and response data, a retrieval engine is built that supports matching corresponding response data and simulated configuration information by command type, operation object, and machine identifier.
[0082] In this embodiment, in response to the server test script executing a test on the server under test, the target configuration information of the server under test is obtained, and the target instructions for the execution of the server test script are obtained based on the target configuration information, including: Obtain the parameter list of the server under test, extract the target machine identifier from the parameter list, and determine the target configuration information of the server under test based on the target machine identifier; The system retrieves the target instructions for the interaction between the server test script and the server under test, obtains the original instruction string of the target instructions, parses and serializes the original instruction string and concatenates it to generate a structured instruction request object. The instruction request object includes a unique identifier, command line string, parsed command body, parameter list, target machine identifier and request timestamp. Encapsulate the instruction request object into a standard format instruction; The standard format instructions are constructed into an instruction syntax tree, in which the main command, subcommands, options and parameters are marked; Semantic analysis is performed based on the instruction syntax tree to identify the instruction type, operation type, target operation object, and parameters as the parsing results.
[0083] In this embodiment, semantic analysis based on the instruction syntax tree includes: Semantic extraction is performed on the instruction syntax tree to obtain the command intent. Based on the pre-built instruction semantic rule base, the command intent is classified into instructions and semantic tags are generated. The semantic similarity is calculated by comparing semantic labels with historical execution samples, and the confidence of instruction classification is dynamically adjusted. When the confidence level is lower than a preset threshold, a secondary judgment is made on the instruction classification based on the context vector, and the semantic label is adjusted according to the judgment result.
[0084] In this embodiment, the retrieval engine determines the response data corresponding to the target instruction in the target database based on the target instruction, including: The query conditions are constructed from the target database based on the parsing results using the retrieval engine. The query conditions include the instruction type, the target operation object, and the target machine identifier (machine ID). First-level filtering is performed based on instruction type and target operation object; A second level of filtering is performed using the target machine identifier to obtain the configuration or status of the target machine; For commands with parameters, the parameters are used as filtering conditions for a third level of screening to obtain response data that matches the target command.
[0085] In this embodiment, determining the response data corresponding to the target instruction in the target database using a retrieval engine based on the target instruction further includes: Set an exit code for querying the target database based on the parsed results using the search engine; If the response data matching the target instruction is successfully obtained, the exit code is returned as the first value; If no response data matching the target instruction is received, the exit code is returned as the second value; If the server being tested does not exist, the exit code is returned as the third value. In response to insufficient permissions, the exit code is returned as the fourth value.
[0086] In this embodiment, dynamically filled data in the response data corresponding to the target instruction is identified, dynamically updated data is generated based on the dynamically filled data, and the dynamically filled data is replaced with the dynamically updated data to obtain the target response data; The execution result of the server test script is determined based on the target response data, and a test verification report is generated based on the execution result, including: Return the response data corresponding to the target instruction as preset static data; Determine whether to add dynamic data content to the preset static data. If yes, execute the dynamic function to update the value of the dynamically populated data in the preset static data to form the dynamically updated data. Replace the dynamically populated data with the dynamically updated data to obtain the query result. Otherwise, use the preset static data as the query result and return the query result. The query results are assembled according to the server response format, and the assembled query results are converted into a text format that the server test script can recognize and used as the simulated response data of the target instruction. Analyze the simulated response data to determine the test results of executing the target instruction.
[0087] In this embodiment, the execution result of the server test script is determined based on the target response data, and a test verification report is generated based on the execution result, including: In response to the target response data obtained from the target instruction, determine whether the server test script has completed the test; If the server test script has completed the test, the execution result of the server test script is determined by comparing the target response data with the expected test result of the server test script, and a test report is generated based on the execution result; If the server test script has not completed the test, the next target instruction of the interaction between the server test script and the server under test is obtained. The next target instruction is standardized, encapsulated, and parsed to obtain the next parsing result. The target database is searched by the retrieval engine according to the next parsing result to obtain the next response data corresponding to the next target instruction. The next response data obtained by the query is dynamically updated, assembled, and formatted. The next response data is analyzed to determine the next test result, and then it is determined again whether the server test script has completed the test.
[0088] In this embodiment, generating a test report includes: The simulated response data of the target instruction is compared with the expected response template at the field level, and the response field matching rate and content similarity are calculated. When the matching rate or similarity falls below a preset threshold, a difference analysis report is output. The difference analysis report includes a description of the field differences, the direction of the deviation, the inferred cause, and suggestions for improvement. The results of each test are stored in a time-series format to generate a test execution chain; Generate a test report based on the test execution chain, including execution statistics, failure distribution, response latency curves, and configuration correlations; Test reports can be filtered, exported, and compared through an interactive interface.
[0089] The aforementioned script testing and verification device acquires the server's simulated configuration information and its response data files for processing test commands. After parsing, it establishes a mapping relationship between test commands and response data, stores the data, and builds a retrieval engine. This virtualizes and automates the script testing and verification process, enabling the verification of test scripts to completely decouple from the dependence on specific models and configurations of real physical servers. It allows for the completion of the entire test script verification process without the need for a real server. The elimination of the need to build and maintain complex physical test environments significantly shortens environment preparation time, reduces hardware costs, and allows for high-frequency, low-cost verification of test scripts. It also shields the device from interference from unstable factors such as network and hardware in the real environment, ensuring that test results are entirely determined by the script logic, facilitating rapid location and repair of script defects.
[0090] For a description of the features in the embodiment corresponding to the script testing and verification device, please refer to the relevant description of the embodiment corresponding to the script testing and verification method, which will not be repeated here.
[0091] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above-described script test verification method embodiments.
[0092] In one embodiment, the electronic device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, this electronic device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores script test and verification data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a script test and verification method.
[0093] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described script test and verification method embodiments at runtime: Obtain the response data file of the server processing test commands, as well as the server's simulation configuration information. Based on the simulation configuration information, parse the response data file to obtain the mapping relationship between test commands and response data. Store simulation configuration information and the mapping relationship between test commands and response data, and build a search engine; When the server test script executes a test on the server under test, the target configuration information of the server under test is obtained, and the target instructions for the server test script are obtained based on the target configuration information. The search engine determines the response data corresponding to the target instruction from the target database based on the target instruction. Identify dynamically populated data in the response data corresponding to the target instruction, generate dynamically updated data based on the dynamically populated data, and replace the dynamically populated data with the dynamically updated data to obtain the target response data; The execution result of the server test script is determined based on the target response data, and a test verification report is generated based on the execution result.
[0094] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.
[0095] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described script testing and verification method embodiments: Obtain the response data file of the server processing test commands, as well as the server's simulation configuration information. Based on the simulation configuration information, parse the response data file to obtain the mapping relationship between test commands and response data. Store simulation configuration information and the mapping relationship between test commands and response data, and build a search engine; When the server test script executes a test on the server under test, the target configuration information of the server under test is obtained, and the target instructions for the server test script are obtained based on the target configuration information. The search engine determines the response data corresponding to the target instruction from the target database based on the target instruction. Identify dynamically populated data in the response data corresponding to the target instruction, generate dynamically updated data based on the dynamically populated data, and replace the dynamically populated data with the dynamically updated data to obtain the target response data; The execution result of the server test script is determined based on the target response data, and a test verification report is generated based on the execution result.
[0096] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described script testing and verification method embodiments: Obtain the response data file of the server processing test commands, as well as the server's simulation configuration information. Based on the simulation configuration information, parse the response data file to obtain the mapping relationship between test commands and response data. Store simulation configuration information and the mapping relationship between test commands and response data, and build a search engine; When the server test script executes a test on the server under test, the target configuration information of the server under test is obtained, and the target instructions for the server test script are obtained based on the target configuration information. The search engine determines the response data corresponding to the target instruction from the target database based on the target instruction. Identify dynamically populated data in the response data corresponding to the target instruction, generate dynamically updated data based on the dynamically populated data, and replace the dynamically populated data with the dynamically updated data to obtain the target response data; The execution result of the server test script is determined based on the target response data, and a test verification report is generated based on the execution result.
[0097] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0098] The script testing and verification method and electronic device provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to help understand the method and core ideas of this application. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.
Claims
1. A script-based testing and verification method, characterized in that, include: Obtain the response data file of the server processing test instructions, as well as the simulation configuration information of the server; parse the response data file according to the simulation configuration information to obtain the mapping relationship between test instructions and response data; The simulation configuration information and the mapping relationship between test commands and response data are stored, and a search engine is built. When a server test script performs a test on the server under test, the target configuration information of the server under test is obtained, and the target instruction for the server test script is obtained based on the target configuration information. The search engine determines the response data corresponding to the target instruction in the target database based on the target instruction. Identify dynamically filled data in the response data corresponding to the target instruction, generate dynamically updated data based on the dynamically filled data, and replace the dynamically filled data with the dynamically updated data to obtain the target response data; The execution result of the server test script is determined based on the target response data, and a test verification report is generated based on the execution result.
2. The script testing and verification method according to claim 1, characterized in that, The process of obtaining the response data file of the server processing test commands and the server's simulation configuration information, and parsing the response data file based on the simulation configuration information to obtain the mapping relationship between test commands and response data, includes: The response data files of the server for processing various test commands are obtained through the data import interface, and the format type of the response data files is identified. Obtain the server configuration information table corresponding to the response data file, construct the configuration information table into simulated configuration information, and determine the test scenario constraints based on the simulated configuration information; In response to the fact that the format of the response data file is an application programming interface specification file, the response data file is parsed to extract the interface endpoints, request methods, request parameters, and the corresponding response body, status code, and response format as extracted information, and the extracted information is converted into instruction mapping entries; Since the format of the response data file is a specification file, the response data file is parsed to generate a simulation data template that conforms to the scenario constraints; Since the response data file is in tabular format, the response data file is parsed to map the rows or columns of the table to request-response pairs of the interface. The instruction mapping entries, the simulated data template, and the request-response pairs are subjected to data normalization processing to convert them into a normalized data model.
3. The script testing and verification method according to claim 1, characterized in that, The storage of the simulation configuration information and the mapping relationship between test commands and response data, and the construction of a retrieval engine, include: Configure the configuration information base and command mapping base in the target database; The simulation configuration information is stored in the configuration information library, and the mapping relationship between various test commands and response data is stored in the command mapping library; Based on the simulated configuration information of various servers and the correlation between various test commands and response data, a retrieval engine is built that supports matching corresponding response data and simulated configuration information by command type, operation object, and machine identifier.
4. The script testing and verification method according to claim 1, characterized in that, When a server test script executes a test on the server under test, the process of obtaining the target configuration information of the server under test and obtaining the target instructions for executing the server test script based on the target configuration information includes: Obtain the parameter list of the server under test, extract the target machine identifier from the parameter list, and determine the target configuration information of the server under test based on the target machine identifier; Obtain the target instruction for the interaction between the server test script and the server under test, obtain the original instruction string of the target instruction, parse and serialize the original instruction string and concatenate it to generate a structured instruction request object. The instruction request object includes a unique identifier, a command line string, a parsed command body, a parameter list, a target machine identifier and a request timestamp. The instruction request object is encapsulated into a standard format instruction; The standard format instructions are constructed into an instruction syntax tree, in which the main command, subcommand, options and parameters are marked; Semantic analysis is performed based on the instruction syntax tree to identify the instruction type, operation type, target operation object, and parameters as the parsing results.
5. The script testing and verification method according to claim 4, characterized in that, The semantic analysis based on the instruction syntax tree includes: Semantic extraction is performed on the instruction syntax tree to obtain the command intent, and the command intent is classified into instructions based on a preset instruction semantic rule base to generate semantic tags; The semantic similarity is calculated by comparing the semantic tags with historical execution samples, and the confidence level of instruction classification is dynamically adjusted. In response to a confidence level lower than a preset threshold, a secondary determination is made on the instruction classification based on the context vector, and the semantic label is adjusted according to the determination result.
6. The script testing and verification method according to claim 4, characterized in that, The step of determining the response data corresponding to the target instruction in the target database through the retrieval engine includes: The search engine constructs query conditions from the target database based on the parsing results. The query conditions include instruction type, target operation object, and target machine identifier. First-level filtering is performed based on the instruction type and the target operation object; A second level of filtering is performed using the target machine identifier to obtain the configuration or status of the target machine; For commands with parameters, the parameters are used as filtering conditions for a third level of screening to obtain response data that matches the target command.
7. The script testing and verification method according to claim 6, characterized in that, The step of determining the response data corresponding to the target instruction in the target database through the retrieval engine based on the target instruction also includes: To set an exit code for the query operation from the target database based on the parsed results using the retrieval engine; In response to successfully obtaining response data that matches the target instruction, the exit code is returned as a first value; If no response data matching the target instruction is obtained, the exit code is returned as a second value; If the server under test does not exist, the exit code is returned as a third value; In response to insufficient permissions, the exit code is returned as the fourth value.
8. The script testing and verification method according to claim 1, characterized in that, The process of identifying dynamically populated data in the response data corresponding to the target instruction, generating dynamically updated data based on the dynamically populated data, and replacing the dynamically populated data with the dynamically updated data to obtain the target response data includes: The response data corresponding to the target instruction is returned as preset static data; Determine whether to add dynamic data content to the preset static data. If yes, execute a dynamic function to update the value of the dynamically populated data in the preset static data to form dynamically updated data. Replace the dynamically populated data with the dynamically updated data to obtain the query result. Otherwise, use the preset static data as the query result and return the query result. The query results are assembled according to the server response format, and the assembled query results are converted into a text format recognizable by the server test script and used as the target response data for the target instruction.
9. The script testing and verification method according to claim 1, characterized in that, The step of determining the execution result of the server test script based on the target response data and generating a test verification report based on the execution result includes: In response to the target response data obtained from the target instruction, determine whether the server test script has completed the test; If the server test script has completed the test, the execution result of the server test script is determined by comparing the target response data with the expected test result of the server test script, and a test report is generated based on the execution result. If the server test script has not completed the test, the next target instruction of the interaction between the server test script and the server under test is obtained. The next target instruction is standardized, encapsulated, and parsed to obtain the next parsing result. The target database is queried by the search engine according to the next parsing result to obtain the next response data corresponding to the next target instruction. The next response data obtained by the query is dynamically updated, assembled, and formatted. The next response data is analyzed to determine the next test result, and the server test script is checked again to see if the test has been completed.
10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the script test verification method as described in any one of claims 1 to 9.