A test script automatic generation method and device
By preprocessing and parsing domain knowledge documents and software requirements specifications, a semantic vector database is generated. By utilizing analysis proxies and coding proxies, the illusion problem in test script generation is solved, and efficient and accurate automatic test script generation is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING INST OF CONTROL ENG
- Filing Date
- 2026-04-24
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies generate test scripts that are prone to creating illusions, cannot effectively meet the testing needs of safety-critical systems, heavily rely on the experience of senior engineers, and are inefficient.
By acquiring domain knowledge documents and software requirements specifications, preprocessing and parsing are performed to generate a semantic vector database. Test scripts are then generated using analysis and coding proxies, including document segmentation, semantic vector generation, atomic requirement item splitting, similarity retrieval, and test plan generation.
It reduces the probability of logical errors and illusions in test scripts, improves the executability and accuracy of test scripts, and reduces the complexity and error risk of manual writing.
Smart Images
Figure CN122432041A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of automated software testing, knowledge engineering, and natural language processing, and particularly to a method and apparatus for automatically generating test scripts. Background Technology
[0002] Software testing is one of the core technical means to ensure the reliability and security of software systems, typically accounting for about 50% of the development cost throughout the complete software lifecycle. Especially in safety-critical fields such as aerospace, automotive electronics, and industrial control, software defects can directly lead to mission failure, equipment damage, and even personal injury or death, with a risk level far exceeding that of general information systems.
[0003] Unlike general-purpose software such as mobile applications and web services, software testing for safety-critical systems has distinct industry characteristics: the test objects are typically closed systems, where testers cannot directly access or modify their internal implementation logic. The testing process is primarily based on verification of external interface behavior and specification requirements. Therefore, software testing for this type of system mainly employs black-box testing and specification-driven testing.
[0004] In practical engineering, testing activities are typically conducted in a Hardware-in-the-Loop (HIL) environment. This involves interacting with the system under test (DUT) through a ground-based test platform or simulation equipment, treating the DUT as a "black box" that can only communicate via an interface. Test scripts send instructions to the DUT via the test platform and receive telemetry data or status information returned by it, thereby determining whether the system meets the predetermined software requirements.
[0005] In the aforementioned HIL testing scenario, generating test scripts requires simultaneously satisfying domain data knowledge, execution tool knowledge, and software semantic requirements. This heavily relies on the experience of senior engineers, becoming a major efficiency bottleneck in current safety-critical system testing processes. In recent years, large language models have been used to generate test scripts; however, these models often rely on probabilistic priors for reasonable guesses, easily leading to the hallucination problem. That is, the generated test script may appear reasonable, but in reality, it represents a non-existent or incompatible interface call. For example, the model might fabricate functions such as `satellite.launch()`, causing the generated test script to fail to execute. Summary of the Invention
[0006] This invention provides a method and apparatus for automatically generating test scripts, which can solve the problem that test scripts generated in related technologies are prone to producing illusions. The technical solution is as follows: On the one hand, a method for automatically generating test scripts is provided, the method comprising: Obtain domain knowledge documents and software requirements specifications; The domain knowledge document is preprocessed to split it into multiple document blocks; For each document block, a corresponding semantic vector is generated to obtain a semantic vector database; the semantic vector database includes multiple tuples, each tuple including a document block and its corresponding semantic vector; Perform requirements analysis on the software requirements specification document to break it down into multiple atomic requirements; Each atomic requirement item is converted into a query vector, and similarity retrieval is performed using the query vector in the semantic vector database to retrieve the multiple document blocks most relevant to the atomic requirement item. The analysis agent method is used to generate corresponding test plans for atomic requirement items and their most related document blocks, and the coding agent method is used to generate test scripts for the test plans, resulting in a set of executable test scripts.
[0007] On the other hand, a test script automatic generation device is provided, the device comprising: The acquisition unit is used to acquire domain knowledge documents and software requirements specifications. The segmentation unit is used to preprocess domain knowledge documents to divide them into multiple document blocks; The vector generation unit is used to generate a corresponding semantic vector for each document block, thereby obtaining a semantic vector database; the semantic vector database includes multiple tuples, each tuple including a document block and its corresponding semantic vector; The parsing unit is used to parse the software requirements specification document to break it down into multiple atomic requirements items; The retrieval unit is used to convert each atomic requirement item into a query vector and perform similarity retrieval in the semantic vector database using the query vector to retrieve the multiple document blocks most relevant to the atomic requirement item. The proxy generation unit is used to generate corresponding test plans for atomic requirement items and their most related document blocks using the analysis proxy method, and to generate test scripts for the test plans using the coding proxy method, resulting in a set of executable test scripts.
[0008] On the other hand, a computer device is provided, the computer device including a memory and a processor, the memory for storing computer programs, and the processor for executing the computer programs stored in the memory to implement the steps of the test script automatic generation method described above.
[0009] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of the above-described test script automatic generation method.
[0010] On the other hand, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the test script automatic generation method described above.
[0011] The technical solution provided by this invention can bring at least the following beneficial effects: This invention, in its embodiments, acquires domain knowledge documents, segments these documents into multiple document blocks, generates semantic vectors for each document block, and stores the document blocks and semantic vectors as tuples in a semantic vector database. Simultaneously, it acquires software requirements specifications and performs requirement parsing to extract multiple atomic requirement items. These atomic requirement items are then converted into query vectors for similarity retrieval in the semantic vector database, yielding document blocks matching the atomic requirement items. Next, a two-stage proxy approach is used: first, an analysis proxy is employed to generate test plans for the atomic requirement items and their document blocks; then, an encoding proxy is used to generate test scripts from the test plans. Therefore, this solution, through a two-stage mechanism of analysis and encoding proxies to generate test scripts, can reduce the probability of logical errors and illusions in the generated test scripts. Attached Figure Description
[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0013] Figure 1 This is a flowchart of a test script automatic generation method provided by an embodiment of the present invention; Figure 2 This is a flowchart of another test script automatic generation method provided by an embodiment of the present invention; Figure 3 This is a structural diagram of a test script automatic generation device provided in an embodiment of the present invention; Figure 4 This is a hardware architecture diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation
[0014] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0015] Please refer to Figure 1 The present invention provides a method for automatically generating test scripts, the method comprising: Step 100: Obtain domain knowledge documents and software requirements specification documents; Step 102: Preprocess the domain knowledge document to divide it into multiple document blocks; Step 104: For each document block, generate a corresponding semantic vector to obtain a semantic vector database; the semantic vector database includes multiple tuples, each tuple including a document block and its corresponding semantic vector; Step 106: Analyze the software requirements specification document to break it down into multiple atomic requirements. Step 108: Convert each atomic requirement item into a query vector, and use the query vector in the semantic vector database to perform similarity retrieval to obtain the multiple document blocks most relevant to the atomic requirement item; Step 110: Use the analysis proxy method to generate corresponding test plans for atomic requirement items and their most related document blocks, and use the coding proxy method to generate test scripts for the test plans to obtain a set of executable test scripts.
[0016] In this embodiment of the invention, a domain knowledge document is acquired, segmented into multiple document blocks, and semantic vectors are generated for each document block. The document blocks and semantic vectors are then stored as tuples in a semantic vector database. Simultaneously, a software requirements specification document is acquired and parsed to extract multiple atomic requirements. These atomic requirements are converted into query vectors, and a similarity search is performed in the semantic vector database to obtain document blocks matching the atomic requirements. Next, a two-stage proxy approach is used: first, an analysis proxy is used to generate test plans for the atomic requirements and their document blocks; then, an encoding proxy is used to generate test scripts from the test plans. Therefore, this solution, through a two-stage mechanism of analysis and encoding proxies to generate test scripts, can reduce the probability of logical errors and illusions in the generated test scripts.
[0017] The following description Figure 1 The execution method for each step is shown.
[0018] First, for step 100, obtain the domain knowledge document and the software requirements specification document.
[0019] Domain knowledge documentation refers to a collection of engineering documents used in a specific professional field (such as aerospace, automotive electronics, or industrial control) to describe technical specifications such as system interfaces, communication protocols, hardware behavior, and the usage of testing tools. These are hard rules that engineers must consult, understand, and strictly follow in their work, rather than general theories or tutorials. Domain knowledge documentation includes Interface Control Documents (ICDs), test platform user manuals, and API documentation, among others.
[0020] The Software Requirements Specification (SPS) document is the core document in software engineering that defines "what the system under test should do." In the testing scenario of safety-critical systems, it describes the functions, performance, interfaces, constraints, and acceptance criteria that the software under test should meet under specific conditions, and serves as the direct basis for test case design.
[0021] In this embodiment of the invention, domain knowledge documents and software requirements specifications can be processed through two independent pipelines: a knowledge document processing pipeline and a requirements-driven generation pipeline. Wherein: The knowledge document processing pipeline is used to transform multi-source heterogeneous domain technical documents into vector representations of high semantic density knowledge that can be uniformly retrieved and understood, and store them in a semantic vector database to solve the problems of low signal-to-noise ratio and complex structure of industrial documents.
[0022] The requirement-driven generation pipeline is used to gradually transform natural language requirements into logically complete and executable test scripts based on the retrieved domain context and through a multi-agent collaboration mechanism.
[0023] The two pipelines mentioned above exchange information through a semantic vector database, thereby achieving efficient alignment between requirements and domain knowledge.
[0024] Then, the following explanations are given regarding step 102, "Preprocessing the domain knowledge document to divide it into multiple document blocks" and step 104, "Generating a corresponding semantic vector for each document block to obtain a semantic vector database".
[0025] In this embodiment of the invention, steps 102 and 104 are processed by a knowledge document processing pipeline.
[0026] Because original domain knowledge documents are typically large in size and complex in structure, while the context window of a large language model is limited, directly inputting the entire document will result in an averaged long document vector representation, failing to highlight key details and leading to inaccurate search results during similarity retrieval. Therefore, domain knowledge documents can be preprocessed into multiple document blocks. A document block refers to several smaller text fragments cut from the original document according to certain rules. Each document block is an independent retrieval unit, which can be vectorized and stored in a semantic vector database. When a retrieval is needed, the most relevant document block will be retrieved and provided as context to the large language model.
[0027] In one embodiment of the present invention, domain knowledge documents can be preprocessed using the following method: Identify structured knowledge in domain knowledge documents that represents the same semantics, and segment this structured knowledge into independent document blocks; among them, any of the following structured knowledge is segmented into independent document blocks: complete protocol command table, complete parameter definition table, complete function definition, and complete API usage example.
[0028] This preprocessing method can automatically segment documents using a large model. Domain knowledge documents and prompts are input into the large model to obtain document blocks output by the model. The prompts include at least the following: a complete protocol command table, a complete parameter definition table, a complete function definition, and a complete API usage example are each segmented as an independent document block.
[0029] When segmenting domain knowledge documents, this large model can use a structured algorithm that can segment based on document object boundaries rather than using a fixed-length window.
[0030] In another embodiment of the present invention, the domain knowledge document can also be preprocessed in other ways: identify and extract the structure tree of the domain knowledge document; and divide the domain knowledge document into multiple document blocks according to the hierarchy of the structure tree.
[0031] By segmenting domain knowledge documents into semantically complete and appropriately sized document blocks, the retrieval system can accurately locate the most relevant paragraphs, table rows, or API descriptions.
[0032] After dividing the domain knowledge document into multiple document blocks, these blocks need to be vectorized to facilitate retrieval. One implementation method is to directly vectorize the document blocks, generating semantic vectors for each block.
[0033] In another implementation, generating a corresponding semantic vector for each document block may specifically include: Each document block is subjected to summary enhancement processing to obtain multiple semantic summaries corresponding to multiple document blocks; the semantic summaries are used to describe the functional semantics, key parameters and constraints of the corresponding document blocks. Each semantic summary is vectorized using an embedding model to generate a semantic vector.
[0034] In this context, the functional semantics in the semantic summary can be the functional meaning of the instruction, the key parameters can include the parameter name, type and value range, and the constraints can include time constraints, sequence constraints, etc.
[0035] It should be noted that this summary enhancement process can be implemented using a language model. The process can be represented as follows: in, This serves as a semantic summary for subsequent semantic retrieval. This is a document block that needs to be fully preserved to provide precise protocol parameters or API definitions.
[0036] In this embodiment of the invention, the document block is first subjected to summary enhancement processing and then vectorized, which can solve the problems of high noise and low density of key information in the document block content, thereby improving the recall effect of high-value, low-text-volume information in the retrieval stage.
[0037] Domain knowledge documents often contain redundant headings, formatting symbols, maintenance records, and repetitive descriptions. For example, an API manual might have 80% sample code, version history, and common error descriptions, while the truly crucial function signatures, parameter types, and return values only account for 20%. Directly vectorizing the original blocks dilutes the semantic weight of key information with this noise. To address this issue, in one embodiment of the present invention, the summary enhancement process may specifically include: Identify and remove non-critical information elements in document blocks; the non-critical information elements include: format characters, redundant headings, version history, maintenance logs, and repetitive descriptive text; Extract key entities and relationships from the remaining text. The key entities include function names, instruction codes, field names, parameter types, and return values. The relationships include parameter mappings and calling constraints. The extracted key entities and relationships are combined into a short text summary.
[0038] By extracting or generating a summary of the document block, formatting characters and irrelevant descriptions are removed, retaining only entities, relationships, and constraints directly relevant to the test. During vectorization, the semantic summary's vector representation focuses more on the core semantics, avoiding being swayed by noise. This reduces noise interference in the document, improving the signal-to-noise ratio.
[0039] Because each row in the ICD table of a domain knowledge document may only contain a few words, but its information value is extremely high, these characters account for a very small proportion in the original document block, and their semantic signal is very weak after vectorization, easily drowned out by long natural language. Furthermore, the model tends to return paragraphs with rich textual descriptions during similarity retrieval, rather than these fewer table rows. To address this issue, in one embodiment of the present invention, the summary enhancement process specifically includes: Parse table document blocks, identify row structure and column headers, and convert each row of data into a natural language sentence; In this process, the value of each cell is combined with the corresponding column header to form a statement whose column name is the value, and all statements are connected in sequence; for numeric or hexadecimal fields, a descriptive context is added during conversion; the descriptive context includes units, precision, or field meaning; Generate the natural language sentence as a semantic summary of the line.
[0040] Transforming table rows into natural language sentences in semantic summarization preserves all key parameters while increasing text length and semantic density, making it more distinguishable in the vector space and easier to match with the query vector to be retrieved.
[0041] In addition to the two summary enhancement methods mentioned above, other methods can be used, such as unifying all types of document blocks into short text summaries in natural language form, thereby eliminating format heterogeneity. The semantic summaries after the vector are all in the same semantic space, and during retrieval, the similarity between the ICD table row summary, the API function summary, and the query vector can be compared fairly, thus achieving joint retrieval across document types.
[0042] After obtaining multiple semantic summaries corresponding to multiple document blocks, an embedding model can be used for vectorization to generate semantic vectors corresponding to the multiple semantic summaries. At this point, multiple document blocks and their corresponding semantic vectors are obtained. These document blocks and their corresponding semantic vectors are stored as tuples in a semantic vector database. Thus, indexing can be performed based on the semantic vectors of the semantic summaries, thereby retrieving the corresponding document blocks.
[0043] Finally, explanations are provided for steps 106 ("Perform requirement analysis on the software requirement specification document to break down into multiple atomic requirement items"), 108 ("Convert each atomic requirement item into a query vector and perform similarity retrieval using the query vector in the semantic vector database to retrieve the multiple document blocks most relevant to the atomic requirement item"), and 110 ("Use the analysis agent to generate corresponding test plans for the atomic requirement items and their document blocks, and use the coding agent to generate test scripts for the test plans to obtain a set of executable test scripts").
[0044] In this embodiment of the invention, steps 106 and 110 are processed by a demand-driven pipeline.
[0045] In this embodiment of the invention, splitting the software requirements specification document into atomic requirement items means decomposing a natural language description that may contain multiple behaviors, conditions, or constraints into the smallest indivisible, single-concern, and independently verifiable requirement unit. In safety-critical testing, atomic requirement items can form precise one-to-one or many-to-one mappings with test cases, ICD parameters, and API calls, avoiding test omissions caused by ambiguous or combined requirements.
[0046] In one implementation, the software requirements specification document is parsed to break it down into multiple atomic requirements, including: Analyze the structural hierarchy of the software requirements specification document to obtain coarse-grained requirements items; Each coarse-grained requirement item is segmented into single sentences based on delimiters; Identify logical connectors in a single sentence; the logical connectors include coordinating connectors, alternative connectors, and conditional connectors; for coordinating connectors, split the single sentence into multiple independent requirements, each retaining its original condition; for alternative and conditional connectors, maintain them as a single atomic requirement; Separate the actions and constraints described in the single sentence. If the constraint is a performance or security indicator that can be independently verified, extract it as an atomic requirement. Otherwise, retain the constraint as an attribute of the action. The preconditions and post-actions described in the single sentence are broken down into multiple atomic requirements in chronological order; For requirements that include enumeration identifiers, each item in the enumeration is treated as an independent atomic requirement.
[0047] Furthermore, it can be verified whether each of the split requirements meets the atomicity criteria, which include single topic, independent verifiability, no logical connectors, and a single triggering condition.
[0048] Each decomposed atomic requirement item corresponds to an independently verifiable test objective. The resulting set of atomic requirements items is R.
[0049] In this embodiment of the invention, context retrieval can be performed for each atomic requirement item. During context retrieval, the atomic requirement item needs to be converted into a query vector, and then the query vector is used to perform a similarity search in a semantic vector database. After the similarity search, multiple document blocks most relevant to the atomic requirement item can be obtained. These most relevant document blocks constitute the domain context of the atomic requirement item, which includes corresponding protocol instructions, parameter definitions, and available test platform APIs and calling conventions.
[0050] The application scenario of the test script in this embodiment of the invention is a system-level black-box testing scenario, where the system under test is regarded as a closed entity whose internal logic cannot be accessed, and testing can only be performed through external interfaces (sending commands, receiving telemetry / output). In this case, directly generating an executable test script from atomic requirements and domain context will lead to logical gaps and illusion problems. In this embodiment of the invention, an analysis agent is introduced. First, an explicit test plan is generated using the analysis agent, so that the implicit domain constraints are fully and explicitly expressed before code generation. Then, the coding agent generates the test script. By generating the test script step by step through two-stage agents, the inherent cognitive burden of black-box testing can be explicitly decoupled into two independently verifiable layers, thereby systematically alleviating the logical gaps and illusion problems.
[0051] In this embodiment of the invention, the test plan includes at least the test preconditions, the test step sequence, the waiting statement or timing constraint, the verification conditions, and the judgment logic.
[0052] In one implementation, the domain, as described above and below, at least includes: an interface control document block and a test platform application programming interface document block; and the generation of corresponding test plans for atomic requirement items and their document blocks using an analysis proxy, including: Extract the external stimulus conditions, input message sequence, and expected output message or telemetry value from the atomic requirement item; Obtain from the interface control document block the instruction code, data field format corresponding to the input message sequence, and telemetry data packet identifier, field offset, value range and precision corresponding to the expected output message or telemetry value; Obtain the instruction sending function signature for sending the input message sequence and the telemetry reading function signature for reading the telemetry data packet from the test platform application programming interface document block; Test preconditions are generated based on the external stimulus conditions, the instruction sending function signature, and the instruction code. The test preconditions include the device power-on sequence and system initialization calls. The test step sequence is generated based on the input message sequence and the instruction sending function signature. The test step sequence includes at least one instruction sending operation and at least one telemetry reading operation arranged in time. Based on the time constraints in the atomic requirement item or the response time specification in the interface control document block, insert wait statements or timing constraint markers into the test step sequence. Based on the telemetry reading function signature, the field offset, the value range, and the precision, verification conditions and judgment logic are generated. The verification conditions include the expected telemetry value range, and the judgment logic includes comparison operators and assertion tolerance. The test preconditions, the test step sequence, the waiting statements or timing constraint markers, the verification conditions, and the decision logic are organized into a test plan.
[0053] Therefore, the generation of test scripts for the test plan using the coding proxy method includes: Map the test preconditions in the test plan to the initialization code blocks in the test script; Map each instruction sending operation in the sequence of the test steps to a real instruction sending function call, and map each telemetry reading operation to a real telemetry reading function call; Map the wait statement or timing constraint marker to a delay function or timeout mechanism in the test script; The verification conditions and decision logic are mapped to assertion statements, wherein the assertion tolerance is used as the allowable error parameter of the assertion statement; Outputs executable test case scripts.
[0054] During the coding proxy phase, atomic requirements, domain context, and test plans can be used as joint inputs to translate the logical steps in the test plan into a target scripting language (such as Python).
[0055] It should be noted that the generation process of test case scripts is subject to the following constraints: Constraint 1: Strictly adhere to the application programming interfaces and parameters defined in the domain context. The application programming interfaces include function names, parameter types, parameter order, and return values. The parameters include instruction codes, data field values, telemetry offsets, and value ranges. Constraint 2: Generating any function names, protocol fields, instruction codes, or telemetry identifiers that do not appear in the domain context is prohibited; Constraint 3: Ensure that the syntax of the generated test script is consistent with the calling specifications specified in the application programming interface document block of the test platform. The calling specifications include parameter passing methods, error handling mechanisms, and synchronous / asynchronous modes.
[0056] In Phase 1, an analytical proxy approach is used to integrate atomic requirements with the domain context, generating test plans that are independent of specific programming languages. This allows for the verification of logical integrity without executing code, and the API calls in the test plan are presented in the form of intent descriptions, avoiding the creation of fictitious function names.
[0057] In Phase Two, a coding proxy approach is used to deterministically translate the test plan into an executable script for a specific test platform. Since no creative decisions are made, the illusion problem can be completely eliminated.
[0058] Please refer to the embodiments of the present invention. Figure 2 By inputting domain knowledge documents and software requirements specifications, and then processing them through a knowledge document processing pipeline and a requirements-driven generation pipeline, a set of executable test scripts can be output. These test scripts can be executed on the target testing platform or in the HIL environment to verify the corresponding software requirements.
[0059] In this embodiment of the invention, the method of "summary-enhanced retrieval + requirement atomization + multi-agent collaborative generation" is used to realize the automated conversion from natural language requirements to executable test scripts, which significantly reduces the complexity and error risk of manually writing test scripts.
[0060] Please refer to Figure 3 This invention provides an automatic test script generation device, which includes: Acquisition unit 300 is used to acquire domain knowledge documents and software requirements specification documents; The segmentation unit 302 is used to preprocess the domain knowledge document to segment it into multiple document blocks; The vector generation unit 304 is used to generate a corresponding semantic vector for each document block to obtain a semantic vector database; the semantic vector database includes multiple tuples, each tuple including a document block and its corresponding semantic vector; Parsing unit 306 is used to parse the software requirements specification document to break it down into multiple atomic requirements items; The retrieval unit 308 is used to convert each atomic requirement item into a query vector and perform similarity retrieval in the semantic vector database using the query vector to retrieve multiple document blocks most relevant to the atomic requirement item. The proxy generation unit 310 is used to generate corresponding test plans for atomic requirement items and their most related document blocks using the analysis proxy method, and to generate test scripts for the test plans using the coding proxy method, thereby obtaining a set of executable test scripts.
[0061] The segmentation unit is specifically used to identify structured knowledge in a domain knowledge document that represents the same semantics, and to segment the structured knowledge that represents the same semantics into independent document blocks; wherein any of the following structured knowledge is segmented into an independent document block: a complete protocol command table, a complete parameter definition table, a complete function definition, or a complete API usage example.
[0062] In one embodiment of the present invention, the vector generation unit is specifically used to: perform summary enhancement processing on each document block to obtain multiple semantic summaries corresponding to multiple document blocks; the semantic summaries are used to describe the functional semantics, key parameters and constraints of the corresponding document blocks; and perform vectorization processing on each semantic summary using an embedding model to generate a semantic vector.
[0063] In one embodiment of the present invention, the plurality of document blocks most relevant to the atomic requirement constitute the domain context of the atomic requirement; the domain context includes at least an interface control document block and a test platform application programming interface document block; When the proxy generation unit executes the method of generating corresponding test plans for atomic requirement items and their most related document blocks using the analysis proxy, it specifically includes: Extract the external stimulus conditions, input message sequence, and expected output message or telemetry value from the atomic requirement item; Obtain from the interface control document block the instruction code, data field format corresponding to the input message sequence, and telemetry data packet identifier, field offset, value range and precision corresponding to the expected output message or telemetry value; Obtain the instruction sending function signature for sending the input message sequence and the telemetry reading function signature for reading the telemetry data packet from the test platform application programming interface document block; Test preconditions are generated based on the external stimulus conditions, the instruction sending function signature, and the instruction code. The test preconditions include the device power-on sequence and system initialization calls. The test step sequence is generated based on the input message sequence and the instruction sending function signature. The test step sequence includes at least one instruction sending operation and at least one telemetry reading operation arranged in time. Based on the time constraints in the atomic requirement item or the response time specification in the interface control document block, insert wait statements or timing constraint markers into the test step sequence. Based on the telemetry reading function signature, the field offset, the value range, and the precision, verification conditions and judgment logic are generated. The verification conditions include the expected telemetry value range, and the judgment logic includes comparison operators and assertion tolerance. The test preconditions, the test step sequence, the waiting statements or timing constraint markers, the verification conditions, and the decision logic are organized into a test plan.
[0064] In one embodiment of the present invention, when the proxy generation unit generates test scripts for the test plan using an coded proxy method, it specifically includes: Map the test preconditions in the test plan to the initialization code blocks in the test script; Map each instruction sending operation in the sequence of the test steps to a real instruction sending function call, and map each telemetry reading operation to a real telemetry reading function call; Map the wait statement or timing constraint marker to a delay function or timeout mechanism in the test script; The verification conditions and decision logic are mapped to assertion statements, wherein the assertion tolerance is used as the allowable error parameter of the assertion statement; Outputs executable test case scripts.
[0065] It should be noted that the test script automatic generation device provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the test script automatic generation device and the test script automatic generation method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.
[0066] Embodiments of this application also provide a computer device, please refer to... Figure 4 The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by the processor to implement the test script automatic generation method provided in the above-described method embodiments.
[0067] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the test script automatic generation method provided in the above-described method embodiments.
[0068] Embodiments of this application also provide a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium and executes the computer program, causing the computer device to perform any of the test script automatic generation methods described in the above embodiments.
[0069] For ease of description, the above systems or devices are described separately as various modules or units based on their functions. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components.
[0070] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.
[0071] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.
[0072] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A method for automatically generating test scripts, characterized in that, The method includes: Obtain domain knowledge documents and software requirements specifications; The domain knowledge document is preprocessed to split it into multiple document blocks; For each document block, a corresponding semantic vector is generated to obtain a semantic vector database; the semantic vector database includes multiple tuples, each tuple including a document block and its corresponding semantic vector; Perform requirements analysis on the software requirements specification document to break it down into multiple atomic requirements; Each atomic requirement item is converted into a query vector, and similarity retrieval is performed using the query vector in the semantic vector database to retrieve the multiple document blocks most relevant to the atomic requirement item. The analysis agent method is used to generate corresponding test plans for atomic requirement items and their most related document blocks, and the coding agent method is used to generate test scripts for the test plans, resulting in a set of executable test scripts.
2. The method according to claim 1, characterized in that, The preprocessing of the domain knowledge document to segment it into multiple document blocks includes: Identify structured knowledge in domain knowledge documents that represents the same semantics, and segment this structured knowledge into independent document blocks; among them, any of the following structured knowledge is segmented into independent document blocks: complete protocol command table, complete parameter definition table, complete function definition, and complete API usage example.
3. The method according to claim 1, characterized in that, The step of generating a corresponding semantic vector for each document block includes: Each document block is subjected to summary enhancement processing to obtain multiple semantic summaries corresponding to multiple document blocks; the semantic summaries are used to describe the functional semantics, key parameters and constraints of the corresponding document blocks. Each semantic summary is vectorized using an embedding model to generate a semantic vector.
4. The method according to claim 1, characterized in that, The multiple document blocks most relevant to the atomic requirement constitute the domain context of the atomic requirement; the domain context includes at least the interface control document block and the test platform application programming interface document block; The method of generating corresponding test plans for atomic requirement items and their most relevant document blocks using the analysis agent includes: Extract the external stimulus conditions, input message sequence, and expected output message or telemetry value from the atomic requirement item; Obtain from the interface control document block the instruction code, data field format corresponding to the input message sequence, and telemetry data packet identifier, field offset, value range and precision corresponding to the expected output message or telemetry value; Obtain the instruction sending function signature for sending the input message sequence and the telemetry reading function signature for reading the telemetry data packet from the test platform application programming interface document block; Test preconditions are generated based on the external stimulus conditions, the instruction sending function signature, and the instruction code. The test preconditions include the device power-on sequence and system initialization calls. The test step sequence is generated based on the input message sequence and the instruction sending function signature. The test step sequence includes at least one instruction sending operation and at least one telemetry reading operation arranged in time. Based on the time constraints in the atomic requirement item or the response time specification in the interface control document block, insert wait statements or timing constraint markers into the test step sequence. Based on the telemetry reading function signature, the field offset, the value range, and the precision, verification conditions and judgment logic are generated. The verification conditions include the expected telemetry value range, and the judgment logic includes comparison operators and assertion tolerance. The test preconditions, the test step sequence, the waiting statements or timing constraint markers, the verification conditions, and the decision logic are organized into a test plan.
5. The method according to claim 4, characterized in that, The method of generating test scripts for the test plan using an encoding proxy includes: Map the test preconditions in the test plan to the initialization code blocks in the test script; Map each instruction sending operation in the sequence of the test steps to a real instruction sending function call, and map each telemetry reading operation to a real telemetry reading function call; Map the wait statement or timing constraint marker to a delay function or timeout mechanism in the test script; The verification conditions and decision logic are mapped to assertion statements, wherein the assertion tolerance is used as the allowable error parameter of the assertion statement; Outputs executable test case scripts.
6. A test script automatic generation device, characterized in that, The device includes: The acquisition unit is used to acquire domain knowledge documents and software requirements specifications. The segmentation unit is used to preprocess domain knowledge documents to divide them into multiple document blocks; The vector generation unit is used to generate a corresponding semantic vector for each document block, thereby obtaining a semantic vector database; the semantic vector database includes multiple tuples, each tuple including a document block and its corresponding semantic vector; The parsing unit is used to parse the software requirements specification document to break it down into multiple atomic requirements items; The retrieval unit is used to convert each atomic requirement item into a query vector and perform similarity retrieval in the semantic vector database using the query vector to retrieve the multiple document blocks most relevant to the atomic requirement item. The proxy generation unit is used to generate corresponding test plans for atomic requirement items and their most related document blocks using the analysis proxy method, and to generate test scripts for the test plans using the coding proxy method, resulting in a set of executable test scripts.
7. The apparatus according to claim 6, characterized in that, The segmentation unit is specifically used to identify structured knowledge in a domain knowledge document that represents the same semantics, and to segment the structured knowledge that represents the same semantics into independent document blocks; wherein any of the following structured knowledge is segmented into an independent document block: a complete protocol command table, a complete parameter definition table, a complete function definition, or a complete API usage example.
8. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the steps of the method according to any one of claims 1-5.
9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of claims 1-5.
10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-5.