A code generation-based simulation software testing method

Abstract

The application discloses a simulation software testing method based on code generation, comprising the following steps: embedding coding and embedding marking are performed on a pre-learned code file; information extraction and feature embedding are performed on a test case; heterogeneous information coding and syntax rule learning are performed on the test case to generate code information; the next syntax rule is predicted according to the generated code information and natural language rules, and finally, a predicted code segment is obtained; the predicted code segment is combined into a predicted code file; and the predicted code file is tested. We use a unique AST encoder, so that the generated code file has higher availability and better technical maturity. Moreover, in terms of the diversity of the generated file, the learning of the specific syntax rule makes the generated simulation file more accurate, and this also makes our technology have a larger test space for the defects of the simulation software.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and in particular to a simulation software testing method based on code generation. Background Technology

[0002] In digital circuit design, register-transfer level (RTL) is an abstract model of synchronous digital circuits. The RTL abstract model is used in hardware description languages ​​such as Verilog and VHDL to create high-level descriptions of actual circuits. In modern digital design, design at the RTL level is the most typical workflow. Simulation software, as a tool for verifying RTL code, outputs waveforms that provide an intuitive description to help developers understand the overall model's performance. This allows developers to pinpoint potential defects in the abstract model at a lower cost. Therefore, ensuring the reliability and stability of simulation software is crucial.

[0003] Currently, the main testing method for simulation software is test case generation testing using fuzz testing, with VeriSmith being a representative example. It generates random Verilog code using an AST-based fuzzy generation method and applies it to the testing of synthesis tools or simulation software.

[0004] In the fields of chip design and high-end manufacturing, there are high requirements for the stability and correctness of development tools. Existing testing technologies have two shortcomings. First, testing tools cannot guarantee their own stability, and the Verilog code generated based on AST has certain unavailability. This leads to false positives in the defects found by the tools. Second, the Verilog code generated based on AST lacks diversity and implements relatively simple functions, which also limits the scope of the tools for defect exploration. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention discloses a simulation software testing method based on code generation, which specifically includes the following steps:

[0006] Embedded encoding and embedding tags are applied to the pre-learned code files;

[0007] Then, information extraction and feature embedding are performed on the test cases;

[0008] Heterogeneous information encoding and syntax rule learning are performed on test cases to generate code information;

[0009] Based on the generated code information and natural language rules, predict the next syntax rule and finally obtain the predicted code segment. Combine the obtained predicted code segments into a predicted code file.

[0010] Test the prediction code file.

[0011] Furthermore, during the embedding encoding process: the input statement is marked and its length is recorded. Each mark is split into characters. All marks and characters are represented as numerical vectors using an embedding method. The numerical vectors are fed back to the encoder and integrated with word embeddings through a sub-layer to obtain an embedding vector. The encoder is then used to encode the embedding vectors.

[0012] Furthermore, for statements with input tags, they are marked as embeddings through a maintained lookup table.

[0013] Furthermore, when extracting information from test cases: the positional information of words is encoded using positional embedding, the positional embedding of each word is calculated, non-linear features are learned through multi-head attention mechanism, the information of character embedding is merged with the non-linear features to obtain merged features, the Gating mechanism is applied to extract local features near each token, and the local features are embedded with the merged features.

[0014] Furthermore, when encoding heterogeneous information for test cases: rule information is encoded using rule IDs, rules in the rule sequence are represented as numerical vectors using a lookup table, syntax rules are used as atomic tags according to the lookup table, the representation of rules is enhanced using the encoding of rule information, the rule content is encoded as a vector and the rule embedding is obtained, and the position embedding is used to represent the position of the syntax rule;

[0015] When learning syntax rules for test cases: the sum of rule embedding and position embedding is input into the self-attention layer of the decoder to extract input features. The gating mechanism of the encoder is used to merge the content encoding rules into the part of extracting input features. The test case syntax tree is regarded as a graph. The adjacency matrix is ​​used to represent the directed graph. The test case syntax tree is multiplied by the adjacency matrix to obtain the feature information of all nodes.

[0016] Furthermore, when predicting the next grammar rule: construct a block stack in the decoder, mark the path from the root to the node to be expanded, represent the nodes in this path as numerical vectors, apply two fully connected layers to extract the node features in the path for prediction, and obtain the next grammar rule.

[0017] When obtaining the prediction code file: new code snippets are formed according to the guidance of the next syntax rule, and this process is repeated until the number of code snippets reaches a set threshold. The obtained prediction codes are then combined into a prediction code file.

[0018] By employing the above technical solutions, this invention provides a simulation software testing method based on code generation. This method can generate highly usable simulation files and, under existing testing conditions, can detect bugs in the simulation software, ensuring the stability of simulation verification tools in chip design and high-end manufacturing fields. Compared to existing technologies, we use a unique AST encoder, making the generated code files more usable and technologically mature. Furthermore, regarding the diversity of generated files, learning specific syntax rules makes our generated simulation files more accurate, which also gives our technology a larger testing scope for simulation software defects. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of the simulation software testing method based on code generation according to the present invention.

[0021] Figure 2 This is a structural block diagram of the encoder and decoder in an embodiment of the present invention. Detailed Implementation

[0022] To make the technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention:

[0023] like Figure 1 The simulation software testing method based on code generation shown includes the following steps:

[0024] S1: Embedding and tagging the pre-learned code files:

[0025] S11: Perform word embedding encoding processing;

[0026] S12: Mark the input statement and record its length.

[0027] S13: Split each token into characters

[0028] S14: Represent all tags and characters as numerical vectors using an embedding method.

[0029] S15: Feed the numerical vector back to the encoder and integrate the numerical vector with the word embedding through a sub-layer to obtain the embedding vector.

[0030] S16: Encode the embedding vector using an encoder.

[0031] S17: For statements with input tags, mark them as embedded using a maintained lookup table.

[0032] S2: After obtaining the embedding tags, we begin information extraction and feature embedding. The specific steps are as follows:

[0033] S21: Encode word position information using position embedding.

[0034] S22: Calculate the position embedding of each word

[0035] S23: Learning Nonlinear Features Through Multi-Head Attention Mechanism

[0036] S24: Merge the embedded character information with nonlinear features to obtain merged features.

[0037] S25: Apply the Gating mechanism to extract local features near each marker.

[0038] S26: Embed local features and merged features into a feature set.

[0039] S3: The program is generated by predicting the order of grammar rules, but considering only these rules lacks a concrete understanding of the program, making it insufficient to predict the next rule. Therefore, we need the AST encoder to consider heterogeneous information and predict rules. The specific steps are as follows:

[0040] S31: Encode rule information using rule ID.

[0041] S32: Represent the rules in the rule sequence as numerical vectors using a lookup table.

[0042] S33: Use syntax rules as atomic tags based on the lookup table.

[0043] S34: Enhancing the representation of rules using the encoding of rule information

[0044] S35: Encode the rule content into a vector and obtain the rule embedding.

[0045] S36: Use positional embedding to represent the position of syntax rules

[0046] S37: Input the sum of rule embeddings and position embeddings into the self-attention layer of the decoder to extract input features.

[0047] S38: Use the encoder's gating mechanism to incorporate content encoding rules into the input feature extraction part.

[0048] S39: Treat the test case syntax tree as a graph, and represent the directed graph using an adjacency matrix M. Multiply the test case syntax tree by the adjacency matrix to obtain the feature information of all nodes.

[0049] S4: After two encoding operations, predict the next syntax rule based on the generated code information and natural language rules, and finally obtain the predicted code fragment.

[0050] S41: Construct a block stack in the decoder, marking the path from the root to the node to be expanded.

[0051] S42: Represent the nodes in this path as numerical vectors.

[0052] S43: Two fully connected layers are applied to extract node features in the path for prediction to obtain the next grammar rule.

[0053] S44: Form new code snippets based on the next syntax rule, and continue to execute them until the number of code snippets reaches the set threshold.

[0054] S46: Combine the obtained prediction codes into a prediction code file.

[0055] S5: After obtaining the prediction code file, we will enter the testing phase. The specific steps are as follows:

[0056] S51: The saved prediction code file is in simulation test file format.

[0057] S52: Generate a pre-compiled file based on the predicted code file for testing.

[0058] S53: The compiled test file

[0059] S54: If a crash or exception occurs during compilation, save the file to the exception folder; otherwise, save it to the normal folder.

[0060] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A simulation software testing method based on code generation, characterized in that... include: Embedded encoding and embedding tags are applied to the pre-learned code files; Then, information extraction and feature embedding are performed on the test cases; Heterogeneous information encoding and syntax rule learning are performed on test cases to generate code information; Based on the generated code information and natural language rules, predict the next syntax rule and finally obtain the predicted code segment. Combine the obtained predicted code segments into a predicted code file. Test the prediction code file; When performing embedding encoding: mark the input statement and record the length of the statement, split each mark into characters, use embedding to represent all marks and characters as numerical vectors, feed the numerical vectors back to the encoder and integrate the numerical vectors with word embeddings through the sub-layer to obtain the embedding vector, and use the encoder to encode the embedding vector; When encoding heterogeneous information for test cases: use rule IDs to encode rule information, use lookup tables to represent rules in the rule sequence as numerical vectors, use lookup tables to treat syntax rules as atomic tags, use the encoding of rule information to enhance the representation of rules, encode rule content into vectors and obtain rule embeddings, and use position embeddings to represent the position of syntax rules; When learning syntax rules for test cases: the sum of rule embedding and position embedding is input into the self-attention layer of the decoder to extract input features. The gating mechanism of the encoder is used to merge the content encoding rules into the part of extracting input features. The test case syntax tree is regarded as a graph. The adjacency matrix is ​​used to represent the directed graph. The test case syntax tree is multiplied by the adjacency matrix to obtain the feature information of all nodes.

2. The simulation software testing method based on code generation according to claim 1, characterized in that: For statements with input tags, they are marked as embeddings using a maintained lookup table.

3. The simulation software testing method based on code generation according to claim 1, characterized in that: When extracting information from test cases: the positional information of words is encoded using positional embedding, the positional embedding of each word is calculated, non-linear features are learned through multi-head attention mechanism, the information of character embedding is merged with the non-linear features to obtain merged features, the Gating mechanism is applied to extract local features near each token, and the local features are embedded with the merged features.

4. The simulation software testing method based on code generation according to claim 1, characterized in that: When predicting the next grammar rule: construct a block stack in the decoder, mark the path from the root to the node to be expanded, represent the nodes in this path as numerical vectors, apply two fully connected layers to extract the node features in the path for prediction, and obtain the next grammar rule. When obtaining the prediction code file: new code snippets are formed according to the guidance of the next syntax rule, and this process is repeated until the number of code snippets reaches a set threshold. The obtained prediction codes are then combined into a prediction code file.

Images