Test data generation method and device, medium and equipment

Abstract

The invention discloses a test data generation method and device, a medium and equipment, and relates to the technical field of tests.The method comprises the steps that a to-be-tested program is obtained; based on the to-be-tested program, performing code modeling to obtain program model information; performing path traversal based on the program model information to obtain at least one test path corresponding to the to-be-tested program, and determining path information of each test path; each test path is a program execution path of the program to be tested; converting the path information of each test path into prompt information corresponding to each test path; and inputting the prompt information corresponding to each test path into the generative language model, and performing text generation processing to obtain test data. According to the method and the device, the automatically constructed prompt information is input into the generative language model, the test data can be more efficiently generated, meanwhile, the coverage rate of the test is ensured through code modeling and path traversal in the process of constructing the prompt information, and the problem of sensitive data leakage is also avoided.

Description

Technical Field

[0001] The present application relates to the field of testing technology, and in particular to a method, device, medium and equipment for generating test data. Background Art

[0002] Artificial Intelligence (AI) is a comprehensive technology in computer science. By studying the design principles and implementation methods of various intelligent machines, it enables machines to have the functions of perception, reasoning and decision-making. Artificial intelligence technology is an interdisciplinary subject that covers a wide range of fields, such as natural language processing, machine learning, deep learning and other major directions. With the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0003] In the field of software testing, after the release of large-scale generative language models represented by ChatGPT (Chat Generative Pre-trained Transformer), large-scale generative language models can be used to generate unit test cases based on prompt words. However, directly inputting the program to be tested as a prompt word into the large model has the risk of code leakage, and the test case coverage rate generated by the large model is low. Summary of the invention

[0004] In order to improve the efficiency and effectiveness of test data generation, the present application provides a test data generation method, device, medium and equipment. The technical solution is as follows:

[0005] In a first aspect, the present application provides a method for generating test data, the method comprising:

[0006] Get the program to be tested;

[0007] Based on the program to be tested, code modeling is performed to obtain program model information; the program model information is described in a domain-specific language;

[0008] Based on the program model information, path traversal is performed to obtain at least one test path corresponding to the program to be tested, and path information of each test path is determined; each test path is a program execution path of the program to be tested;

[0009] Converting the path information of each test path into prompt information corresponding to each test path, wherein the prompt information is described in a natural language;

[0010] The prompt information corresponding to each test path is input into the generative language model, and text generation processing is performed to obtain test data.

[0011] Optionally, performing code modeling based on the program to be tested to obtain program model information includes:

[0012] Based on a static code analysis tool, static analysis is performed on the program to be tested to obtain program feature information;

[0013] Based on the grammatical rules and preset data structure of the domain-specific language, the program feature information is parsed and assembled to obtain the program model information.

[0014] Optionally, the static analysis of the program to be tested based on a static code analysis tool to obtain program feature information includes:

[0015] Based on a static code analysis tool, semantic and grammatical analysis is performed on the program to be tested to obtain an identifier attribute table; the identifier attribute table represents attribute information of multiple identifiers;

[0016] Performing rule analysis and state collection on the identifier attribute table to obtain identifier state information, wherein the identifier state information represents state information of the multiple identifiers during the rule analysis process of the identifier attribute table;

[0017] When the quality inspection result of the identifier status information meets a preset condition, the identifier status information is used as the program feature information.

[0018] Optionally, performing path traversal based on the program model information to obtain at least one test path corresponding to the program to be tested, and determining path information of each test path includes:

[0019] Determine node information of each first node in at least one first node from the program model information, wherein the node information includes scope information; each first node corresponds to a data structure element in the program to be tested;

[0020] Based on the scope information in the node information of each first node, the program to be tested is divided into at least one basic block, each basic block is a group of continuous program instructions, and each basic block includes an entry instruction and an exit instruction;

[0021] Generate a directed graph according to the entry instruction in each basic block and the exit instruction in each basic block, wherein the directed graph represents the control flow of the program to be tested, and the directed graph includes at least one second node and at least one directed edge between the at least one second node; wherein each second node corresponds to a basic block;

[0022] Based on the directed graph, traverse the program execution path to obtain the at least one test path;

[0023] According to at least one basic block associated with each test path, path information of each test path is determined, wherein the path information includes path start condition information and path end condition information.

[0024] Optionally, converting the path information of each test path into prompt information corresponding to each test path includes:

[0025] Determine test data condition information of each test path according to the path start condition information in the path information of each test path; the test data condition information is described in a structured programming language;

[0026] Based on the grammar conversion rules between the program language and the natural language, the test data generation condition information is converted into the prompt information corresponding to each test path.

[0027] Optionally, the method further includes:

[0028] Obtain a historical test program, where the historical test program is a historical version of the program to be tested;

[0029] Determine an incremental program to be tested of the program to be tested relative to the historical test program;

[0030] At least one incremental test path associated with the incremental program to be tested is determined to generate corresponding test data based on the at least one incremental test path; each incremental test path is any test path of the at least one test path.

[0031] Optionally, the method further includes:

[0032] Performing rule checking on the test data corresponding to each test path to obtain rule checking results corresponding to the test data;

[0033] In the case where the rule checking result indicates that the test data does not conform to the logic rule indicated by the corresponding test path, generating context information corresponding to the test data; the context information indicates that the test data is excluded;

[0034] According to the context information, the corresponding prompt information is updated.

[0035] Optionally, the method further includes:

[0036] Get the preset unit test case template;

[0037] Fill the test data corresponding to each test path into the unit test case template to obtain at least one unit test case;

[0038] The at least one unit test case and the preset annotation instructions are input into the generative language model, and text generation processing is performed to obtain annotation information corresponding to each unit test case.

[0039] Optionally, the method further includes:

[0040] In the simulated test environment, executing the at least one unit test case to obtain an execution result corresponding to each unit test case;

[0041] According to the execution result corresponding to each unit test case, evaluation information of the at least one unit test case is determined, the evaluation information including indicator information corresponding to a plurality of evaluation indicators, and the plurality of evaluation indicators including test coverage and test pass rate.

[0042] In a second aspect, the present application provides a device for generating test data, the device comprising:

[0043] An acquisition module is used to acquire the program to be tested;

[0044] A structural analysis module, used to perform code modeling based on the program to be tested to obtain program model information; the program model information is described in a domain-specific language;

[0045] A test path determination module, used to perform path traversal based on the program model information, obtain at least one test path corresponding to the program to be tested, and determine the path information of each test path; each test path is a program execution path of the program to be tested;

[0046] A prompt information generating module, used for converting the path information of each test path into prompt information corresponding to each test path, wherein the prompt information is described in a natural language;

[0047] The test data generation module is used to input the prompt information corresponding to each test path into the generative language model, perform text generation processing, and obtain test data.

[0048] In a third aspect, the present application provides a computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement a method for generating test data as described in the first aspect.

[0049] In a fourth aspect, the present application provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement a test data generation method as described in the first aspect.

[0050] In a fifth aspect, the present application provides a computer program product, wherein the computer program product includes computer instructions, and when the computer instructions are executed by a processor, a method for generating test data as described in the first aspect is implemented.

[0051] The test data generation method, device, medium and equipment provided in this application have the following technical effects:

[0052] The solution provided by the present application is based on the program to be tested, and code modeling is performed to obtain program model information described in a domain-specific language; based on the program model information, at least one corresponding test path can be generated and the path information of each test path can be determined, and the test path is the program execution path of the program to be tested; the path information of each test path is converted into corresponding prompt information described in natural language, and then the prompt information corresponding to each test path is input into the generative language model to perform text generation processing, so that the test data required by the program to be tested can be obtained. In the solution provided by the present application, by code modeling, path traversal and language conversion of the program to be tested, the prompt information corresponding to the program to be tested can be automatically generated to meet the input requirements of the generative language model in the task of generating test data, and improve the generation efficiency of test data; in the process of code modeling and path traversal, the desensitization of business data and the protection of code privacy are also realized, which reduces the security risk of using the generative language model, and for the program to be tested containing large functions and complex reference relationships, the path coverage can also be improved through code modeling and path traversal to ensure the comprehensiveness of the test.

[0053] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 It is a schematic diagram of an implementation environment of a method for generating test data provided in an embodiment of the present application;

[0056] Figure 2 It is a flowchart of a method for generating test data provided in an embodiment of the present application;

[0057] Figure 3It is a flowchart of a code modeling provided by an embodiment of the present application;

[0058] Figure 4 It is a schematic diagram of a static analysis process provided by an embodiment of the present application;

[0059] Figure 5 It is a schematic diagram of the architecture of an improved static analysis tool provided in an embodiment of the present application;

[0060] Figure 6 It is a schematic diagram of a path traversal process provided by an embodiment of the present application;

[0061] Figure 7 It is a flowchart of generating test data for an incremental program to be tested provided by an embodiment of the present application;

[0062] Figure 8 It is a schematic diagram of a process for determining path information of an incremental test path provided by an embodiment of the present application;

[0063] Fig. 9 This is a flow chart of a test data inspection method provided by an embodiment of the present application;

[0064] Fig.10 It is a schematic diagram of a flow chart of generating and evaluating a unit test case provided in an embodiment of the present application;

[0065] Fig.11 It is a structural diagram of a unit test case template provided in an embodiment of the present application;

[0066] Fig.12 It is a programming language schematic diagram of a unit test case template provided in an embodiment of the present application;

[0067] Fig.13 It is a processing flow diagram of a unit test case evaluation system provided by an embodiment of the present application;

[0068] Fig.14 It is an interactive flow chart for generating unit test cases provided by an embodiment of the present application;

[0069] Fig.15 It is a flowchart of generating unit test cases from a program to be tested provided by an embodiment of the present application;

[0070] Fig.16 is a schematic diagram of a test data generating device provided in an embodiment of the present application;

[0071] Fig.17 It is a schematic diagram of the hardware structure of a device for implementing a method for generating test data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] In order to improve the generation efficiency and effectiveness of test data, the embodiments of the present application provide methods, devices, media and equipment for generating test data. Below, in conjunction with the drawings in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of this application. Examples of the embodiments are shown in the drawings, where the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions.

[0073] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0074] In order to facilitate understanding of the technical solutions described in the embodiments of the present application and the technical effects produced therefrom, the embodiments of the present application explain the relevant professional terms involved:

[0075] The solution provided in the embodiments of the present application involves technologies such as machine learning (ML) and deep learning (DL) of artificial intelligence (AI).

[0076] Artificial intelligence: It is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level technology and software-level technology. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics.

[0077] Machine learning is a multi-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.

[0078] Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0079] Deep learning is a major research direction in the field of machine learning. It is introduced into machine learning to make it closer to its original goal - artificial intelligence. The concept of deep learning originated from the study of artificial neural networks. Multilayer perceptrons with multiple hidden layers are a type of deep learning structure. Deep learning combines low-level features to form more abstract high-level representation attribute categories or features to discover the distributed feature representation of data. Deep learning is the inherent laws and representation levels of learning sample data. The information obtained in the learning process is of great help in the interpretation of data such as text, images and sounds. Its ultimate goal is to enable machines to have analytical learning capabilities like humans and to be able to recognize data such as text, images and sounds. Deep learning has achieved many results in search technology, data mining, machine learning, machine translation, natural language processing, multimedia learning, speech, recommendation and personalization technology, as well as other related fields. Deep learning enables machines to imitate human activities such as seeing, hearing and thinking, solves many complex pattern recognition problems, and makes great progress in artificial intelligence related technologies.

[0080] GPT: Generative Pre-Trained Transformer, a generative pre-trained Transformer model, is a deep learning model for text generation based on the Internet and trained with available data.

[0081] ChatGPT: Chat Generative Pre-trained Transformer, a language generation model based on Transformer trained with big data in multiple languages. The main function of this model is to have real-time conversations with people. It is a natural language processing tool driven by artificial intelligence technology. It can communicate by understanding and learning human language, and can also interact according to the context of the chat.

[0082] Prompt: Prompt refers to the questions or topics that users enter when interacting with large-scale generative language models such as ChatGPT. These prompts are used as input so that the model can generate answers or responses.

[0083] DSL: Domain-Specific Language, a programming language specifically designed to solve problems in a specific domain. Compared with general programming languages ​​(such as C++, Java, etc.), DSL is more focused on a specific domain, so it is easier to use and understand. DSL usually has concise and clear syntax and semantics specifically for a specific domain, and can provide more efficient and flexible solutions in that domain. Examples of DSL include HTML (HyperText Markup Language), CSS (Cascading Style Sheets), SQL (Structured Query Language), etc.

[0084] IDE: Integrated Development Environment, is an application used to provide a program development environment. It generally includes tools such as code editors, compilers, debuggers, and graphical user interfaces. It integrates an integrated development software service suite including code writing, analysis, compilation, and debugging functions.

[0085] Cppcheck: is a static code checking tool that supports C and C++ codes. As a supplementary check of the compiler, Cppcheck performs strict logical checks on the source code of the product.

[0086] Unit testing: unit testing refers to checking and verifying the smallest testable unit in the software. Generally speaking, the meaning of the unit in unit testing should be determined according to the actual situation. For example, in C language, a unit refers to a function, in Java, a unit refers to a class, and in graphical software, it can refer to a window or a menu. In general, a unit is the smallest functional module to be tested that is artificially defined.

[0087] The solution provided in the embodiment of the present application can be deployed in the cloud, which also involves cloud technology, etc.

[0088] Cloud technology: refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to realize data computing, storage, processing and sharing. It can also be understood as a general term for network technology, information technology, integration technology, management platform technology and application technology based on cloud computing business model applications, which can form a resource pool and be used on demand, flexible and convenient. The back-end services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, each item may have its own identification mark in the future, and all need to be transmitted to the back-end system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support, so cloud technology needs to be supported by cloud computing. Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, expanded at any time, and paid for by use. As a cloud computing basic capability provider, a cloud computing resource pool platform will be established, referred to as a cloud platform, generally referred to as Infrastructure as a Service (IaaS), and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (which can be virtualized machines, including operating systems), storage devices, and network devices.

[0089] It is understandable that in the specific implementation of this application, related data such as user information and device information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0090] See also Figure 1 , which is a schematic diagram of an implementation environment of a method for generating test data provided in an embodiment of the present application, such as Figure 1 As shown, the implementation environment may include at least a client 01 and a server 02 .

[0091] Specifically, the client 01 may include devices such as smart phones, desktop computers, tablet computers, laptops, vehicle-mounted terminals, digital assistants, smart wearable devices, and voice interaction devices. It may also include software running in the device, such as web pages provided to users by some service providers, or applications provided to users by these service providers. Specifically, the client 01 may run an integrated development environment for program developers to write programs to be tested, generate trigger operations in response to test data generated by program developers for the programs to be tested, call plug-ins in the code editor, and perform structural analysis of the programs to be tested, path traversal, and conversion of path information into prompt information described in natural language for processing. The client 01 sends the prompt information corresponding to each test path to the server 02.

[0092] Specifically, the server 02 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server 02 may include a network communication unit, a processor, and a memory, etc. The terminal and the server may be directly or indirectly connected by wired or wireless communication, and this application is not limited here. Specifically, the server 02 can be used to run a generative language model, which can learn the grammar and semantics of natural language, and respond and reply according to the input statement. The server 02 receives the prompt information corresponding to each test path sent by the client 01, and inputs each prompt information into the generative language model, performs text generation processing respectively, and obtains test data for the program to be tested. The server 02 sends the generated test data to the client 01, and the program developer can use the test data to construct a unit test case to perform unit testing on the program to be tested.

[0093] The embodiment of the present application can also be implemented in combination with cloud technology. Specifically, the server 02 and the database are located in the cloud, and the server 02 can be a physical machine or a virtualized machine. The above is an exemplary embodiment provided by the present application. The method for generating test data provided by the embodiment of the present application can also have other implementation environments, such as the server 02 being a cluster server, etc., and the present application does not limit this.

[0094] See also Figure 2, which is a flowchart of a method for generating test data provided in an embodiment of the present application. The present application provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment). Figure 2 As shown, a method for generating test data provided in an embodiment of the present application may include the following steps:

[0095] S210: Obtain the program to be tested.

[0096] In the embodiment of the present application, the program to be tested is a software code that has not been tested yet, and can be written based on any programming language.

[0097] Generally, software testing can be performed on the program to be tested by constructing test data and generating test cases. After the release of large-scale generative language models represented by ChatGPT (Chat Generative Pre-trained Transformer), the large-scale generative language model can be used to automatically generate unit test cases based on prompt words. However, directly inputting the program to be tested as a prompt word into a large-scale generative language model will have the risk of leaking business data and code privacy, and it is also unrealistic in terms of time cost. At the same time, large-scale generative language models are trained to better understand natural language, and the lack of understanding of programming language leads to unstable quality of generated test cases, low test coverage, and failure to effectively improve test efficiency.

[0098] Therefore, in response to the above-mentioned problems, the embodiments of the present application propose a method for automatically obtaining prompt information described in a natural language from a program to be tested described in a programming language, thereby improving the efficiency of generating test data through a large-scale generative language model, while avoiding the leakage of business data and code privacy, and improving the quality and coverage of test data.

[0099] S220: Based on the program to be tested, code modeling is performed to obtain program model information; the program model information is described in a domain-specific language.

[0100] In the embodiment of the present application, code modeling is performed based on the program to be tested, mainly to build a corresponding data structure model or control behavior model according to the code structure of the program to be tested, and the obtained program model information may include class information, function information, call information, variable information, execution path information, etc. Code modeling can also hide specific and sensitive business data in the program to be tested to protect code privacy.

[0101] Exemplarily, the domain-specific language may be HTML (HyperText Markup Language), CSS (Cascading Style Sheets), SQL (Structured Query Language), etc. The specific domain-specific language may be selected according to the programming language of the program to be tested or the business type corresponding to the program to be tested.

[0102] In one embodiment of the present application, Figure 3 As shown, code modeling of the program to be tested may include the following steps:

[0103] S221: Based on the static code analysis tool, static analysis is performed on the program to be tested to obtain program feature information.

[0104] Among them, static analysis refers to the analysis of the test program itself without running the test program. It can be the feature extraction processing of various program elements. The obtained program feature information represents the attribute information and status information of various program elements, or various identifiers. Identifiers include but are not limited to constant names, function names, procedure names, domain names, variable names, etc. The attribute information and status information can represent the scope of the identifier, logical expression, reference relationship, dependency relationship, calling relationship, etc. between identifiers.

[0105] Exemplarily, the static code analysis tool may be Cppcheck or the like. The format suffix of the original file generated by Cppcheck is .dump, and its structure may be as follows:

[0106]

[0107] Among them, the explanation of each information is as follows: rawtoken: unprocessed token (identifier); suppression: prohibited item; standards: language standard; directivelist: reference table; tokenlist: identifier attribute table; scopes: interval; variables: variable table in function; valueflow: data flow; the identifier attribute table tokenlist in the original file can be used as program feature information.

[0108] S222: Based on the grammatical rules and preset data structure of the domain-specific language, the program feature information is parsed and assembled to obtain program model information.

[0109] The preset data structure is a data structure customized based on a domain-specific language to describe program model information. The parsing and assembly of program feature information is to re-describe the program feature information using the domain-specific language as the description language and the preset data structure as the description structure. It can be understood that by using the domain-specific language and the preset data structure, complex logical expressions can be uniformly simplified and business variable information can be uniformly desensitized.

[0110] Exemplarily, the program model information (referred to as DSL) may be defined as follows, wherein the numbers 1-9 merely symbolically refer to the set identifiers of each set in the entire program model information:

[0111]

[0112] The node StmNode corresponds to a data structure element in the program to be tested, which usually contains data and pointers to other nodes. A node can be an independent individual or a part of a data structure. In computer programs, nodes are usually used to represent data and structures. For example, each element in a data structure such as a linked list, tree, and graph is a node. The node information of the node StmNode can be defined as follows, where numbers 1-12 only symbolically refer to the attribute identifiers of each attribute in the node information:

[0113]

[0114]

[0115] In actual business scenarios, large functions in the program to be tested usually have a volume of 1,000 to 2,000 lines. By using the identifier attribute table generated by the Cppcheck tool and combining it with the specified data structure, program model information described in DSL is generated, which includes classes, functions, call information, variables, paths and other information. It can cope with complex situations such as public libraries, protocol buffer definitions, and cross-project references in large C++ projects, thereby being able to handle large functions in actual business scenarios and has wide applicability.

[0116] In the above embodiment, the program to be tested is converted into program model information described in a domain-specific language, which retains the data structure information and control behavior information required for complete program testing, thereby ensuring the coverage of subsequently generated test data; by using the domain-specific language and preset data structure, the complex logical expressions in the program to be tested can be uniformly simplified, and the business variable information in the program to be tested can be uniformly desensitized, thereby improving processing efficiency and reducing the risk of leakage of business data or code privacy.

[0117] Considering that the main function of static analysis tools such as Cppcheck is code quality inspection rather than code modeling, open source static analysis tools such as cppcheck have shortcomings in actual business scenarios, which are specifically reflected in:

[0118] 1. The syntax analysis is not powerful enough. Cppcheck cannot recognize relatively low-frequency syntax scenarios, such as library functions, struct structures, and enumerations with scopes.

[0119] 2. It takes a long time. Affected by the size of the source code (for example, the source code is more than 1GB), running a full analysis takes about 10 minutes+ and generates 0.5GB+ of analysis data;

[0120] 3. Insufficient decoupling. Cppcheck has many quality check rule couplings, resulting in low efficiency in code structure analysis.

[0121] Therefore, in one embodiment of the present application, Cppcheck is improved to improve the efficiency and accuracy of static analysis. Specifically, Figure 4 As shown, step S221 may include:

[0122] S2211: Based on the static code analysis tool, semantic and grammatical analysis is performed on the program to be tested to obtain an identifier attribute table; the identifier attribute table represents attribute information of multiple identifiers.

[0123] Static analysis can be a semantic analysis and grammatical analysis of the program under test without running it, including type analysis of various conditional expressions such as whether they are Boolean and extraction of identifier related information. It can check semantic errors, grammatical errors and construct identifier attribute tables.

[0124] Figure 5 is a schematic diagram of the architecture of an improved static analysis tool provided according to an embodiment of the present application, such as Figure 5 As shown in the figure, according to the processing logic, the static analysis tool is divided into the basic layer, the rule layer and the quality inspection plug-in layer. The improved static analysis tool is decoupled in the logical structure, which is convenient for rapid iteration and improves the analysis efficiency of the code structure. Figure 5 h / c files in the Cppcheck, and can also include configuration files such as Confd, etc., and then perform static analysis such as initialization, preprocessing, and symbolization to obtain the identifier attribute table Tokenlist. Compared with the original Cppcheck, the preprocessing process is added, which can effectively improve the analysis efficiency and reduce unnecessary quality inspection processes.

[0125] S2212: Perform rule analysis and state collection on the identifier attribute table to obtain identifier state information, where the identifier state information represents state information of multiple identifiers during the rule analysis process of the identifier attribute table.

[0126] Among them, rule analysis is equivalent to program debugging, in which program debugging is the process of testing the compiled program manually or by compiling the program before putting it into actual operation to correct syntax errors and logical errors. The identifier status information can be a .dump file, which is a memory image file that reflects the program execution status during the debugging process.

[0127] like Figure 5 As shown, the rule layer performs the collection of various types of information according to the Tokenlist and generates a file in .dump format, which is the identifier status information. In the rule layer, the C++ code element information is split by field, which is convenient for quickly expanding the collection of other C++ code elements and improving the usefulness of the output information. In response to the problem that some C++ syntax is not supported, the syntax error reporting logic such as checking library function macro definitions and Template (template) can also be adjusted, and the extraction of some C++ code elements has been newly added. The C++ code elements in the dark box part are also the new parts of the embodiment of this application compared to the open source Cppcheck. In response to the problem of insufficient performance of static parsing of large engineering codes, after streamlining static analysis and information splitting and collection, the generation efficiency of .dump files has been improved. In a specific application embodiment, the size of the .dump file is reduced from 120M to 11M, and the generation time is reduced from 15 minutes to 10 seconds.

[0128] S2213: When the quality inspection result of the identifier status information meets the preset conditions, the identifier status information is used as program feature information.

[0129] Program feature information can include attribute information and status information of multiple identifiers, retaining the data structure information and control behavior information required for program complete testing, and ensuring the coverage of the test data generated subsequently. Figure 5 As described above, the .dump file can be handed over to the quality check plug-in layer for quality checks such as security scanning and performance scanning.

[0130] In the above embodiment, the improved static analysis tool decouples the processing logic, simplifies the static analysis, splits the information collection, improves the performance of semantic and syntactic analysis, improves the usefulness of the output information, reduces the time consumption, and improves the efficiency of code structure analysis.

[0131] S230: Perform path traversal based on the program model information to obtain at least one test path corresponding to the program to be tested, and determine path information of each test path; each test path is a program execution path of the program to be tested.

[0132] It can be understood that the program model information represents the data structure information and control behavior information of the program to be tested. The program control logic can be determined based on the program model information, and the program execution path can be traversed according to the program control logic to obtain at least one test path. The path information of the test path represents the program control logic corresponding to the test path. The path information includes but is not limited to path start condition information and path end condition information. The path start condition information corresponds to the trigger logic of the program control logic, and the path end condition information corresponds to the termination logic of the program control logic.

[0133] In one embodiment of the present application, Figure 6 As shown, based on the embodiment shown in S221-S222, and using a directed graph to represent the program control logic, the path traversal can be specifically implemented as follows:

[0134] S231: Determine node information of each first node in at least one first node from program model information, where the node information includes scope information; each first node corresponds to a data structure element in the program to be tested.

[0135] The first node is the aforementioned StmNode. The node information of StmNode may be as shown above, including scope information scope and controlledScope, etc.

[0136] S232: Based on the scope information in the node information of each first node, divide the program to be tested into at least one basic block, each basic block is a group of continuous program instructions, and each basic block includes an entry instruction and an exit instruction.

[0137] According to the parent-child relationship between scope and controlledScope in the scope information of each first node, the program to be tested is divided into at least one basic block, where a basic block refers to a group of continuous program instructions and has only one entry instruction and one exit instruction. Once the entry instruction in the basic block is executed, the remaining instructions will also be executed until the exit instruction is executed.

[0138] The definition of a basic block is as follows:

[0139]

[0140] S233: Generate a directed graph according to the entry instruction in each basic block and the exit instruction in each basic block, wherein the directed graph represents the control flow of the program to be tested, and the directed graph includes at least one second node and at least one directed edge between at least one second node; wherein each second node corresponds to a basic block.

[0141] The directed graph, also called the Control Flow Graph (CFG), is used to represent the control flow of a program. The nodes in the CFG represent the basic blocks of the program, and the directed edges represent that for two basic blocks associated by the directed edge, one basic block can jump to another basic block.

[0142] S234: Based on the directed graph, traverse the program execution path to obtain at least one test path.

[0143] Among them, the program execution path refers to a set of instruction sequences executed during the execution of the program. If there is at least one input that can make the program execute a specific path, then the path is called a reachable program execution path. The traversal of the program execution path based on the directed graph starts from the first second node in the directed graph, and according to the exit instruction of the first second node, the second second node that can be jumped to through the first second node is determined. According to the exit instruction of the second second node, the above process is repeated until no more jumps can be made. Then, a program execution path is constructed by one or more second nodes and the jump sequence between them, which corresponds to a test path.

[0144] S235: Determine path information of each test path according to at least one basic block associated with each test path, where the path information includes path start condition information and path end condition information.

[0145] According to at least one basic block included in each test path, the path information of each test path is determined, and the path information includes path start condition information and path end condition information. The path start condition information can be determined based on the entry instruction of the first basic block corresponding to the test path, and the path end condition information can be determined based on the exit instruction of the last basic block corresponding to the test path.

[0146] The path information of each test path, that is, the data structure of the test path, can be defined as follows:

[0147]

[0148] In the above embodiment, a directed graph representing the program control logic can be constructed completely, accurately and efficiently based on the program model information, so that the program execution path is traversed according to the directed graph, and the at least one test path obtained can cover all boundary conditions and program execution paths, thereby improving the comprehensiveness of the test; at the same time, by determining the path start condition information and the path end condition information in the path information of each test path, the condition requirements for generating test data corresponding to the test path are met, and the path information does not involve business data and code privacy, thereby avoiding the risk of leakage of business data or code privacy.

[0149] S240: Convert the path information of each test path into prompt information corresponding to each test path, where the prompt information is described in a natural language.

[0150] It can be understood that the path information of each test path represents the program control logic corresponding to the test path, and the program control logic includes trigger logic, which corresponds to the path start condition information in the path information. Inputting test data that meets the path start condition information can trigger the execution of the corresponding test path, so the path start condition information in the path information can indicate the generation constraint of the corresponding test data. In addition, the path end condition information corresponding to the termination logic can indicate the constraint of the corresponding test return value.

[0151] In the related art, a greedy algorithm is used to directly solve the constraints of the logical expression in the path start condition information to obtain test data. This method only supports the generation of three types of test data: global variables, member variables, and input parameters. It cannot perceive the dynamic variables and temporary variables in the path execution process. In addition, the algorithm has a disadvantage: if the same variables exist in the logical expression, the test data generated previously will be overwritten by the test data generated later. For example, the test data that meets the condition of "a>1&&(a<0||b<0)" can normally be a=2, b=-1, but the current algorithm may generate a set of test data such as a=-1, b=-1, where && represents AND logic and || represents OR logic.

[0152] The embodiment of the present application utilizes the understanding and computing capabilities of the generative language model to solve the above problems. In order for the generative language model to better understand the input text content and generate test data that meets the test requirements, the structured path information is converted into prompt information described in natural language, and the prompt information is used as the text content input to the generative language model.

[0153] In the embodiment of the present application, the prompt information corresponding to each test path may include: task subject information, such as "generate test data that meets the following rules", the rules that the test data needs to meet, the data type of the test data, background information, the test data evaluation indicators used, etc. The prompt information may also include context information, and the context information may indicate avoiding one or a category of test data. The prompt information also indicates the generation constraints of the test data.

[0154] In one embodiment of the present application, specifically, the language conversion may include the following steps:

[0155] S241: Determine test data condition information of each test path according to path start condition information in path information of each test path; the test data condition information is described in a structured programming language.

[0156] Taking the embodiment shown in S231 to S235 as an example, the path information of each test path includes path start condition information, that is, the entry expression (Expr or simpExpr) of the test path.

[0157] It is feasible to obtain the test data condition information by describing the path start condition information based on a preset data structure.

[0158] Exemplarily, the data structure of the test data condition information can be defined as follows:

[0159]

[0160] S242: Based on the grammar conversion rules between the program language and the natural language, the test data generation condition information is converted into prompt information corresponding to each test path.

[0161] Taking the above embodiment as an example, the prompt information described in natural language after conversion can be expressed as:

[0162] Purpose: Generate test data that meets the following rules and requires data to be returned in JSON format;

[0163] Rule: 'a>b&&b>c&&c>d&&d>e' is true, and a is of type unsigned int and b is of type char;

[0164] The return result is data in json format.

[0165] In the above embodiment, the path start condition information indicating the test data generation constraint condition is first converted into the test condition information described in a preset data structure, and then converted into the prompt information described in natural language, so that the prompt information described in natural language can be automatically and accurately generated, eliminating the operation of manually constructing the prompt information according to the path start condition information, thereby improving the test data generation efficiency; the preset data structure is used to express the test data generation constraint condition in a structured manner, ensuring that key information is not missing during the conversion process, and also providing auxiliary information such as task subject, context information, task result format, etc., so that the converted prompt information is clearer, more complete, and more readable, so that the generative language model can better understand the task requirements; at the same time, the prompt information is described in natural language, which can enable the generative language model to better understand the input content and quickly and accurately generate test data that meets the prompt information requirements.

[0166] S250: Input the prompt information corresponding to each test path into the generative language model, perform text generation processing, and obtain test data.

[0167] In an embodiment of the present application, the generative language model can automatically generate text content that meets the prompt information requirements based on the prompt information.

[0168] Taking the prompt information shown in the above embodiment as an example, the prompt information "Purpose: Generate test data that meets the following rules, requiring the return of data in json format; rule: 'a>b&&b>c&&c>d&&d>e' is true, and a is of unsigned int type, b is of char type; the return result is data in json format." is input into the generative language model, and the test data obtained is a key-value pair in json format, which can be shown as follows:

[0169] {

[0170] "a":4294967295,

[0171] "b":"z",

[0172] "c":"y",

[0173] "d":"x",

[0174] "e":"w"

[0175] }

[0176] Among them, the value range of unsigned int is 0 to 4294967295, and the generative language model selects the maximum value 4294967295 as the qualified test data a. The value range of char type is -128 to 127, so the generative language model selects the lowercase letter z as the qualified test data. In addition, the number of test data corresponding to each test path can also be specified.

[0177] It can be seen from the above embodiments that the present application provides a method for generating test data, by performing code modeling based on the program to be tested, to obtain program model information described in a domain-specific language; based on the program model information, at least one corresponding test path can be generated and the path information of each test path can be determined, and the test path is the program execution path of the program to be tested; the path information of each test path is converted into corresponding prompt information described in natural language, and then the prompt information corresponding to each test path is input into the generative language model, and text generation processing is performed to obtain the test data required by the program to be tested. In the scheme provided by the present application, by code modeling, path traversal and language conversion of the program to be tested, the prompt information corresponding to the program to be tested can be automatically generated to meet the input requirements of the generative language model in the task of generating test data, and improve the generation efficiency of test data; in the process of code modeling and path traversal, the desensitization of business data and the protection of code privacy are also realized, which reduces the security risk of using the generative language model, and for the program to be tested containing large functions and complex reference relationships, the path coverage can also be improved through code modeling and path traversal to ensure the comprehensiveness of the test.

[0178] In another embodiment of the present application, Figure 7 As shown, the method may further include:

[0179] S261: Obtain a historical test program, where the historical test program is a historical version of the program to be tested.

[0180] S262: Determine the incremental program to be tested relative to the historical test program.

[0181] Specifically, a diff algorithm may be used to compare the differences between the program to be tested and the historical test program, thereby determining an incremental program to be tested. The incremental program to be tested may be a newly added program to the program to be tested relative to the historical test program, or may be a modified program to be tested relative to the historical test program.

[0182] S263: Determine at least one incremental test path associated with the incremental program to be tested, so as to generate corresponding test data based on the at least one incremental test path; each incremental test path is any test path of the at least one test path.

[0183] There is a mapping relationship between the test path and the code lines of the program to be tested. According to the number of code lines of the incremental program to be tested, at least one incremental test path associated with it can be determined, that is, the program execution path that needs to be supplemented and the corresponding test data can be determined.

[0184] Specifically, Figure 8 As shown, with reference to the aforementioned embodiment, the node information of multiple first nodes stmNodes is obtained according to the program model information, and a directed graph is constructed according to the node information, and the data storage format of the directed graph graph can be an adjacency matrix. Generate adjacent edge set edges according to the directed graph, and the adjacent edge also corresponds to the directed edge in the directed graph, and the directed edge is associated with two nodes adjacent to the path. After the path traversal, all reachable paths validPaths are also at least one test path. At this time, according to the code line of the test path and the program to be tested, there is a mapping relationship, and the test path that does not contain the diff line (that is, the incremental program to be tested) is eliminated, and the test path corresponding to the diff line is retained, as an incremental test path, the incremental test path uses pathID as the path identifier, and the corresponding path information Pathdetails is obtained by assembling with a preset data structure, and the data structure of the path information of the incremental test path can refer to the aforementioned embodiment. The process of generating corresponding prompt information according to the path information of the incremental test path and generating corresponding test data according to the prompt information can also refer to the aforementioned embodiment, which will not be repeated here.

[0185] In the above embodiment, test data is generated for the incremental program to be tested, which reduces the amount of test data required to be generated for subsequent versions of the program to be tested, further speeds up the testing efficiency, and also improves the reusability of the test data corresponding to the historical test program.

[0186] In one embodiment of the present application, Fig. 9 As shown, the method may further include:

[0187] S271: Perform rule verification on the test data corresponding to each test path to obtain the rule verification result corresponding to the test data.

[0188] S272: When the rule checking result indicates that the test data does not conform to the logic rule indicated by the corresponding test path, generate context information corresponding to the test data; the context information indicates that the test data is excluded.

[0189] S273: Update corresponding prompt information according to the context information.

[0190] In the above embodiment, the test data may be subjected to a logic rule check, for example, to verify whether the logic expression indicated in the path start condition is satisfied. When the rule check result indicates that the test data does not meet the logic rule indicated by the corresponding test path, context information in the prompt information is generated to prompt the generative language model to exclude the group of test data when generating the test data.

[0191] For example, for the following test data:

[0192] {

[0193] "a":4294967295,

[0194] "b":"z",

[0195] "c":"y",

[0196] "d":"x",

[0197] "e":"w"

[0198] },

[0199] The govaluate library function of the go language is used for verification. The following exemplary verification code is also provided in the embodiment of the present application:

[0200]

[0201] For the above test data, the error message under govaluate's check rule is +$err, which means that the rule check result fails. At this time, you can assemble the prompt information again, add this set of test data as error information to the context information in the prompt information, and re-input the updated prompt information into the generative language model until the test data that passes the rule check result is given. The updated prompt information can be as follows:

[0202] Purpose: Generate test data that meets the following rules and requires the data to be returned in JSON format.

[0203] Rule: 'a>b&&b>c&&c>d&&d>e' is true, and a is of type unsigned int and b is of type char

[0204] Tip: Cannot provide {

[0205] "a":4294967295,

[0206] "b":"z",

[0207] "c":"y",

[0208] "d":"x",

[0209] "e":"w"

[0210] }

[0211] In the above embodiment, the rule check is performed through the test data, and the prompt information is updated when the rule check result fails, so that the generative language model excludes the test data that has been checked as error information when regenerating the test data, thereby improving the accuracy of the test data generation and ensuring the pass rate of the test data.

[0212] In one embodiment of the present application, corresponding unit test cases are automatically generated based on the test data to more efficiently apply the test data to software testing. Fig.10 As shown, the method may further include:

[0213] S281: Obtain a preset unit test case template.

[0214] like Fig.11 As shown, the structure of the preset unit test case template includes the function header and the driving function of the function body, which may include but is not limited to the following template elements: the name of the package / class / function under test, the return type, the mark case information, the parameter declaration assignment, the global variable assignment, the member variable assignment, the constructor assignment, the method of calling the function under test, the simulation function information, etc.

[0215] The programming language of the preset unit test case template can be as follows Fig.12 As shown, the function header is Test_F, and the function body includes declaration statements, call statements, return statements, etc. for each template element.

[0216] S282: Fill the unit test case template with the test data corresponding to each test path to obtain at least one unit test case.

[0217] In an exemplary embodiment, a unit test case for an XML data request program for an advertising service in a social network software (Social Network Services, SNS) may be as follows:

[0218]

[0219] Among them, the test data can be used to create a new object req of the AdInfoXmlReq class and a new sns_xml string new_sns_xml, and to create a new object curTestObj of the AdXmlMgr class and its new strings canvas_xml, ad_xml, and sns_xml. The above unit test case tests the function ProcessSnsXml to test whether it can respond to the XML data request of the request object req for the advertisement and return the canvas_xml, ad_xml, and sns_xml data of the advertisement, and save the data in ret to check whether the data in ret meets the response requirements.

[0220] S283: Input at least one unit test case and a preset annotation instruction into a generative language model, perform text generation processing, and obtain annotation information corresponding to each unit test case.

[0221] As can be seen from the above example, the readability of unit test cases is relatively poor, and the generative language model can be used to generate corresponding Chinese comments. The preset comment instruction can be described in natural language: "Generate the following unit test case code comment". Since the generated unit test case code does not have business privacy, it can be directly provided to the generative language model.

[0222] Exemplarily, a unit test case and its corresponding annotation information can be expressed as:

[0223]

[0224]

[0225] In the process of generating unit test cases matching the business, some basic capabilities of the business, such as mock capabilities, can also be retained.

[0226] S284: Execute at least one unit test case in the simulated test environment to obtain an execution result corresponding to each unit test case.

[0227] S285: Determine evaluation information of at least one unit test case according to the execution result corresponding to each unit test case, where the evaluation information includes indicator information corresponding to a plurality of evaluation indicators, and the plurality of evaluation indicators include test coverage and test pass rate.

[0228] In one embodiment of the present application, an evaluation system is built to evaluate in real time whether the generated unit test cases are effective. Fig.13As shown, taking the WeChat advertising module as an example, after submitting the program to be tested, one or more unit test cases are automatically generated in the constructed LINUX environment according to the above process, and then in the constructed private machine cluster, the WeChat advertising background module compilation plug-in is called to perform the compilation and execution of the unit test cases, collect test results, etc., and judge whether the generated unit test cases can run normally and whether the incremental coverage of the unit test cases meets the standards according to the test results, determine in real time whether the generated unit test cases are valid, and generate an archive report. Unit test cases that fail or do not meet the incremental coverage standards can be returned to the test data generation link for iteration, and can be assisted by manual intervention.

[0229] Specifically, an embodiment of the present application also introduces various indicators as shown in the following table to more fully measure the effectiveness and economy of unit test cases.

[0230] Table 1 Unit test case evaluation indicators

[0231]

[0232]

[0233] In the above embodiment, the text generation capability of the generative language model is used to add comments to the unit test cases, and the code specifications can also be checked to increase the readability of the unit test cases and improve the language specification procedures of the unit test cases. At the same time, the generated unit tests are evaluated in real time through the evaluation system to improve the effectiveness of the unit test cases finally used and the efficiency of the unit tests.

[0234] Fig.14 This is an interactive flow chart for generating unit test cases provided by an embodiment of the present application. Fig.14 As shown, on the local side, users can use VSCode (a lightweight code editor) to edit the program. In response to the user's creation of a unit test case operation for the selected function area, that is, the program to be tested, the background logic is triggered by calling the preset plug-in in the code editor, including generating a DSL file representing the code model structure according to the selected function area, that is, the program model information; determining the code change line number of the selected function area, traversing the incremental path according to the program model information, and determining the path start condition of the incremental path; constructing prompt information according to the path start condition of the incremental path, and inputting the prompt information into the large model of the server to generate test data. The local side assembles the program code of the unit test case according to the received test data. In order to further improve the code, the program code of the unit test case can be input into the large model again to annotate, standardize and automatically correct the unit test case.

[0235] Fig.14 The specific implementation steps of steps 2-4 in can be as follows Fig.15 As shown, after all incrementally reachable paths are obtained through deep traversal, the path can be optimized to solve the path explosion problem, and 100% incremental test coverage can be achieved with fewer incremental test paths, reducing the amount of subsequent testing. Other specific execution steps can refer to the aforementioned embodiments and will not be repeated here.

[0236] The method of generating test data based on the generative language model in this solution is not limited to the unit test case generation scenario, but can also be applied in scenarios such as traffic screening, compilation analysis, and interface testing.

[0237] The present application embodiment also provides a test data generating device 1600, such as Fig.16 As shown, the device may include:

[0238] An acquisition module 1610 is used to acquire a program to be tested;

[0239] The structure analysis module 1620 is used to perform code modeling based on the program to be tested to obtain program model information; the program model information is described in a domain-specific language;

[0240] The test path determination module 1630 is used to perform path traversal based on the program model information, obtain at least one test path corresponding to the program to be tested, and determine the path information of each test path; each test path is a program execution path of the program to be tested;

[0241] A prompt information generating module 1640, configured to convert the path information of each test path into prompt information corresponding to each test path, wherein the prompt information is described in a natural language;

[0242] The test data generation module 1650 is used to input the prompt information corresponding to each test path into the generative language model, perform text generation processing, and obtain test data.

[0243] In one embodiment of the present application, the structure analysis module 1620 may include:

[0244] A static analysis unit, used to perform static analysis on the program to be tested based on a static code analysis tool to obtain program feature information;

[0245] The program model information generating unit is used to parse and assemble the program feature information based on the grammatical rules and preset data structure of the domain-specific language to obtain the program model information.

[0246] In one embodiment of the present application, the static analysis unit may include:

[0247] A semantic analysis subunit, used for performing semantic and grammatical analysis on the program to be tested based on a static code analysis tool to obtain an identifier attribute table; the identifier attribute table represents attribute information of multiple identifiers;

[0248] A rule analysis and status collection subunit, configured to perform rule analysis and status collection on the identifier attribute table to obtain identifier status information, wherein the identifier status information represents status information of the multiple identifiers during the rule analysis process of the identifier attribute table;

[0249] The quality inspection subunit is used to use the identifier status information as the program feature information when the quality inspection result of the identifier status information meets a preset condition.

[0250] In one embodiment of the present application, the test path determination module 1630 may include:

[0251] A node determination unit, configured to determine node information of each first node in at least one first node from the program model information, wherein the node information includes scope information; each first node corresponds to a data structure element in the program to be tested;

[0252] A basic block determination unit, configured to divide the program to be tested into at least one basic block based on the scope information in the node information of each first node, each basic block being a group of continuous program instructions, and each basic block including an entry instruction and an exit instruction;

[0253] A directed graph determining unit, configured to generate a directed graph according to the entry instruction in each basic block and the exit instruction in each basic block, wherein the directed graph represents the control flow of the program to be tested, and the directed graph includes at least one second node and at least one directed edge between the at least one second node; wherein each second node corresponds to a basic block;

[0254] A path traversal unit, configured to traverse the program execution path based on the directed graph to obtain the at least one test path;

[0255] The path information determination unit is used to determine the path information of each test path according to at least one basic block associated with each test path, wherein the path information includes path start condition information and path end condition information.

[0256] In one embodiment of the present application, the prompt information generating module 1640 may include:

[0257] a condition information determination unit, configured to determine the test data condition information of each test path according to the path start condition information in the path information of each test path; the test data condition information is described in a structured programming language;

[0258] A language conversion unit is used to convert the test data generation condition information into the prompt information corresponding to each test path based on the grammar conversion rules between the program language and the natural language.

[0259] In one embodiment of the present application, the apparatus 1600 may further include:

[0260] A historical version acquisition unit, used for acquiring a historical test program, wherein the historical test program is a historical version of the program to be tested;

[0261] An incremental program determination unit, used to determine an incremental program to be tested of the program to be tested relative to the historical test program;

[0262] The incremental path determination unit is used to determine at least one incremental test path associated with the incremental program to be tested, so as to generate corresponding test data based on the at least one incremental test path; each incremental test path is any test path of the at least one test path.

[0263] In one embodiment of the present application, the apparatus 1600 may further include:

[0264] A rule detection unit, used to perform rule detection on the test data corresponding to each test path to obtain the rule detection result corresponding to the test data;

[0265] A context information generating unit, configured to generate context information corresponding to the test data when the rule checking result indicates that the test data does not conform to the logic rule indicated by the corresponding test path; the context information indicates that the test data is excluded;

[0266] The prompt information updating unit is used to update the corresponding prompt information according to the context information.

[0267] In one embodiment of the present application, the apparatus 1600 may further include:

[0268] A test case template acquisition unit is used to acquire a preset unit test case template;

[0269] A case generation unit, used to fill the test data corresponding to each test path into the unit test case template to obtain at least one unit test case;

[0270] The use case annotation unit is used to input the at least one unit test case and preset annotation instructions into the generative language model, perform text generation processing, and obtain annotation information corresponding to each unit test case.

[0271] In one embodiment of the present application, the apparatus 1600 may further include:

[0272] A case execution unit, used to execute the at least one unit test case in a simulated test environment to obtain an execution result corresponding to each unit test case;

[0273] The use case evaluation unit is used to determine the evaluation information of at least one unit test case according to the execution result corresponding to each unit test case, wherein the evaluation information includes indicator information corresponding to multiple evaluation indicators, and the multiple evaluation indicators include test coverage and test pass rate.

[0274] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0275] An embodiment of the present application provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement a method for generating test data as provided in the above method embodiment.

[0276] Fig.17 A schematic diagram of the hardware structure of a device for implementing a method for generating test data provided in an embodiment of the present application is shown. The device may participate in constituting or include the apparatus or system provided in an embodiment of the present application. Fig.17 As shown, the device 10 may include one or more (1002a, 1002b, ..., 1002n are used to illustrate) processors 1002 (the processor 1002 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1004 for storing data, and a transmission device 1006 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Fig.17The structure shown is for illustration only and does not limit the structure of the electronic device. Fig.17 More or fewer components as shown, or with Fig.17 Different configurations are shown.

[0277] It should be noted that the one or more processors 1002 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the device 10 (or mobile device). As involved in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0278] The memory 1004 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the methods in the embodiments of the present application. The processor 1002 executes various functional applications and data processing by running the software programs and modules stored in the memory 1004, that is, to realize the above-mentioned method for generating test data. The memory 1004 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1004 may further include a memory remotely arranged relative to the processor 1002, and these remote memories may be connected to the device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0279] The transmission device 1006 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the device 10. In one example, the transmission device 1006 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 1006 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0280] The display may be, for example, a touch screen liquid crystal display (LCD) that may enable a user to interact with a user interface of device 10 (or mobile device).

[0281] An embodiment of the present application also provides a computer-readable storage medium, which can be set in a server to store at least one instruction or at least one program related to a test data generation method in a method embodiment, and the at least one instruction or the at least one program is loaded and executed by the processor to implement a test data generation method provided in the above method embodiment.

[0282] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0283] The embodiment of the present invention further provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes a method for generating test data provided in the above various optional implementations.

[0284] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of the present application are described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0285] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0286] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0287] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for generating test data, characterized in that: The method comprises: Get the program to be tested; Based on the program to be tested, code modeling is performed to obtain program model information; the program model information is described in a domain-specific language; Based on the program model information, path traversal is performed to obtain at least one test path corresponding to the program to be tested, and path information of each test path is determined; each test path is a program execution path of the program to be tested; Converting the path information of each test path into prompt information corresponding to each test path, wherein the prompt information is described in a natural language; The prompt information corresponding to each test path is input into the generative language model, and text generation processing is performed to obtain test data.

2. The method according to claim 1, characterized in that The code modeling is performed based on the program to be tested to obtain program model information, including: Based on a static code analysis tool, static analysis is performed on the program to be tested to obtain program feature information; Based on the grammatical rules and preset data structure of the domain-specific language, the program feature information is parsed and assembled to obtain the program model information.

3. The method according to claim 2, characterized in that The static code analysis tool is used to perform static analysis on the program to be tested to obtain program feature information, including: Based on a static code analysis tool, semantic and grammatical analysis is performed on the program to be tested to obtain an identifier attribute table; the identifier attribute table represents attribute information of multiple identifiers; Performing rule analysis and state collection on the identifier attribute table to obtain identifier state information, wherein the identifier state information represents state information of the multiple identifiers during the rule analysis process of the identifier attribute table; When the quality inspection result of the identifier status information meets a preset condition, the identifier status information is used as the program feature information.

4. The method according to claim 1, characterized in that The performing path traversal based on the program model information to obtain at least one test path corresponding to the program to be tested, and determining path information of each test path, includes: Determine node information of each first node in at least one first node from the program model information, wherein the node information includes scope information; each first node corresponds to a data structure element in the program to be tested; Based on the scope information in the node information of each first node, the program to be tested is divided into at least one basic block, each basic block is a group of continuous program instructions, and each basic block includes an entry instruction and an exit instruction; Generate a directed graph according to the entry instruction in each basic block and the exit instruction in each basic block, wherein the directed graph represents the control flow of the program to be tested, and the directed graph includes at least one second node and at least one directed edge between the at least one second node; wherein each second node corresponds to a basic block; Based on the directed graph, traverse the program execution path to obtain the at least one test path; According to at least one basic block associated with each test path, path information of each test path is determined, wherein the path information includes path start condition information and path end condition information.

5. The method according to claim 4, characterized in that The converting the path information of each test path into prompt information corresponding to each test path includes: Determine test data condition information of each test path according to the path start condition information in the path information of each test path; the test data condition information is described in a structured programming language; Based on the grammar conversion rules between the program language and the natural language, the test data generation condition information is converted into the prompt information corresponding to each test path.

6. The method according to claim 1, characterized in that The method further comprises: Obtain a historical test program, where the historical test program is a historical version of the program to be tested; Determine an incremental program to be tested of the program to be tested relative to the historical test program; At least one incremental test path associated with the incremental program to be tested is determined to generate corresponding test data based on the at least one incremental test path; each incremental test path is any test path of the at least one test path.

7. The method according to claim 1, characterized in that The method further comprises: Performing rule checking on the test data corresponding to each test path to obtain rule checking results corresponding to the test data; In the case where the rule checking result indicates that the test data does not conform to the logic rule indicated by the corresponding test path, generating context information corresponding to the test data; the context information indicates that the test data is excluded; According to the context information, the corresponding prompt information is updated.

8. The method according to claim 1, characterized in that The method further comprises: Get the preset unit test case template; Fill the test data corresponding to each test path into the unit test case template to obtain at least one unit test case; The at least one unit test case and the preset annotation instructions are input into the generative language model, and text generation processing is performed to obtain annotation information corresponding to each unit test case.

9. The method according to claim 8, characterized in that The method further comprises: In the simulated test environment, executing the at least one unit test case to obtain an execution result corresponding to each unit test case; According to the execution result corresponding to each unit test case, evaluation information of the at least one unit test case is determined, the evaluation information including indicator information corresponding to a plurality of evaluation indicators, and the plurality of evaluation indicators including test coverage and test pass rate.

10. A device for generating test data, characterized in that: The device comprises: An acquisition module is used to acquire the program to be tested; A structural analysis module, used to perform code modeling based on the program to be tested to obtain program model information; the program model information is described in a domain-specific language; A test path determination module, used to perform path traversal based on the program model information, obtain at least one test path corresponding to the program to be tested, and determine the path information of each test path; each test path is a program execution path of the program to be tested; A prompt information generating module, used for converting the path information of each test path into prompt information corresponding to each test path, wherein the prompt information is described in a natural language; The test data generation module is used to input the prompt information corresponding to each test path into the generative language model, perform text generation processing, and obtain test data.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by a processor to implement a method for generating test data according to any one of claims 1 to 9.

12. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement a test data generation method as described in any one of claims 1 to 9.

Images