Software automatic testing method and system, storage medium and equipment

Abstract

The invention provides an automatic software test method and system, a storage medium and equipment, and the method comprises the steps: obtaining a target code, extracting related information and test requirements, and generating a test case according to the target code and the test requirements through a trained large model; in response to the code change information, evaluating the influence range of code change on the test case, performing static analysis, determining the change of a code structure, performing dynamic analysis, and determining the change of a code execution path; according to the evaluation result of the influence range of the code change on the test case, determining a test case automatic adjustment strategy, running the adjusted test case, obtaining log data in the running process, extracting error information in the log data, performing code mapping on the error information, and determining an error position. According to the method, a large model technology is utilized, and the processes of automatic generation, intelligent adjustment, automatic result analysis, accurate problem positioning and the like of the test case are optimized, so that the efficiency and the accuracy of software testing are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of software automatic testing, and in particular relates to a software automatic testing method, system, storage medium and equipment. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] In the complex environment of software development and maintenance, automated testing plays a vital role. However, existing automated testing methods have certain limitations, specifically:

[0004] Traditionally, test cases have to be manually written by test engineers based on requirements and design documents. This process is not only time-consuming and labor-intensive, but also requires the test engineers to possess deep expertise and experience. As software functionality continues to expand and its complexity increases, the number and complexity of test cases also increase, further increasing the cost of writing them.

[0005] During the software development process, code changes occur frequently. Whenever the code changes, the relevant test cases need to be manually updated to reflect the new code logic and functions. This is not only tedious but also prone to errors. As a result, the test cases are inconsistent with the latest state of the code, thus reducing the accuracy and reliability of the test.

[0006] On the other hand, after the automatic test run is completed, the test results mainly rely on manual analysis, which has low accuracy, is prone to inconsistencies, and is time-consuming. For complex test results, manual analysis often finds it difficult to quickly locate the problem, thereby delaying the repair of the problem and the release of the software. If a test failure is found, error location is also more difficult, and code line-level location is often impossible to achieve. Summary of the Invention

[0007] In order to solve the above problems, the present invention proposes a software automatic testing method, system, storage medium and device. The present invention uses big model technology to optimize the processes of automatic generation of test cases, intelligent adjustment, automated result analysis, precise problem location and repair suggestions, thereby significantly improving the efficiency and accuracy of software testing.

[0008] According to some embodiments, the present invention adopts the following technical solutions:

[0009] A software automatic testing method comprises the following steps:

[0010] Obtain historical test cases and related data for various types of software codes, preprocess the acquired data, extract key features, use the key features to train large models, and adjust and optimize the parameters of the large models;

[0011] Obtain the target code, extract relevant information and test requirements, and use the trained large model to generate test cases based on the target code and test requirements;

[0012] Perform syntax and logic consistency checks on the generated test cases;

[0013] In response to code change information, evaluate the impact of code changes on test cases, perform static analysis to determine changes in code structure, and perform dynamic analysis to determine changes in code execution paths;

[0014] Determine the test case automatic adjustment strategy based on the impact of code changes on test cases, and adjust test cases in order of priority according to the test case automatic adjustment strategy;

[0015] Run the adjusted test case, obtain log data during the running process, extract error information from the log data, perform code mapping on the error information, and determine the error location.

[0016] As an optional implementation method, the relevant data of the software code includes a code library, interface documents and design documents, and the extracted key features include code structure features, test scenario features and test case templates.

[0017] As an optional implementation method, the process of performing static analysis includes: parsing the changed code into an abstract syntax tree, traversing the abstract syntax tree, locating the function definition node, extracting the function name, parameter list and return type, finding the variable declaration node, extracting the variable type information, comparing the variable types before and after the change, and detecting whether there is a type change; generating a control flow graph based on the abstract syntax tree, analyzing the execution path of the code and the variable definition-usage relationship, comparing the information before and after the change, identifying changes in the variable definition or usage location, and changes in the code block.

[0018] As an optional implementation method, the process of dynamic analysis includes: inserting probes into the code when the code is compiled or run to collect information during code execution. The probes are inserted at several positions before and after function calls and at the entry and exit of conditional branches. The execution order of the functions and the results of the conditional branches are recorded by the probes. Based on the information collected by the code instrumentation, a dynamic call chain is constructed to reflect the actual execution path of the code. The dynamic call chains before and after the change are compared to detect changes in the code execution path.

[0019] As an optional implementation, the process of evaluating the scope of impact of code changes on test cases also includes counting the coverage of the code by the test cases, where the coverage includes statement coverage, branch coverage, and path coverage, and identifying code paths that are not covered by the test cases through coverage analysis.

[0020] As an optional implementation method, based on the evaluation results of the impact scope of code changes on test cases, the process of determining the automatic adjustment strategy of test cases includes: adjusting the expected result type of the test case based on the change of function signature or variable type in the static analysis results, and adding test scenarios to cover the newly added boundaries based on the code boundary change results in the static analysis results; deleting related test cases that are no longer executed and adding new test cases to cover the new code path based on the code path changes in the dynamic analysis results.

[0021] As an optional implementation, the process of sequentially adjusting the test cases according to the test case automatic adjustment strategy according to the priority includes: assigning a higher priority to the test case that is more affected by the change, and processing the high-priority test case first;

[0022] For situations where the change situation is complex or uncertain, the user's input or selection is received through the interactive interface to adjust the test case.

[0023] As an optional implementation method, running the adjusted test case, obtaining log data during the running process, and extracting error information from the log data includes: preprocessing the obtained log data, including log cleaning, format conversion and feature extraction, using a pre-trained model to identify the error type and error location in the log data, classifying the identified error information into predefined error types, and using a large model to establish a mapping relationship between the natural language description in the error information and the elements in the code, analyzing the keywords in the error information through the large model, and matching possible code elements in the code library according to the mapping relationship.

[0024] A software automatic testing system, comprising:

[0025] The large model training module is configured to obtain historical test cases and related data of various types of software codes, preprocess the obtained data, extract key features, use the key features to train the large model, and adjust and optimize the parameters of the large model;

[0026] The test case generation module is configured to obtain the target code, extract relevant information and test requirements, and use the trained large model to generate test cases based on the target code and test requirements;

[0027] A preliminary check module is configured to perform syntax check and logic consistency check on the generated test cases;

[0028] a change analysis module configured to respond to code change information, evaluate the scope of impact of code changes on test cases, perform static analysis to determine changes in code structure, and perform dynamic analysis to determine changes in code execution paths;

[0029] The intelligent maintenance module is configured to determine a test case automatic adjustment strategy based on an evaluation result of the impact of the code change on the test case, and adjust the test cases in sequence according to the test case automatic adjustment strategy based on the priority;

[0030] The intelligent analysis module is configured to run the adjusted test case, obtain log data during the running process, extract error information in the log data, perform code mapping on the error information, and determine the error location.

[0031] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the steps in the above method are completed.

[0032] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the above method are completed.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] By leveraging large-scale modeling technology, this system can automatically generate high-quality test cases, significantly reducing the time and cost of manually writing test cases. Intelligent analysis capabilities can quickly and accurately identify errors and issues during testing, providing detailed error cause analysis and remediation suggestions, thereby improving test efficiency and quality.

[0035] The present invention can detect code changes in real time, automatically evaluate the impact of changes on test cases, and promptly adjust test cases to adapt to the new code status, ensuring the continued effectiveness and accuracy of the test. The automated maintenance and intelligent adjustment functions enable test cases to be automatically updated as the code changes, reducing the workload of manual maintenance of test cases and lowering the cost of test maintenance.

[0036] Through error location technology and repair suggestion generation algorithm, the present invention can quickly and accurately locate problems and provide preliminary solutions, significantly improving the efficiency of problem location and repair. It can generate more comprehensive and accurate test cases based on comprehensive multi-source data training and optimization of large models, thereby improving test coverage and ensuring software quality.

[0037] This invention automates the generation, maintenance, and analysis of test cases, supporting rapid iteration and continuous delivery, and improving the efficiency and flexibility of software development. This not only significantly improves testing efficiency and quality, reduces test maintenance costs, but also enhances the adaptability of code changes, the efficiency of problem location and repair, and test coverage and accuracy. Furthermore, the system supports continuous integration and continuous delivery, and prioritizes data security and privacy protection, bringing comprehensive optimization and upgrades to the software development and testing fields.

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0040] Figure 1 is a flowchart of a software automatic testing method according to an embodiment;

[0041] Figure 2 is a schematic structural diagram of a software automatic testing system according to an embodiment;

[0042] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment. DETAILED DESCRIPTION

[0043] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0044] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0045] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0046] In the absence of conflict, the embodiments and features in the embodiments of this application can be combined with each other.

[0047] Example 1

[0048] A software automatic testing method, such as Figure 1 As shown, the following steps are included:

[0049] Obtain historical test cases and related data for various types of software codes, preprocess the acquired data, extract key features, use the key features to train large models, and adjust and optimize the parameters of the large models;

[0050] Obtain the target code, extract relevant information and test requirements, and use the trained large model to generate test cases based on the target code and test requirements;

[0051] Perform syntax and logic consistency checks on the generated test cases;

[0052] In response to code change information, evaluate the impact of code changes on test cases, perform static analysis to determine changes in code structure, and perform dynamic analysis to determine changes in code execution paths;

[0053] Determine the test case automatic adjustment strategy based on the impact of code changes on test cases, and adjust test cases in order of priority according to the test case automatic adjustment strategy;

[0054] Run the adjusted test case, obtain log data during the running process, extract error information from the log data, perform code mapping on the error information, and determine the error location.

[0055] First, when generating test cases, this embodiment collects multi-source data such as historical test cases, code libraries, interface documents, and design documents to build a rich training data set.

[0056] The dataset must cover software projects of varying complexity and domains to ensure the model's generalizability. The collected data must be preprocessed to extract key features, such as code structure, test scenario characteristics, and test case templates. These features will serve as input to the model.

[0057] Use a deep learning framework (such as TensorFlow, PyTorch, etc.), select an appropriate model architecture (such as Transformer, BERT, etc.), and train on a preprocessed dataset. During training, adjust model parameters and optimizer settings to improve model accuracy and efficiency.

[0058] Use static code analysis tools (such as AST parser) to parse newly submitted code and extract information such as code structure, function signatures, variable types, etc.; combine interface documents and design documents to identify test requirements, including functional testing, boundary testing, performance testing, etc.

[0059] The parsed code information and test requirements are input into the pre-trained large model, which generates test cases covering key functions and boundary conditions based on the input information. The generated test cases are output in natural language descriptions or structured formats (such as XML and JSON).

[0060] Generated test cases are first syntax-checked using natural language processing tools to ensure correct syntax. By comparing the generated test cases with the code logic, the system verifies that the expected test results are consistent with the code implementation. For low-quality test cases, the system automatically provides optimization suggestions, such as adding test scenarios and adjusting test data. Users can manually adjust the test cases based on these suggestions.

[0061] This embodiment can integrate with version control systems such as Git to obtain code change information in real time. Static analysis and dynamic analysis techniques can be used to assess the impact of code changes on test cases. Static analysis focuses on changes in code structure, such as function signatures and variable types; dynamic analysis focuses on changes in code execution paths.

[0062] As described in detail below, static analysis primarily focuses on changes in code structure, such as function signatures and variable types. Tools (such as ANTLR and JavaParser) are used to parse the modified code into an AST. The AST is an abstract representation of the source code, preserving the syntactic structure and semantics of the code while removing unnecessary details (such as brackets and spaces). The AST is traversed to locate the function definition node, extracting information such as the function name, parameter list, and return type. By comparing the ASTs before and after the modification, any changes in the function signature can be detected. For example, changes in the type or number of function parameters or changes in the function name may occur.

[0063] Find the variable declaration node in the AST and extract the variable type information. Compare the variable types before and after the change to detect whether there is a type change. For example, if a variable changes from int to float.

[0064] Next, a control flow graph (CFG) is generated based on the AST. A CFG is a directed graph where nodes represent basic code blocks and edges represent control flow paths. The CFG allows analysis of the code execution path and variable definition-usage relationships. By traversing the CFG, the definition points (such as variable declarations and assignment statements) and usage points (such as variable references in expressions) of variables are recorded. By comparing the data flow information before and after the change, changes in variable definition or usage locations can be identified and their impact on the test case can be assessed.

[0065] Based on the data flow analysis results, identify the variables and functions affected by the code changes. For example, a change in the type of a variable may affect all functions that use the variable. These functions and their associated test cases need to be re-evaluated.

[0066] In addition, you can compare the CFGs before and after the change to detect changes in the code structure, such as adding or deleting code blocks, changing the execution order of code blocks, etc. These changes may affect the execution path of the code, requiring adjustments to related test cases.

[0067] Next, we'll introduce dynamic analysis, which focuses on changes in the code execution path. In this embodiment, probes are inserted into the code during compilation or runtime to collect information about the code's execution. Probes can be inserted before and after function calls, at the entry and exit points of conditional branches, and at other locations.

[0068] Probes record information such as the execution order of functions and the results of conditional branches. For example, they record whether a function is called or whether a conditional branch is executed.

[0069] Based on the information collected by code instrumentation, a dynamic call chain is constructed to reflect the actual execution path of the code. By comparing the dynamic call chain before and after the change, changes in the code execution path can be detected. For example, a function is no longer called after the change, or the execution result of a conditional branch has changed.

[0070] In some embodiments, statistical tools (such as JaCoCo, Cobertura, etc.) can also be used to calculate the code coverage of test cases. Coverage includes statement coverage, branch coverage, path coverage, etc. Through coverage analysis, code paths that are not covered by test cases are identified. These uncovered paths may be new paths introduced by code changes, requiring redesign of test cases to cover them.

[0071] In terms of specific implementation, the above analysis / statistics tools can be integrated, for example, through Git's hook mechanism and API interface. For example, the git diff command can be used to obtain code changes, and the gitlog command can be used to obtain commit history and other information.

[0072] Through static analysis and dynamic analysis, the impact of code changes on test cases can be comprehensively assessed, thereby intelligently maintaining test cases and ensuring the adequacy and effectiveness of testing.

[0073] Based on the results of the change impact assessment, an automatic test case adjustment strategy is developed. For affected test cases, the system provides automatic update suggestions, such as modifying expected results, adding new test scenarios, or deleting no longer relevant test cases.

[0074] Based on static analysis results, the system automatically modifies the expected results of test cases based on changes in function signatures or variable types. For example, if the return type of a function changes from int to float, the assertion of the function's return value in the test case will be automatically adjusted.

[0075] Based on the dynamic analysis results, if a code path is no longer executed after the change, the related test cases are automatically deleted. If the change introduces new code paths or execution scenarios, new test cases are automatically added to cover these paths or scenarios.

[0076] During adjustments, test cases are prioritized based on the results of the change impact assessment. Test cases that are most affected by the change (such as those covering the changed code paths) are given higher priority. During the automated adjustment process, high-priority test cases are prioritized to ensure they can adapt to code changes in a timely manner.

[0077] For complex or uncertain change situations, this embodiment also provides a user interaction interface to guide the user to make manual adjustments or confirm automatic adjustment suggestions.

[0078] Specifically, by analyzing the results of the change impact assessment, complex or uncertain change scenarios can be identified, which may involve multiple code modules, complex logic changes, or potential compatibility issues.

[0079] In this embodiment, a user interaction interface is provided to display detailed information about complex changes, automatic adjustment suggestions, and possible consequences. The interface design follows the principle of simplicity to avoid information overload. Only key information and necessary operation options are displayed. The interface operation should be simple and intuitive so that users can quickly understand and operate. For example, clear operation buttons such as "Confirm automatic adjustment", "Manual adjustment" and "Cancel" are provided. When the user performs an operation, the system should provide real-time feedback, such as displaying the adjustment progress, success or failure information, etc.

[0080] The system automatically provides adjustment suggestions based on the results of the change impact assessment. The suggestions should be clear and concise, including the specific content of the adjustment, the possible impact, and the recommended action steps.

[0081] For complex changes, the system guides users through manual adjustments. For example, it provides a comparison view of the code before and after the change, as well as the relationship between test cases, to help users understand the impact of the change and make appropriate adjustments.

[0082] After adjusting test cases, the system automatically runs the adjusted test cases to verify the correctness of the changes. For code changes involving multiple modules or complex functions, the system automatically triggers regression testing to ensure that the changes do not introduce new defects.

[0083] During the test run, key information such as test logs and server logs is automatically collected, and the collected logs are preprocessed, including log cleaning, format conversion, feature extraction, etc. The preprocessed log data will be used as the input for subsequent analysis. Then, natural language processing and text mining techniques are utilized to identify error information in the logs, including error types, error locations, etc.; combined with code logic and test case information, the reasons for the errors are analyzed. The system provides an error cause analysis report, including possible problem roots, repair suggestions, etc. The analysis results are visually presented in the form of charts, lists, etc., facilitating users to quickly understand the test results and error situations.

[0084] In this embodiment, regular expressions are used to correct records with inconsistent formats, script logic is used to remove duplicates, and statistical methods are used to identify outliers. For example, records that are all empty in the log or incorrect JSON records can be filtered out. Ensure that each log record contains necessary key fields, such as eventid, sessionId, event, etc., and records lacking any key fields should be filtered out.

[0085] In this embodiment, the forward maximum matching method, reverse maximum matching method or bidirectional maximum matching method is used to segment the log text. For example, the natural language description part in the log is segmented into words or phrases for subsequent analysis. And a predefined stop word list is used to remove common meaningless words, such as "de", "shi", "zai", etc.

[0086] On the other hand, the words are sorted according to their word frequencies, and high-frequency words are removed, which are often stop words.

[0087] The log text is represented as a set of words, ignoring the order and grammar of the words. For example, each log record is converted into a word vector, and each element of the vector represents the number of occurrences of the corresponding word in the log.

[0088] The TF-IDF value of each word is calculated to evaluate its importance in the log. The higher the TF-IDF value, the more important the word is for distinguishing different log records. The words are mapped to a low-dimensional vector space, retaining the semantic information of the words. For example, tools such as Word2Vec or GloVe are used to train a word embedding model, and each word is represented as a real number vector.

[0089] In this example, we selected an appropriate NLP model based on the task requirements, such as a recurrent neural network (RNN), a long short-term memory network (LSTM), or a Transformer. These models excel in natural language processing tasks and can capture complex features in text. Consider using pre-trained NLP models, such as BERT and GPT, which have been pre-trained on large amounts of text data and provide better text representation and feature extraction capabilities.

[0090] Use labeled log data to train the model. The labeled data should include information such as error type and error location. Improve the recognition accuracy of the model by adjusting model parameters and optimizing algorithms.

[0091] This embodiment utilizes an NLP model to identify entity information in logs, such as error type and location. For example, named entity recognition (NER) technology is used to identify error codes and exception information in logs. The identified error information is classified into predefined error types, such as syntax errors, runtime errors, and logical errors. Classification algorithms such as support vector machines (SVMs) and random forests can be used for classification.

[0092] Leveraging the semantic understanding and code analysis capabilities of the large model, we can pinpoint the error message to a specific line of code or code segment. To pinpoint the error message to a specific line of code or code segment, we can leverage the semantic understanding and code analysis capabilities of the large model, combined with code analysis technology, to achieve precise code-level positioning.

[0093] First, leveraging the semantic understanding capabilities of the large model, a mapping relationship is established between the natural language description in the error message and elements such as variables, functions, and classes in the code. For example, when an error message mentions a "null pointer exception," the model understands that this is often related to uninitialized variables or object accesses in the code. The large model analyzes keywords in the error message (such as exception type and error description) and matches possible code elements (such as variable names, function names, and class names) in the code base.

[0094] Use static analysis tools to analyze the code structure and dependencies to understand its functionality and logic. This helps pinpoint errors to specific lines or sections of code. While the code is running, use dynamic analysis tools to collect information about the code execution, such as execution paths and variable values. This helps understand the code at runtime and more accurately pinpoint errors.

[0095] Choose the appropriate large model based on the task requirements. The GPT series of models performs well in natural language processing tasks and can generate high-quality text. The BERT model performs well in text understanding and classification tasks and can capture the deep semantic information of the text.

[0096] The model is trained using annotated error message-code pairs. The annotated data should include information such as the error message, the corresponding code line or code segment, and the error type. In some implementations, the model's localization accuracy can be improved by adjusting model parameters and optimizing algorithms. For example, algorithms such as attention mechanisms and graph neural networks can be used to help the model better understand the structure and semantics of the code.

[0097] Static analysis tools analyze the structure and dependencies of code to understand its functionality and logic. For example, they analyze function call relationships and variable scope within the code. Static analysis tools can detect potential errors in the code, such as syntax errors, incorrect variable usage, and memory leaks, helping to pinpoint errors to specific lines or sections of code.

[0098] Dynamic analysis tools can track the execution path of code while it's running, recording information like variable values ​​and function calls. This helps us understand the code at runtime and more accurately locate errors. Dynamic analysis tools can capture exceptions in the code and analyze their cause and location. This helps pinpoint errors to specific lines of code or sections of code.

[0099] Leveraging the semantic understanding and code analysis capabilities of the large model, combined with the results of static and dynamic analysis, the accuracy of code location can be improved. For example, the large model can infer the possible error location and cause based on the natural language description in the error message and the results of code analysis.

[0100] In some embodiments, large models can also be optimized, such as by introducing an attention mechanism into the model to help it better understand the structure and semantics of the code. The attention mechanism can make the model pay more attention to the parts of the code related to the error information.

[0101] Alternatively, graph neural networks (GNNs) can be used to process the structural information of code. Code can be represented as a graph structure, where nodes represent code elements (such as variables and functions) and edges represent the relationships between code elements. GNNs can capture the complex structure and dependencies in code.

[0102] Based on the results of error location, the system provides preliminary code modification suggestions. These suggestions may include modifying code logic, updating dependent library versions, adjusting configuration parameters, etc. The suggestion generation algorithm combines domain knowledge and historical repair cases to improve the accuracy and practicality of the suggestions.

[0103] Users can modify the code based on the repair suggestions provided by the system and rerun the test cases to verify the repair results. The system will optimize and adjust the repair suggestions based on the verification results.

[0104] During the data processing and analysis process, the system implements strict data encryption and access control policies to ensure the security and privacy of user data. Furthermore, the system supports data desensitization and anonymization to protect sensitive user information from being leaked.

[0105] Example 2

[0106] A software automatic testing system, such as Figure 2 Shown, including:

[0107] The large model training module is configured to obtain historical test cases and related data of various types of software codes, preprocess the obtained data, extract key features, use the key features to train the large model, and adjust and optimize the parameters of the large model;

[0108] The test case generation module is configured to obtain the target code, extract relevant information and test requirements, and use the trained large model to generate test cases based on the target code and test requirements;

[0109] A preliminary check module is configured to perform syntax check and logic consistency check on the generated test cases;

[0110] a change analysis module configured to respond to code change information, evaluate the scope of impact of code changes on test cases, perform static analysis to determine changes in code structure, and perform dynamic analysis to determine changes in code execution paths;

[0111] The intelligent maintenance module is configured to determine a test case automatic adjustment strategy based on an evaluation result of the impact of the code change on the test case, and adjust the test cases in sequence according to the test case automatic adjustment strategy based on the priority;

[0112] The intelligent analysis module is configured to run the adjusted test case, obtain log data during the running process, extract error information in the log data, perform code mapping on the error information, and determine the error location.

[0113] Example 3

[0114] A computer-readable storage medium (Memory) is a memory device in an electronic device used to store programs and data. It is understood that the computer-readable storage medium herein can include both built-in storage media in an electronic device and, of course, extended storage media supported by the electronic device. The computer-readable storage medium provides storage space that stores the processing system of the electronic device.

[0115] Furthermore, the storage space also stores one or more instructions suitable for being loaded and executed by the processor. These instructions may be one or more computer programs (including program code). It should be noted that the computer-readable storage medium herein may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage; optionally, it may be at least one computer-readable storage medium located remotely from the aforementioned processor.

[0116] In one embodiment, the computer-readable storage medium stores one or more instructions; the processor loads and executes the one or more instructions stored in the computer-readable storage medium to implement the following process:

[0117] Obtain historical test cases and related data for various types of software codes, preprocess the acquired data, extract key features, use the key features to train large models, and adjust and optimize the parameters of the large models;

[0118] Obtain the target code, extract relevant information and test requirements, and use the trained large model to generate test cases based on the target code and test requirements;

[0119] Perform syntax and logic consistency checks on the generated test cases;

[0120] In response to code change information, evaluate the impact of code changes on test cases, perform static analysis to determine changes in code structure, and perform dynamic analysis to determine changes in code execution paths;

[0121] Determine the test case automatic adjustment strategy based on the impact of code changes on test cases, and adjust test cases in order of priority according to the test case automatic adjustment strategy;

[0122] Run the adjusted test case, obtain log data during the running process, extract error information from the log data, perform code mapping on the error information, and determine the error location.

[0123] Example 4

[0124] An electronic device, such as Figure 3 As shown, the electronic device includes a processor 1001, a communication interface 1002, and a computer-readable storage medium 1003. The processor 1001, the communication interface 1002, and the computer-readable storage medium 1003 may be connected via a bus or other means.

[0125] Among them, the communication interface 1002 is used to receive and send data, the computer-readable storage medium 1003 can be stored in the memory of the electronic device, the computer-readable storage medium 1003 is used to store a computer program, the computer program includes program instructions, and the processor 1001 is used to execute the program instructions stored in the computer-readable storage medium 1003.

[0126] The processor 1001 (or CPU (Central Processing Unit)) is the computing core and control core of the electronic device, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to implement corresponding method processes or corresponding functions.

[0127] The processor 1001 is configured to execute the following process:

[0128] Obtain historical test cases and related data for various types of software codes, preprocess the acquired data, extract key features, use the key features to train large models, and adjust and optimize the parameters of the large models;

[0129] Obtain the target code, extract relevant information and test requirements, and use the trained large model to generate test cases based on the target code and test requirements;

[0130] Perform syntax and logic consistency checks on the generated test cases;

[0131] In response to code change information, evaluate the impact of code changes on test cases, perform static analysis to determine changes in code structure, and perform dynamic analysis to determine changes in code execution paths;

[0132] Determine the test case automatic adjustment strategy based on the impact of code changes on test cases, and adjust test cases in order of priority according to the test case automatic adjustment strategy;

[0133] Run the adjusted test case, obtain log data during the running process, extract error information from the log data, perform code mapping on the error information, and determine the error location.

[0134] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0138] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made by those skilled in the art that fall within the spirit and principles of the present invention and do not require creative effort are intended to be within the scope of protection of the present invention.

Claims

1. A software automatic testing method, characterized in that: The following steps are involved: Obtain historical test cases and related data for various types of software codes, preprocess the acquired data, extract key features, use the key features to train large models, and adjust and optimize the parameters of the large models; Obtain the target code, extract relevant information and test requirements, and use the trained large model to generate test cases based on the target code and test requirements; Perform syntax and logic consistency checks on the generated test cases; In response to code change information, evaluate the impact of code changes on test cases, perform static analysis to determine changes in code structure, and perform dynamic analysis to determine changes in code execution paths; Determine the test case automatic adjustment strategy based on the impact of code changes on test cases, and adjust test cases in order of priority according to the test case automatic adjustment strategy; Run the adjusted test case, obtain log data during the running process, extract error information from the log data, perform code mapping on the error information, and determine the error location.

2. A software automatic testing method as claimed in claim 1, characterized in that: The process of performing static analysis includes: parsing the changed code into an abstract syntax tree, traversing the abstract syntax tree, locating the function definition node, extracting the function name, parameter list and return type, finding the variable declaration node, extracting the variable type information, comparing the variable types before and after the change, and detecting whether there is a type change; generating a control flow graph based on the abstract syntax tree, analyzing the execution path of the code and the variable definition-usage relationship, comparing the information before and after the change, identifying changes in the variable definition or usage location, and changes in code blocks.

3. A software automatic testing method as claimed in claim 1, characterized in that: The process of dynamic analysis includes: inserting probes into the code when the code is compiled or run to collect information during code execution. The probes are inserted before and after function calls and at the entry and exit of conditional branches. The execution order of functions and the results of conditional branches are recorded by the probes. Based on the information collected by code instrumentation, a dynamic call chain is constructed to reflect the actual execution path of the code. The dynamic call chain before and after the change is compared to detect changes in the code execution path.

4. A software automatic testing method as claimed in claim 1, characterized in that: The process of evaluating the impact of code changes on test cases also includes statistically analyzing the code coverage of test cases, which includes statement coverage, branch coverage, and path coverage. Through coverage analysis, code paths that are not covered by test cases are identified.

5. A software automatic testing method as claimed in claim 1, characterized in that: Based on the evaluation results of the impact of code changes on test cases, the process of determining the automatic adjustment strategy of test cases includes: adjusting the expected result type of the test case according to the changes in function signatures or variable types in the static analysis results; adding test scenarios to cover the newly added boundaries according to the code boundary changes in the static analysis results; deleting related test cases that are no longer executed and adding new test cases to cover the new code paths according to the code path changes in the dynamic analysis results.

6. A software automatic testing method as claimed in claim 1, characterized in that: The process of adjusting test cases in sequence according to the priority and the test case automatic adjustment strategy includes: assigning higher priority to test cases that are more affected by the change and processing high-priority test cases first; For situations where the change situation is complex or uncertain, the user's input or selection is received through the interactive interface to adjust the test case.

7. A software automatic testing method as claimed in claim 1, characterized in that: The process of running the adjusted test case, obtaining log data during the running process, and extracting error information from the log data includes: preprocessing the obtained log data, including log cleaning, format conversion and feature extraction, using a pre-trained model to identify the error type and error location in the log data, classifying the identified error information into predefined error types, and using a large model to establish a mapping relationship between the natural language description in the error information and the elements in the code, analyzing the keywords in the error information through the large model, and matching possible code elements in the code library according to the mapping relationship.

8. A software automatic testing system, characterized in that: include: The large model training module is configured to obtain historical test cases and related data of various types of software codes, preprocess the obtained data, extract key features, use the key features to train the large model, and adjust and optimize the parameters of the large model; The test case generation module is configured to obtain the target code, extract relevant information and test requirements, and use the trained large model to generate test cases based on the target code and test requirements; A preliminary check module is configured to perform syntax check and logic consistency check on the generated test cases; a change analysis module configured to respond to code change information, evaluate the scope of impact of code changes on test cases, perform static analysis to determine changes in code structure, and perform dynamic analysis to determine changes in code execution paths; The intelligent maintenance module is configured to determine a test case automatic adjustment strategy based on an evaluation result of the impact of the code change on the test case, and adjust the test cases in sequence according to the test case automatic adjustment strategy based on the priority; The intelligent analysis module is configured to run the adjusted test case, obtain log data during the running process, extract error information in the log data, perform code mapping on the error information, and determine the error location.

9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps of the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The method comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein the steps of the method according to any one of claims 1 to 7 are completed when the computer instructions are executed by the processor.

Images