Software testing method and device, medium and computer program product

Abstract

The invention discloses a software testing method and device, a medium and a computer program product, and relates to the technical field of software testing, a test case is generated according to input test requirement information for tested software, a code analysis tool is utilized to analyze the complexity score of the test case, and the complexity score of the test case is analyzed; and calling the first agent to decompose the test case into a plurality of sub-test tasks according to the complexity score, and calling a plurality of second agents to correspondingly execute the sub-test tasks so as to comprehensively cover the test scene, in the test, the second intelligent agent corresponding to the associated software module is triggered to repeatedly test the associated software module according to the configuration information change of the target software module of the tested software and the dependency graph between the software modules, so that the system change is automatically adapted, the overall repeated test is reduced, and hidden problems are found; the problems of incomplete test coverage, difficulty in discovering hidden problems and low efficiency in related technologies are solved, and the test efficiency and reliability of the management operation and maintenance software of the data center server are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular, to a software testing method, device, medium, and computer program product. Background Art

[0002] With the continuous expansion of the scale of data center server management and operation and maintenance software, its functional requirements and complexity are also continuously increasing. However, there are many problems in the current testing methods for data center server management and operation and maintenance software. For example, testing tools and processes often lag behind the rapidly changing business requirements, resulting in incomplete test coverage or low efficiency. Some potential problems are difficult to expose before the software goes online, and a large amount of manual operation is still required during the testing process, which not only increases the time cost but also easily introduces human errors.

[0003] How to improve the reliability of the test results of data center server management and operation and maintenance software is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The present invention provides a software testing method, device, medium, and computer program product to at least solve the problem of low reliability of test results of data center server management and operation and maintenance software in related technologies.

[0005] The present invention provides a software testing method, including: Generating test cases according to the input test requirement information of the software to be tested; Analyzing with a code analysis tool to obtain the complexity score of the test cases; Invoking a first intelligent agent to decompose the test cases into multiple sub-test tasks according to the complexity score; Invoking multiple second intelligent agents to correspondingly execute the sub-test tasks. During the execution process, if there is a change in the configuration information of the target software module of the software to be tested, then according to the dependency graph between the software modules of the software to be tested, triggering the second intelligent agent corresponding to the associated software module of the target software module to perform repeated testing on the associated software module.

[0006] The present invention also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of any of the above software testing methods when executing the computer program.

[0007] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above software testing methods.

[0008] The present invention also provides a computer program product, including a computer program which, when executed by a processor, implements the steps of any of the above software testing methods.

[0009] Through the present invention, a generative method is adopted to generate test cases according to the input test requirement information of the software to be tested, reducing the time and labor costs of manually writing test cases and the probability of introducing human errors; a code analysis tool is used to analyze and obtain the complexity score of the test cases, and a first intelligent agent is called to decompose the test cases into multiple sub-test tasks according to the complexity score, and multiple second intelligent agents are called to correspondingly execute the sub-test tasks, so as to comprehensively cover the test scenarios in a real and complex environment through the collaborative work of multiple intelligent agents. By triggering the second intelligent agent corresponding to the associated software module to perform repeated testing on the associated software module according to the configuration information change of the target software module of the software to be tested and the dependency graph between software modules during the test, automatic adaptation to the changes in the system where the software to be tested is located is realized, overall repeated testing is reduced and hidden problems are discovered, and the problems of incomplete test coverage, difficulty in discovering hidden problems, and low test efficiency in the related art are solved, thereby improving the test efficiency and reliability of the data center server management and operation and maintenance software. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0011] Figure 1 It is a flowchart of a software testing method provided by an embodiment of the present invention; Figure 2 It is a flowchart of a model training method provided by an embodiment of the present invention; Figure 3 It is a flowchart of another software testing method provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present invention.

[0013] It should be noted that in the description of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present invention are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0014] To enable those skilled in the art of this technology to better understand the solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0015] Some key terms used in the embodiments of the present invention will be explained here first.

[0016] Pre-training is a strategy for training deep learning models. Its core lies in using a large-scale dataset to preliminarily train the model so that the model can learn general feature representations. This process is similar to the basic learning stage of humans before learning new knowledge, where they accumulate experience through extensive reading and observation.

[0017] A pre-trained language model (PLM) generally refers to a large-scale neural network algorithm structure and parameters obtained by designing language model training tasks and training a large-scale neural network algorithm structure based on a large-scale corpus (including language training materials such as sentences, paragraphs, etc.). For subsequent other tasks, feature extraction or task fine-tuning can be performed on the basis of this model to achieve specific task purposes. The idea of pre-training is to first train a task to obtain a set of model parameters, then use this set of model parameters to initialize the network model parameters, and then use the initialized network model to train other tasks to obtain models adapted to other tasks. By pre-training on a large-scale corpus, the neural language representation model can learn powerful language representation capabilities and extract rich syntactic and semantic information from the text. The pre-trained language model can provide word elements (tokens) containing rich semantic information and sentence-level features for downstream tasks, and can also directly perform fine-tuning for downstream tasks on the pre-trained model to conveniently and quickly obtain downstream-specific models.

[0018] The neural network algorithm structure for pre-training a language model can be a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Long Short-Term Memory (LSTM) network, etc., or it can also be a model constructed with an attention network, such as a Transformer model, a Bidirectional Encoder Representations from Transformers (BERT) model, a Contrastive Language-Image Pre-training (CLIP) model, etc. The present invention does not limit this here. An attention network refers to a network model that uses an attention mechanism for training. This model assigns different weights to each part of the input sequence, thereby extracting more important feature information from the input sequence, enabling the model to finally obtain a more accurate output. II. Fine-tuning Definition: Fine-tuning refers to further training on a dataset for a specific task on the basis of using a pre-trained model to adjust the model parameters so that it better adapts to the target task. During the fine-tuning process, most layers of the pre-trained model are usually frozen, and only the newly added layers are trained or a small number of key layers are adjusted. This can not only retain the features learned by the pre-trained model but also quickly adapt to the specific requirements of the new task. In addition, choosing appropriate learning rates and the number of training epochs is also the key to successful fine-tuning.

[0019] Fine-tuning refers to small-scale training on the basis of a pre-trained model for a specific task objective (downstream task) and task data (downstream data) to achieve minor adjustments to the parameters of the pre-trained model, ultimately obtaining a model adapted to the specific task and data.

[0020] The data center server management and operation and maintenance software usually runs on the management node server of the data center and the baseboard management controller (BMC) or other management controllers of other servers. It usually has functional modules such as monitoring management, asset management, capacity management, online / offline management, wiring management, inspection management, operation and maintenance management, security control, energy efficiency management, visualization management, automated operation and maintenance, and work order management.

[0021] Among them, the monitoring and management module is used for real-time monitoring of the server (real-time collection and processing of operation status data of the server, network devices, storage devices, etc., including central processor utilization rate, memory usage, storage space utilization rate, etc.), environment monitoring (monitoring the computer room environment, such as temperature, humidity, water immersion, access control, etc., to ensure the stability of the physical environment of the data center), traffic monitoring (analyzing network traffic, identifying potential bottlenecks, and optimizing bandwidth usage), etc.

[0022] The asset management module is used for life cycle management of the server (full life cycle management from equipment warehousing to scrapping, providing basic data for capacity analysis, maintenance, repair, etc.), asset classification and query (classifying and managing assets, supporting asset query and statistics), etc.

[0023] The capacity management module is used for resource planning of the server (managing resources such as space, power, refrigeration, network, etc. in the data center to prevent overloading), capacity analysis (providing display of current capacity and capacity trend analysis), etc.

[0024] The online / offline management module is used for resource allocation and recovery of the server (allocating resources such as cabinets, U positions, network cables, etc. to the server when going online), and recycling resources for reuse when going offline, etc.

[0025] The wiring management module is used for network planning of the server, including managing the path from the server to the switch, and supporting graphical interface for wiring planning.

[0026] The patrol management module is used for performing automated patrols (supporting technologies such as positioning and QR code recognition, and performing patrol tasks online or offline through mobile devices), patrol plans (formulating daily patrol plans, recording patrol results, and supporting preventive maintenance), etc.

[0027] The operation and maintenance management module is used for task management of the server (batch issuing of policies such as configurations and passwords, supporting immediate, timed, and periodic triggering of tasks), alarm management (when the device triggers an alarm rule, notifying operation and maintenance personnel through multiple channels (such as emails, text messages, WeChat)), fault handling (supporting processes such as fault diagnosis, emergency handling, and maintenance application), etc.

[0028] The security control module is used for vulnerability management of the server (supporting active vulnerability protection and unified management of multiple vulnerabilities), permission control (managing user permissions to ensure that operations comply with security specifications), etc.

[0029] The energy efficiency management module is used for energy consumption monitoring of the server (real-time monitoring of the energy consumption of the device and analyzing the energy consumption trend), energy saving optimization (reducing the energy consumption of the data center through energy consumption alarms and optimization suggestions), etc.

[0030] The visualization management module is used for the three-dimensional visualization function of the server (such as providing an intuitive 3D computer room view to help operation and maintenance personnel quickly locate problems), and the data visualization function (displaying real-time monitoring data, alarm information, and statistical reports through graphs and charts).

[0031] The automated operation and maintenance module is used for the automated monitoring of the server (reducing manual operations through automatic monitoring, anomaly detection, and problem diagnosis) and intelligent diagnosis, etc.

[0032] The work order management module is used for event management, change management, and problem management based on specifications, and is associated with the configuration management module.

[0033] The work order processing module is used to support the creation, approval, execution, and recording of work orders.

[0034] These functions enable the data center server management and operation and maintenance software to comprehensively cover the daily operation and maintenance needs of the data center, improving management efficiency and system stability.

[0035] It can be seen that the software scale of the data center server management and operation and maintenance software and the hardware scale involved in the deployment are continuously expanding, the functional requirements are gradually increasing, the functional complexity is continuously rising, and there are associations and influences among the software modules. The traditional testing scheme for testing individual software modules separately has problems of incomplete coverage and low efficiency, and it is difficult to discover some potential problems.

[0036] Therefore, for the software testing scheme provided by the embodiments of the present invention, first, a generative method is used to generate test cases according to the input test requirement information of the software to be tested, reducing the time and labor costs of manually writing test cases and the probability of introducing human errors; a code analysis tool is used to analyze and obtain the complexity score of the test cases, and a first intelligent agent is called to decompose the test cases into multiple sub-test tasks according to the complexity score, and multiple second intelligent agents are called to execute the sub-test tasks correspondingly, so as to comprehensively cover the test scenarios in a real and complex environment through the collaborative work of multiple intelligent agents. By triggering the second intelligent agent corresponding to the associated software module to repeat the test of the associated software module according to the configuration information change of the target software module of the software to be tested and the dependency graph among the software modules during the test, automatic adaptation to the changes of the system where the software to be tested is located is realized, reducing overall repeated testing and discovering hidden problems, solving the problems of incomplete test coverage, difficulty in discovering hidden problems, and low test efficiency in the related technology, thereby improving the test efficiency and reliability of the data center server management and operation and maintenance software.

[0037] Embodiments of the present invention provide a software testing method. Combining with the execution process of the software testing method, the method will be described in detail below.

[0038] Figure 1The flowchart of a software testing method provided by an embodiment of the present invention.

[0039] As Figure 1 shown, the software testing method provided by an embodiment of the present invention includes:

[0040] S101: Generate test cases according to the input test requirement information of the software under test.

[0041] S102: Analyze using a code analysis tool to obtain the complexity score of the test cases.

[0042] S103: Call the first intelligent agent to decompose the test cases into multiple sub-test tasks according to the complexity score.

[0043] S104: Call multiple second intelligent agents to correspondingly execute the sub-test tasks. During the execution process, if there is a change in the configuration information of the target software module of the software under test, then according to the dependency graph between the software modules of the software under test, trigger the second intelligent agent corresponding to the associated software module of the target software module to perform repeated testing on the associated software module.

[0044] The software testing method provided by an embodiment of the present invention can be deployed on a computing device (such as a server or a computer), or can be deployed on a computing cluster composed of multiple computing devices. In some optional implementation manners of the embodiment of the present invention, if it is deployed on a computing device, this computing device serves as a test host and is communicatively connected to the server or server cluster on which the software under test is deployed, and multiple intelligent agents are deployed through multi-threading on this computing device to implement software automated testing based on models and intelligent agents. If it is deployed on a computing cluster, different intelligent agents can also be deployed on different computing devices.

[0045] In specific implementation, for S101, the present invention provides a generation method to generate test cases, reducing the manual writing cost. Specifically, generating test cases according to the input test requirement information of the software under test in S101 may include: using a first pre-trained model to generate test cases according to the test requirement information.

[0046] As introduced in the above embodiment of the present invention, the first pre-trained model can adopt a pre-trained language model, and the ability of the first pre-trained model to generate test cases according to test requirements can be trained through the historical test cases of the software for managing and operating and maintaining the data center server.

[0047] For S102, since the scale of the software under test such as the software for managing and operating and maintaining the data center server is large, the test process that the test cases need to execute is long, and if executed as a whole, potential problems therein may be ignored. Therefore, the embodiment of the present invention decomposes the test cases into multiple sub-test tasks.

[0048] In the embodiments of the present invention, test case decomposition is performed based on the code complexity of test cases, and the complexity score of the test cases is obtained by analyzing using a code analysis tool. Specifically, a static code analysis tool can be used to calculate the complexity score of the test cases.

[0049] In some alternative embodiments of the embodiments of the present invention, obtaining the complexity score of the test cases by analyzing using a code analysis tool may include: evaluating the function nesting depth score of the test cases and the number of decision points of the test cases using the code analysis tool; calculating the complexity score based on the function nesting depth score and the number of decision points. Thus, for the high-complexity code parts, a finer-grained test process is generated to ensure test coverage, so that the key parts are fully tested, thereby reducing the later maintenance cost.

[0050] For S103, an agent refers to an entity that can perceive the environment and take actions to achieve specific goals. It can be software, hardware, or a system, and has autonomy, adaptability, and interaction capabilities. In the embodiments of the present invention, with a model as the base, through multi-agent collaboration, test scenarios are comprehensively covered in a real and complex environment, the system changes are automatically adapted, resources and response time are monitored in real time, and the verification of the normal flow and the abnormal flow is concerned, which not only improves the test efficiency and coverage rate, but also can effectively identify hidden defects, reduce the workload of repeated tests, and thus comprehensively optimize the test quality.

[0051] In the embodiments of the present invention, based on the code complexity of the test cases, the first agent (which can be denoted as the decomposition Agent) is guided to decompose the test cases into multiple sub-test tasks, realizing adaptive test case decomposition that adapts to system changes.

[0052] For S104, in the face of complex test scenarios, the embodiments of the present invention use multi-agent collaborative testing. By calling multiple second agents to execute different sub-test tasks respectively, the second agents can use automated test tools such as Selenium and Junit to perform specific test work and output test results.

[0053] It can be understood that there is a corresponding relationship between the sub-test tasks and the software modules of the software under test, and there are dependency relationships between different software modules, and this dependency relationship can be represented by a dependency graph. In the embodiments of the present invention, when the second agent executes the sub-test task, if it is found that the configuration information of the software module it is responsible for has changed or the second agent has changed the configuration information of the software module it is responsible for, at this time, the second agent determines the associated software module according to the dependency graph and triggers the corresponding second associated software module to perform repeated tests.

[0054] In an embodiment of the present invention, if there is a change in the configuration information of the target software module of the software under test in S104, triggering the second agent corresponding to the associated software module of the target software module to perform repeated testing on the associated software module according to the dependency graph between the software modules of the software under test may include: deploying the dependency graph to the second agent, so that after changing the configuration information of the target software module based on the second agent, determining the associated software module and the associated second agent corresponding to the associated software module according to the dependency graph, and triggering the associated second agent to execute repeated testing on the associated software module based on the second agent.

[0055] In practical applications, if the second agent 1 is responsible for the software module X and makes changes to it, it notifies the second agent 2 responsible for the software module Y because the software module Y depends on the software module X. The notified second agent 2 will then perform targeted testing on the software module Y to ensure compatibility and proper functionality with the updated software module X. This collaborative approach not only reduces repeated testing but also uncovers hidden issues that are not easily detected when testing individual modules separately, such as functional anomalies caused by interactions between modules.

[0056] The software testing method provided by the embodiments of the present invention generates test cases according to the input test requirement information of the software under test by adopting a generative method, reducing the time and labor costs of manually writing test cases and the probability of introducing human errors; analyzing the complexity score of the test cases by using a code analysis tool, calling the first agent to decompose the test cases into multiple sub-test tasks according to the complexity score, and calling multiple second agents to execute the sub-test tasks correspondingly, so as to comprehensively cover the test scenarios in a real and complex environment through the collaborative work of multiple agents. By triggering the second agent corresponding to the associated software module to perform repeated testing on the associated software module according to the change in the configuration information of the target software module of the software under test and the dependency graph between the software modules during the test, it can comprehensively cover the test scenarios in a real and complex environment, automatically adapt to system changes, not only improve the test efficiency and coverage rate, but also effectively identify hidden vulnerabilities, reduce the workload of repeated testing, and thus improve the test efficiency and reliability of the data center server management and operation and maintenance software.

[0057] Based on the above embodiments, the embodiments of the present invention further illustrate the generation steps of the test cases.

[0058] As introduced in the above embodiments, a first pre-trained model can be used to generate test cases according to the input test requirement information. On this basis, the embodiments of the present invention provide a solution for generating test cases by using a retrieval-augmented generation method.

[0059] In some alternative embodiments of the embodiments of the present invention, using a first pre-trained model to generate test cases according to test requirement information may include: retrieving, from a first knowledge base according to the test requirement information, associated test case samples corresponding to multiple expert test case sample classes; and using the first pre-trained model to generate test cases corresponding to different expert test case sample classes according to the test requirement information and the associated test case samples.

[0060] Among them, the first knowledge base may include a requirement knowledge base and a test case knowledge base. The requirement knowledge base is used to store development knowledge bases such as requirement documents. The test case knowledge base is used to store previous test cases.

[0061] By performing data preprocessing and vectorization processing on historical test data (including historical test requirement information and historical test cases), and storing the generated vectors into a vector database (such as Pinecone, FAISS, etc.), a first knowledge base is constructed. Among them, the vectorization processing can be performed using an embedding model. The embedding model can be a pre-trained Transformer model or a model optimized specifically for search.

[0062] By widely collecting historical test requirements and historical test cases, the background, experience, and thinking modes of different experts can be utilized, enabling the first pre-trained model to generate diverse test cases. For example, some testers may be more proficient in designing boundary condition cases, while others may tend to construct complex multi-step test scenarios. Such diverse inputs provide rich training materials for the model. At the same time, modifying the test cases can further enhance their differences, such as by adjusting the expression, input conditions, or expected results, eliminating duplicate content, and improving data quality. This not only avoids the overfitting problem caused by single data for the model but also improves its generalization ability, enabling it to better adapt to changing requirements and actual scenarios. A model trained with diverse data can handle complex user requirements more flexibly and exhibit higher robustness and practicality in different application environments.

[0063] Therefore, in the embodiments of the present invention, when using the first pre-trained model to generate test cases, retrieve, from the first knowledge base according to the test requirement information, associated test case samples corresponding to multiple expert test case sample classes, and generate a prompt by using the test requirement information, the associated test case samples, and a generation instruction for generating multiple test cases corresponding to different expert test case sample classes, and input the prompt into the first pre-trained model to generate diverse test cases.

[0064] In some other alternative embodiments of the embodiments of the present invention, using the first pre-trained model to generate test cases corresponding to different expert test case sample classes according to the test requirement information and associated test case samples may include: retrieving a corresponding associated code annotation file from the second knowledge base according to the test requirement information and associated test case samples; generating a first prompt word according to the test requirement information, associated test case samples and associated code annotation files; inputting the first prompt word into the first pre-trained model so that the first pre-trained model outputs multiple test cases corresponding to different expert test case sample classes.

[0065] Test cases are described by code, and code annotation information is the annotation and explanation of the code. In addition to establishing the first database of requirements - test cases, the embodiments of the present invention can further enhance the ability of the first pre-trained model to generate test cases by constructing the second database of code - annotations.

[0066] In the embodiments of the present invention, inputting the first prompt word into the first pre-trained model so that the first pre-trained model outputs multiple test cases corresponding to different expert test case sample classes may include: inputting the first prompt word into the first pre-trained model so that the first pre-trained model outputs multiple test cases corresponding to different expert test case sample classes and the annotation information of the test cases. By allowing the first pre-trained model to generate test cases and their annotation information according to the test requirement information, associated test case samples and associated code annotation files, it is convenient for testers to check.

[0067] In the embodiments of the present invention, inputting the first prompt word into the first pre-trained model so that the first pre-trained model outputs multiple test cases corresponding to different expert test case sample classes may also include: inputting the first prompt word into the first pre-trained model so that the first pre-trained model outputs multiple test cases corresponding to different expert test case sample classes and the inference content for generating the test cases. In addition to using the first pre-trained model to generate test cases and their annotation information, by setting the prompt word to make the first pre-trained model output the inference content, the first pre-trained model can give the inference basis while generating test cases and their annotation information according to the test requirement information, associated test case samples and associated code annotation files, and guide the first pre-trained model to generate test cases more accurately through the way of the chain of thought.

[0068] In the embodiments of the present invention, the training steps of the first pre-trained model may include: sampling test case samples from historical software test data; inputting the test requirement samples of the test case samples into a reinforcement learning model to obtain a first test case; calculating a reward score according to the first test case and the test case samples; updating the parameters of the reinforcement learning model according to the reward score until the iteration end condition is reached to obtain the first pre-trained model.

[0069] Among them, the historical software test data can be sourced from the above-mentioned first knowledge base and / or second knowledge base.

[0070] Figure 2 It is a flowchart of a model training method provided by an embodiment of the present invention.

[0071] As Figure 2 shown, input the test requirement sample into the reinforcement learning model, output the first test case, and compare the first test case with the test case sample.

[0072] In an embodiment of the present invention, calculating the reward score according to the first test case and the test case sample may include: calculating the similarity score of the first test case and the test case sample using the second pre-trained model; calculating the text n-gram matching degree score of the first test case and the test case sample; calculating the text overlap degree score of the first test case and the test case sample; calculating the reward score according to the similarity score, the text n-gram matching degree score, and the text overlap degree score.

[0073] In the related art, usually only text similarity, semantic similarity, etc. are used for the training evaluation of the pre-trained language model, but this method has limitations and cannot effectively express the semantic situation of sentences.

[0074] As Figure 2 shown, an embodiment of the present invention evaluates the similarity between the first test case and the test case sample by combining the text n-gram matching degree (Bilingual Evaluation Understudy, BLEU) score, the text overlap degree (Recall-Oriented Understudy for Gisting Evaluation, ROUGE) score, and the similarity score, thereby improving the reliability of the evaluation. And the similarity score is calculated by the second pre-trained model, and the second pre-trained model can adopt a pre-trained language model or a semantic similarity model, so as to obtain a reference similarity instead of a relative comparison based on human preference.

[0075] Then the training process of the first pre-trained model is as Figure 2 shown, input the test requirement sample into the reinforcement learning model, and output the first test case. Use the second pre-trained model to generate the similarity score S of the first test case and the test case sample. Calculate the BLEU and ROUGE values of the first test case and the test case sample.

[0076] In embodiments of the present invention, calculating a reward score based on a similarity score, a text n-gram matching degree score, and a text overlap degree score may include: performing a weighted calculation based on the similarity score, the text n-gram matching degree score, and the text overlap degree score to obtain the reward score; the weight coefficients of the similarity score, the text n-gram matching degree score, and the text overlap degree score are updated as the reinforcement learning model is trained.

[0077] Then the above process can be represented by the following formula: 。

[0078] Wherein, represents the reward score, represents the weight coefficient of the text n-gram matching degree score, represents the text n-gram matching degree score, represents the weight coefficient of the text overlap degree score, represents the text overlap degree score, represents the similarity score between the first test case and the test case sample.

[0079] During the training process of the reinforcement learning model, adjust and values, so as to find the optimal parameter combination to further improve the performance of the model. BLEU evaluates the n-gram matching degree between the generated text and the reference text, reflecting the accuracy of the generated text. ROUGE evaluates the quality of the generated text by calculating the overlap degree between the generated text and the reference text. This reward function combines three different evaluation metrics and can more comprehensively evaluate the quality of the generated test cases.

[0080] Use the reward score R to train the reinforcement learning model. After multiple training iterations, when the preset number of iterations is reached or the reward score meets the preset score threshold, output the reinforcement learning model as the first pre-trained model.

[0081] Based on the above embodiments, embodiments of the present invention further illustrate the automated testing process of the software under test.

[0082] Figure 3 is a flowchart of another software testing method provided by embodiments of the present invention.

[0083] The software testing method provided by the embodiment of the present invention may further include: during the execution of the sub-test task, invoking the third intelligent agent to monitor the running state information of the software module and the running state information of the second intelligent agent, and triggering the fourth intelligent agent to execute the corresponding exception handling operation after identifying the abnormal information therein; the types of the exception handling operation at least include restarting the second intelligent agent corresponding to the abnormal software module or replacing the second intelligent agent corresponding to the abnormal software module.

[0084] In the embodiment of the present invention, the intelligent agent can also be used to record the vulnerabilities and repair results in the software testing process, predict the version differences and chain effects through the pre-trained model, and the multiple intelligent agents are divided into different tasks to monitor and repair the software module and related modules, reducing repeated testing and timely discovering hidden problems.

[0085] Specifically, the third intelligent agent (which can be denoted as the monitoring Agent) can collect the running data in the software testing process in real time, including the running state information of the software modules of the software under test (such as resource consumption, response time) and the running state information of the second intelligent agent, analyze the abnormal trend through the pre-trained model and give an early warning to improve the system reliability.

[0086] Furthermore, the third intelligent agent reports the abnormal information to the fourth intelligent agent (which can be denoted as the management Agent), and the fourth intelligent agent is responsible for managing resources and coordinating the interaction between the second intelligent agents, realizing load balancing and exception handling. When the second intelligent agent fails to execute the sub-test task, the fourth intelligent agent reassigns the task or restarts the second intelligent agent, and at the same time ensures the efficient communication between the intelligent agents, for example, exchanging data through the message queue.

[0087] The third intelligent agent can also implement the alarm function, report the abnormal information monitored in the testing process of the software under test to the tester, and trigger the fourth intelligent agent to re-schedule after the tester confirms. After receiving the confirmation from the tester, the fourth intelligent agent starts the repair process. After the repair is completed, it triggers the predefined automated test cases to verify whether the repair is effective and ensure that no new problems are introduced.

[0088] Based on the above embodiments, the software testing method provided by the embodiment of the present invention may further include: obtaining the test results output by each second intelligent agent; using the third pre-trained model to generate a test report of the software under test according to the test results.

[0089] In the embodiment of the present invention, using the third pre-trained model to generate a test report of the software under test according to the test results may include: retrieving the associated information from the third knowledge base according to the test results, and the associated information includes at least one of historical test data, known defect patterns, test coverage criteria, and relevant business logic description information; using the third pre-trained model to generate a test report according to the test results and the associated information.

[0090] As shown Figure 3 in the figure, the sixth intelligent agent can be used to generate a test report. The sixth intelligent agent collects the test results output by each second intelligent agent during the test execution. The test results may include test case ID, execution status (pass / fail), error log, execution time, etc.

[0091] The sixth intelligent agent can combine with the third knowledge base to generate a test report. The third knowledge base may include data such as historical test data, known defect patterns, test coverage criteria, and relevant business logic descriptions, etc., to help the sixth intelligent agent better analyze the current test results.

[0092] In addition, the first knowledge base, the second knowledge base, and the third knowledge base are updated regularly to ensure consistency with the latest test data and project requirements, which can be achieved through automated scripts or manual reviews.

[0093] In the embodiments of the present invention, the retrieval-enhanced generation technology is adopted to retrieve information related to the test results from the third knowledge base, such as historical defect patterns or the execution of similar test cases. This step ensures the context relevance of the report content. Combining the retrieved information and the current test results, the sixth intelligent agent uses a natural language generation model to generate the text content of the test report, by designing prompt words (such as "generate a defect analysis based on the following test results and historical data") to ensure that the generated content meets the expectations.

[0094] Specifically, the workflow of the sixth intelligent agent may include: report template, which pre-defines the structure of the test report, such as including parts like summary, overview of test results, defect analysis, test coverage evaluation, and improvement suggestions. Dynamic filling: The sixth intelligent agent automatically fills in the various parts of the report according to the test results and the content generated by retrieval enhancement. For example, calculate the test pass rate, list high-priority defects, etc.

[0095] Then as Figure 3 shown, the software testing method provided by the embodiments of the present invention realizes an efficient automated test process through a series of collaborative intelligent agents based on a pre-trained model.

[0096] When new test requirement information arrives, the fifth intelligent agent (which can be denoted as the generation Agent) uses the first pre-trained model to generate diverse test cases.

[0097] Next, the first intelligent agent decomposes these test cases into smaller parts according to the complexity of the code (such as cyclomatic complexity).

[0098] The fourth agent then coordinates these decomposed test cases and assigns them to the second agent (which can be denoted as the execution agent). The second agent uses tools such as Selenium or JUnit to run the tests in parallel.

[0099] Meanwhile, the third agent monitors performance metrics in real time (such as CPU usage, memory usage, and execution time), and notifies the fourth agent when anomalies are detected.

[0100] After the tests are completed, the sixth agent generates a detailed report, including key metrics such as test coverage and defect rate, and presents them in the form of charts to help users quickly evaluate the test quality.

[0101] Throughout the process, the fourth agent is responsible for resource management and coordination to ensure the efficient operation of the system.

[0102] The following are the functional descriptions of each agent:

[0103] The fifth agent (generation agent): When new test requirement information arrives, the fifth agent calls the above-mentioned first pre-trained model to generate diverse test cases.

[0104] The first agent (decomposition agent): According to the complexity of the test cases and the code, it decomposes them into several sub-test tasks. It calls a static code analysis tool to evaluate the complexity of the code (such as the depth of function nesting, the number of decision points) to guide the decomposition strategy. For highly complex code parts, it generates a more fine-grained test process to ensure test coverage, so that key parts are fully tested, thereby reducing the later maintenance cost.

[0105] The second agent (second agents 0, 1...n): After decomposition, it calls automation testing tools such as Selenium, JUnit, etc. to perform specific test work and feedback the results.

[0106] The third agent (monitoring agent): It monitors performance metrics in real time during the test execution process (such as CPU usage, memory usage, execution time), monitors not only the resource occupancy of the code, but also the performance of each agent itself, and timely discovers abnormal resource occupancy of the code and performance bottlenecks of the agents. This third agent can record logs, analyze the reasons for anomalies (such as memory leaks), and notify the fourth agent in time for intervention.

[0107] The sixth agent (report agent): After the tests are completed, it generates a detailed test report, analyzes key metrics such as test coverage and defect rate, and presents the results in a visual form (such as charts) to help users quickly evaluate the test quality.

[0108] Fourth Agent (Management Agent): The fourth agent plays a core role in the entire system, responsible for managing resources and coordinating the interactions among agents. It not only implements a load balancing algorithm to allocate tasks according to the load of agents and the complexity of the test segments, but also provides an error handling mechanism. For example, when the second agent fails, it reallocates tasks or restarts the second agent, and at the same time ensures efficient communication among the second agents, such as data exchange through a message queue.

[0109] In addition, this solution also establishes a feedback loop. After the test results are determined manually, they are fed back to the fifth agent and the first agent to optimize the test strategy, which can ensure the accuracy of the test and its applicability to different scenarios.

[0110] Through the description of the above implementation manners, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation manner.

[0111] An embodiment of the present invention also provides a software testing device, which may include: a generating unit for generating test cases according to the input test requirement information of the software to be tested; an analyzing unit for analyzing the complexity score of the test cases by using a code analysis tool; a decomposing unit for calling the first agent to decompose the test cases into multiple sub-test tasks according to the complexity score; a testing unit for calling multiple second agents to execute the sub-test tasks correspondingly. During the execution process, if there is a change in the configuration information of the target software module of the software to be tested, then according to the dependency graph between the software modules of the software to be tested, the second agent corresponding to the associated software module of the target software module is triggered to perform repeated testing on the associated software module.

[0112] In the embodiment of the present invention, the analyzing unit uses a code analysis tool to analyze the complexity score of the test cases, which may include: using the code analysis tool to evaluate the function nesting depth score of the test cases and the number of decision points of the test cases; calculating the complexity score according to the function nesting depth score and the number of decision points.

[0113] In the embodiment of the present invention, when there is a change in the configuration information of the target software module of the software to be tested, the testing unit triggers the second agent corresponding to the associated software module of the target software module to perform repeated testing on the associated software module according to the dependency graph between the software modules of the software to be tested, which may include: deploying the dependency graph to the second agent, so that after the configuration information of the target software module is changed based on the second agent, the associated software module and the associated second agent corresponding to the associated software module are determined according to the dependency graph, and the associated second agent is triggered based on the second agent to execute the repeated testing on the associated software module.

[0114] The software testing device provided by an embodiment of the present invention may further include: a monitoring unit, configured to, during the execution of a sub-test task, call the running state information of the third intelligent agent monitoring software module and the running state information of the second intelligent agent, and trigger the fourth intelligent agent to execute a corresponding exception handling operation after identifying the abnormal information therein; the types of the exception handling operations at least include restarting the second intelligent agent corresponding to the abnormal software module or replacing the second intelligent agent corresponding to the abnormal software module.

[0115] In an embodiment of the present invention, the generating unit generating test cases according to the input test requirement information of the software to be tested may include: generating test cases according to the test requirement information by using a first pre-trained model.

[0116] In an embodiment of the present invention, the generating unit generating test cases according to the test requirement information and associated test case samples by using a first pre-trained model may include: retrieving associated test case samples corresponding to multiple expert test case sample classes from a first knowledge base according to the test requirement information; generating test cases corresponding to different expert test case sample classes according to the test requirement information and the associated test case samples by using the first pre-trained model.

[0117] In an embodiment of the present invention, the generating unit generating test cases corresponding to different expert test case sample classes according to the test requirement information and the associated test case samples by using a first pre-trained model may include: retrieving a corresponding associated code annotation file from a second knowledge base according to the test requirement information and the associated test case samples; generating a first prompt word according to the test requirement information, the associated test case samples, and the associated code annotation file; inputting the first prompt word into the first pre-trained model so that the first pre-trained model outputs multiple test cases corresponding to different expert test case sample classes.

[0118] The software testing provided by an embodiment of the present invention may further include: a training unit, configured to sample test case samples from historical software test data; input the test requirement samples of the test case samples into a reinforcement learning model to obtain a first test case; calculate a reward score according to the first test case and the test case samples; update the parameters of the reinforcement learning model according to the reward score until an iteration end condition is reached to obtain a first pre-trained model.

[0119] In an embodiment of the present invention, the training unit calculating the reward score according to the first test case and the test case samples may include: calculating a similarity score between the first test case and the test case samples by using a second pre-trained model; calculating a text n-gram matching degree score between the first test case and the test case samples; calculating a text overlap degree score between the first test case and the test case samples; calculating the reward score according to the similarity score, the text n-gram matching degree score, and the text overlap degree score.

[0120] In an embodiment of the present invention, the training unit calculates a reward score based on the similarity score, the text n-gram matching score, and the text overlap score, which may include: performing weighted calculation based on the similarity score, the text n-gram matching score, and the text overlap score to obtain the reward score; the weight coefficients of the similarity score, the text n-gram matching score, and the text overlap score are updated as the reinforcement learning model is trained.

[0121] The software testing apparatus provided by the embodiment of the present invention may further include: a report generation unit, configured to obtain the test results output by each second agent; and generate a test report of the software under test by using a third pre-trained model according to the test results.

[0122] In an embodiment of the present invention, the report generation unit generates a test report of the software under test by using a third pre-trained model according to the test results, which may include: retrieving associated information from a third knowledge base according to the test results, where the associated information includes at least one of historical test data, known defect patterns, test coverage criteria, and relevant business logic description information; and generating a test report by using the third pre-trained model according to the test results and the associated information.

[0123] For the description of the features in the corresponding embodiment of the software testing apparatus, reference may be made to the relevant description in the corresponding embodiment of the software testing method, which will not be elaborated here.

[0124] An embodiment of the present invention further provides an electronic device, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above embodiments of the software testing method.

[0125] An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the steps in any one of the above embodiments of the software testing method when running.

[0126] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disc, and other media that can store computer programs.

[0127] An embodiment of the present invention further provides a computer program product, where the computer program product includes a computer program, and the computer program, when executed by a processor, implements the steps in any one of the above embodiments of the software testing method.

[0128] An embodiment of the present invention further provides another computer program product, including a non-volatile computer-readable storage medium storing a computer program, where the computer program, when executed by a processor, implements the steps in any of the above software testing method embodiments.

[0129] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0130] The above has introduced in detail a software testing method, device, medium, and computer program product provided by the present invention. Specific examples are used herein to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A software testing method, characterized in that, Including: Generating test cases according to the input test requirement information of the software under test; Analyzing the complexity score of the test cases by using a code analysis tool; Invoking a first intelligent agent to decompose the test cases into multiple sub-test tasks according to the complexity score; Invoking multiple second intelligent agents to execute the sub-test tasks correspondingly. During the execution process, if there is a change in the configuration information of the target software module of the software under test, then according to the dependency graph between the software modules of the software under test, triggering the second intelligent agent corresponding to the associated software module of the target software module to perform repeated testing on the associated software module.

2. The software testing method according to claim 1, wherein Analyzing the complexity score of the test cases by using a code analysis tool, including: Evaluating the function nesting depth score of the test cases and the number of decision points of the test cases by using the code analysis tool; Calculating the complexity score according to the function nesting depth score and the number of decision points.

3. The software testing method according to claim 1, characterized in that If there is a change in the configuration information of the target software module of the software under test, then according to the dependency graph between the software modules of the software under test, triggering the second intelligent agent corresponding to the associated software module of the target software module to perform repeated testing on the associated software module, including: Deploying the dependency graph to the second intelligent agent, so that after changing the configuration information of the target software module based on the second intelligent agent, determining the associated software module and the associated second intelligent agent corresponding to the associated software module according to the dependency graph, and triggering the associated second intelligent agent to perform repeated testing on the associated software module based on the second intelligent agent.

4. The software testing method according to claim 1, wherein Also including: During the execution process of the sub-test tasks, invoking a third intelligent agent to monitor the running state information of the software module and the running state information of the second intelligent agent, and triggering a fourth intelligent agent to perform corresponding exception handling operations after identifying the abnormal information therein; The types of the exception handling operations at least include restarting the second intelligent agent corresponding to the abnormal software module or replacing the second intelligent agent corresponding to the abnormal software module.

5. The software testing method according to claim 1, wherein Generating test cases according to the input test requirement information of the software under test, including: Generating the test cases by using a first pre-trained model according to the test requirement information.

6. The software testing method according to claim 5, wherein Generating the test cases by using a first pre-trained model according to the test requirement information, including: Retrieving associated test case samples corresponding to multiple expert test case sample classes from a first knowledge base according to the test requirement information; Generating the test cases corresponding to different expert test case sample classes by using the first pre-trained model according to the test requirement information and the associated test case samples.

7. The software testing method according to claim 5, characterized in that, Generating the test cases corresponding to different expert test case sample classes by using the first pre-trained model according to the test requirement information and the associated test case samples, including: Retrieving the corresponding associated code annotation file from a second knowledge base according to the test requirement information and the associated test case samples; Generating a first prompt word according to the test requirement information, the associated test case samples and the associated code annotation file; Input the first prompt into the first pre-trained model so that the first pre-trained model outputs multiple test cases corresponding to different expert test case sample classes.

8. The software testing method according to claim 5, wherein The training steps of the first pre-trained model include: Sampling test case samples from historical software test data; Inputting the test requirement samples of the test case samples into a reinforcement learning model to obtain a first test case; Calculating a reward score based on the first test case and the test case samples; Updating the parameters of the reinforcement learning model according to the reward score until the iteration end condition is reached to obtain the first pre-trained model.

9. The software testing method according to claim 8, wherein, Calculating a reward score based on the first test case and the test case samples includes: Calculating the similarity score between the first test case and the test case samples using a second pre-trained model; Calculating the text n-gram matching score between the first test case and the test case samples; Calculating the text overlap score between the first test case and the test case samples; Calculating the reward score based on the similarity score, the text n-gram matching score, and the text overlap score.

10. The software testing method according to claim 9, wherein Calculating the reward score based on the similarity score, the text n-gram matching score, and the text overlap score includes: Performing weighted calculation based on the similarity score, the text n-gram matching score, and the text overlap score to obtain the reward score; The weight coefficients of the similarity score, the text n-gram matching score, and the text overlap score are updated as the reinforcement learning model is trained.

11. The software testing method according to claim 1, wherein It also includes: Obtaining the test results output by each second agent; Generating a test report for the software under test using a third pre-trained model based on the test results.

12. The software testing method according to claim 11, wherein, Generating a test report for the software under test using a third pre-trained model based on the test results includes: Retrieving associated information from a third knowledge base according to the test results, where the associated information includes at least one of historical test data, known defect patterns, test coverage criteria, and relevant business logic description information; Generating the test report using the third pre-trained model based on the test results and the associated information.

13. An electronic device, characterized in that, It includes: A memory for storing a computer program; A processor for implementing the steps of the software test method according to any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, where the computer program, when executed by a processor, implements the steps of the software test method according to any one of claims 1 to 12.

15. A computer program product, comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the software test method according to any one of claims 1 to 12.

Images