Software testing method, device and equipment based on heterogeneous multi-agent and medium

Abstract

The invention relates to the technical field of testing, can be applied to the fields of financial science and technology and medical health, and discloses a software testing method, device, equipment and medium based on heterogeneous multi-agent, the method comprises the following steps: receiving a to-be-tested task, and analyzing the to-be-tested task to generate a task vector; for each heterogeneous agent in an agent pool, constructing a capability vector of the heterogeneous agent, and determining a capability matrix according to the capability vector and the task vector; generating a test workflow of the directed acyclic graph according to the task vector and the capability matrix; aiming at each task node in the test workflow, performing graph enhancement retrieval from a domain knowledge base to obtain a retrieval result, and injecting the retrieval result into a control instruction of the task node as enhancement knowledge; and calling a preset test tool to execute a test case generated by the test workflow. And the test quality and the test efficiency of the complex software system are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of testing, which can be applied to the fields of financial technology and medical health, and in particular relates to a software testing method, device and equipment based on heterogeneous multi-agent and a medium. BACKGROUND

[0002] With the increasing complexity of software systems, especially in the fields of financial technology and medical health, software testing faces great challenges. Traditional software testing methods mainly include manual testing and automated testing. Manual testing relies on the experience and skills of testers, although it is highly flexible, it has problems such as high cost, low efficiency, and easy to make mistakes, and it is difficult to meet the needs of modern software rapid iteration. Automated testing improves testing efficiency by writing test scripts to automatically execute test cases, but has problems such as maintenance difficulty and poor adaptability, especially when facing frequent changes in requirements and complex business logic, the maintenance cost of test scripts is often higher than the benefits it brings.

[0003] In recent years, with the development of artificial intelligence technology, intelligent testing methods based on large language models (LLM) have begun to attract attention. However, existing intelligent testing methods still have the following shortcomings: (1) only relying on a single general large language model to perform all testing tasks, resulting in poor performance on specific tasks, and generating test cases with uneven quality, resulting in low testing quality; (2) difficulty in understanding and using deep domain knowledge, resulting in test cases that cannot effectively cover core business logic and compliance requirements; (3) the collaboration process of multi-agent is often pre-set and static, with low collaboration efficiency; (4) most existing solutions use an open-loop "generate-execute" mode, which does not make full use of test execution results. SUMMARY

[0004] The present application provides a software testing method, device and equipment based on heterogeneous multi-agent to solve the technical problem of low testing quality and efficiency of existing complex software systems.

[0005] In a first aspect, a software testing method based on heterogeneous multi-agent is provided, comprising:

[0006] receiving a task to be tested and parsing the task to be tested to generate a task vector;

[0007] for each heterogeneous agent in the agent pool, constructing a capability vector of the heterogeneous agent, and determining a capability matrix according to the capability vector and the task vector;

[0008] generating a directed acyclic graph test workflow according to the task vector and the capability matrix;

[0009] For each task node in the test workflow, a retrieval result is obtained by performing graph-enhanced retrieval from a domain knowledge base, and the retrieval result is injected as enhanced knowledge into control instructions of the task node.

[0010] A preset test tool is invoked to execute a test case generated by the test workflow.

[0011] In a second aspect, a software testing device based on heterogeneous multi-agent is provided, comprising:

[0012] A receiving and parsing unit is configured to receive a to-be-tested task and parse the to-be-tested task to generate a task vector;

[0013] A construction determining unit is configured to construct a capability vector of each heterogeneous agent in an agent pool, and determine a capability matrix according to the capability vector and the task vector;

[0014] A generation unit is configured to generate a directed acyclic graph test workflow according to the task vector and the capability matrix;

[0015] A retrieval and injection unit is configured to, for each task node in the test workflow, obtain a retrieval result by performing graph-enhanced retrieval from a domain knowledge base, and inject the retrieval result as enhanced knowledge into control instructions of the task node.

[0016] A calling generation unit is configured to invoke a preset test tool to execute a test case generated by the test workflow.

[0017] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned software testing method based on heterogeneous multi-agent when executing the computer program.

[0018] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program implements the steps of the above-mentioned software testing method based on heterogeneous multi-agent when executed by a processor.

[0019] The scheme realized by the software testing method, device, equipment and medium based on the heterogeneous multi-agent can receive a to-be-tested task, analyze the to-be-tested task to generate a task vector, construct a capability vector of each heterogeneous agent in an agent pool, and determine a capability matrix according to the capability vector and the task vector, generate a directed acyclic graph test workflow according to the task vector and the capability matrix, perform graph-enhanced retrieval from a domain knowledge base to obtain a retrieval result for each task node in the test workflow, inject the retrieval result into control instructions of the task node as enhanced knowledge, and call a preset test tool to execute a test case generated by the test workflow. In the present application, the to-be-tested task is analyzed to generate a task vector, the capability vector of each heterogeneous agent in the agent pool is constructed, and the capability matrix is determined according to the capability vector and the task vector. Then, the test workflow is generated according to the task vector and the capability matrix, and enhanced knowledge retrieved from the domain knowledge base is injected into each task node in the test workflow. Finally, the test case generated by the test workflow is executed by calling the preset test tool to perform software testing on the complex software system, which not only understands and uses deep domain knowledge, effectively covers the core business logic and compliance requirements, but also improves the test quality and test efficiency of the complex software system. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0021] Figure 1 is a flowchart of a software testing method based on a heterogeneous multi-agent in an embodiment of the present application;

[0022] Figure 2 is a whole schematic diagram of a software testing system based on a heterogeneous multi-agent in an embodiment of the present application;

[0023] Figure 3 is Figure 1 is a specific embodiment flowchart of step S110 in

[0024] Figure 4 is Figure 1 is a specific embodiment flowchart of step S120 in

[0025] Figure 5 is Figure 1 is a specific embodiment flowchart of step S130 in

[0026] Figure 6 is a flowchart of a software testing method based on heterogeneous multi-agent in another embodiment of the present application;

[0027] Figure 7 is a schematic block diagram of a software testing device based on heterogeneous multi-agent in an embodiment of the present application;

[0028] Figure 8 is a structural schematic diagram of a computer device in an embodiment of the present application;

[0029] Figure 9 is another structural schematic diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0031] The software testing method based on heterogeneous multi-agent provided by the embodiments of the present application can be applied to a client or a server. The client can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The server can be implemented by an independent server or a server cluster composed of multiple servers. At present, in the fields of financial technology and medical health, the testing quality and testing efficiency of existing complex software systems are both low. In view of the above problems, the present application provides a software testing method based on heterogeneous multi-agent. The method first analyzes a to-be-tested task to generate a task vector, constructs a capability vector for each heterogeneous agent in an agent pool, and determines a capability matrix according to the capability vector and the task vector. Then, a test workflow is generated according to the task vector and the capability matrix, and enhanced knowledge retrieved from a domain knowledge base is injected into each task node in the test workflow. Finally, a test case generated by the test workflow is executed by calling a preset test tool to perform software testing of a complex software system. Not only deep domain knowledge is understood and used, but also the core business logic and compliance requirements are effectively covered, and the testing quality and testing efficiency of the complex software system are improved. The present application will be described in detail through specific embodiments.

[0032] Please refer to Figure 1 shown, Figure 1 is a flowchart of a software testing method based on heterogeneous multi-agent in another embodiment of the present application;

[0033] S110, receiving a to-be-tested task and parsing the to-be-tested task to generate a task vector.

[0034] Specifically, for the convenience of understanding, first, the software testing method based on heterogeneous multi-agent is applied to the software system based on heterogeneous multi-agent, and the software testing system based on heterogeneous multi-agent is as follows Figure 2 As shown, it includes a task parsing module, an ability modeling and storage module, a dynamic arrangement and optimization module, a graph enhancement knowledge base, an automated execution adaptation layer, a quality measurement and credit distribution module, and an audit and integration module. It should be noted that the task parsing module is used to parse the to-be-tested task to generate a task vector; the ability modeling and storage module is used to store and maintain an ability matrix composed of ability vectors of multiple heterogeneous agents; the dynamic arrangement and optimization module is used to generate a directed acyclic graph test workflow containing at least two heterogeneous agents based on the task vector and the ability matrix; the graph enhancement knowledge base contains a vector index library and a graph database, which is used to perform graph enhancement retrieval to provide enhanced knowledge; the automated execution adaptation layer is used to provide a unified tool calling interface and a sandbox execution environment; the quality measurement and credit distribution module is used to collect quality indicators and evaluate the contribution of heterogeneous agents to update the ability matrix and the strategy learning method; the audit and integration module is used to provide access control and operation log recording. The task parsing unit receives the original to-be-tested task (such as requirement document, user story, etc. natural language description), parses the to-be-tested task to generate a task vector, wherein the parsing includes preprocessing, encoding, classification and conversion processing.

[0035] As shown in Figure 3 In step S110, the following steps are included: S111-S113:

[0036] S111, preprocessing the to-be-tested task, and using a pre-trained Transformer language model to encode the to-be-tested task after preprocessing to obtain an encoding vector;

[0037] S112, using a pre-set classifier to classify the encoding vector to obtain a classification result;

[0038] S113, converting the classification result into a numerical task vector.

[0039] Specifically, the to-be-tested task is preprocessed, where the preprocessing is to perform word segmentation, stop word removal, and other processing on the text corresponding to the to-be-tested task, and to retain key semantic features. A pre-trained Transformer language model (such as BERT or GPT) is used to encode the preprocessed to-be-tested task to obtain an encoded vector. A preset classifier is used to classify the encoded vector to obtain a classification result, where the preset classifier includes a task type classifier, a complexity classifier, a priority classifier, and a risk domain classifier. The task type classifier identifies types such as E2E, unit, and compliance through a multi-classification model (such as Softmax) and outputs a test task scenario type. The complexity classifier outputs a complexity score of 0-1 (1 being the highest complexity) through a regression model. The priority classifier outputs a priority score of 0-1 (1 being the highest priority) through a regression model. The risk domain classifier labels domain keywords (such as “anti-money laundering” and “loan approval” in the financial technology field) through an entity recognition model, where the entity recognition model integrates named entity recognition (NER) technology and can accurately extract domain keywords. Finally, the classification result is converted into a numerical task vector. For ease of understanding, an example is given: for a to-be-tested task of “implementing end-to-end testing of the new loan approval process”, the generated task vector is V_task = [type: 'E2E', complexity: 0.8, priority: 0.9, risk_domain: 'CreditAudit'], where type represents the test task scenario type, complexity represents the complexity score, priority represents the priority score, and risk_domain represents the domain keyword. For a to-be-tested task of “verifying that the transfer system complies with anti-money laundering regulations”, the generated task vector is V_task = [type: 'Compliance', complexity: 0.9, risk_domain: 'AML'], and it can be understood that the complexity score and the priority score can be determined whether to be output according to actual needs.

[0040] S120, for each heterogeneous agent in the agent pool, constructing a capability vector of the heterogeneous agent, and determining a capability matrix according to the capability vector and the task vector.

[0041] Specifically, the agent pool is a schedulable resource set composed of multiple functional agents, and its core value lies in providing on-demand agent resources for dynamic orchestration units. The heterogeneous functional agent refers to an agent that has significant differences in at least one dimension of model architecture type, field expertise, tool stack, inference ability, and context processing, and solves complex test tasks through complementary cooperation. Through the implementation of the above steps S110-S120, precise and dynamic heterogeneous agent scheduling is achieved, that is, by introducing a computable capability matrix and a task vector, the fuzzy heterogeneous agent selection problem is transformed into a solvable optimization problem, precise matching of specific tasks and optimal heterogeneous agent combination is achieved, and test efficiency and test quality are significantly improved.

[0042] wherein, as shown in Figure 4 S120 includes the following steps: S121-S122:

[0043] S121, constructing the capability vector of the heterogeneous agent in multiple dimensions, wherein the multiple dimensions include model architecture type, field expertise, tool proficiency, inference ability, and context processing ability;

[0044] S122, all the capability vectors constitute an old capability matrix, and according to the old capability matrix and the capability performance vector corresponding to the last test task, the capability matrix is determined by an exponential moving average method.

[0045] Specifically, the model architecture type (model): a one-hot vector encoding represents the domain fine-tuned model; the domain expertise (domain): based on the accuracy score of the agent on the evaluation set in a specific domain (for example, the financial technology field, the medical health field); the reasoning ability (reasoning): the score on the standard logical reasoning or code generation benchmark test; the tool proficiency (tool): a vector representing the success rate and historical effect score of the call to different test tool APIs; the context processing ability (context): the normalized score of the effective context window length of the agent. The calculation formula of the ability vector is: V_agent=w1xF_model+w2xF_domain+w3xF_tool+w4xF_reasoning+w5xF_context, wherein w1, w2, w3, w4 and w5 are learnable weights, w1 is the weight of the model architecture type dimension, w2 is the weight of the domain expertise dimension, w3 is the weight of the tool proficiency dimension, w4 is the weight of the reasoning ability dimension, and w5 is the weight of the context processing ability dimension, F_model is the feature function of the model architecture type dimension, F_domain is the feature function of the domain expertise dimension, F_tool is the feature function of the tool proficiency dimension, F_reasoning is the feature function of the reasoning ability dimension, and F_context is the feature function of the context processing ability dimension. Understandably, the ability vector is the weighted sum of multiple dimensions such as model architecture type, domain expertise, tool proficiency, reasoning ability and context processing ability. It should be noted that the weight parameter is dynamically adjusted through closed-loop learning to determine the contribution ratio of each dimension to the total ability, w1 (model architecture type weight): control the influence intensity of model difference. For example, in the financial test task, if the fine-tuned model performs better, w1 is automatically increased (such as from 0.2 to 0.5), and the importance of model selection is strengthened; w2 (domain expertise weight): adjust the priority of domain knowledge. For example, in the anti-money laundering test, if the domain expertise contributes significantly to the result, w2 is learned and optimized from 0.3 to 0.6. w3 (tool proficiency weight): weight factor of tool call success rate. For example, when an agent frequently successfully calls a tool framework with test functions, the value of w3 is increased to reflect the value of the tool dimension. w4 (reasoning ability weight): emphasize the weight of logical reasoning. For example, when dealing with complex compliance rules, if the reasoning ability determines the success or failure of the task, the value of w4 is increased (for example, 0.4→0.7). w5 (context processing weight): priority of context length. For example, in long document testing, if the context ability is critical, the value of w5 is adjusted up to match the demand. It should be further noted that the ability matrix is dynamically updated using the exponential moving average method: V_agent_new=α×

[0046] V agent old + (1 - a) x V agent performance, wherein a is a smoothing factor, V agent performance is a capability performance vector corresponding to the last test task, and it can be understood that, assuming V agent new is the Nth capability matrix, V agent old is the old capability matrix, i.e., the (N-1)th capability matrix, and V agent performance is the (N-1)th capability performance vector, which will be described in detail below, for the sake of simplicity of description, details are not described herein.

[0047] S130, generating a directed acyclic graph test workflow according to the task vector and the capability matrix.

[0048] Specifically, the test workflow at least contains two cooperative heterogeneous functional agents, a directed acyclic graph (DAG) is a special graph structure composed of task nodes (Node) and directed edges (Directed Edge), and meets: 1, directedness: edge direction, indicating the dependency relationship between nodes (such as A→B indicating that B depends on A); 2, acyclicity: there is no cycle path, i.e., it is impossible to return to itself through directed edges from any node. By implementing the above step S130, the quality and robustness of the collaborative output are improved, that is, by forcing the test workflow to at least contain two heterogeneous agents and introducing an evaluation / arbitration agent mechanism, the advantages of multi-heterogeneous agent collaboration are fully utilized, and high-quality test results that cannot be achieved by a single heterogeneous agent are output.

[0049] As shown in FIG. 13, step S130 includes the following steps: S131-S135. Figure 5

[0050] S131, selecting an initial DAG template from a workflow template library according to the test task scene type in the task vector;

[0051] S132, for each task node in the initial DAG template, calculating the cosine similarity between the capability matrix and the requirement vector corresponding to the task node to obtain a similarity value;

[0052] S133, evaluating the adaptation degree of each of the heterogeneous agents to the task node according to the similarity value to obtain an adaptation degree score;

[0053] S134, using an integer linear programming method to model the allocation of the heterogeneous agents as a total utility function composed of the adaptation degree score under a preset constraint condition;

[0054] ​S135, exploring and optimizing the optimal agent allocation scheme generated when maximizing the total utility function using a policy learning algorithm to generate a directed acyclic graph test workflow.

[0055] Specifically, the preset constraints include resource constraints, dependency constraints, and quality constraints. According to the test task scenario type (such as financial compliance testing) identified in the task vector, an initial DAG template is intelligently matched from a pre-built workflow template library, for example, the anti-money laundering task automatically selects a three-node chain template of "rule analysis → case generation → execution assertion". Then, for each task node (such as the rule analysis node) in the initial DAG template, its complexity, risk domain, and other attributes are extracted to generate a demand vector, and the cosine similarity between all heterogeneous agent capability vectors in the computing power matrix and the demand vector (such as the similarity between the financial expertise agent and the rule analysis node is 0.92) is calculated to obtain an adaptation score. Based on this, an integer linear programming method is used for constraint optimization: binary variables are used to represent the heterogeneous agent-task node allocation relationship, the total adaptation score is maximized as the total utility function, and the optimal agent allocation scheme is generated under the constraints of resource constraints, dependency constraints, and quality constraints. Finally, the optimal agent allocation scheme is deeply optimized through a policy learning algorithm (such as the Monte Carlo tree search algorithm, the policy gradient algorithm, and the context multi-armed bandit algorithm): exploring topology structure adjustment (such as adding parallel branches) based on the initial allocation, estimating the utility improvement value (such as the total adaptation degree is improved by 20%) through simulation execution, and outputting the final DAG test workflow containing the binding relationship of heterogeneous agents and dynamic dependency edges, achieving the dual optimization goals of "accurate matching + flexible expansion".

[0056] S140, for each task node in the test workflow, performing graph-enhanced retrieval from a domain knowledge base to obtain retrieval results, and injecting the retrieval results as enhanced knowledge into the control instructions of the task node.

[0057] Specifically, for each task node in the test workflow, a retrieval result is obtained by performing graph-enhanced retrieval based on an inference method from the domain knowledge base, wherein the inference method includes a path inference method, a pattern inference method and a rule inference method, the domain knowledge base is a database including a graph database and a vector index library, the graph database is a database including a knowledge graph, and the retrieval result is injected into a control instruction of the task node as enhanced knowledge. It should be noted that the graph database is a database including a knowledge graph constructed by automatically extracting entities and relationships from domain documents using a named entity recognition and relationship extraction model; the path inference method is to find a path from a specific entity to a target task node through a specific relationship; the pattern inference method is to match a predefined graph pattern to identify potential risk points; and the rule inference method is to apply logical rules in a rule engine for multi-hop inference. It should be further noted that the control instruction refers to a prompt or a parameter, that is, the retrieval result is converted into a natural language description or a structured constraint and injected into the prompt or the parameter of the heterogeneous agent. By implementing the above step S140, the utilization of domain knowledge is deepened, a graph-enhanced mechanism is proposed, semantic retrieval and graph inference are combined, deep relationships and rules among knowledge can be mined, highly relevant and accurate context can be provided for the heterogeneous agent, model hallucination can be effectively inhibited, and the coverage depth for complex business and compliance scenarios is improved.

[0058] S150, invoking a preset test tool to execute the test case generated by the test workflow.

[0059] Specifically, by automatically executing the adaptation layer, the preset test tool is invoked in the isolated sandbox environment to execute the test case generated by the test workflow. It should be noted that during the execution of the test, the test data involving sensitive information is desensitized; the test is executed in a containerized sandbox environment, which is isolated from the production system; and a unified API interface is provided for different test tools to facilitate the invocation of different test tools. It should be further noted that by executing the above step S150, the enterprise-level security and compliance are ensured: the compliance and security are deeply integrated into the technical architecture, and through modules such as sandbox execution, data desensitization and audit traceability, the practical obstacles in the deployment of high-standard industries such as finance are solved.

[0060] Figure 6 A flowchart of a software testing method based on a heterogeneous multi-agent in another embodiment of the application is shown in FIG. 1. Figure 6 As shown in the figure, in the embodiment, the method includes steps S110-S170. That is, in the embodiment, the method further includes steps S160-S170 after step S150 of the above embodiment.

[0061] S160, collect the quality indicators after the test is completed, wherein the quality indicators include code coverage, defect retrieval rate, false positive rate, and execution delay;

[0062] S170, calculate a new capability performance vector of the to-be-tested task according to the quality indicators and the heterogeneous agents in the test workflow by a Shapley value algorithm, and update the capability matrix according to the new capability performance vector.

[0063] Specifically, the code coverage (for example, collected by a tool with a collection function, such as 85% -> 0.85); the defect detection rate (statistically counted by a defect tracking system, such as 1 real defect detected -> 1.0); the false positive rate (statistically counted by a human verification, such as the number of false positives / total number of reports -> 0.1); and the execution delay (clock difference value monitored by a tool with a monitoring function, such as 2.3 seconds -> take the reciprocal 0.43) are collected, and the new capability performance vector of the to-be-tested task is calculated by the Shapley value algorithm according to the collected code coverage, defect retrieval rate, false positive rate, execution delay, and heterogeneous agents in the test workflow, specifically including: randomly sorting the heterogeneous agents in the test workflow, adding the heterogeneous agents one by one in order for each sorting, and calculating the change value of the quality indicators before and after the addition; summarizing the change values in all sortings, calculating the average contribution of each heterogeneous agent to each quality indicator, and obtaining the contribution vector of each heterogeneous agent; performing weighted calculation on the contribution vector to obtain a weighted contribution vector, and combining the weighted contribution vector by dimension to generate the new capability performance vector. It should be noted that the new capability performance vector is used as a capability performance vector to update the capability matrix. It should also be noted that the strategy learning method can also be updated according to the new capability performance vector. Understandably, by implementing steps S160-S170, a closed-loop self-optimizing learning capability is constructed: through the closed-loop learning mechanism of "execution-metric-credit allocation-update", the system can continuously learn from historical test tasks, automatically optimize the scheduling strategy and heterogeneous agent capability cognition, and realize continuous iteration of performance, that is, the quality and efficiency of subsequent tests can be improved.

[0064] For the convenience of understanding, the implementation process of steps S110-S170 will be illustrated as follows: the to-be-tested task is to verify whether the newly online transfer system meets the anti-money laundering regulations; the execution process is as follows: step 1: the task analysis module receives the to-be-tested task "verify whether the newly online transfer system meets the anti-money laundering regulations", and generates a task vector: V_task = [type: 'Compliance', risk_domain: 'AML', complexity: 0.9]; step 2: the dynamic arrangement and optimization module identifies the "Compliance" and "AML" labels, selects the heterogeneous intelligent agents with high scores in the financial technology field and reasoning ability from the capability matrix, and generates a test workflow of DAG: [compliance analysis heterogeneous intelligent agent] → [negative case generation heterogeneous intelligent agent] → [execution and assertion heterogeneous intelligent agent]; step 3: the compliance analysis heterogeneous intelligent agent queries the graph-enhanced knowledge base, vector retrieval recalls "AML regulation documents", and the graph extension infers the specific rule "transactions over 200,000 RMB per day need to be reported" from the "large transaction" node; step 4: the rule is injected into the prompt of the negative case generation intelligent agent as a hard constraint; step 5: the negative case generation intelligent agent generates a test case: "user A transfers 5 times to user B within 8 hours, with a total amount of 201,000 RMB, and asserts that the system should generate a reporting record"; step 6: the automated execution adaptation layer executes the test case in a sandbox environment; step 7: the quality measurement and credit allocation module collects quality indicators: code coverage 85%, defect detection rate 1, false positive rate 0%, and execution time delay 2.3 seconds; step 8: using the Shapley value algorithm, high contribution is attributed to the graph reasoning step that provides key rules and the negative case generation intelligent agent that accurately generates the test case, and accordingly the capability performance vector of the intelligent agent in the anti-money laundering (AML) task is improved.

[0065] In the software testing method based on heterogeneous multi-agent in the application, the to-be-tested task is analyzed to generate a task vector, the capability vector of each heterogeneous intelligent agent in the intelligent agent pool is constructed, and the capability matrix is determined according to the capability vector and the task vector; then the test workflow is generated according to the task vector and the capability matrix, and the enhanced knowledge retrieved from the domain knowledge base is injected into each task node in the test workflow; finally, the test case generated by the test workflow is executed by calling the pre-set test tool, so as to perform software testing on the complex software system, not only the deep domain knowledge is understood and used, and the core business logic and compliance requirements are effectively covered, but also the test quality and test efficiency of the complex software system are improved.

[0066] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0067] The non-company software tools or components appearing in the embodiments of the present application are only illustrative and do not represent actual use.

[0068] In an embodiment, a software testing device 200 based on heterogeneous multi-agent is provided, which corresponds to the software testing method based on heterogeneous multi-agent in the above embodiments. As shown in the figure, the software testing device 200 based on heterogeneous multi-agent includes a receiving and analyzing unit 201, a building and determining unit 202, a generating unit 203, a retrieval and injection unit 204, and a calling and generating unit 205. The detailed description of each functional module is as follows: Figure 7

[0069] The receiving and analyzing unit 201 is configured to receive a to-be-tested task and analyze the to-be-tested task to generate a task vector;

[0070] The building and determining unit 202 is configured to build a capability vector of each heterogeneous agent in an agent pool and determine a capability matrix according to the capability vector and the task vector;

[0071] The generating unit 203 is configured to generate a directed acyclic graph test workflow according to the task vector and the capability matrix;

[0072] The retrieval and injection unit 204 is configured to, for each task node in the test workflow, perform graph-enhanced retrieval from a domain knowledge base to obtain a retrieval result, and inject the retrieval result as enhanced knowledge into a control instruction of the task node;

[0073] The calling and generating unit 205 is configured to call a preset test tool to execute a test case generated by the test workflow.

[0074] In an embodiment, the receiving and analyzing unit 201 is specifically configured to:

[0075] preprocess the to-be-tested task, and encode the preprocessed to-be-tested task using a pre-trained Transformer language model to obtain an encoding vector;

[0076] classify the encoding vector using a preset classifier to obtain a classification result;

[0077] convert the classification result into a numerical task vector.

[0078] In an embodiment, the building and determining unit 202 is specifically configured to:

[0079] ​constructing the capability vector of the heterogeneous intelligent agent, wherein the multi-dimension includes a model architecture type, a field expertise, a tool proficiency, an inference ability, and a context processing ability;

[0080] All the capability vectors constitute an old capability matrix, and according to the old capability matrix and a capability performance vector corresponding to a last test task, an exponential moving average method is used to determine the capability matrix.

[0081] In an embodiment, the generating unit 203 is specifically configured to:

[0082] selecting an initial DAG template from a workflow template library according to a test task scenario type in the task vector;

[0083] For each task node in the initial DAG template, a cosine similarity between the capability matrix and a requirement vector corresponding to the task node is calculated to obtain a similarity value;

[0084] According to the similarity value, an adaptation degree of each heterogeneous intelligent agent to the task node is evaluated to obtain an adaptation score;

[0085] An integer linear programming method is used to model the allocation of the heterogeneous intelligent agent as a total utility function composed of the adaptation scores under a preset constraint condition;

[0086] An optimal intelligent agent allocation scheme generated when the total utility function is maximized is explored and optimized by using a policy learning algorithm to generate a directed acyclic graph test workflow.

[0087] In an embodiment, the retrieval injection unit 204 is further configured to:

[0088] performing graph enhancement retrieval based on an inference method from the domain knowledge base to obtain a retrieval result, wherein the inference method includes a path inference method, a pattern inference method, and a rule inference method, and the domain knowledge base is a database including a graph database and a vector index library, and the graph database is a database including a knowledge graph.

[0089] In an embodiment, the software testing apparatus 200 based on heterogeneous multi-agent further includes:

[0090] The acquisition unit is configured to acquire a quality index after test completion, wherein the quality index includes a code coverage, a defect retrieval rate, a false positive rate, and an execution time delay;

[0091] The updating unit is configured to calculate a new capability performance vector of the to-be-tested task by using a Shapley value algorithm according to the quality index and the heterogeneous intelligent agent in the test workflow, and update the capability matrix according to the new capability performance vector.

[0092] In an embodiment, the updating unit is further configured to:

[0093] randomly ordering the heterogeneous agents in the test workflow, adding the heterogeneous agents one by one in order for each ordering, and calculating the change value of the quality indicators before and after the addition of each heterogeneous agent;

[0094] summarizing the change values in all orderings, calculating the average contribution of each of the heterogeneous agents to each of the quality indicators to obtain a contribution vector of each of the heterogeneous agents;

[0095] performing weighted calculation on the contribution vectors to obtain a weighted contribution vector, and combining the weighted contribution vector by dimension to generate the new capability performance vector.

[0096] The software testing device based on the heterogeneous multi-agent in the application first analyzes a to-be-tested task to generate a task vector, constructs a capability vector of each heterogeneous agent in an agent pool, and determines a capability matrix according to the capability vector and the task vector. Then, the software testing device generates a test workflow according to the task vector and the capability matrix, and injects enhanced knowledge retrieved from a domain knowledge base into each task node in the test workflow. Finally, the software testing device calls a preset test tool to execute a test case generated by the test workflow to perform software testing on a complex software system. The software testing device not only understands and uses deep domain knowledge, effectively covers core business logic and compliance requirements, but also improves the testing quality and testing efficiency of the complex software system.

[0097] The specific limitations of the software testing device based on the heterogeneous multi-agent can be referred to the limitations of the software testing method based on the heterogeneous multi-agent in the above, and will not be repeated here. Each unit in the software testing device based on the heterogeneous multi-agent can be realized by software, hardware, and a combination thereof, in whole or in part. Each unit can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0098] In an embodiment, a computer device can be a server, and an internal structure diagram of the computer device can be as shown in Figure 8As shown in the figure. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media, internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external client through the network connection. The computer program is executed by the processor to realize the functions or steps of the server side of the heterogeneous multi-agent based software testing method.

[0099] In one embodiment, a computer device is provided, which can be a client, and its internal structure diagram can be as shown in the figure. Figure 9 As shown in the figure. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile storage media, internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external server through the network connection. The computer program is executed by the processor to realize the functions or steps of the client side of the heterogeneous multi-agent based software testing method.

[0100] In one embodiment, a computer device is provided, which includes a memory, a processor and a computer program stored on the memory and executable on the processor, and the processor executes the computer program to realize the steps of the above-mentioned heterogeneous multi-agent based software testing method.

[0101] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by the processor to realize the steps of the above-mentioned heterogeneous multi-agent based software testing method.

[0102] It should be noted that the functions or steps that the above-mentioned computer readable storage medium or computer device can realize can be referred to the related description of the server side and the client side in the foregoing method embodiment. To avoid repetition, they will not be described one by one here.

[0103] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchl ink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0104] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified. In actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above-described functions.

[0105] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not limit it. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features. Such modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A software testing method based on heterogeneous multi-agent, characterized in that: include: Receiving a task to be tested, and parsing the task to be tested to generate a task vector; For each heterogeneous agent in the agent pool, construct a capability vector of the heterogeneous agent, and determine a capability matrix based on the capability vector and the task vector; Generating a test workflow of a directed acyclic graph according to the task vector and the capability matrix; For each task node in the test workflow, a graph enhancement search is performed from a domain knowledge base to obtain a search result, and the search result is injected into the control instruction of the task node as enhanced knowledge; A preset test tool is called to execute the test cases generated by the test workflow.

2. The software testing method based on heterogeneous multi-agents according to claim 1, characterized in that: The step of parsing the task to be tested to generate a task vector includes: Preprocessing the task to be tested, and encoding the preprocessed task to be tested using a pretrained Transformer language model to obtain an encoding vector; Using a preset classifier to classify the encoding vector to obtain a classification result; The classification result is converted into the numerical task vector.

3. The software testing method based on heterogeneous multi-agents according to claim 1, characterized in that: The step of constructing the capability vector of the heterogeneous agent and determining the capability matrix according to the capability vector and the task vector includes: Constructing the multi-dimensional capability vector of the heterogeneous agent, wherein the multi-dimensional vector includes model architecture type, domain expertise, tool proficiency, reasoning ability, and context processing ability; All the capability vectors constitute an old capability matrix, and the capability matrix is ​​determined by an exponential moving average method based on the old capability matrix and the capability performance vector corresponding to the last test task.

4. The software testing method based on heterogeneous multi-agents according to claim 1, characterized in that: The step of generating a test workflow of a directed acyclic graph according to the task vector and the capability matrix includes: Selecting an initial DAG template from a workflow template library according to the test task scenario type in the task vector; For each task node in the initial DAG template, calculating the cosine similarity between the capability matrix and the requirement vector corresponding to the task node to obtain a similarity value; Evaluate the fitness of each of the heterogeneous agents to the task node according to the similarity value to obtain a fitness score; Using an integer linear programming method, under preset constraints, the allocation of the heterogeneous agents is modeled as a total utility function consisting of the fitness scores; A policy learning algorithm is used to explore and optimize the optimal agent allocation solution generated when maximizing the total utility function to generate the test workflow as a directed acyclic graph.

5. The software testing method based on heterogeneous multi-agents according to claim 1, characterized in that: The step of performing graph-enhanced retrieval from the domain knowledge base to obtain retrieval results includes: A retrieval result is obtained by performing graph-enhanced retrieval based on a reasoning method from the domain knowledge base, wherein the reasoning method includes a path reasoning method, a pattern reasoning method, and a rule reasoning method. The domain knowledge base is a database including a graph database and a vector index library, and the graph database is a database including a knowledge graph.

6. The heterogeneous multi-agent based software testing method according to any one of claims 1 to 5, characterized in that: After the step of calling a preset test tool to execute the test case generated by the test workflow, the method further includes: Collecting quality indicators after the test is completed, wherein the quality indicators include code coverage, defect retrieval rate, false alarm rate, and execution delay; A new capability performance vector of the task to be tested is calculated according to the quality index and the heterogeneous agents in the test workflow using a Shapley value algorithm, and the capability matrix is ​​updated according to the new capability performance vector.

7. The software testing method based on heterogeneous multi-agents according to claim 6, characterized in that: The step of calculating the new capability performance vector of the task to be tested by using a Shapley value algorithm based on the quality indicator and the heterogeneous agents in the test workflow includes: Randomly sorting the heterogeneous agents in the test workflow, adding the heterogeneous agents one by one in order for each sorting, and calculating the change value of the quality index before and after the heterogeneous agents are added; Summarizing the change values ​​in all rankings, calculating the average contribution of each of the heterogeneous agents to each of the quality indicators, and obtaining a contribution vector of each of the heterogeneous agents; A weighted contribution vector is obtained by performing a weighted calculation on the contribution vectors, and the weighted contribution vectors are combined according to dimensions to generate the new capability performance vector.

8. A software testing device based on heterogeneous multi-agent, characterized in that: include: A receiving and parsing unit, configured to receive a task to be tested and parse the task to be tested to generate a task vector; A construction and determination unit is used to construct a capability vector of each heterogeneous agent in the agent pool, and determine a capability matrix according to the capability vector and the task vector; A generating unit, configured to generate a test workflow of a directed acyclic graph according to the task vector and the capability matrix; A retrieval injection unit is used to perform a graph enhancement retrieval from a domain knowledge base for each task node in the test workflow to obtain a retrieval result, and inject the retrieval result as enhanced knowledge into the control instructions of the task node; The calling generation unit is used to call a preset test tool to execute the test case generated by the test workflow.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the software testing method based on heterogeneous multi-agents as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the software testing method based on heterogeneous multi-agents as described in any one of claims 1 to 7 are implemented.

Images