Software system testing method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a software system testing method and device, equipment and a storage medium, and relates to the field of artificial intelligence and the application field of artificial intelligence in the field of financial science and technology. The method comprises the steps that in the software system testing process, when the accumulated duration is larger than the preset duration, the running process of the software system is analyzed according to at least one obtained target testing case, a target testing result corresponding to each target testing case and a target running log of the software system, and the expected running state of the software system is obtained; the target test case is automatically generated according to information in a pre-created case information base; generating a verification case according to the expected operation state; the verification case is operated, a target result of the software system is obtained according to the obtained operation result and the expected operation state, and the target result is used for representing whether the software system needs to be improved or not. In this way, the robustness of long-term operation of the system is tested through the automatically generated case in advance.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a software system testing method and device, equipment and a storage medium. BACKGROUND

[0002] Before a software system is released, it needs to be tested so that the software system can bring a better experience to users.

[0003] At present, the functions of a software system are tested by using fixed test cases written by artificial to test whether the functions of the software system need to be improved.

[0004] However, at present, only single functions or businesses of the software system are tested, and defects in the long-term running process of the software system cannot be tested, which may result in poor robustness of the software system. SUMMARY

[0005] The present application provides a software system testing method, device, equipment and storage medium, which can test defects in the long-term running process of the software system and improve the robustness of the software system.

[0006] In a first aspect, the present application provides a software system testing method, comprising:

[0007] In the software system testing process, when the cumulative duration reaches a first preset duration, at least one target test case running in the cumulative duration, a target test result corresponding to each target test case, and a target running log of the software system are obtained; wherein the target test case is automatically generated according to information in a pre-created case information library;

[0008] The running process of the software system is analyzed by a large model according to at least one target test case, a target test result corresponding to each target test case, and a target running log of the software system, and an expected running state of the software system is obtained;

[0009] According to the expected running state, a verification case is generated;

[0010] The verification case is run, and a target result of the software system is obtained according to the obtained running result and the expected running state, which is used to represent whether the software system needs to be improved.

[0011] In a possible implementation, the generating a verification case according to the expected running state comprises:

[0012] At least one target business to be verified is determined by the large model according to the expected running state;

[0013] obtaining, by the large model, information of each target business in the case information library, and generating a verification case corresponding to each target business, to obtain at least one verification case.

[0014] In a possible implementation, the case information library includes a business process knowledge base, a verification rule knowledge base, and a script knowledge base.

[0015] The obtaining, by the large model, information of each target business in the case information library, and generating a verification case corresponding to each target business includes:

[0016] obtaining, by the large model, target node information of each target business in the business process knowledge base;

[0017] obtaining, by the large model, corresponding target verification rules in the verification rule knowledge base and target scripts corresponding to the target node information in the script knowledge base according to the target node information;

[0018] generating, by the large model, the verification case according to the target node information, the target verification rules, and the target scripts.

[0019] In a possible implementation, the running the verification case, and obtaining a target result of the software system according to the obtained running result and the expected running state includes:

[0020] running, by the large model, the verification case to obtain a running result;

[0021] comparing, by the large model, the running result with a corresponding result in the expected running state to obtain a deviation value corresponding to the verification case;

[0022] when the deviation value is greater than a preset deviation value, obtaining a first target result, the first target result being used to represent that the software system needs to be improved;

[0023] when the deviation value is less than or equal to the preset deviation value, obtaining a second target result, the second target result being used to represent that the software system does not need to be improved.

[0024] In a possible implementation, the method further includes:

[0025] obtaining, by the large model, information of at least one business in a pre-created case information library, and generating a current test case;

[0026] determining, by the large model, similarities of the test case and a plurality of first test cases respectively, to obtain a plurality of similarities, wherein the plurality of first test cases are test cases generated within a second preset time length before the current time;

[0027] When at least one of the plurality of similarities is greater than the similarity threshold, a new test case is regenerated until the plurality of similarities corresponding to the new test are all less than or equal to the similarity threshold;

[0028] When the plurality of similarities are all less than or equal to the similarity threshold, the test case is run, and a test result corresponding to the test case and a running log of the software system are generated.

[0029] In a possible implementation, the case information library includes a business process knowledge base, a verification rule knowledge base, and a script knowledge base.

[0030] The information of at least one business is obtained from the pre-created case information library, and a current test case is generated, including:

[0031] The business and node information of the business are obtained from the business process knowledge base according to a case generation strategy by using the large model;

[0032] The corresponding verification rule is obtained from the verification rule knowledge base according to the node information by using the large model, and the script corresponding to the node information is obtained from the script knowledge base;

[0033] The test case is generated according to the node information, the verification rule, and the script by using the large model.

[0034] In a possible implementation, the method further includes:

[0035] The historical data are learned by using the large model, and a case generation strategy and a state inference module are optimized, the state inference module being a module in the large model for obtaining an expected running state of the software system; the historical data at least include a test case, a test result, and a running state.

[0036] In a second aspect, the present application provides a software system testing device, including:

[0037] The obtaining module is configured to, in a software system testing process, when the accumulated time length is greater than a first preset time length, obtain at least one target test case running in the accumulated time length, a target test result corresponding to each target test case, and a target running log of the software system; wherein the target test case is automatically generated according to information in a pre-created case information library.

[0038] The processing module is configured to analyze a running process of the software system according to at least one target test case, a target test result corresponding to each target test case, and a target running log of the software system by using the large model, to obtain an expected running state of the software system; and the generating module is configured to generate a verification case according to the expected running state; and the processing module is further configured to run the verification case, to obtain a target result of the software system according to an obtained running result and the expected running state, and to use the target result to represent whether the software system needs to be improved.

[0039] In a possible implementation, the generating module is specifically configured to determine at least one target business to be verified according to the expected running state by using the large model, to obtain information of each target business in the case information library by using the large model, and to generate a verification case corresponding to each target business to obtain at least one verification case.

[0040] In a possible implementation, the case information library includes a business process knowledge base, a verification rule knowledge base, and a script knowledge base; and the generating module is specifically configured to obtain target node information of each target business in the business process knowledge base by using the large model, to obtain a target verification rule corresponding to the target node information in the verification rule knowledge base by using the large model, and to obtain a target script corresponding to the target node information in the script knowledge base by using the large model, and to generate the verification case according to the target node information, the target verification rule, and the target script by using the large model.

[0041] In a possible implementation, the processing module is specifically configured to obtain a running result by running the verification case by using the large model, to compare the running result with a corresponding result in the expected running state by using the large model, to obtain a deviation value corresponding to the verification case, to obtain a first target result when the deviation value is greater than a preset deviation value, and to use the first target result to represent that the software system needs to be improved, and to obtain a second target result when the deviation value is less than or equal to the preset deviation value, and to use the second target result to represent that the software system does not need to be improved.

[0042] In a possible implementation, the generation module is further configured to: acquire, by the large model, information of at least one service in a pre-created case information library, and generate a current test case. The large model is configured to determine similarities between the test case and a plurality of first test cases respectively, to obtain a plurality of similarities, wherein the plurality of first test cases are test cases generated within a second preset time length before a current time. When at least one similarity in the plurality of similarities is greater than a similarity threshold, a new test case is re-generated until similarities corresponding to the new test case are all less than or equal to the similarity threshold. When the plurality of similarities are all less than or equal to the similarity threshold, the test case is run, and a test result corresponding to the test case and a running log of the software system are generated.

[0043] In a possible implementation, the case information library includes a business process knowledge base, a verification rule knowledge base, and a script knowledge base. The generation module is further configured to acquire, by the large model, service and node information of the service in the business process knowledge base according to a case generation strategy. The large model is configured to acquire corresponding verification rules in the verification rule knowledge base according to the node information, and acquire scripts corresponding to the node information in the script knowledge base. The large model is configured to generate the test case according to the node information, the verification rules, and the scripts.

[0044] In a possible implementation, the apparatus further includes an optimization module configured to learn historical data by the large model, and optimize a case generation strategy and a state inference module. The state inference module is a module in the large model used to obtain an expected running state of the software system. The historical data at least includes test cases, test results, and running states.

[0045] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0046] The memory stores computer execution instructions.

[0047] The processor executes the computer execution instructions stored in the memory, so that the processor performs the first aspect and / or various possible implementation manners of the first aspect.

[0048] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the first aspect and / or various possible implementation manners of the first aspect.

[0049] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the first aspect and / or various possible implementation manners of the first aspect.

[0050] The software system testing method, device, equipment and storage medium provided by the present application can obtain at least one target test case running in the accumulated time length, the target test result corresponding to each target test case, and the target running log of the software system when the accumulated time length is greater than the first preset time length in the software system testing process. The target test case is automatically generated according to the information in the pre-created case information library. The running process of the software system is analyzed according to the at least one target test case, the target test result corresponding to each target test case, and the target running log of the software system, and the expected running state of the software system is obtained. The verification case is generated according to the expected running state. The verification case is run, and the target result of the software system is obtained according to the obtained running result and the expected running state. The target result is used to represent whether the software system needs to be improved. In this way, the long-term running process of the system can be tested by the pre-automatically generated case, and the robustness and user experience of the software system obtained after testing are improved. BRIEF DESCRIPTION OF DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0052] Figure 1 A flowchart of a software system testing method provided by the present application;

[0053] Figure 2 A flowchart of a method for generating a verification case provided by an embodiment of the present application;

[0054] Figure 3 A flowchart of a method for generating a test case provided by an embodiment of the present application;

[0055] Figure 4 A structural diagram of a software system testing method provided by an embodiment of the present application;

[0056] Figure 5 A flowchart of a complete software system testing method provided by an embodiment of the present application;

[0057] Figure 6 A structural diagram of a software system testing device provided by an embodiment of the present application;

[0058] Figure 7 A structural diagram of an electronic device provided by an embodiment of the present application.

[0059] The specific embodiments of the application will be described in detail below with reference to the drawings. These drawings and the associated description are not intended to limit the scope of the application in any way, but merely to illustrate certain embodiments of the application. DETAILED DESCRIPTION

[0060] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The same reference numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments are not meant to represent all embodiments consistent with the application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the application as detailed in the appended claims.

[0061] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards of relevant countries and regions, necessary security measures are taken, public order and good customs are not violated, and appropriate operation portals are provided for users to choose authorization or refusal.

[0062] It should be noted that the software system testing method, device, equipment and storage medium provided by the present application can be used in the field of artificial intelligence, and can also be used in any field other than artificial intelligence, and can also be used in the application field of artificial intelligence in the field of financial technology. The application field of the software system testing method, device, equipment and storage medium in the present application is not limited.

[0063] Before the software system is released, it needs to be tested to enable the software system to bring better experience to users.

[0064] At present, the functions of the software system are tested by using the fixed test cases written by artificial to test whether the functions of the software system need to be improved.

[0065] However, at present, only single function or business of the software system is tested, and defects in the long-term running process of the software system cannot be tested. For example, some systems may have problems such as memory leakage and data consistency error under high-frequency operation or specific process combination. The robustness of the software system obtained after testing is poor, which affects the user experience.

[0066] Based on this, the application provides a software system testing method. In the software system testing process, test cases can be automatically generated according to information in a pre-created case information library. When the cumulative duration is greater than a first preset duration, at least one target test case running in the cumulative duration, a target test result corresponding to each target test case, and a target running log of the software system are analyzed to obtain an expected running state of the software system. The expected running state is further verified by a verification model to verify the long-term running of the software system and determine whether the software system needs to be improved. In this way, the long-term running defects of the system can be tested by the pre-generated test cases, and the robustness and user experience of the software system obtained after testing can be improved.

[0067] The technical solutions of the application and how the technical solutions solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the application will be described below with reference to the drawings.

[0068] Figure 1 A flowchart of a software system testing method provided by the application is shown in FIG. 1. Figure 1 The method comprises the following steps.

[0069] In the software system testing process, at least one target test case running in a cumulative duration, a target test result corresponding to each target test case, and a target running log of the software system are obtained when the cumulative duration reaches a first preset duration.

[0070] It should be noted that in the software system testing process, different test cases need to be continuously executed to test the running of the software system for the test cases.

[0071] In the application, the long-term running of the software system can be automatically detected every first preset duration. The first preset duration can be one week or 10 days, and can be set according to the actual situation of the software system. The embodiments of the application do not limit this.

[0072] For example, when the first preset duration is one week, at least one target test case running in one week, a target test result corresponding to each target test case, and a target running log of the software system can be obtained.

[0073] The number of target test cases and target test results obtained by the embodiments of the application is not limited.

[0074] It should be noted that the target running log is a log generated when each test case is run.

[0075] The target test case is automatically generated according to information in a pre-created case information library.

[0076] It should be noted that the test case can be used to test the running of a certain business process of the software system.

[0077] For example, during the testing of the software system, the test case can be automatically generated and executed without interruption or at certain time intervals to detect the running of each test case of the software system.

[0078] S102, analyzing, by the large model, the running process of the software system according to the at least one target test case, the target test result corresponding to each target test case, and the target running log of the software system, to obtain an expected running state of the software system.

[0079] In the present application, the large model can be a large language model or a multi-modal model, and the embodiments of the present application do not limit this. The state inference module in the large model can obtain the expected running state of the software system according to the at least one target test case, the target test result corresponding to each target test case, and the target running log of the software system.

[0080] S103, generating a verification case according to the expected running state.

[0081] For example, the verification case can be a case for verifying at least one key business in the expected running state, and the embodiments of the present application do not limit this.

[0082] S104, running the verification case, and obtaining a target result of the software system according to the obtained running result and the expected running state, the target result being used to represent whether the software system needs to be improved.

[0083] In the present application, the target result can include a first target result used to represent that the software system needs to be improved, or a second target result used to represent that the software system does not need to be improved.

[0084] It should be understood that when the target result is the second target result, the result can be recorded. When the target result is the first target result, the result can be recorded and a user can be notified so that the user can improve the software system based on the target result.

[0085] For example, the user can be notified by sending a notification message to the terminal device of the user, or sending an email to the device of the user, and the embodiments of the present application do not limit the specific way of notifying the user.

[0086] In this way, in the software system test process, the test case can be automatically generated according to the information in the pre-created case information library. When the accumulated time length is greater than the first preset time length, the expected running state of the software system is obtained by analyzing at least one target test case running in the accumulated time length, the target test result corresponding to each target test case, and the target running log of the software system. Further, the expected running state is verified by the verification model to verify the long-term running condition of the software system and determine whether the software system needs to be improved. In this way, the long-term running defects of the system can be tested by the pre-automatically generated case, and the robustness and user experience of the software system obtained after the test are improved.

[0087] In the present application, the method of generating verification cases can refer to Figure 2 , Figure 2 A method of generating verification cases provided by an embodiment of the present application is shown in a flowchart.

[0088] As Figure 2 shown, generating verification cases can include:

[0089] S201, determining at least one target business to be verified according to the expected running state by a large model.

[0090] Exemplarily, after obtaining the expected running state, at least one target business to be verified can be determined in the expected running state according to a pre-set strategy by a large model.

[0091] It should be understood that the pre-set strategy can be to obtain a relatively critical target business of the software system in the expected running state. The relatively critical target business can be a business that has a greater impact on users when defects occur. The pre-set strategy is not limited in the present application.

[0092] S202, obtaining information of each target business in the case information library by a large model, and generating a verification case corresponding to each target business to obtain at least one verification case.

[0093] Exemplarily, the case information library includes a business process knowledge base, a verification rule knowledge base, and a script knowledge base.

[0094] The business process knowledge base can store the operation process nodes of each functional module of the software system, and record the node dependency relationship in a directed graph structure. The business process knowledge base stores the correspondence between multiple business processes and nodes.

[0095] The verification rule knowledge base can be obtained by integrating business logic constraints. The verification rule knowledge base stores the correspondence between multiple nodes and verification rules.

[0096] The script knowledge base contains API call scripts, UI automation scripts and the like corresponding to each operation node. The embodiments of the present application do not specifically limit the type of script in the script knowledge base. The script knowledge base stores the correspondence between multiple nodes and scripts.

[0097] For example, the three knowledge bases can store data in a structured data format, support dynamic updating and expansion.

[0098] In the case information base, the information of each target business is obtained, and the verification case corresponding to each target business is generated, including: obtaining the target node information of each target business in the business process knowledge base through the large model; obtaining the corresponding target verification rule in the verification rule knowledge base and the target script corresponding to the target node information in the script knowledge base through the large model according to the target node information; and generating the verification case according to the target node information, the target verification rule and the target script through the large model.

[0099] For example, the target node information of each target business can include one, two or more target nodes, and the embodiments of the present application do not limit this.

[0100] The target node information of each target business is obtained in the business process knowledge base through the large model, which can generate a directed acyclic graph based on the dependency relationship of each target node to ensure the feasibility of the process logic. For example, node 1→node 2→node 3.

[0101] After determining the target node information of the target business, the target script corresponding to each target node can be obtained in the script knowledge base, and the relationship between each target script can be obtained according to the directed relationship between the target nodes, i.e. the dependency relationship.

[0102] In this way, by obtaining information in different knowledge bases, the script corresponding to each node of the business is finally obtained, which can be an automated execution script, so that the accuracy of the obtained script is higher. And it can realize the automatic execution script, i.e. automatically run the verification case.

[0103] In the present application, the verification is generated through the large model, so that the accuracy of the generated verification case is higher.

[0104] In combination with the above embodiments, generating the target result can include: after the large model generates the verification case, continuing to run the verification case by the large model to obtain a running result; comparing the running result with a corresponding result in the expected running state by the large model to obtain a deviation value corresponding to the verification case. When the deviation value is greater than a preset deviation value, a first target result is obtained, and the first target result is used to represent that the software system needs to be improved; when the deviation value is less than or equal to the preset deviation value, a second target result is obtained, and the second target result is used to represent that the software system does not need to be improved.

[0105] The preset deviation value can be set according to the actual situation of the software system, for example, the preset deviation value can be 5%, or can be other numerical values. The embodiments of the present application do not make specific limitations on this.

[0106] In this way, if the deviation value is greater than the preset deviation value, it means that the deviation between the expected result processed by the large model and the actual result is large, which can be caused by the long-term running of the software system, that is, the long-term running of the software system has certain defects, therefore, the first target result that the software system needs to be improved is obtained. If the deviation value is less than or equal to the preset deviation value, it means that the deviation between the expected result processed by the large model and the actual result is small, that is, the long-term running of the running system will not affect the business of the software, or the impact on the business of the software system is small, therefore, the second target result that the software system does not need to be improved is obtained. Therefore, the target result obtained in the present application can accurately guide the user.

[0107] Based on the above embodiments, during the testing of the software system, the test case can be automatically generated and executed without interruption or at certain time intervals. Next, the generation of the test case is described.

[0108] Figure 3 A method flow diagram for generating a test case is provided in the embodiments of the present application.

[0109] As shown in Figure 3 , the generation of the test case can include:

[0110] S301, obtaining information of at least one business in a pre-created case information library by a large model to generate a current test case.

[0111] As described in the above embodiments, the case information library includes a business process knowledge base, a verification rule knowledge base and a script knowledge base.

[0112] The related descriptions of the business process knowledge base, the verification rule knowledge base and the script knowledge base can be referred to the descriptions in the above embodiments, which will not be repeated here.

[0113] The current test case can be generated by the large model to obtain the business and node information of the business in the business process knowledge base according to the case generation strategy; the corresponding verification rule is obtained in the verification rule knowledge base according to the node information by the large model, and the script corresponding to the node information is obtained in the script knowledge base; the test case is generated by the large model according to the node information, the verification rule and the script.

[0114] Exemplarily, the number of nodes included in each business node information can also be 1, 2 or more, and the embodiments of the present application do not limit this.

[0115] The business is obtained in the business process knowledge base by the large model according to the case generation strategy, and the corresponding node information of the business is obtained, which can generate a directed acyclic graph based on the dependency relationship of each node to ensure the feasibility of the process logic. For example, node A→node B→node C.

[0116] Exemplarily, the case generation strategy can be the ranking of the business to be obtained, or the probability of each business needing to be tested, and the embodiments of the present application do not limit the case generation strategy.

[0117] After determining the node information of the business, the script corresponding to each node can be obtained in the script knowledge base, and the relationship between each script can be obtained according to the directed relationship between the nodes, i.e. the dependency relationship.

[0118] In this way, by obtaining information in different knowledge bases, the script corresponding to each node of the business is finally obtained, which can be an automated execution script, so that the accuracy of the obtained script is high. And it can realize the automatic execution of the script, i.e. automatically running the verification case.

[0119] S302, the similarity of the test case to a plurality of first test cases is determined by the large model, and a plurality of similarities are obtained.

[0120] Among them, the plurality of first test cases are test cases generated within a second preset time length before the current time.

[0121] The second preset time length can be 10 days, 20 days or 30 days, and the embodiments of the present application do not limit the second preset time length.

[0122] Exemplarily, the semantic similarity algorithm (such as cosine similarity) can be used to compare the current test case generated and the test case generated in the past 30 days to obtain the similarity between the current test case and each test case before.

[0123] The similarity can be obtained by other ways, and the embodiments of the present application do not limit this.

[0124] S303, when there is at least one similarity greater than the similarity threshold value in the plurality of similarities, a new test case is regenerated until the plurality of similarities corresponding to the new test are all less than or equal to the similarity threshold value.

[0125] When there is at least one similarity greater than the similarity threshold value in the plurality of similarities, it indicates that the current test case may have been generated similarly within the previous second preset time length, and therefore, in order to improve the coverage of the test case on the business of the software system, a new case can be regenerated.

[0126] The method of regenerating a new test case can refer to the method of generating the current test case in the above embodiments, which is not limited in the embodiments of the present application.

[0127] S304, when the plurality of similarities are all less than or equal to the similarity threshold value, the test case is run to generate a test result corresponding to the test case and a running log of the software system.

[0128] When the plurality of similarities are all less than or equal to the similarity threshold value, it indicates that the current test case has no similar test case within the previous second preset time length, and therefore, the test case can be directly run.

[0129] In this way, by controlling the similarity of the newly generated test case with the test cases generated within the previous second preset time length, the similarity is low, which can reduce the situation that the same several businesses of the software system are repeatedly tested, and the number of tests on other businesses is small, and improve the coverage of the test case on the business of the software system in the software testing process.

[0130] In the present application, in the software system testing process, the historical data can also be learned by the large model, and the case generation strategy and the state inference module are optimized.

[0131] The state inference module is a module in the large model for obtaining the expected running state of the software system.

[0132] The historical data at least includes test cases, test results, and running states.

[0133] In this way, by optimizing the case generation strategy, the situation that the same several businesses of the software system are repeatedly tested, and the number of tests on other businesses is small, can be reduced, and the coverage of the test case on the business of the software system in the software testing process is improved. Moreover, by optimizing the state inference module, the accuracy of the obtained expected running state is improved.

[0134] Figure 4 A structure schematic diagram for testing a software system is provided in the embodiments of the present application.

[0135] AsFigure 4 As shown, the interval time for generating the test case is taken as an example of the intermittent duration. When the cumulative duration reaches the intermittent duration, the test case is generated by the large model, and the test case is executed to obtain the test result.

[0136] When the cumulative duration reaches the first preset duration, the expected running state is generated by the large model, and the target result is obtained based on the expected running state.

[0137] The specific processes can be referred to the descriptions of the above embodiments, which will not be repeated here.

[0138] Figure 5 A flowchart of a complete software system testing method provided by the embodiments of the present application is shown.

[0139] As shown Figure 5 The complete software system testing method can include system initialization, daily test case generation, periodic verification mechanism, report generation and pushing, and test data management.

[0140] Next, the contents of each part will be described.

[0141] System initialization: establish a business process knowledge base, establish a verification rule knowledge base, and establish a script knowledge base.

[0142] The related descriptions of each knowledge base can be referred to the descriptions of the above embodiments, which will not be repeated here.

[0143] Daily test case generation: the large model randomly extracts business and nodes of the business from the knowledge base. A directed acyclic graph is generated in combination with the dependency relationship. Test cases are generated by matching verification rules and executing scripts. The non-repetition of the test cases is ensured by a semantic similarity algorithm.

[0144] The specific processes can be referred to the descriptions of the above embodiments, which will not be repeated here.

[0145] Periodic verification mechanism: the large model generates an expected running state, and generates a verification case based on the expected running state. The result obtained after running the verification case is compared with the actual result to identify defects.

[0146] The specific processes can be referred to the descriptions of the above embodiments, which will not be repeated here.

[0147] Test data management: distributed storage architecture classifies and manages test data.

[0148] For example, a distributed storage architecture is designed, and test content (case script), test data (input parameter, expected result), and test result (execution log, actual output) are stored in a database and a log system according to time dimension.

[0149] Large model continuous learning optimization case generation strategy.

[0150] For example, if a certain type of defect is found in multiple tests, the model will automatically increase the priority of case generation in that scenario.

[0151] Report generation and push: Design a report generation engine to aggregate test results daily / weekly / monthly, generate visual reports with trend analysis, and automatically push to the user's mailbox.

[0152] For example, the report generation engine is designed to implement the following functions:

[0153] Data aggregation: Aggregate test case execution results by day / week / month, record pass rate, failure case distribution (such as function module proportion), defect type (such as interface error, data validation failure).

[0154] Trend analysis: Based on historical data, generate visual charts (such as pass rate trend chart, defect quantity change), large model for semantic interpretation of trend anomalies.

[0155] Email template and push: Pre-set multi-dimensional report templates (development version, management version), development version includes detailed error logs and code positioning, management version focuses on key indicator overview. The system automatically sends reports to the specified mailbox at the preset period, supports custom recipient grouping and alarm level filtering.

[0156] It should be noted that the defects described above are areas for improvement.

[0157] In this way, the software testing method of the embodiment of the application can generate a large number of non-repetitive test cases by randomly combining business processes through a large model, covering edge scenarios that are difficult to reach through traditional manual testing. It can improve the coverage and generation efficiency of test cases. The periodic system state inference mechanism can find long-term running cumulative implicit problems. The application ensures the diversity and randomness of test cases through the automatic generation and uniqueness verification of the large model, avoiding the "inertial testing" caused by the remaining vulnerabilities. Based on the continuous learning of historical test data, the large model can establish a dynamic model of system state changes, realize long-term detection and recording of system health, and provide data support for system optimization. Compared with traditional single-point testing, it is more forward-looking.

[0158] Figure 6 The structure of a software system testing device provided by the embodiment of the application is shown in FIG. 1. Figure 6 As shown in FIG. 1, the software system testing device 60 provided by the embodiment of the application comprises:

[0159] The acquisition module 601 is configured to acquire at least one target test case running in the accumulated time length, a target test result corresponding to each target test case, and a target running log of the software system when the accumulated time length is greater than the first preset time length during the software system test process, wherein the target test case is automatically generated according to information in a pre-created case information library.

[0160] The processing module 602 is configured to analyze a running process of the software system according to the at least one target test case, the target test result corresponding to each target test case, and the target running log of the software system by using the large model, to obtain an expected running state of the software system.

[0161] In a possible implementation, the generation module 603 is specifically configured to determine at least one target business to be verified according to the expected running state by using the large model, to acquire information of each target business in the case information library by using the large model, and to generate a verification case corresponding to each target business to obtain at least one verification case.

[0162] In a possible implementation, the case information library includes a business process knowledge base, a verification rule knowledge base, and a script knowledge base.

[0163] In a possible implementation, the processing module 602 is specifically configured to obtain a running result by running the verification case by using the large model, to compare the running result with a corresponding result in the expected running state by using the large model, to obtain a deviation value corresponding to the verification case, to obtain a first target result when the deviation value is greater than a preset deviation value, and to obtain a second target result when the deviation value is less than or equal to the preset deviation value.

[0164] In a possible implementation, the generation module 603 is further configured to obtain information of at least one service in a pre-created case information library by using the large model, and generate the current test case. The large model is used to determine similarities of the test case and a plurality of first test cases respectively, to obtain a plurality of similarities, where the plurality of first test cases are test cases generated within a second preset time length before the current time. When at least one similarity in the plurality of similarities is greater than a similarity threshold, a new test case is re-generated until similarities corresponding to the new test case are all less than or equal to the similarity threshold. When the plurality of similarities are all less than or equal to the similarity threshold, the test case is run, and a test result corresponding to the test case and a running log of the software system are generated.

[0165] In a possible implementation, the case information library includes a business process knowledge base, a verification rule knowledge base, and a script knowledge base. The generation module 603 is further configured to obtain service and node information of the service in the business process knowledge base according to a case generation strategy by using the large model, obtain corresponding verification rules in the verification rule knowledge base according to the node information by using the large model, and obtain scripts corresponding to the node information in the script knowledge base by using the large model. The generation module 603 is further configured to generate the test case according to the node information, the verification rules, and the scripts by using the large model.

[0166] In a possible implementation, the apparatus further includes an optimization module 604, configured to learn historical data by using the large model, and optimize a case generation strategy and a state inference module. The state inference module is a module in the large model used to obtain an expected running state of the software system. The historical data at least includes test cases, test results, and running states.

[0167] The software system testing apparatus provided in this embodiment can perform the method provided in the method embodiments, and has similar implementation principles and technical effects. Details are not described herein again.

[0168] Figure 7 FIG. 1 shows a structural schematic diagram of an electronic device provided in an embodiment of the present application. As shown in FIG. 1, the electronic device 70 provided in this embodiment includes at least one processor 701 and a memory 702. Optionally, the device 70 further includes a communication component 703. The processor 701, the memory 702, and the communication component 703 are connected through a bus 704. Figure 7

[0169] In the specific implementation process, the at least one processor 701 executes computer execution instructions stored in the memory 702, so that the at least one processor 701 performs the method described above.

[0170] The specific implementation process of the processor 701 can refer to the method embodiments described above, and has similar implementation principles and technical effects. Details are not described herein again. ​

[0171] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application does not limit to only one bus or one type of bus.

[0172] The present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the method described above.

[0173] The present application also provides a computer readable storage medium having computer executable instructions stored therein, which, when executed by a processor, implement the method described above.

[0174] The readable storage medium described above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0175] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0176] It should be noted that for the foregoing method embodiments, in order to simplify the description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the order of the described actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0177] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0178] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0179] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0180] When an integrated unit / module is implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0181] If the integrated units / modules are implemented in the form of software program modules and sold or used as independent products, they can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a number of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The aforementioned memory includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0182] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0183] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. The application is intended to cover any variations, uses or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice in the art to which the application pertains or can relate. The specification and examples are to be regarded as exemplary only, and the true scope and spirit of the application are indicated by the following claims.

[0184] It should be understood that the application is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the application is limited only by the claims that follow.

Claims

1. A software system testing method, characterized in that, include: During software system testing, when the cumulative time reaches a first preset time, at least one target test case that has run within the cumulative time, the target test result corresponding to each target test case, and the target running log of the software system are obtained; wherein, the target test case is automatically generated based on information in a pre-created case information database; The large model analyzes the operation process of the software system based on at least one target test case, the target test results corresponding to each target test case, and the target operation log of the software system, and obtains the expected operation state of the software system. Based on the expected operating state, generate verification cases; The verification case is run, and the target result of the software system is obtained based on the running results and the expected running state. The target result is used to characterize whether the software system needs to be improved.

2. The method according to claim 1, characterized in that, The step of generating verification cases based on the expected operating state includes: Based on the expected operating state, the large model determines at least one target service to be verified. The large model retrieves information about each target business from the case information database and generates a verification case for each target business, thus obtaining at least one verification case.

3. The method according to claim 2, characterized in that, The case information database includes a business process knowledge base, an inspection rule knowledge base, and a script knowledge base; The step of obtaining information about each target service from the case information database and generating a verification case corresponding to each target service includes: The target node information for each target business is obtained from the business process knowledge base using the large model. The large model retrieves the corresponding target verification rule from the verification rule knowledge base based on the target node information, and retrieves the target script corresponding to the target node information from the script knowledge base; The large model generates the verification case based on the target node information, the target verification rules, and the target script.

4. The method according to any one of claims 1-3, characterized in that, The verification case is run, and based on the obtained running results and the expected running state, the target result of the software system is obtained, including: The validation cases are run using the large model to obtain the results. The large model is used to compare the running results with the corresponding results in the expected running state to obtain the deviation value corresponding to the verification case. When the deviation value is greater than the preset deviation value, a first target result is obtained, which is used to characterize the software system as needing improvement; When the deviation value is less than or equal to the preset deviation value, a second target result is obtained, which is used to characterize that the software system does not need to be improved.

5. The method according to claim 1, characterized in that, The method further includes: The large model retrieves information on at least one business from a pre-created case information database and generates the current test case. The similarity between the test case and multiple first test cases is determined by the large model, resulting in multiple similarity scores; wherein, the multiple first test cases are test cases generated within a second preset time period before the current time. When at least one of the multiple similarities is greater than the similarity threshold, a new test case is generated until all the similarities corresponding to the new test are less than or equal to the similarity threshold. When multiple similarities are all less than or equal to the similarity threshold, the test case is run, and the test results and the software system's running logs corresponding to the test case are generated.

6. The method according to claim 5, characterized in that, The case information database includes a business process knowledge base, an inspection rule knowledge base, and a script knowledge base; The step of retrieving at least one business information from a pre-created case information database and generating the current test case includes: According to the case generation strategy, the large model obtains the business and the node information of the business from the business process knowledge base; The large model retrieves the corresponding verification rules from the verification rule knowledge base based on the node information, and retrieves the script corresponding to the node information from the script knowledge base; The test cases are generated using the large model based on the node information, the verification rules, and the script.

7. The method according to claim 6, characterized in that, The method further includes: The large model learns from historical data to optimize the case generation strategy and the state inference module. The state inference module is the module in the large model used to obtain the expected operating state of the software system. The historical data includes at least test cases, test results, and operating states.

8. A software system testing apparatus, comprising: The acquisition module is used to acquire, during the software system testing process, at least one target test case running within the cumulative time, the target test result corresponding to each target test case, and the target running log of the software system when the cumulative time exceeds a first preset time; wherein, the target test case is automatically generated based on information in a pre-created case information database; The processing module is used to analyze the operation process of the software system through a large model based on at least one target test case, the target test result corresponding to each target test case, and the target operation log of the software system, so as to obtain the expected operation state of the software system. The generation module is used to generate verification cases based on the expected operating state; The processing module is also used to run the verification case, and based on the obtained running results and the expected running state, obtain the target result of the software system. The target result is used to characterize whether the software system needs to be improved.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 7.

Images