System testing method and device, computer equipment and readable storage medium

Abstract

The invention relates to a system testing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps of obtaining a unified test case corresponding to a to-be-tested multi-interaction-mode system; according to each interaction mode, performing corresponding conversion processing on the operation steps in the unified test case to obtain a target operation step corresponding to each interaction mode; according to the target operation step corresponding to each interaction mode, testing the multi-interaction mode system to obtain a test result corresponding to each interaction mode; and obtaining an expected result corresponding to each interaction mode, and determining a target test result of the multi-interaction mode system according to the test result corresponding to each interaction mode and the expected result. By adopting the method, the test cost of a multi-interaction mode system can be reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a system testing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] A multi-interaction system refers to a system that can provide multiple interaction modes. In order to ensure the stable operation of the multi-interaction system, it needs to be tested.

[0003] In traditional technology, when testing a multi-interaction mode system, a set of test cases is generally designed for each interaction mode of the multi-interaction mode system, and the multi-interaction mode system is tested using the test case set corresponding to each interaction mode; however, designing a set of test cases for each interaction mode will result in a large number of cases, which will in turn cause a high testing cost for the multi-interaction mode system. Summary of the invention

[0004] Based on this, it is necessary to provide a system testing method, apparatus, computer equipment, computer-readable storage medium and computer program product that can reduce the testing cost of a multi-interaction system in response to the above technical problems.

[0005] In a first aspect, the present application provides a system testing method, comprising:

[0006] Obtain unified test cases corresponding to the multi-interaction mode system to be tested;

[0007] According to each interaction mode, corresponding conversion processing is performed on the operation steps in the unified test case to obtain target operation steps corresponding to each interaction mode;

[0008] Testing the multi-interaction mode system according to the target operation steps corresponding to each interaction mode to obtain a test result corresponding to each interaction mode;

[0009] The expected result corresponding to each interaction mode is obtained, and the target test result of the multi-interaction mode system is determined according to the test result and the expected result corresponding to each interaction mode.

[0010] In one embodiment, the operation steps in the unified test case are converted according to each interaction mode to obtain target operation steps corresponding to each interaction mode, including:

[0011] According to each interaction mode, the operation type and operation parameters of the operation steps in the unified test case are subjected to corresponding format conversion processing to obtain initial operation steps corresponding to each interaction mode;

[0012] According to each interaction mode, corresponding assignment processing is performed on the operation type and operation parameters of the initial operation steps corresponding to each interaction mode to obtain the target operation steps corresponding to each interaction mode.

[0013] In one embodiment, the multi-interaction mode system is tested according to the target operation steps corresponding to each interaction mode to obtain the test results corresponding to each interaction mode, including:

[0014] Determine a test framework corresponding to each of the described interaction modes;

[0015] Through the test framework corresponding to each interaction mode, based on the target operation steps corresponding to each interaction mode, the multi-interaction mode system is subjected to the interaction operation corresponding to each interaction mode to obtain the test result corresponding to each interaction mode.

[0016] In one embodiment, determining the target test result of the multi-interaction mode system according to the test result and expected result corresponding to each interaction mode includes:

[0017] Determine a judgment method corresponding to each interaction method;

[0018] According to the judgment method corresponding to each interaction mode, judging whether the test result corresponding to each interaction mode is consistent with the expected result corresponding to each interaction mode, and obtaining the judgment result corresponding to each interaction mode;

[0019] According to the judgment results corresponding to each interaction mode, the target test results of the multi-interaction mode system are determined.

[0020] In one embodiment, the determining of the judgment method corresponding to each interaction method includes:

[0021] Obtain keywords for each interaction mode;

[0022] According to the keywords of each interaction mode, the corresponding relationship between the keywords and the judgment modes is queried to obtain the judgment mode corresponding to each interaction mode.

[0023] In one embodiment, obtaining a unified test case corresponding to the multi-interaction mode system to be tested includes:

[0024] Extracting common processing logic information of each interaction mode of the multi-interaction mode system to be tested;

[0025] A test case corresponding to the common processing logic information is obtained as the unified test case.

[0026] In a second aspect, the present application also provides a system testing device, comprising:

[0027] A test case acquisition module is used to acquire unified test cases corresponding to the multi-interaction mode system to be tested;

[0028] A conversion processing module, used to perform corresponding conversion processing on the operation steps in the unified test case according to each interaction mode, and obtain target operation steps corresponding to each interaction mode;

[0029] A system testing module, which tests the multi-interaction mode system according to the target operation steps corresponding to each interaction mode, and obtains a test result corresponding to each interaction mode;

[0030] The result determination module is used to obtain the expected result corresponding to each interaction mode, and determine the target test result of the multi-interaction mode system according to the test result and the expected result corresponding to each interaction mode.

[0031] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0032] Obtain unified test cases corresponding to the multi-interaction mode system to be tested;

[0033] According to each interaction mode, corresponding conversion processing is performed on the operation steps in the unified test case to obtain target operation steps corresponding to each interaction mode;

[0034] Testing the multi-interaction mode system according to the target operation steps corresponding to each interaction mode to obtain a test result corresponding to each interaction mode;

[0035] The expected result corresponding to each interaction mode is obtained, and the target test result of the multi-interaction mode system is determined according to the test result and the expected result corresponding to each interaction mode.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0037] Obtain unified test cases corresponding to the multi-interaction mode system to be tested;

[0038] According to each interaction mode, corresponding conversion processing is performed on the operation steps in the unified test case to obtain target operation steps corresponding to each interaction mode;

[0039] Testing the multi-interaction mode system according to the target operation steps corresponding to each interaction mode to obtain a test result corresponding to each interaction mode;

[0040] The expected result corresponding to each interaction mode is obtained, and the target test result of the multi-interaction mode system is determined according to the test result and the expected result corresponding to each interaction mode.

[0041] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0042] Obtain unified test cases corresponding to the multi-interaction mode system to be tested;

[0043] According to each interaction mode, corresponding conversion processing is performed on the operation steps in the unified test case to obtain target operation steps corresponding to each interaction mode;

[0044] Testing the multi-interaction mode system according to the target operation steps corresponding to each interaction mode to obtain a test result corresponding to each interaction mode;

[0045] The expected result corresponding to each interaction mode is obtained, and the target test result of the multi-interaction mode system is determined according to the test result and the expected result corresponding to each interaction mode.

[0046] The above-mentioned system testing method, device, computer equipment, computer-readable storage medium and computer program product first obtain a unified test case corresponding to the multi-interaction mode system to be tested, and according to each interaction mode, perform corresponding conversion processing on the operation steps in the unified test case to obtain the target operation steps corresponding to each interaction mode, and then test the multi-interaction mode system according to the target operation steps corresponding to each interaction mode to obtain the test results corresponding to each interaction mode, and finally obtain the expected results corresponding to each interaction mode, and determine the target test results of the multi-interaction mode system according to the test results and expected results corresponding to each interaction mode. In this way, when testing the multi-interaction mode system to be tested, the testing purpose of the multi-interaction mode system can be achieved through the unified test case corresponding to the multi-interaction mode system, and there is no need to design a set of test cases for each interaction mode, thereby reducing the number of test cases and further reducing the testing cost of the multi-interaction mode system; at the same time, the testing process of the multi-interaction mode system is simplified, which is conducive to improving the testing efficiency of the multi-interaction mode system. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 An application environment diagram of a system testing method in an embodiment;

[0049] Figure 2 A schematic diagram of a flow chart of a system testing method in an embodiment;

[0050] Figure 3 A flowchart of steps for determining target test results of a multi-interaction mode system in one embodiment;

[0051] Figure 4 is a flow chart of a system testing method in another embodiment;

[0052] Figure 5 A flowchart of a method for reducing the cost of automatic verification of a multi-interaction mode system in one embodiment;

[0053] Figure 6 is a structural block diagram of a system testing device in one embodiment;

[0054] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0057] The current era is one of informatization and intelligence, and various informatization systems have penetrated into all aspects of production and life. In order to enable natural persons to exchange information effectively with informatization systems, it is necessary to design various forms of human-computer interaction. Common human-computer interaction methods include command line mode, interface mode, client mode, web page mode, etc. The command line mode is mainly used by infrastructure operation and maintenance personnel to perform system maintenance at the operating system level or even lower level; the interface mode provides the possibility for different information systems to interact with each other, which is convenient for the expansion and integration of information systems; and the client mode and web page form provide users with rich graphical interface interaction capabilities, which are relatively friendly to non-professional ordinary users, and can also display images and charts, enhancing the expression ability of the informatization system.

[0058] In order to adapt to the needs of different roles, a considerable number of information systems provide the ability to interact in multiple ways. The forms of multiple interaction methods seem to be very different, but in fact, their internal logic is basically similar in many cases. After the system receives user instructions through different interaction methods, it processes them uniformly in the background, and then returns the response content of the corresponding interaction method according to the different interaction methods after the processing is completed. For such a system with multiple interaction methods, effective automated verification is more complicated than that of a system with a single interaction method. Testers need to select different test frameworks when verifying different interaction methods. For the use case set with the same verification logic, it needs to be implemented multiple times on different test frameworks, resulting in an exponential increase in the scale of automated use cases, which is not conducive to tracking and maintenance. In addition, due to the isolation of different frameworks, testers are unable to perform automated verification of mixed use of multiple interaction methods, which reduces the ability to discover problems in complex scenarios.

[0059] The main steps of the traditional automated verification method for multi-interaction systems are: designing corresponding use case sets based on different test frameworks, including designing use case scenarios, using the framework to perform user operations, obtaining relevant output results, and then using the corresponding test framework to verify the results. Obviously, under the traditional solution, testers need to design a set of test case sets for each interaction mode, and they also need to maintain them separately. The number of use cases is relatively large, and the maintenance cost is correspondingly high.

[0060] Therefore, for the current relatively simple multi-interaction mode automated verification method, this solution considers the underlying adaptation of multiple interaction modes, hiding the specific interaction form from the tester, and providing a unified upper-level abstract method to facilitate the tester to design the use case logic. When actually performing automated verification, it is only necessary to change the test configuration to select a specific interaction mode. At the same time, it also supports the tester to use multiple interaction modes in one use case, assisting them in designing complex scenario use cases and enhancing the ability to discover deep-level defects. Based on this, this application proposes a system testing method that can reduce testing costs.

[0061] The system testing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown in FIG. 1 , the test device 101 communicates with the multi-interaction mode system 102 to be tested via a network. Specifically, refer to Figure 1 , the test device 101 obtains a unified test case corresponding to the multi-interaction mode system 102 to be tested; according to each interaction mode, the operation steps in the unified test case are converted accordingly to obtain the target operation steps corresponding to each interaction mode; according to the target operation steps corresponding to each interaction mode, the multi-interaction mode system 102 is tested to obtain the test results corresponding to each interaction mode; the expected results corresponding to each interaction mode are obtained, and the target test results of the multi-interaction mode system 102 are determined according to the test results and expected results corresponding to each interaction mode. Among them, the test device 101 refers to a device for testing a multi-interaction mode system, specifically refers to a device deployed with an abstract operation layer and a test framework corresponding to different interaction modes. The multi-interaction mode system 102 refers to a system that can provide multiple interaction modes.

[0062] In an exemplary embodiment, Figure 2 As shown, a system testing method is provided, which is applied to Figure 1 The test device in is taken as an example to illustrate, including the following steps S201 to S204.

[0063] in:

[0064] Step S201, obtaining a unified test case corresponding to the multi-interaction mode system to be tested.

[0065] Among them, multiple interactive modes include command line mode, interface mode, client mode, web page mode, etc.

[0066] Among them, unified test cases refer to test cases with unified multi-interaction modes, specifically, abstract expression test cases, which are used to describe the behavior of the test cases. Figure 5 The abstract layer only provides the testers with the common processing logic of abstract operations, and the testers design use cases based on the abstracted common processing logic.

[0067] Among them, reference Figure 5The unified test case includes scenario definition (i.e., use case scenario definition), operation execution (i.e., operation step execution), result acquisition (i.e., operation result acquisition), and expected comparison (i.e., expected result comparison). Among them, the use case scenario definition is a combination of a set of operation processes and expected responses planned by the tester according to the requirements and design; the operation step execution is to perform the operation according to the designed scenario using the method provided by the abstract test framework; the operation result acquisition is to wait for the operation to be completed and obtain the specific test result returned by the underlying abstract test framework method; the expected result comparison uses the expected test output to compare with the actual test data to determine whether the test case execution has passed.

[0068] It should be noted that traditional test cases (such as test cases corresponding to command lines) are specific operation descriptions. For example, the command line is to enter an operation string in the terminal to start the program, while this operation in the unified test case is an abstract method to start the program, rather than a specific human-computer interaction operation. The main difference between the two is that traditional test cases have implementation details and content that is strongly related to the test method and test framework; while unified test cases have no implementation details and no content that is strongly related to the test method and test framework.

[0069] Exemplarily, the testing device obtains a unified test case corresponding to the multi-interaction mode system to be tested from a local database, so as to facilitate subsequent testing of the multi-interaction mode system to be tested according to the unified test case.

[0070] Step S202: According to each interaction mode, corresponding conversion processing is performed on the operation steps in the unified test case to obtain target operation steps corresponding to each interaction mode.

[0071] Among them, reference Figure 5 ,By designing a unified test case set for multiple interaction modes, the ,number of repeated logic and use cases can be reduced. ,By designing an abstract operation layer and adapting to the ,underlying test framework, it supports the flexible conversion of the unified ,use case set into a use case set with a specific interaction form, or ,mixing multiple interaction modes in the same use case, the test intensity and ,exploration depth can be enhanced.

[0072] Among them, the operation steps in the unified test case refer to a set of operation processes planned according to requirements and design.

[0073] The corresponding conversion processing is performed on the operation steps in the unified test case in order to convert the unified test case into a test case of a specific interaction mode.

[0074] The target operation steps corresponding to each interaction mode are used to characterize the specific operation steps corresponding to each interaction mode, and can also be used to characterize the specific test cases corresponding to each interaction mode.

[0075] Exemplarily, the test device obtains the operation steps in the unified test case, and performs corresponding conversion processing on the operation steps in the unified test case according to each interaction mode, to obtain the converted operation steps corresponding to each interaction mode as the target operation steps corresponding to each interaction mode.

[0076] For example, the test equipment performs corresponding conversion processing on the operation steps in the unified test case according to the interaction methods such as command line method, interface method, client method, and web page form, and obtains the target operation steps corresponding to the command line method, the target operation steps corresponding to the interface method, the target operation steps corresponding to the client method, the target operation steps corresponding to the web page form, etc.

[0077] Step S203: testing the multi-interaction mode system according to the target operation steps corresponding to each interaction mode, and obtaining the test results corresponding to each interaction mode.

[0078] The test result corresponding to each interaction mode refers to the actual response content corresponding to each interaction mode.

[0079] Exemplarily, the testing device drives the testing framework corresponding to each interaction mode, such as a command line testing framework, an interface testing framework, a web testing framework, etc., according to the target operation steps corresponding to each interaction mode, so that the testing framework corresponding to each interaction mode performs the interaction operation of the corresponding interaction mode with the multi-interaction mode system, and obtains the test results corresponding to each interaction mode.

[0080] Step S204, obtaining the expected result corresponding to each interaction mode, and determining the target test result of the multi-interaction mode system according to the test result and the expected result corresponding to each interaction mode.

[0081] The expected result corresponding to each interaction mode refers to the expected response content corresponding to each interaction mode.

[0082] The expected result corresponding to each interaction mode is included in the unified test case. When the unified test case is actually executed, the expected result of the specific interaction mode will be selected according to the configuration information.

[0083] The target test result of the multi-interaction mode system refers to whether the test passes or fails.

[0084] Exemplarily, the testing device obtains the expected results corresponding to each interaction mode from a unified test case, and determines whether the test results corresponding to each interaction mode are consistent with the corresponding expected results, obtains the judgment results corresponding to each interaction mode, and finally determines the target test results of the multi-interaction mode system based on the judgment results corresponding to each interaction mode.

[0085] For example, see Figure 5 First, design a unified test case for multiple interaction modes. Then, the abstract operation layer in the test device performs different abstract processing on the operation type and operation input parameters of the operation steps in the test case according to different interaction modes, and configures and assigns values ​​to the abstract operation type and operation input parameters. Then, use it to drive the corresponding test framework, so that the corresponding test framework performs the corresponding interaction mode with the tested system according to the corresponding interaction mode, so as to obtain the corresponding actual test results. Finally, the abstract operation layer receives the actual test results returned by the test framework, and selects the appropriate judgment method according to the keyword index of the interaction mode to determine whether the actual test results are consistent with the expected results in the test case, so as to determine whether the execution of this test case is passed, and then obtain the test results of the multi-interaction mode system.

[0086] In the above system testing method, firstly, a unified test case corresponding to the multi-interaction mode system to be tested is obtained, and according to each interaction mode, the operation steps in the unified test case are converted accordingly to obtain the target operation steps corresponding to each interaction mode, and then the multi-interaction mode system is tested according to the target operation steps corresponding to each interaction mode to obtain the test results corresponding to each interaction mode, and finally the expected results corresponding to each interaction mode are obtained, and the target test results of the multi-interaction mode system are determined according to the test results and expected results corresponding to each interaction mode. In this way, when the multi-interaction mode system to be tested is tested, the testing purpose of the multi-interaction mode system can be achieved through the unified test case corresponding to the multi-interaction mode system, and there is no need to design a set of test cases for each interaction mode, thereby reducing the number of test cases and further reducing the testing cost of the multi-interaction mode system; at the same time, the testing process of the multi-interaction mode system is simplified, which is conducive to improving the testing efficiency of the multi-interaction mode system.

[0087] In an exemplary embodiment, the above-mentioned step S202, according to each interaction mode, performs corresponding conversion processing on the operation steps in the unified test case to obtain the target operation steps corresponding to each interaction mode, and specifically includes the following contents: according to each interaction mode, performs corresponding format conversion processing on the operation type and operation parameters of the operation steps in the unified test case to obtain the initial operation steps corresponding to each interaction mode; according to each interaction mode, performs corresponding assignment processing on the operation type and operation parameters of the initial operation steps corresponding to each interaction mode to obtain the target operation steps corresponding to each interaction mode.

[0088] Among them, the operation steps in the unified test case have corresponding operation types and operation parameters (ie, operation input parameters).

[0089] The corresponding format conversion processing is performed on the operation type and operation parameters of the operation steps in the unified test case in order to convert the operation type and operation parameters of the operation steps into the operation type format and operation parameter format adapted to the corresponding interaction mode.

[0090] The initial operation steps corresponding to each interaction mode refer to the operation steps after format conversion corresponding to each interaction mode, including the operation type format and operation parameter format adapted to the corresponding interaction mode.

[0091] The corresponding assignment processing is performed on the operation type and operation parameters of the initial operation steps corresponding to each interaction mode in order to convert the operation type and operation parameters of the initial operation steps corresponding to each interaction mode into specific content.

[0092] Among them, the target operation steps corresponding to each interaction mode refer to the operation steps after the assignment processing corresponding to each interaction mode, including the specific content of the operation type and the specific content of the operation parameters adapted to the corresponding interaction mode. For example, for the implementation of the command line framework, it is necessary to use a character interaction tool to interact with the command line dialogue, and when executing the command, it is necessary to convert the operation type and operation parameters into specific character content; for the implementation of the interface framework; taking the http interface as an example, it is necessary to use the http request response framework to convert the operation type into a specific request method and request address, and convert the operation parameters into request parameters or key-value pairs in the request body, and then initiate an http request; for the web side, it is necessary to use the web page test framework to drive the test, identify the various control elements on the interactive page, and encapsulate multiple control operations into meaningful operation steps, and perform orderly operations on the elements in the encapsulation order when executing the test.

[0093] Exemplarily, the test device performs corresponding format conversion processing on the operation type and operation parameters of the operation steps in the unified test case according to each interaction mode, so as to obtain the operation type format and operation parameter format adapted to the corresponding interaction mode, thereby obtaining the operation steps after format conversion corresponding to each interaction mode, as the initial operation steps corresponding to each interaction mode; then, according to each interaction mode, the test device performs corresponding assignment processing on the operation type and operation parameters of the initial operation steps corresponding to each interaction mode, so as to obtain the specific content of the operation type and the specific content of the operation parameters adapted to the corresponding interaction mode, thereby obtaining the operation steps after assignment processing corresponding to each interaction mode, as the target operation steps corresponding to each interaction mode.

[0094] In this embodiment, according to each interaction mode, the operation steps in the unified test case are converted accordingly to obtain the target operation steps corresponding to each interaction mode. There is no need to design a set of test cases for each interaction mode, thereby simplifying the system testing process and improving the system testing efficiency.

[0095] In an exemplary embodiment, the above-mentioned step S203 tests the multi-interaction mode system according to the target operation steps corresponding to each interaction mode to obtain the test results corresponding to each interaction mode, and specifically includes the following contents: determining the test framework corresponding to each interaction mode; performing the interaction operation corresponding to each interaction mode on the multi-interaction mode system based on the target operation steps corresponding to each interaction mode through the test framework corresponding to each interaction mode, and obtaining the test results corresponding to each interaction mode.

[0096] Among them, different interaction modes correspond to different test frameworks; for example, the test framework corresponding to the command line mode is the command line test framework, the test framework corresponding to the interface mode is the interface test framework, and the test framework corresponding to the web page form is the web test framework.

[0097] Exemplarily, the test device determines the test framework corresponding to each interaction mode based on the correspondence between the interaction mode and the test framework; then uses the target operation steps corresponding to each interaction mode to drive the test framework corresponding to each interaction mode, so that the test framework corresponding to each interaction mode performs the interaction operation corresponding to each interaction mode on the multi-interaction mode system according to the target operation steps corresponding to each interaction mode, thereby obtaining the test results corresponding to each interaction mode.

[0098] In this embodiment, the multi-interaction mode system is tested according to the target operation steps corresponding to each interaction mode, and the test results corresponding to each interaction mode are obtained, thereby achieving the purpose of flexibly calling multiple interaction modes in a unified test case to test the multi-interaction mode system, which is conducive to further improving the testing efficiency of the multi-interaction mode system.

[0099] In an exemplary embodiment, Figure 3 As shown, the above step S204, according to the test results and expected results corresponding to each interaction mode, determines the target test results of the multi-interaction mode system, including the following steps S301 to S303. Among them:

[0100] Step S301, determining a judgment method corresponding to each interaction method.

[0101] Step S302, judging whether the test result corresponding to each interaction mode is consistent with the expected result corresponding to each interaction mode according to the judgment method corresponding to each interaction mode, and obtaining the judgment result corresponding to each interaction mode.

[0102] Step S303: determining a target test result of the multi-interaction mode system according to the judgment result corresponding to each interaction mode.

[0103] Among them, different interaction methods correspond to different judgment methods; for example, for the command line interaction method, the output string information is compared and verified; for the interface http interaction method, the obtained response body is parsed to determine whether the status code and each field in the response body meet the requirements; for the web side, it is necessary to obtain the pop-up window, page and other prompt information elements after the operation is completed to verify the correctness of the element content.

[0104] The judgment result corresponding to each interaction mode refers to a test pass (indicating that the test result is consistent with the expected result) or a test fail (indicating that the test result is inconsistent with the expected result).

[0105] Exemplarily, the testing device obtains the judgment method corresponding to each interaction mode from the local database, and performs corresponding format conversion processing on the expected results in the unified test case according to each interaction mode, so as to obtain the expected results corresponding to each interaction mode; then, according to the judgment method corresponding to each interaction mode, it is judged whether the test results corresponding to each interaction mode are consistent with the expected results corresponding to each interaction mode, so as to obtain the judgment results corresponding to each interaction mode; finally, according to the judgment results corresponding to each interaction mode, the target test results of the multi-interaction mode system are determined; for example, if the judgment results corresponding to each interaction mode are consistent with the expected results corresponding to each interaction mode, then it is determined that the multi-interaction mode system has passed the test; otherwise, it is determined that the multi-interaction mode system has failed the test.

[0106] In this embodiment, the target test results of the multi-interaction mode system are determined based on the test results and expected results corresponding to each interaction mode; in this way, when performing system testing, the test results and expected results corresponding to each interaction mode are comprehensively considered, which is conducive to improving the test accuracy of the multi-interaction mode system.

[0107] In an exemplary embodiment, the above step S301 determines the judgment method corresponding to each interaction method, which specifically includes the following contents: obtaining keywords for each interaction method; according to the keywords for each interaction method, querying the correspondence between the keywords and the judgment method to obtain the judgment method corresponding to each interaction method.

[0108] Each interaction mode has a corresponding keyword; for example, the keyword corresponding to the command line mode is command line, the keyword corresponding to the interface mode is interface, and the keyword corresponding to the web page mode is web.

[0109] There is a one-to-one correspondence between keywords and judgment methods.

[0110] Exemplarily, the test device performs keyword recognition processing on each interaction mode according to the keyword recognition instruction to obtain the keyword of each interaction mode; then, according to the keyword of each interaction mode, the correspondence between the keyword and the judgment method is queried to obtain the judgment method corresponding to the keyword of each interaction mode as the judgment method corresponding to each interaction mode.

[0111] In this embodiment, according to the keywords of each interaction method, the correspondence between the query keywords and the judgment methods can be quickly obtained, and the judgment method corresponding to each interaction method can be obtained without manual selection, thereby improving the efficiency of determining the judgment method corresponding to each interaction method.

[0112] In an exemplary embodiment, the above step S201, obtaining a unified test case corresponding to the multi-interaction mode system to be tested, specifically includes the following contents: extracting the common processing logic information of each interaction mode of the multi-interaction mode system to be tested; obtaining the test case corresponding to the common processing logic information as the unified test case.

[0113] Among them, for the multi-interaction mode system, the processing logic of each interaction mode is basically similar; for example, after the multi-interaction mode system receives user instructions through different interaction modes, it processes them uniformly in the background, and after the processing is completed, it returns the response content of the corresponding interaction mode according to the different interaction modes.

[0114] Exemplarily, the test device analyzes the processing logic information of each interaction mode of the multi-interaction mode system to be tested, and obtains the common processing logic information of each interaction mode of the multi-interaction mode system; then, according to the test case generation instruction, a test case corresponding to the common processing logic information is generated, including use case scenario definition, operation step execution, operation result acquisition, and expected result comparison; finally, the generated test case is used as a unified test case corresponding to the multi-interaction mode system to be tested.

[0115] In this embodiment, by extracting the common processing logic information of each interaction mode of the multi-interaction mode system to be tested, and generating a unified test case corresponding to the multi-interaction mode system to be tested based on the common processing logic information, it is beneficial to achieve the testing purpose of the multi-interaction mode system according to the unified test case later, without having to design a set of test cases for each interaction mode, thereby reducing the testing cost of the multi-interaction mode system.

[0116] In an exemplary embodiment, Figure 4 As shown, another system testing method is provided, which is described by taking the method applied to a testing device as an example, and includes the following steps S401 to S409. Among them:

[0117] Step S401: extracting common processing logic information of each interaction mode of the multi-interaction mode system to be tested.

[0118] Step S402: acquiring a test case corresponding to the common processing logic information as a unified test case corresponding to the multi-interaction mode system to be tested.

[0119] Step S403: According to each interaction mode, the operation type and operation parameters of the operation steps in the unified test case are converted into corresponding formats to obtain initial operation steps corresponding to each interaction mode.

[0120] Step S404: According to each interaction mode, corresponding assignment processing is performed on the operation type and operation parameters of the initial operation step corresponding to each interaction mode to obtain the target operation step corresponding to each interaction mode.

[0121] Step S405: determine a test framework corresponding to each interaction mode.

[0122] Step S406, respectively, through the test framework corresponding to each interaction mode, based on the target operation steps corresponding to each interaction mode, the multi-interaction mode system is subjected to the interaction operation corresponding to each interaction mode, and the test result corresponding to each interaction mode is obtained.

[0123] Step S407, obtaining a keyword for each interaction mode; searching for a corresponding relationship between the keyword and the judgment mode according to the keyword for each interaction mode, and obtaining a judgment mode corresponding to each interaction mode.

[0124] Step S408, judging whether the test result corresponding to each interaction mode is consistent with the expected result corresponding to each interaction mode according to the judgment method corresponding to each interaction mode, and obtaining the judgment result corresponding to each interaction mode.

[0125] Step S409: determining a target test result of the multi-interaction mode system according to the judgment result corresponding to each interaction mode.

[0126] In the above-mentioned system testing method, when testing the multi-interaction mode system to be tested, the testing purpose of the multi-interaction mode system can be achieved through a unified test case corresponding to the multi-interaction mode system, without the need to design a set of test cases for each interaction mode, thereby reducing the number of test cases and further reducing the testing cost of the multi-interaction mode system; at the same time, the testing process of the multi-interaction mode system is simplified, which is conducive to improving the testing efficiency of the multi-interaction mode system.

[0127] In order to more clearly illustrate the system testing method provided by the embodiment of the present application, the system testing method is specifically described below with a specific embodiment. In an exemplary embodiment, Figure 5As shown, the present application also proposes a method for reducing the cost of automated verification of a multi-interaction mode system, and its main process includes: first, designing a unified test case set, clarifying the execution scenario, operation steps, result acquisition and expected comparison of the case; second, calling the underlying test framework through the abstract operation layer, the abstract operation layer includes abstracting user operations, unifying the execution mode of the case, adapting the operation of the underlying test framework, adapting the output result verification of the underlying test framework, and designing a multi-type comparison method; finally, the underlying test framework is driven by the abstract operation layer to perform different interactive mode operations, obtain the corresponding operation results and return to the abstract operation layer for comparison and verification. In this way, the differences in different interactive modes are hidden by performing unified abstract processing on user operations, and a unified top-level operation mode is provided for testers, reducing the repeated implementation of different interactive mode use case sets, reducing the maintenance cost of automated use cases, and improving the efficiency of automated verification; in addition, testers can easily select multiple interactive modes in the same use case set, providing the ability of complex interactive use cases and enhancing the depth of automated verification.

[0128] refer to Figure 5 ,The specific steps of the method for reducing the cost of automated verification of multi-interaction systems are as follows:

[0129] 1. Design test cases with unified multi-interaction modes. The specific steps include use case scenario definition, operation step execution, operation result acquisition, and expected result comparison.

[0130] 2. At the bottom of the unified test case, an abstract operation layer needs to be added to adapt the underlying test framework. This includes operation abstraction, unified execution mode, framework operation adaptation, framework output result verification adaptation, and multi-type comparison mode.

[0131] Operation abstraction includes operation type abstraction and operation input parameter abstraction. Operation type abstraction summarizes all current commands, enumerates all command operations, and unifies the format. Operation input parameter abstraction abstracts the parameters corresponding to different operations as a whole, forms the input parameter format required by each, standardizes command input parameters, prevents missing or incorrect parameters, and reduces the degree of coupling between use cases and underlying operations.

[0132] The unified execution method is to break down and divide all the steps of human-computer interaction, define each operation step in a unified format, and standardize the top-level call. Generally, it can be divided into the following execution steps: operation node specification, operation type specification, operation parameter specification, actual operation execution, and operation result acquisition.

[0133] The main purpose of framework adaptation is to adapt and implement the unified operation mode at the test framework layer, hiding the specific implementation from the tester so that he only cares about the operation scenario instead of the specific interaction mode. When implementing, all abstract operation types need to be implemented, and different operation parameters need to be configured and assigned, and then the corresponding test frameworks need to be called to perform the actual operation.

[0134] The framework outputs result verification and adaptation. For test results in different formats obtained through different interaction methods, a unified keyword index is designed. When verifying the results, an appropriate judgment method is selected based on the keywords to determine whether the actual output results are consistent with the expected results.

[0135] To meet the diverse comparison and verification needs in actual testing, multiple comparison methods such as strict comparison, relational comparison, and inclusion comparison should be provided. For example, after each unified use case is executed, the output results can be compared in a strict consistency manner, a variety of expected or possible ways, a method that includes a certain string, and other ways. This is a more flexible expected comparison capability.

[0136] 3. The test framework executes test cases. After receiving the abstract operations from the abstract operation layer, different test frameworks drive the corresponding test frameworks to interact with the tested system in different ways, obtain relevant test results and feed them back to the abstract operation layer for processing.

[0137] The above embodiments can achieve the following technical effects: (1) Establish an abstract operation layer to reduce the maintenance cost of test cases: This method systematically establishes an abstract operation layer for systems with multiple interaction modes, supports multiple use case sets with different interaction modes to be tested using the same set of operation components, reduces the repeated logic of the use case set and the overall use case scale, and improves the maintainability of the test cases; (2) Supports the simultaneous use of multiple interaction modes, allowing the design of more complex scenario use cases: Compared with the original independent test cases, the method described in this scheme supports testers to use multiple interaction modes in the same use case to verify the tested system, improves the verification strength of the scenario use case, and is conducive to discovering deeper problems; (3) The use case scale is small, the maintenance cost is low, and it supports the design of complex use cases.

[0138] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0139] Based on the same inventive concept, the embodiment of the present application also provides a system testing device for implementing the system testing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more system testing device embodiments provided below can refer to the limitations on the system testing method above, and will not be repeated here.

[0140] In an exemplary embodiment, Figure 6 As shown, a system testing device is provided, including: a use case acquisition module 610, a conversion processing module 620, a system testing module 630 and a result determination module 640, wherein:

[0141] The use case acquisition module 610 is used to acquire a unified test case corresponding to the multi-interaction mode system to be tested.

[0142] The conversion processing module 620 is used to perform corresponding conversion processing on the operation steps in the unified test case according to each interaction mode, and obtain the target operation steps corresponding to each interaction mode.

[0143] The system testing module 630 tests the multi-interaction mode system according to the target operation steps corresponding to each interaction mode, and obtains the test results corresponding to each interaction mode.

[0144] The result determination module 640 is used to obtain the expected result corresponding to each interaction mode, and determine the target test result of the multi-interaction mode system according to the test result and the expected result corresponding to each interaction mode.

[0145] In an exemplary embodiment, the conversion processing module 620 is also used to perform corresponding format conversion processing on the operation type and operation parameters of the operation steps in the unified test case according to each interaction mode, so as to obtain the initial operation steps corresponding to each interaction mode; according to each interaction mode, perform corresponding assignment processing on the operation type and operation parameters of the initial operation steps corresponding to each interaction mode, so as to obtain the target operation steps corresponding to each interaction mode.

[0146] In an exemplary embodiment, the system testing module 630 is also used to determine a test framework corresponding to each interaction mode; through the test framework corresponding to each interaction mode, based on the target operation steps corresponding to each interaction mode, the multi-interaction mode system is subjected to interaction operations corresponding to each interaction mode to obtain test results corresponding to each interaction mode.

[0147] In an exemplary embodiment, the result determination module 640 is also used to determine the judgment method corresponding to each interaction mode; according to the judgment method corresponding to each interaction mode, it is judged whether the test result corresponding to each interaction mode is consistent with the expected result corresponding to each interaction mode, and the judgment result corresponding to each interaction mode is obtained; according to the judgment result corresponding to each interaction mode, the target test result of the multi-interaction mode system is determined.

[0148] In an exemplary embodiment, the result determination module 640 is further used to obtain keywords for each interaction mode; based on the keywords for each interaction mode, query the correspondence between the keywords and the judgment modes to obtain the judgment modes corresponding to each interaction mode.

[0149] In an exemplary embodiment, the use case acquisition module 610 is further used to extract common processing logic information of each interaction mode of the multi-interaction mode system to be tested; and obtain test cases corresponding to the common processing logic information as unified test cases.

[0150] Each module in the above system test device can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0151] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 7As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, a system testing method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0152] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0153] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0154] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0155] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0156] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0157] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0158] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A system testing method, characterized in that: The method comprises: Obtain unified test cases corresponding to the multi-interaction mode system to be tested; According to each interaction mode, corresponding conversion processing is performed on the operation steps in the unified test case to obtain target operation steps corresponding to each interaction mode; Testing the multi-interaction mode system according to the target operation steps corresponding to each interaction mode to obtain a test result corresponding to each interaction mode; The expected result corresponding to each interaction mode is obtained, and the target test result of the multi-interaction mode system is determined according to the test result and the expected result corresponding to each interaction mode.

2. The method according to claim 1, characterized in that: According to each interaction mode, the operation steps in the unified test case are converted accordingly to obtain target operation steps corresponding to each interaction mode, including: According to each interaction mode, the operation type and operation parameters of the operation steps in the unified test case are subjected to corresponding format conversion processing to obtain initial operation steps corresponding to each interaction mode; According to each interaction mode, corresponding assignment processing is performed on the operation type and operation parameters of the initial operation steps corresponding to each interaction mode to obtain the target operation steps corresponding to each interaction mode.

3. The method according to claim 1, characterized in that: The step of testing the multi-interaction mode system according to the target operation steps corresponding to each interaction mode to obtain the test results corresponding to each interaction mode includes: Determine a test framework corresponding to each of the described interaction modes; Through the test framework corresponding to each interaction mode, based on the target operation steps corresponding to each interaction mode, the multi-interaction mode system is subjected to the interaction operation corresponding to each interaction mode to obtain the test result corresponding to each interaction mode.

4. The method according to claim 1, characterized in that: Determining the target test result of the multi-interaction mode system according to the test result and expected result corresponding to each interaction mode includes: Determine a judgment method corresponding to each interaction method; According to the judgment method corresponding to each interaction mode, judging whether the test result corresponding to each interaction mode is consistent with the expected result corresponding to each interaction mode, and obtaining the judgment result corresponding to each interaction mode; According to the judgment results corresponding to each interaction mode, the target test results of the multi-interaction mode system are determined.

5. The method according to claim 4, characterized in that The determining of the judgment method corresponding to each interaction method includes: Obtain keywords for each interaction mode; According to the keywords of each interaction mode, the corresponding relationship between the keywords and the judgment modes is queried to obtain the judgment mode corresponding to each interaction mode.

6. The method according to any one of claims 1 to 5, characterized in that: The obtaining of a unified test case corresponding to the multi-interaction mode system to be tested includes: Extracting common processing logic information of each interaction mode of the multi-interaction mode system to be tested; A test case corresponding to the common processing logic information is obtained as the unified test case.

7. A system testing device, characterized in that: The device comprises: A use case acquisition module, used to acquire unified test cases corresponding to the multi-interaction mode system to be tested; A conversion processing module, used to perform corresponding conversion processing on the operation steps in the unified test case according to each interaction mode, so as to obtain target operation steps corresponding to each interaction mode; A system testing module, which tests the multi-interaction mode system according to the target operation steps corresponding to each interaction mode, and obtains a test result corresponding to each interaction mode; The result determination module is used to obtain the expected result corresponding to each interaction mode, and determine the target test result of the multi-interaction mode system according to the test result and the expected result corresponding to each interaction mode.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images