Abnormality test method and device, readable medium, electronic equipment and product

Abstract

The invention discloses an anomaly test method and device, a readable medium, electronic equipment and a product. The method comprises the steps that forward testing is carried out on a tested object, service dependency information of the tested object is determined, and the service dependency information comprises services called by the tested object and service types of all the services; for a to-be-tested target service, according to the target service type, a target business type corresponding to the target service and a to-be-tested target exception type, generating a target exception test case by using a pre-generated case generation model, the target service being a service included in the service dependency information; and according to the target exception test case, performing exception test on the tested object, and determining an exception test result corresponding to the target exception test case. Therefore, the identification of the abnormal service link dependence, the design and verification of the abnormal scene and the generation and judgment of the test result can be completed automatically, the accuracy can be ensured, and the overall efficiency of the abnormal test can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to an exception testing method, apparatus, readable medium, electronic device, and product. Background Art

[0002] Currently, in testing services, it is usually necessary to cover exception scenarios, which depends on manually written exception test cases judged by humans. That is, exception scenarios mainly rely on manual construction by testers. As business scenarios become increasingly complex, involving more middleware and microservices, the difficulty for testers to construct exception scenarios gradually increases. In addition, there are factors such as limited human resources, making the current manual-based exception testing have problems in terms of efficiency and coverage that are difficult to guarantee. Summary of the Invention

[0003] This Summary of the Invention section is provided to introduce concepts in a brief form, which will be described in detail in the subsequent Detailed Description section. This Summary of the Invention section is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

[0004] In a first aspect, the present disclosure provides an exception testing method, which includes: Determine service dependency information of the object under test by performing a positive test on the object under test, where the service dependency information includes services called by the object under test and service types of each service; For a target service to be tested, generate a target exception test case according to the target service type, the target business type corresponding to the target service, and the target exception type to be tested, by using a pre-generated use case generation model, where the target service is a service included in the service dependency information; Perform an exception test on the object under test according to the target exception test case, and determine an exception test result corresponding to the target exception test case.

[0005] In a second aspect, the present disclosure provides an exception testing apparatus, which includes: A first determination module, configured to determine service dependency information of the object under test by performing a positive test on the object under test, where the service dependency information includes services called by the object under test and service types of each service; A use case generation module, configured to generate a target exception test case for a target service to be tested according to the target service type, the target business type corresponding to the target service, and the target exception type to be tested, by using a pre-generated use case generation model, where the target service is a service included in the service dependency information; A second determination module, configured to perform an exception test on the object under test according to the target exception test case, and determine an exception test result corresponding to the target exception test case.

[0006] In a third aspect, the present disclosure provides a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processing device, the steps of the method described in the first aspect of the present disclosure are implemented.

[0007] In a fourth aspect, the present disclosure provides an electronic device, including: A storage device, on which a computer program is stored; A processing device, configured to execute the computer program in the storage device to implement the steps of the method described in the first aspect of the present disclosure.

[0008] In a fifth aspect, the present disclosure provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect of the present disclosure are implemented.

[0009] Through the above technical solutions, a positive test is first performed on the object under test to obtain the service dependency information of the object under test, and the services called by the object under test and their service types are clarified. Thus, the automatic identification of the service call link is realized through the positive test, and the accuracy can be ensured while identifying the abnormal link dependency. After that, based on the services included in the service dependency information, the target service to be tested is selected, and then based on the target service type, the target service corresponding target service and the target exception type, the target exception test case is generated by using the use case generation model. Thus, the abnormal test cases that meet the requirements are automatically generated by the use case generation model, and the efficiency of generating the abnormal test cases is higher and the coverage is more comprehensive. And, the object under test is subjected to an abnormal test according to the generated target abnormal test case to obtain the corresponding abnormal test result. Thus, the automatic verification of the abnormal test and the generation of the test result can be realized based on the target abnormal test case. In this way, the identification of the abnormal service link dependency, the design of the abnormal test scenario, the verification of the abnormal test scenario, and the judgment of the test result can all be completed in an automated manner. Compared with the manual intervention method, not only the accuracy can be ensured, but also the overall efficiency of the abnormal test can be greatly improved.

[0010] Other features and advantages of the present disclosure will be described in detail in the subsequent specific implementation part. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Combined with the drawings and referring to the following specific implementation manners, the above and other features, advantages and aspects of the embodiments of the present disclosure will become more obvious. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic, and the original elements and elements are not necessarily drawn to scale. In the drawings: Figure 1 It is a flowchart of an exception testing method provided according to an embodiment of the present disclosure; Figure 2 In the exception testing method provided by the present disclosure, it is an exemplary flowchart for determining service dependency information of an object under test; Figure 3 It is a block diagram of an exception testing device provided according to an embodiment of the present disclosure; Figure 4 It shows a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure. Detailed implementation manners

[0012] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0013] It should be understood that the various steps recorded in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0014] As used herein, the term "including" and its variations are open-ended, that is, "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0015] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependent relationships.

[0016] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly stated in the context, it should be understood as "one or more".

[0017] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0018] It is understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to users in an appropriate manner and the authorization of users should be obtained in accordance with relevant laws and regulations.

[0019] For example, when responding to an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application program, a server, or a storage medium that executes the operation of the technical solution of the present disclosure according to the prompt message.

[0020] As an optional but non-limiting implementation manner, the manner of sending a prompt message to the user in response to receiving an active request from the user may be, for example, in the form of a pop-up window, and the prompt message may be presented in text in the pop-up window. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0021] It is understood that the above process of notifying and obtaining user authorization is only illustrative and does not limit the implementation manner of the present disclosure, and other manners that meet relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0022] At the same time, it is understood that the data involved in the technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws, regulations and related regulations.

[0023] Figure 1 is a flowchart of an exception testing method provided according to an embodiment of the present disclosure. The method provided by the present disclosure can be applied to an object with testing capabilities, and the object can perform exception testing on the object to be tested based on the method provided by the present disclosure. Exemplarily, the method provided by the present disclosure can be applied to a testing platform, a testing framework (for example, an automated testing framework), etc. As Figure 1 shown, the exception testing method provided by the present disclosure may include step 11 to step 13.

[0024] In step 11, by performing a positive test on the object to be tested, the service dependency information of the object to be tested is determined.

[0025] Among them, the service dependency information may include the services called by the object to be tested and the service types of each service. The object to be tested is the object to be tested, such as the system to be tested, the application program to be tested, etc.

[0026] In an automated test framework, positive test cases related to the object under test are usually maintained to verify whether the object under test meets the expected test scenarios under normal inputs and processes (for example, successful submission after correctly filling out a form). Executing a positive test on the object under test means running the pre-designed positive test cases through the automated test framework.

[0027] By executing a positive test on the object under test, the services called when the object under test is running normally can be identified. These services need to be concerned during the process of performing abnormal tests on the object under test, which facilitates targeted abnormal tests in the follow-up.

[0028] Optionally, the service dependency information may also include the call dependency relationships between the services called by the object under test. For example, Service A → Service B → Service C means that Service A calls Service B, and Service B calls Service C. Among them, Service B is the downstream service of Service A, and Service C is the downstream service of Service B.

[0029] In a possible implementation manner, step 11 may include the following steps: Use the generated positive test cases to test the object under test to obtain the log identification information returned by the object under test; According to the log identification information, determine the services carrying the log identification information and the call dependency relationships between the services carrying the log identification information in the log corresponding to the object under test for the positive test case; Based on the correspondence between the service type and the service name keyword, determine the service type of each service according to the service name corresponding to each service carrying the log identification information.

[0030] Optionally, the test platform may send a positive test request to the interface indicated by the positive test case based on the positive test case, so as to initiate a call to the object under test through the interface and execute the positive test case. And, the test platform will receive the response message returned by the object under test for the positive test request. There is a globally unique log identification information in the response message, and the log identification information can be extracted from the response message. Exemplarily, the log identification information may be the log identifier logid.

[0031] According to the log identification information, the log corresponding to the object under test for the positive test case can be located. For example, the test platform can initiate a query to the log storage system based on the log identification information to obtain the full-link log information corresponding to the current positive test request. Furthermore, all services carrying the log identification information and the call dependency relationships between services can be determined from this log. During the operation of the object under test, the called services will carry the log identification information and be recorded in the log. Therefore, through the log identification information, the services called during the execution of the positive test case can be directly determined, and the call dependency relationships between services can also be determined. Optionally, the test management platform can determine the call dependency relationships between services through a link tracing tool (such as Zipkin) or a service registry (such as Consul).

[0032] In the present disclosure, a service can be identified by a service name. Optionally, the service name can be PSM (Product Service Mark), which is a mark used to uniquely identify a service in a distributed system (especially a microservices architecture) and is usually defined by the internal naming convention of an enterprise.

[0033] Optionally, the service types can include middleware and external services. Middleware is a basic setup component inside the object under test and is used to support communication or data processing between services. Optionally, middleware can include, but is not limited to, database middleware (such as MySQL), message queues (such as Kafka), cache middleware (such as Redis), API (Application Programming Interface) gateways, load balancers, etc. External services are services provided by third parties and are usually called through HTTP / RPC. Optionally, external services can include, but are not limited to, payment interfaces, map APIs, cloud storage services, third-party logins, weather services, etc.

[0034] Generally, there is a corresponding relationship between the service type and the service name keyword, which can be pre-constructed according to the actual application situation. For example, which service name keyword or keywords a service type such as middleware corresponds to, and which service name keyword or keywords an external service type corresponds to can be determined. For example, the name of a service based on the cache middleware Redis usually carries the service name keyword Redis. Therefore, the service type of middleware corresponds to the service name keyword Redis. In this way, the service type of a service can be determined by the service name. For example, the PSM of a service usually carries keywords related to the service type. Therefore, the service type corresponding to the service can be further determined according to the PSM of the service.

[0035] Based on the correspondence between service types and service name keywords, the service name of a service can be matched with the service name keywords, and the service type with a successful match can be determined as the service type corresponding to the service. By way of example, if there is a correspondence between the service type of middleware and the service name keyword of Redis in the correspondence between service types and service name keywords, then if the service name of the service carries the string Redis, it can be considered a successful match, and the service type of the service is determined to be middleware.

[0036] By way of example, the process of determining the service dependency information of the object under test can be as Figure 2 shown. In Figure 2 , the test platform executes positive test cases by calling the interfaces of the object under test, determines the services called by the object under test and the call dependency relationships between the services by obtaining log identification information (logid), and determines the service type (whether it is an external service or middleware) based on PSM to determine the service dependency information.

[0037] Through the above method, by performing positive testing on the object under test, it is possible to automatically determine the services called by the object under test, the service types of each service, and the call dependency relationships between the services through the log identification information returned by the object under test, which is accurate and efficient, can quickly prepare for subsequent exception testing, and is conducive to improving the efficiency of exception testing.

[0038] Returning to Figure 1 , in step 12, for the target service to be tested, according to the target service type, the target business type corresponding to the target service, and the target exception type to be tested, a target exception test case is generated using the pre-generated use case generation model.

[0039] Among them, the target service is the service included in the service dependency information.

[0040] In the present disclosure, after determining the service dependency information including the services called by the object under test through step 11, the target service can be determined specifically to perform exception testing on the target service. By way of example, several services can be selected from the services included in the service dependency information according to actual needs, and these selected services are respectively used as target services for exception testing. For another example, each service included in the service dependency information can be respectively used as a target service for exception testing.

[0041] For the target service to be tested, according to the target service type, the target business type corresponding to the target service, and the target exception type to be tested, a target exception test case can be generated using the use case generation model.

[0042] The target service type can be the service type of the target service and / or the service type of the downstream service of the target service. Among them, the downstream service of the target service is determined through the call dependency. The downstream service of the target service is the service called after the target service is called in the call dependency. For example, if in the call dependency service A → service B (service A calls service B), then service B is the downstream service of service A.

[0043] The target exception type is the exception type tested when performing an exception test on the target service. The target exception type usually includes a first exception type for characterizing data exceptions, a second exception type for characterizing service exceptions, or a third exception type for characterizing network exceptions.

[0044] Optionally, data exceptions can include but are not limited to interface input parameter exceptions, interface output parameter exceptions, etc.; service exceptions can include but are not limited to CPU (Central Processing Unit) full load, memory leak, database master-slave delay, Redis cache penetration, etc.; network exceptions can include but are not limited to network rejection, network timeout, network packet loss, etc.

[0045] Based on this, in a possible implementation manner, the method provided by the present disclosure may further include the following steps: If the target exception type is the first exception type, determine that the target service type is the service type of the target service; If the target exception type is the second exception type or the third exception type, determine that the target service type includes the service type of the downstream service of the target service.

[0046] If the target exception type is the first exception type, it indicates that the exception test has a higher correlation with the service itself. When generating the corresponding exception test case, it is necessary to base on the service type of the target service itself, and more appropriate exception test cases can be obtained. Therefore, it can be determined that the target service type is the service type of the target service.

[0047] If the exception type is the second exception type or the third exception type, it indicates that the exception test has a higher correlation with the downstream service depending on the target service. When generating the corresponding exception test case, it is necessary to base on the service type of the downstream service of the target service, and more appropriate exception test cases can be obtained. Therefore, it can be determined that the target service type is the service type of the downstream service of the target service.

[0048] Optionally, if the exception type is the second exception type or the third exception type, the service type of the target service and the service types of the downstream services of the target service can also be jointly included in the target service type, that is, the target service type includes both the service type of the target service itself and the service types of the downstream services of the target service, so as to make the generation of exception test cases have richer reference information.

[0049] The target business type is the business type corresponding to the target service, and the business type can be set according to actual needs. Optionally, the target business type may include, but is not limited to, order services, billing services, etc.

[0050] In a possible implementation manner, the mapping relationship between the service type, the business type, the exception type, and the exception test case can be pre-constructed to form a use case generation model. In this way, according to the mapping relationship indicated by the use case generation model, the exception test case corresponding to the target service type, the target business type, and the target exception type can be determined as the target exception test case.

[0051] In another possible implementation manner, step 12 may include the following steps: Input the target service type, the target business type, and the target exception type into the use case generation model to obtain at least one exception test case output by the use case generation model; Determine the target exception test case from the at least one exception test case.

[0052] Optionally, the use case generation model may be a model generated based on a machine learning algorithm.

[0053] In a possible embodiment, the use case generation model can be generated in the following manner: Obtain multiple groups of sample data, where a group of sample data includes a service type sample, a business type sample, an exception type sample, and at least one exception test case sample; Train the large model by taking the service type sample, the business type sample, and the exception type sample as the input of the model and taking at least one exception test case sample corresponding to the service type sample, the business type sample, and the exception type sample as the target output of the model, so as to obtain the trained use case generation model.

[0054] Among them, multiple groups of sample data can be obtained from the abnormal test knowledge base. The abnormal test knowledge base can be constructed based on the abnormal test cases used in historical abnormal tests and their corresponding service types, business types, and abnormal types. For the abnormal tests that have been carried out, the abnormal test cases used in the abnormal tests, as well as the business type, service type of the service targeted by the abnormal test, and the abnormal type tested by the abnormal test, can be collected and stored in the abnormal test knowledge base in an associated manner. On this basis, by tracing back the abnormal traffic, the input parameters and output parameters in the case of data anomalies can be associated and expanded, and the common anomalies between services and middleware anomalies can be summarized to generate new abnormal test cases to enrich the information in the abnormal test knowledge base. In the abnormal test knowledge base, the same service type, business type, and abnormal type can correspond to diverse abnormal test cases.

[0055] Based on the constructed abnormal test knowledge base, the associated service type, business type, abnormal type, and at least one abnormal test case stored therein can be obtained as service type samples, business type samples, abnormal type samples, and abnormal test case samples to obtain multiple groups of sample data.

[0056] After obtaining the sample data, the training of the model can be carried out. The use case generation model can be trained based on a large model. By obtaining an existing large model (such as, ChatGPT), and then using the obtained sample data to train the large model to optimize the weight parameters in the model to adapt to the generation scenario of abnormal test cases. Among them, the service type samples, business type samples, and abnormal type samples are used as the input of the model, and at least one abnormal test case sample corresponding to the service type samples, business type samples, and abnormal type samples is used as the target output of the model to train the large model to obtain the trained use case generation model. Exemplarily, in each training process, the large model can obtain an output result for the input service type samples, business type samples, and abnormal type samples, and update the parameters inside the large model by calculating the loss function between the input result and the target output until the training is completed.

[0057] The trained use case generation model can output several available abnormal test cases related to the input service type, business type, and abnormal type for the user to select. When training the use case generation model, using the service type of the service instead of the service itself (for example, the service name) can enhance the generalization ability of the use case generation model.

[0058] In this way, when the target service type, target business type and target exception type are input into the use case generation model, at least one exception test case output by the use case generation model can be obtained, and then the target exception test case is determined in the at least one exception test case. For example, one or more of the exception test cases output by the use case generation model can be selected as exception test cases according to actual needs, and used in subsequent exception tests. For another example, each of the exception test cases output by the use case generation model can be used as a target exception test case and used in subsequent exception tests.

[0059] Optionally, for the first exception type, when constructing the exception test knowledge base, the field type of the input parameter of the interface corresponding to the object under test can also be recorded. Accordingly, when obtaining sample data, the field type can be used as a field type sample to form a set of sample data with a service type sample, a business type sample, an exception type sample, and at least one exception test case sample. Then, during training, the field type sample, the service type sample, the business type sample, and the exception type sample are used as the input of the model, and the corresponding at least one exception test case sample is used as the target output of the model to train the large model to obtain a trained use case generation model. In this way, if the target exception type is the first exception type, while inputting the target service type, the target business type, and the target exception type into the use case generation model, the field type can also be input together to obtain a more suitable target exception test case.

[0060] Through the above method, based on the use case generation model, exception test cases can be generated specifically for the target service based on the target service type, target business type and target exception type, with high generation efficiency and wide coverage.

[0061] In step 13, an abnormal test is performed on the object under test according to the target abnormal test case, and an abnormal test result corresponding to the target abnormal test case is determined.

[0062] After determining the target abnormal test case, the abnormal test can be performed on the object under test to verify whether the actual performance of the object under test meets the expected performance of the target abnormal test case design, and then determine the abnormal test result. If the actual performance of the object under test is consistent with the expected performance corresponding to the target abnormal test case, the abnormal test result can be determined as passed; if the actual performance of the object under test is inconsistent with the expected performance corresponding to the target abnormal test case, the abnormal test result can be determined as failed.

[0063] Optionally, the target exception test case may include a target exception injection method and target input parameters. Among them, exception injection means artificially injecting a preset fault into the object under test (such as service unavailability, latency, etc.) to simulate an exception scenario that may occur in a real environment. The target exception injection method is used to indicate the method required to construct the exception scenario corresponding to the target exception test case. Exemplarily, the target exception injection method may be network layer injection (such as simulating network latency, packet loss, interruption, etc. through a proxy or firewall rules), service layer injection (such as intercepting service requests, returning error codes or timeouts, etc.), data layer injection (such as inputting abnormal parameters, forced failure, etc.), etc. The target input parameter is the input parameter for calling the interface of the object under test in the exception test.

[0064] In one possible implementation manner, step 13 may include the following steps Perform exception injection on the object under test according to the target exception injection method; Send an exception test request to the object under test that has been injected with an exception, and obtain the target response result returned by the object under test. The exception test request carries the target input parameter; Determine the exception test result according to the target response result.

[0065] According to the target exception injection method indicated by the target exception test case, first, according to the target exception injection method, perform exception injection on the object under test. Exemplarily, if the target exception type is the first exception type, the exception injection may be embodied as a way of providing incorrect input parameters, that is, setting the target input parameter as an abnormal parameter. For another example, if the target exception type is the second exception type or the third exception type, the exception injection may be implemented by injecting an abnormal service, that is, constructing the running environment of the object under test as the abnormal environment indicated by the target exception test case.

[0066] After the exception injection is completed, an exception test request carrying the target input parameter may be sent to the object under test that has been injected with an exception to trigger an exception test to verify the target exception test case. The object under test returns a target response result for the exception test request. Based on this target response result, the exception test result can be further determined.

[0067] Optionally, the target response result may include the log information of the object under test corresponding to the exception test case. The log information may include the business result and program performance returned by the object under test. The business result is whether the output of the object under test in terms of functional logic meets the expectation, which can reflect whether the business rules can be correctly implemented. For example, whether the login is successful. The program performance is a series of performances of the object under test during the execution of the target exception test case. In one possible implementation manner, according to the non-functional performance of the target response at runtime, it can reflect the performance, stability, etc. of the system, such as performance indicators, resource consumption, etc.

[0068] In a possible implementation manner, determining an abnormal test result according to a target response result may include the following steps: Obtain a verification rule, where the verification rule includes at least one verification item corresponding to the target abnormal type, and each verification item corresponds to a verification method; For at least one verification item corresponding to the target abnormal type, according to the log information, use the verification method corresponding to the verification item to determine whether the target response result passes the verification; In response to determining that there is a target verification item that fails to pass the verification, generate an abnormal test result for indicating that the target verification item fails to pass; In response to determining that all at least one verification item corresponding to the target abnormal type passes the verification, generate an abnormal test result for indicating that the abnormal test passes.

[0069] Among them, the verification rule can be pre-designed and usually includes at least one verification item and its verification method. Each verification item corresponds to a determination dimension for determining whether the performance of the object under test passes in the abnormal test. For a verification item, a ternary correspondence relationship can be set among the input parameter, the abnormal type, and the reference response result (that is, the service result and the program performance). This ternary correspondence relationship stipulates the reference response result that the object under test should return in the abnormal test with the specified input parameter and the specified abnormal type, so as to compare with the actual target response result of the object under test, and then determine whether the corresponding verification item passes the verification.

[0070] Optionally, the verification method corresponding to the verification item can be used to verify the following items: Strong dependencies are expected between services but are actually weak dependencies; weak dependencies are expected between services but are actually strong dependencies; service panic; concurrent requests are not idempotent; abnormal returns are not processed, etc.

[0071] For example, for the verification item that weak dependencies are expected between services but are actually strong dependencies, its reference response result can be that when the downstream service is unavailable, its upstream service returns an error or times out. In this way, it is easy to determine whether the verification passes according to the actual performance of the target response result.

[0072] In addition, the verification item and its verification method can also be customized according to actual needs. Exemplarily, the verification rule can be implemented by designing a rule engine to maintain the above ternary correspondence relationship.

[0073] Thus, for at least one verification item corresponding to the target abnormal type, according to the log information (including the service result and the program performance), using the verification method corresponding to this verification item, it can be determined whether the target response result passes the verification for this verification item.

[0074] In this way, based on the target response result, for each verification item corresponding to the target exception type, a verification result indicating whether the verification passes can be obtained respectively. Furthermore, if there is a target verification item that fails the verification, an exception test result indicating that the target verification item fails can be generated. If each verification item corresponding to the target exception type passes the verification, an exception test result indicating that the exception test passes can be generated. Exemplarily, the exception test result can be presented in the form of a test report.

[0075] Through the above technical solution, first, a positive test is performed on the object under test to obtain the service dependency information of the object under test, and the services called by the object under test and their service types are clarified. Thus, the automated identification of the service call link is achieved through the positive test, and the accuracy can be ensured while identifying the abnormal link dependency. After that, based on the services included in the service dependency information, the target service to be tested is selected, and then based on the target service type, the target business corresponding to the target service, and the target exception type, the target exception test case is generated by using the pre-generated use case generation model. Thus, the exception test cases that meet the requirements are automatically generated by the use case generation model, and the efficiency of generating the exception test cases is higher and the coverage is more comprehensive. Moreover, the object under test is subjected to an exception test according to the generated target exception test case to obtain the corresponding exception test result. Thus, the automated verification of the exception test and the generation of the test result can be achieved based on the target exception test case. In this way, the identification of the abnormal service link dependency, the design of the abnormal test scenario, the verification of the abnormal test scenario, and the judgment of the test result can all be completed in an automated manner. Compared with the manual intervention method, not only the accuracy can be ensured, but also the overall efficiency of the exception test can be greatly improved.

[0076] Figure 3 is a block diagram of an exception test device provided according to an embodiment of the present disclosure. As Figure 3 shown, the device 30 may include: A first determination module 31, configured to determine the service dependency information of the object under test by performing a positive test on the object under test, where the service dependency information includes the services called by the object under test and the service types of each service; A use case generation module 32, configured to generate a target exception test case for the target service to be tested according to the target service type, the target business type corresponding to the target service, and the target exception type to be tested, by using a pre-generated use case generation model, where the target service is the service included in the service dependency information; A second determination module 33, configured to perform an exception test on the object under test according to the target exception test case and determine the exception test result corresponding to the target exception test case.

[0077] Optionally, the service dependency information further includes the call dependency relationships between the services invoked by the object under test; The first determination module 31 includes: A first test sub-module, configured to test the object under test by using the generated positive test cases, and obtain the log identification information returned by the object under test; A first determination sub-module, configured to determine, according to the log identification information, the services carrying the log identification information and the call dependency relationships between the services carrying the log identification information in the log corresponding to the object under test for the positive test cases; A second determination sub-module, configured to determine the service types of the respective services based on the correspondence between the service type and the service name keyword, according to the service names corresponding to the services carrying the log identification information.

[0078] Optionally, the service dependency information further includes the call dependency relationships between the services invoked by the object under test; The target service type is the service type of the target service and / or the service type of the downstream service of the target service, and the downstream service of the target service is determined through the call dependency relationship.

[0079] Optionally, the target exception type includes a first exception type for characterizing data exceptions, a second exception type for characterizing service exceptions, or a third exception type for characterizing network exceptions; The apparatus 30 further includes: A third determination module, configured to determine that the target service type is the service type of the target service if the target exception type is the first exception type; A fourth determination module, configured to determine that the target service type includes the service type of the downstream service of the target service if the target exception type is the second exception type or the third exception type.

[0080] Optionally, the use case generation module 32 includes: A processing sub-module, configured to input the target service type, the target service type, and the target exception type into the use case generation model, and obtain at least one exception test case output by the use case generation model; A third determination sub-module, configured to determine a target exception test case from the at least one exception test case.

[0081] Optionally, the use case generation model is generated by the following modules: An acquisition module, configured to acquire multiple groups of sample data, where a group of sample data includes a service type sample, a service type sample, an exception type sample, and at least one exception test case sample; A training module, which is used to train a large model by taking the service type sample, the business type sample, and the exception type sample as the input of the model, and taking at least one exception test case sample corresponding to the service type sample, the business type sample, and the exception type sample as the target output of the model, so as to obtain a trained use case generation model.

[0082] Optionally, the target exception test case includes a target exception injection method and target input parameters; The second determination module 33 includes: An exception injection sub-module, which is used to inject an exception into the object under test according to the target exception injection method; A second test sub-module, which is used to send an exception test request to the object under test that has been injected with an exception, and obtain a target response result returned by the object under test, where the exception test request carries the target input parameters; A fourth determination sub-module, which is used to determine the exception test result according to the target response result.

[0083] Optionally, the target response result includes log information of the object under test corresponding to the exception test case; The fourth determination sub-module includes: An acquisition sub-module, which is used to acquire a verification rule, where the verification rule includes at least one verification item corresponding to the target exception type, and each verification item corresponds to a verification method; A fifth determination sub-module, which is used to determine whether the target response result passes the verification for at least one verification item corresponding to the target exception type according to the log information and using the verification method corresponding to the verification item; A first generation sub-module, which is used to generate an exception test result for indicating that the target verification item fails to pass in response to determining that there is a target verification item that fails to pass the verification; A second generation sub-module, which is used to generate an exception test result for indicating that the exception test passes in response to determining that all at least one verification item corresponding to the target exception type passes the verification.

[0084] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0085] Based on the same inventive concept, an embodiment of the present disclosure also provides a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processing device, it implements the steps of the exception test method described in any embodiment of the present disclosure.

[0086] Based on the same inventive concept, an embodiment of the present disclosure also provides an electronic device, including: a storage device storing a computer program thereon; a processing device configured to execute the computer program in the storage device to implement the steps of the abnormal test method according to any embodiment of the present disclosure.

[0087] Based on the same inventive concept, an embodiment of the present disclosure also provides a computer program product including a computer program, which when executed by a processor implements the steps of the abnormal test method according to any embodiment of the present disclosure.

[0088] Reference is now made to Figure 4 , which shows a schematic structural diagram of an electronic device 600 suitable for implementing an embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0089] As Figure 4 shown, the electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0090] Generally, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 4 shows the electronic device 600 having various devices, it should be understood that it is not required to implement or include all the shown devices. More or fewer devices may be alternatively implemented or included.

[0091] In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program including program code for performing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by a processing device 601, the above-described functions defined in the method of the embodiment of the present disclosure are performed.

[0092] It should be noted that the above-mentioned computer-readable medium in the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. And in the present disclosure, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0093] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0094] The above computer-readable medium can be included in the above electronic device; or it can exist separately without being assembled into the electronic device.

[0095] The above computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: Determine the service dependency information of the object under test by performing a positive test on the object under test, where the service dependency information includes the services called by the object under test and the service types of each service; For a target service to be tested, generate a target exception test case using a pre-generated use case generation model according to the target service type, the target service type corresponding to the target service, and the target exception type to be tested, where the target service is a service included in the service dependency information; Perform an exception test on the object under test according to the target exception test case and determine the exception test result corresponding to the target exception test case.

[0096] Computer program code for performing the operations of the present disclosure can be written in one or more programming languages or combinations thereof. The above programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, and C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0097] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0098] The modules described in the embodiments of the present disclosure can be implemented in software or in hardware. In some cases, the name of a module does not constitute a limitation on the module itself. For example, the first determination module can also be described as "the module that determines the service dependency information of the object under test by performing a positive test on the object under test".

[0099] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. By way of example, and without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGA), Application Specific Integrated Circuits (ASIC), Application Specific Standard Products (ASSP), System on a Chip (SOC), Complex Programmable Logic Devices (CPLD), and the like.

[0100] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM or Flash memory), an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0101] According to one or more embodiments of the present disclosure, an exception testing method is provided, and the method includes: By performing a positive test on the object under test, determine the service dependency information of the object under test, where the service dependency information includes the services called by the object under test and the service types of each service; For a target service to be tested, according to the target service type, the target business type corresponding to the target service, and the target exception type to be tested, use a pre-generated use case generation model to generate a target exception test case, where the target service is a service included in the service dependency information; Perform an exception test on the object under test according to the target exception test case, and determine an exception test result corresponding to the target exception test case.

[0102] According to one or more embodiments of the present disclosure, an exception testing method is provided, and the service dependency information further includes the call dependency relationship between the services called by the object under test; The step of determining the service dependency information of the object under test by performing a positive test on the object under test includes: Use the generated positive test cases to test the object under test to obtain the log identification information returned by the object under test; According to the log identification information, determine the services carrying the log identification information and the call dependency relationship between the services carrying the log identification information in the log corresponding to the positive test case of the object under test; Based on the correspondence between the service type and the service name keyword, determine the service type of each service according to the service name corresponding to each service carrying the log identification information.

[0103] According to one or more embodiments of the present disclosure, an exception testing method is provided, and the service dependency information further includes the call dependency relationship between the services called by the object under test; The target service type is the service type of the target service and / or the service type of the downstream service of the target service, and the downstream service of the target service is determined through the call dependency relationship.

[0104] According to one or more embodiments of the present disclosure, an exception testing method is provided, and the target exception type includes a first exception type for characterizing data exceptions, a second exception type for characterizing service exceptions, or a third exception type for characterizing network exceptions; The method further includes: If the target exception type is the first exception type, determine that the target service type is the service type of the target service; If the target exception type is the second exception type or the third exception type, determine that the target service type includes the service types of the downstream services of the target service.

[0105] According to one or more embodiments of the present disclosure, an exception testing method is provided. According to the target service type, the target service type corresponding to the target service, and the target exception type to be tested, using a pre-generated use case generation model, generate a target exception test case, including: Input the target service type, the target business type, and the target exception type into the use case generation model, and obtain at least one exception test case output by the use case generation model; Determine the target exception test case from the at least one exception test case.

[0106] According to one or more embodiments of the present disclosure, an exception testing method is provided. The use case generation model is generated in the following manner: Obtain multiple groups of sample data, where a group of sample data includes a service type sample, a business type sample, an exception type sample, and at least one exception test case sample; Train a large model by using the service type sample, the business type sample, and the exception type sample as the input of the model, and using at least one exception test case sample corresponding to the service type sample, the business type sample, and the exception type sample as the target output of the model, so as to obtain a trained use case generation model.

[0107] According to one or more embodiments of the present disclosure, an exception testing method is provided. The target exception test case includes a target exception injection method and target input parameters; Perform exception testing on the object under test according to the target exception test case, and determine the exception test result corresponding to the target exception test case, including: Perform exception injection on the object under test according to the target exception injection method; Send an exception test request to the object under test that has been injected with an exception, and obtain a target response result returned by the object under test. The exception test request carries the target input parameters; Determine the exception test result according to the target response result.

[0108] According to one or more embodiments of the present disclosure, an exception testing method is provided. The target response result includes the log information of the object under test corresponding to the exception test case; Determine the exception test result according to the target response result, including: Obtain the verification rules, where the verification rules include at least one verification item corresponding to the target exception type, and each verification item corresponds to a verification method; For at least one verification item corresponding to the target exception type, according to the log information, use the verification method corresponding to the verification item to determine whether the target response result passes the verification; In response to determining that there is a target verification item that fails to pass the verification, generate an exception test result for indicating that the target verification item fails; In response to determining that all at least one verification item corresponding to the target exception type passes the verification, generate an exception test result for indicating that the exception test passes.

[0109] According to one or more embodiments of the present disclosure, there is provided an exception testing device, the device includes: A first determination module, configured to determine the service dependency information of the object under test by performing a positive test on the object under test, where the service dependency information includes the services called by the object under test and the service types of each service; A use case generation module, configured to, for a target service to be tested, according to the target service type, the target service type corresponding to the target service, and the target exception type to be tested, use a pre-generated use case generation model to generate a target exception test case, where the target service is a service included in the service dependency information; A second determination module, configured to perform an exception test on the object under test according to the target exception test case and determine an exception test result corresponding to the target exception test case.

[0110] According to one or more embodiments of the present disclosure, there is provided a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processing device, the steps of the exception testing method described in any embodiment of the present disclosure are implemented.

[0111] According to one or more embodiments of the present disclosure, there is provided an electronic device, including: A storage device, on which a computer program is stored; A processing device, configured to execute the computer program in the storage device to implement the steps of the exception testing method described in any embodiment of the present disclosure.

[0112] According to one or more embodiments of the present disclosure, there is provided a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the exception testing method described in any embodiment of the present disclosure are implemented.

[0113] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0114] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0115] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

Claims

1. An abnormal test method, characterized in that, The method includes: Determining service dependency information of the object under test by performing a positive test on the object under test, where the service dependency information includes the services called by the object under test and the service types of each service; For a target service to be tested, according to the target service type, the target business type corresponding to the target service, and the target exception type to be tested, using a pre-generated use case generation model to generate a target exception test case, where the target service is a service included in the service dependency information; Performing an exception test on the object under test according to the target exception test case and determining an exception test result corresponding to the target exception test case.

2. The method according to claim 1, characterized in that The service dependency information further includes the call dependency relationships between the services called by the object under test; The determining the service dependency information of the object under test by performing a positive test on the object under test includes: Testing the object under test using the generated positive test cases to obtain log identification information returned by the object under test; According to the log identification information, determining, in the log of the object under test corresponding to the positive test case, the services carrying the log identification information and the call dependency relationships between the services carrying the log identification information; Based on the correspondence between the service type and the service name keyword, determining the service type of each service according to the service names corresponding to the services carrying the log identification information.

3. The method according to claim 1, wherein The service dependency information further includes the call dependency relationships between the services called by the object under test; The target service type is the service type of the target service and / or the service type of the downstream service of the target service, and the downstream service of the target service is determined through the call dependency relationship.

4. The method according to claim 3, wherein The target exception type includes a first exception type for characterizing data exception, a second exception type for characterizing service exception, or a third exception type for characterizing network exception; The method further includes: If the target exception type is the first exception type, determining that the target service type is the service type of the target service; If the target exception type is the second exception type or the third exception type, determining that the target service type includes the service type of the downstream service of the target service.

5. The method according to claim 1, wherein The generating the target exception test case by using the pre-generated use case generation model according to the target service type, the target business type corresponding to the target service, and the target exception type to be tested includes: Inputting the target service type, the target business type, and the target exception type into the use case generation model to obtain at least one exception test case output by the use case generation model; Determining a target exception test case from the at least one exception test case.

6. The method according to claim 1, wherein The use case generation model is generated by the following method: Obtaining multiple groups of sample data, where a group of sample data includes a service type sample, a business type sample, an exception type sample, and at least one exception test case sample; The large model is trained by taking the service type sample, the business type sample, and the exception type sample as the input of the model, and taking at least one exception test case sample corresponding to the service type sample, the business type sample, and the exception type sample as the target output of the model, so as to obtain a trained use case generation model.

7. The method according to claim 1, wherein The target exception test case includes a target exception injection method and target input parameters; The performing an exception test on the object under test according to the target exception test case and determining an exception test result corresponding to the target exception test case includes: Performing exception injection on the object under test according to the target exception injection method; Sending an exception test request to the object under test that has been injected with an exception to obtain a target response result returned by the object under test, where the exception test request carries the target input parameters; Determining the exception test result according to the target response result.

8. The method according to claim 7, wherein The target response result includes log information of the object under test corresponding to the exception test case; The determining the exception test result according to the target response result includes: Obtaining a verification rule, where the verification rule includes at least one verification item corresponding to the target exception type, and each verification item corresponds to a verification method; For at least one verification item corresponding to the target exception type, determining whether the target response result passes the verification by using the verification method corresponding to the verification item according to the log information; In response to determining that there is a target verification item that fails to pass the verification, generating an exception test result for indicating that the target verification item fails to pass; In response to determining that at least one verification item corresponding to the target exception type passes the verification, generating an exception test result for indicating that the exception test passes.

9. An abnormal test device, characterized in that, The device includes: A first determination module, configured to determine service dependency information of the object under test by performing a positive test on the object under test, where the service dependency information includes services called by the object under test and service types of each service; A use case generation module, configured to generate a target exception test case for a target service to be tested according to the target service type, the target business type corresponding to the target service, and the target exception type to be tested, by using a pre-generated use case generation model, where the target service is a service included in the service dependency information; A second determination module, configured to perform an exception test on the object under test according to the target exception test case and determine an exception test result corresponding to the target exception test case.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processing device, it implements the steps of the method according to any one of claims 1-8.

11. An electronic device, characterized in that, including: A storage device, on which a computer program is stored; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1-8.

12. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-8.

Images