Test method, device, electronic equipment, storage medium and program product

Abstract

Embodiments of the present application disclose a test method, device, electronic equipment, storage medium and program product. The test method generates a sample data set, stores the sample data set to a benchmark database, generates a test case according to the sample data set, determines a link identifier corresponding to the test case when a version of a target program is changed and the target program after the version change is triggered to run the test case, and quickly obtains the corresponding sample data set from the benchmark database according to the link identifier, so as to facilitate comparison of the test data set and the sample data set and obtain a test result. The test method can realize fast full-link automatic testing, achieve fine testing effect, improve the accuracy and reliability of the test result, and improve the stability and robustness of the target program running. The test method provided by the embodiments of the present application can be applied to test scenes of programs such as cloud technology, artificial intelligence, intelligent transportation and auxiliary driving.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a testing method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] With the continuous development of internet communication technology, providing various business services using different programs based on internet communication technology has become a mainstream trend. To ensure the effective implementation of business functions, relevant tests need to be conducted on the target program.

[0003] In related technologies, testing the operation of a target program often requires manually writing test cases. After running the test cases, the actual running results of the test cases are compared with the expected running results in the test cases to determine the operation of the target program. However, the reliability of the test results obtained by testing the target program in the above way needs to be improved. Summary of the Invention

[0004] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0005] This invention provides a testing method, apparatus, electronic device, storage medium, and program product that can achieve rapid, end-to-end automated testing, achieve refined testing results, improve the accuracy and reliability of test results, and enhance the stability and robustness of the target program.

[0006] On one hand, embodiments of the present invention provide a testing method, including:

[0007] A sample data set is generated and stored in a benchmark database. The sample data set includes processing data of multiple service modules called when processing business requests of the target program. The processing data of each service module includes a link identifier, which is used to identify the service call link formed by multiple service modules.

[0008] Test cases are generated based on the sample data set;

[0009] When the version of the target program changes, the target program with the changed version is triggered to run the test cases, obtain a test data set, determine the link identifier corresponding to the test cases, obtain the corresponding sample data set from the benchmark database based on the link identifier, compare the test data set with the sample data set, and obtain the test results.

[0010] On the other hand, embodiments of the present invention also provide a testing apparatus, including:

[0011] A data generation module is used to generate a sample data set and store the sample data set in a benchmark database. The sample data set includes processing data from multiple service modules called when processing business requests of the target program. The processing data of each service module includes a link identifier, which is used to identify a service call link formed by multiple service modules.

[0012] The test case generation module is used to generate test cases based on the sample data set.

[0013] The data comparison module is used to trigger the target program after version change to run the test cases when the version of the target program changes, obtain the test data set, determine the link identifier corresponding to the test cases, obtain the corresponding sample data set from the benchmark database according to the link identifier, compare the test data set with the sample data set, and obtain the test results.

[0014] Furthermore, the aforementioned data generation module is specifically used for:

[0015] Obtain the request data of each service module in the service call chain, and forward the request data to the corresponding service module;

[0016] Obtain the response data returned based on the request data, and forward the response data to the corresponding service module;

[0017] The request data and response data of each service module are merged according to a preset data type to obtain the processing data of each service module;

[0018] The sample data set is obtained by merging the processing data from multiple service modules.

[0019] Furthermore, the aforementioned data generation module is specifically used for:

[0020] The target function is invoked to serialize the sample data set, resulting in the target data set.

[0021] Obtain the target file compiled based on the preset file format, and convert the target data set into the target file format according to the target file;

[0022] The target data set, converted into the target file format, is stored in the benchmark database.

[0023] Furthermore, the aforementioned testing apparatus also includes a data extraction module, which is used for:

[0024] Add the sample data set to the task list;

[0025] In response to a task processing instruction, the sample data set is retrieved from the task list and sent to the thread pool;

[0026] In response to a thread processing instruction, the sample data set is extracted from the thread pool and the sample data set is verified.

[0027] Furthermore, the aforementioned data extraction module is specifically used for at least one of the following:

[0028] Based on the sample data set, the multiple service modules called by the target program when processing the business request are connected in series to obtain the service call chain. The service call chain is then compared with a preset call chain to determine the integrity of the service call chain.

[0029] or,

[0030] A first tag for identifying the sample data set is extracted from the sample data set, and multiple second tags are extracted from the benchmark database. Based on the matching relationship between the first tag and the multiple second tags, the storage status of the sample data set in the benchmark database is determined, wherein the storage status is either stored in the benchmark database or not stored in the benchmark database.

[0031] Furthermore, the aforementioned use case generation module is specifically used for:

[0032] Based on the sample data set, the request data and response data of each service module in the service call chain, as well as the link identifier corresponding to the service call chain, are obtained;

[0033] Obtain a preset test case template, and add the request data and the response data to the corresponding positions in the preset test case template to obtain the test case;

[0034] The test cases are identified based on the link identifier.

[0035] Furthermore, the aforementioned use case generation module is also used for:

[0036] The response data of each service module in the service call chain are merged to obtain the result simulation data;

[0037] Generate the result simulation identifier corresponding to the result simulation data;

[0038] Generate key-value pairs based on the simulated data and the simulated identifier, and add the key-value pairs to the test case.

[0039] Furthermore, the aforementioned data comparison module is also used for:

[0040] The target program, after triggering the version change, generates a test request based on the test cases;

[0041] Determine the first service module currently invoked when processing the test request, and obtain the simulation status of the second service module bypassed by the first service module. The simulation status is used to characterize whether the response data returned by the second service module to the first service module needs to be simulated or not.

[0042] When the simulation state indicates that the response data returned by the second service module to the first service module needs to be simulated, the result simulation data is obtained according to the result simulation identifier, and the result simulation data is used as the response data returned by the second service module to the first service module.

[0043] Furthermore, the aforementioned data comparison module is also used for:

[0044] The test cases are run in the continuous integration platform at a preset frequency.

[0045] The total number of times the test cases were successfully run in the continuous integration platform;

[0046] When the number of successful runs is greater than or equal to a preset threshold, the test case is added to the test case library of the continuous integration platform.

[0047] Furthermore, the aforementioned data comparison module is also used for:

[0048] Obtain the historical runtime of the test case when it is successfully run in the continuous integration platform, and determine the target runtime based on the historical runtime.

[0049] Obtain the benchmark data set obtained by running the test cases at the target runtime, and use the benchmark data set to update the sample data set in the benchmark database.

[0050] Furthermore, the aforementioned data comparison module is also used for:

[0051] The test data set is compared with the request data of the same service module in the sample data set;

[0052] or,

[0053] The test dataset is compared with the response data of the corresponding service module in the sample dataset.

[0054] Furthermore, the aforementioned data comparison module is also used for:

[0055] According to the preset filtering rules, data to be removed is filtered from the test data set and the sample data set, and the data to be removed is then removed.

[0056] The step of selecting data to be removed from the test data set and the sample data set according to preset filtering rules includes at least one of the following:

[0057] Data with random numbers as its data attribute is selected as the data to be removed from the test data set and the sample data set.

[0058] or,

[0059] The data to be removed is selected from the test data set and the sample data set using wildcards;

[0060] or,

[0061] Variable strings are extracted from the target type of data, and the variable strings are used as the data to be removed. The data to be removed is then selected from the test data set and the sample data set.

[0062] Furthermore, the target program generates the business request in the first operating environment, and runs the test case in the second operating environment. The data generation module is also used for:

[0063] Determine the request type of the business request;

[0064] The risk level of the request type is determined according to the preset business classification rules;

[0065] When the risk level is less than or equal to a preset level threshold, the sample data set is determined as the data set to be generated.

[0066] On the other hand, embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described testing method.

[0067] On the other hand, embodiments of the present invention also provide a computer-readable storage medium storing a program that is executed by a processor to implement the above-described testing method.

[0068] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the test method described above.

[0069] The embodiments of this invention offer at least the following beneficial effects: By generating a sample data set and storing it in a benchmark database, test cases are generated based on the sample data set. Since the sample data set includes processing data from multiple service modules invoked when handling business requests of the target program, when the version of the target program changes, triggering the execution of test cases in the target program after the version change, testing of the target program throughout the entire service call chain can be achieved, thereby obtaining a test data set. Next, the link identifier corresponding to the test cases is determined. Since the processing data of each service module includes a link identifier, and the link identifier is used to identify the service call chain formed by multiple service modules, the corresponding sample data set can be quickly obtained from the benchmark database based on the link identifier. This facilitates comparison between the test data set and the sample data set to obtain test results. Therefore, the testing method provided by the embodiments of this invention can achieve rapid, end-to-end automated testing, achieving refined testing effects, which is beneficial for improving the accuracy and reliability of test results and enhancing the stability and robustness of the target program.

[0070] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0071] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0072] Figure 1 A schematic diagram of an implementation environment provided for an embodiment of the present invention;

[0073] Figure 2 An architecture diagram of the testing system provided in an embodiment of the present invention;

[0074] Figure 3 A flowchart of the testing method provided in the embodiments of the present invention;

[0075] Figure 4 This is a schematic diagram of the data management interface provided in an embodiment of the present invention;

[0076] Figure 5 This is a schematic diagram illustrating the processing procedure of the relay packet capture service provided in an embodiment of the present invention;

[0077] Figure 6This is a schematic diagram illustrating the data extraction service provided in an embodiment of the present invention.

[0078] Figure 7 This is a schematic diagram illustrating the processing steps of the use case generation service provided in this embodiment of the invention.

[0079] Figure 8 This is a partial schematic diagram of a continuous integration pipeline provided in an embodiment of the present invention;

[0080] Figure 9 This is a schematic diagram illustrating the processing procedure of the automated execution module provided in an embodiment of the present invention;

[0081] Figure 10 This is a schematic diagram of the test case statistics interface provided in an embodiment of the present invention;

[0082] Figure 11 This is a schematic diagram illustrating the overall working principle of the testing method provided in this embodiment of the invention;

[0083] Figure 12 A complete flowchart of the testing method provided in the embodiments of the present invention;

[0084] Figure 13 This is a schematic diagram illustrating an example of a service call chain provided in an embodiment of the present invention;

[0085] Figure 14 This is a schematic diagram of the structure of the testing device provided in an embodiment of the present invention;

[0086] Figure 15 This is a structural block diagram of a portion of the server provided in an embodiment of the present invention. Detailed Implementation

[0087] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0088] Before providing a further detailed description of the embodiments of the present invention, the nouns and terms involved in the embodiments of the present invention are explained, and the nouns and terms involved in the embodiments of the present invention are subject to the following interpretations:

[0089] Continuous integration (CI) refers to the frequent and automatic integration of code into the trunk and production environment, thereby determining whether new code and existing code can be correctly integrated together.

[0090] Daily build is a continuous integration method that is triggered on a scheduled basis.

[0091] A test case is a description of a specific program product for testing purposes, reflecting the test plan, methods, techniques, and strategies.

[0092] The target user can be either the user of the target program or the account of the target program.

[0093] In related technologies, testing the operation of a target program often requires manually writing test cases. After running the test cases, the actual running results are compared with the expected running results in the test cases to determine the operation of the target program. However, the reliability of test results obtained by manually writing test cases needs to be improved. For example, manually writing test cases is prone to errors and omissions, or manually writing test cases often only considers the overall operation of the target program and cannot perform more detailed testing.

[0094] Based on this, embodiments of the present invention provide a testing method, apparatus, electronic device, storage medium, and program product, which can achieve rapid, end-to-end automated testing, achieve refined testing effects, improve the accuracy and reliability of test results, and enhance the stability and robustness of the target program.

[0095] The testing method provided in this embodiment of the invention can be applied to testing scenarios of cloud technology, artificial intelligence, smart transportation, and driver assistance programs. The specific principles of the testing method provided in this embodiment of the invention will be described in detail below with reference to the accompanying drawings.

[0096] Reference Figure 1 , Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present invention. The implementation environment includes a server 101 and a terminal 102, wherein the terminal 102 and the server 101 are connected through a communication network 103.

[0097] Server 101 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0098] In addition, server 101 can also be a node server in a blockchain network.

[0099] Terminal 102 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, vehicle terminal, etc., but is not limited to these. Terminal 102 and server 101 can be directly or indirectly connected via wired or wireless communication, and this embodiment of the invention does not impose any limitations.

[0100] in, Figure 1 The terminal 102 shown can be a terminal used for testing. The terminal 102 can run the target program under test in the test environment and then send a business request to the server 101. During the process of processing the business request, the server 101 generates a sample data set containing link identifiers, stores the sample data set in the benchmark database, and then generates test cases based on the sample data set. When the version of the target program changes, the target program with the changed version is triggered to run the test cases, and the server 101 obtains the test data set, then determines the link identifier corresponding to the test case, retrieves the corresponding sample data set from the benchmark database based on the link identifier, compares the test data set with the sample data set, and obtains the test results.

[0101] Understandably, terminal 102 can also run the target program under test in a production environment.

[0102] Alternatively, the target program can also be run directly on server 101, in which case the test of the target program can be completed using only server 101.

[0103] Reference Figure 2 , Figure 2The diagram illustrates the architecture of a testing system provided in this embodiment of the invention. The system includes a packet capture service, a data extraction service, a test case generation service, and an automated execution module. The automated execution module primarily comprises a test case execution service, a data comparison service, a data reading service, a configuration center, and a report generation service. The packet capture service captures processing data from multiple service modules invoked during the processing of business requests from the target program, generates a sample dataset, merges the datasets, and stores the sample data set. The data extraction service verifies and filters the sample data set according to preset filtering rules. The test case generation service automatically generates test cases. The automated execution module performs test case admission, data comparison, noise reduction, and test report generation. The relay packet capture service, data extraction service, test case generation service, and automated execution module work together to achieve automated testing. Specifically, the sample data set of the relay packet capture service is automatically transmitted to the data extraction service for verification. The sample data set that meets the conditions is automatically transmitted to the test case generation service to generate test cases. The automated execution module continuously monitors the version of the target program. Once the version of the target program changes, the automated execution module will trigger the target program with the changed version to run test cases in order to obtain test results.

[0104] based on Figure 1 The implementation environment shown and Figure 2 The test system architecture shown below will be explained in detail below, along with the principles of the test method provided in this embodiment of the invention. (Refer to...) Figure 3 , Figure 3 This is a flowchart of a testing method provided in an embodiment of the present invention. Exemplarily, this testing method can be performed by… Figure 1 The server 101 executes this, or it can be performed by... Figure 1 The terminal 102 can execute it, or it can be executed by... Figure 1 The test is executed jointly by server 101 and terminal 102. The following description takes the test method executed by server 101 as an example. The test method includes, but is not limited to, the following steps 301 to 303.

[0105] Step 301: Generate a sample data set and store the sample data set in the benchmark database.

[0106] This step can be done by Figure 2 The relay packet capture service shown is executed, where the sample data set is used to automatically generate test cases later. The sample data set is generated by the server when processing the business requests of the target program. The target program is the program to be tested. The business requests of the target program are used to implement corresponding functions in the target program, such as displaying promotional information, sending instant messaging messages, quick payment, game operation, etc.

[0107] In one possible implementation, after receiving a business request, the server needs to call the corresponding service through the service interface to process the business request. This service typically has multiple service modules, meaning that multiple service modules need to be called to process the business request. In this case, multiple service modules can form a service call chain. For example, in the process of processing the business request, service modules S1, S2, S3, and S4 are called. S1-S2-S3-S4 can form a service call chain. Of course, the order and number of service modules in the service call chain need to be determined according to the actual situation. Furthermore, multiple service call chains can be invoked to process the same business request. Accordingly, in this embodiment of the invention, the sample data set includes processing data from multiple service modules invoked when processing business requests from the target program. Each service module's processing data can be request data or response data. Request data refers to data sent by a service module to a downstream service module, and response data refers to data sent by a service module to an upstream service module. Taking the aforementioned service call chain S1-S2-S3-S4 as an example, the request data from service module S2 is data sent to service module S3, while the request data from service module S2 is data sent to service module S1. It can be understood that for service module S1, its response data can be data returned to the target program, i.e., it can be the return data from the entire service interface. For service module S4, its processing data may only include the return data.

[0108] In addition, the processing data of each service module includes a link identifier, which is used to identify the service call link formed by multiple service modules. Taking the above service call link S1-S2-S3-S4 as an example, the processing data of service module S1, service module S2, service module S3 and service module S4 all include the same link identifier. Therefore, the above service call link S1-S2-S3-S4 can be quickly and accurately determined using the link identifier.

[0109] After generating the sample dataset, it can be stored in a benchmark database. The benchmark database stores data that will serve as a comparison benchmark for subsequent tests, facilitating data comparison by retrieving the sample dataset from the benchmark database later. In practical applications, there can be multiple sample datasets; therefore, the benchmark database can store multiple datasets that will serve as comparison benchmarks for subsequent tests.

[0110] Step 302: Generate test cases based on the sample data set.

[0111] This step can be done by Figure 2The test case generation service execution shown can be achieved by first obtaining the request and response data of each service module in the service call chain, as well as the link identifier corresponding to the service call chain, based on the sample data set. Then, a preset test case template is obtained, and the request and response data are added to the corresponding positions in the preset test case template to obtain the test cases. Finally, the test cases are identified according to the link identifier.

[0112] One method is to generate test cases using preset test case templates, which allows for automated test case generation and offers high efficiency. The preset test case templates can be configured with corresponding fields for request and response data; when generating test cases, the request or response data is simply added to the corresponding fields.

[0113] For example, the request data or response data may include one or more of the following: service module name, interface name, interface identifier, request source IP address, request source port, request destination IP address, request destination port, request plaintext, response plaintext, request type, routing value, request parsing plaintext, response parsing plaintext, link identifier, and request time. This embodiment of the invention does not impose limitations on these categories. Based on this, the preset use case template can also have multiple different fields. Request data or response data can be added to the corresponding fields. For example, the service module name can be added to the service module name field, the interface name to the interface name field, and so on.

[0114] In addition, the preset use case template can be in the form of a table, code, etc., and this embodiment of the invention does not limit the form.

[0115] Step 303: When the version of the target program changes, the test cases of the target program after the version change are triggered to obtain the test data set, the link identifier corresponding to the test case is determined, the corresponding sample data set is obtained from the benchmark database according to the link identifier, the test data set is compared with the sample data set to obtain the test results.

[0116] This involves a change in the version of the target program, which could mean updating at least a portion of the target program's code. This step can be performed by... Figure 2 The automated execution module shown in the diagram executes the test cases when the version of the target program changes. This triggers the execution of test cases in the target program after the version change, resulting in a test data set. The test data set is the result of the test cases. Similar to the sample data set, the test data set also includes the processing data of multiple service modules called when running the test cases.

[0117] Next, since the benchmark database stores multiple sample data sets, the link identifier corresponding to the test case is determined first, and the corresponding sample data set is obtained from the benchmark database based on the link identifier. Then, the test data set is compared with the sample data set. Specifically, since both the test dataset and the sample dataset include processing data from multiple service modules, when comparing the test dataset and the sample dataset, the processing data corresponding to each service module can be compared separately. That is, the test dataset can be compared with the request data of the same service module in the sample dataset, or the test dataset can be compared with the response data of the same service module in the sample dataset. For example, taking the above service call chain S1-S2-S3-S4 as an example, the request data of the service module S1 in the sample dataset is Req1 and the response data is Res1, while the request data of the service module S1 in the test dataset is Req2 and the response data is Res2. Then, Req1 is compared with Req2, and Res1 is compared with Res2. The comparison principle of the processing data of other service modules is similar and will not be elaborated here. In this embodiment of the invention, both the request data in the test dataset and the sample dataset are compared, as are the response data in the test dataset and the sample dataset. By comparing the response data, the operation verification of different types of target programs can be performed, which is business-related, thereby realizing refined testing of the target program. For example, by comparing the response data in the test dataset and the sample dataset, log checks can be performed during the business request processing.

[0118] Steps 301 to 303 above generate a sample data set, store it in a benchmark database, and generate test cases based on the sample data set. Since the sample data set includes processing data from multiple service modules called when handling business requests of the target program, when the target program version changes, triggering the execution of test cases in the updated version allows for testing of the target program across the entire service call chain, thus obtaining a test data set. Next, the link identifier corresponding to the test cases is determined. Because each service module's processing data includes a link identifier, and the link identifier identifies the service call chain formed by multiple service modules, the corresponding sample data set can be quickly retrieved from the benchmark database based on the link identifier. This facilitates comparison between the test data set and the sample data set to obtain test results. Therefore, the testing method provided by this embodiment of the invention can achieve rapid, end-to-end automated testing, achieving refined testing effects, which helps improve the accuracy and reliability of test results, and enhances the stability and robustness of the target program.

[0119] The target program can generate business requests in a first operating environment and run test cases in a second operating environment; alternatively, the target program can also generate business requests and run test cases in the second operating environment. The first operating environment is the production environment, where the target program is used normally. The second operating environment is the test environment, specifically set up for testing the target program. In related technologies, to improve the stability of the target program, business requests are typically generated and test cases are run in the test environment.

[0120] In this embodiment of the invention, the target program generates business requests in the first operating environment. Accordingly, before generating the sample data set, the request type of the business request is determined. Based on preset business classification rules, the risk level of the request type is determined. When the risk level is less than or equal to a preset level threshold, the sample data set is determined as the data set to be generated. For example, the request type of the business request can be a payment request, an information sending request, a promotional information retrieval request, a game operation request, etc. The business classification rules include the risk levels corresponding to different request types. For example, based on the risk level from high to low, the risk levels can be level three, level two, and level one. The risk level of the payment request and information sending request can be level three, the risk level of the promotional information retrieval request can be level two, and the risk level of the game operation request can be level one. The preset level threshold can be level two. Based on this, the sample data set corresponding to the business request is only generated when the request type of the business request is a promotional information retrieval request or a game operation request. This not only enriches the source of the sample data set and improves the reliability of the test, but also reduces the impact of generating business requests in the production environment on the stability of the target program.

[0121] It is understood that the above business classification rules and preset level thresholds can be determined according to the actual situation, and the embodiments of the present invention do not limit them.

[0122] In one possible implementation, in step 301 above, generating a sample data set can specifically involve obtaining the request data of each service module in the service call chain, forwarding the request data to the corresponding service module, obtaining the response data returned based on the request data, forwarding the response data to the corresponding service module, merging the request data and response data of each service module according to a preset data type to obtain the processing data of each service module, and merging the processing data of multiple service modules to obtain a sample data set.

[0123] In related technologies, a dual-engine regression testing method is generally used to obtain sample data. This method is based on a dual-engine regression platform, which copies data generated during the normal use of the target program by a user and uses it as sample data. However, this method lacks flexibility because the target program's code needs to be modified due to the data copying requirement, thus exhibiting business intrusion. In this embodiment of the invention, the relay packet capture service acts as a relay station for request or response data, forwarding the data. Therefore, no code modification is required for the target program. Thus, this embodiment of the invention does not exhibit business intrusion when generating the sample data set, thereby reducing development and integration costs and improving the versatility of the testing method.

[0124] Furthermore, after generating the sample dataset, it can be managed through a graphical user interface. For example, refer to... Figure 4 , Figure 4 This is a schematic diagram of the data management interface provided in an embodiment of the present invention. The interface allows retrieval of corresponding sample data sets using information such as service module name, interface name, routing value, link identifier, request source IP address, request source port, request destination IP address, request destination port, and request time. After retrieving the corresponding sample data set, a data information area can be displayed. This data information area displays specific data from the sample data set, such as service module name, interface name, status, routing value, link identifier, request source IP address, request source port, request destination IP address, request destination port, business request time, environment type, container, data creation time, etc. Furthermore, the data information area can include operation buttons 401 for exporting, copying, and other operations. Additionally, the data information area also includes a plaintext display sub-area 402, which displays the parsed plaintext of request or response data. The display can be switched by clicking "Request Parsing Plaintext" or "Response Parsing Plaintext". Furthermore, a copy button 403 can be provided in the plaintext display sub-area 402. The parsed plaintext can be copied by using the copy button 403, which facilitates subsequent data processing and improves efficiency.

[0125] In one possible implementation, in step 301 above, the sample data set is stored in the benchmark database. Specifically, the target function can be called to serialize the sample data set to obtain the target data set, obtain the target file compiled based on the preset file format, convert the target data set into the target file format according to the target file, and store the target data set converted into the target file format in the benchmark database.

[0126] Specifically, serializing the sample data set improves data scalability, reduces data footprint, and increases storage efficiency. The objective function can be the Protobuf `ParseFromString` function. Protobuf is a high-efficiency protocol data exchange format library. The default file format for the target file can be Protobuf format. Converting the target data set to the target file format can be done by converting the target data set to JSON format. It is understood that this embodiment of the invention does not limit the objective function, the default file format, or the target file format.

[0127] The working process of the relay packet capture service in the embodiments of the present invention will be described below with reference to the accompanying drawings. Figure 5 , Figure 5 This is a schematic diagram illustrating the processing flow of the relay packet capture service provided in this embodiment of the invention. The relay packet capture service mainly includes a data capture stage and a data processing stage. First, in the data capture stage, the relay packet capture service obtains request data, forwards the request data to the corresponding service module, and then obtains the response data returned by the corresponding service module. All request data and response data are stored uniformly using a ServiceProto object to generate a sample data set, which is then published to a Redis message queue. Next, in the data processing stage, the service subscribes to the corresponding Redis message queue, extracts the sample data set from the Redis message queue, and parses the sample data set. The parsing method corresponds to the storage method of the ServiceProto object mentioned above. After parsing, request data and response data from multiple service modules are obtained. Each service module's request data and response data include a link identifier. Then, using the Protobuf ParseFromString function and a pre-compiled Protobuf format file, the request data and response data are converted into JSON format and stored in a baseline database. The specific fields of the request data and response data have been described in detail previously and will not be repeated here. The relay packet capture service publishes the sample data set to the Redis message queue after generating it. This makes it easier to subscribe to the corresponding Redis message queue during the data processing stage and extract the sample data set from it. On the other hand, it also allows the data extraction service to extract the sample data set from the Redis message queue for data verification and filtering, thus making the testing process simpler and smoother and improving testing efficiency.

[0128] In one possible implementation, the sample dataset can be validated before being stored in the benchmark database. Only after the validation is successful can the sample dataset be stored in the benchmark database. Figure 5 In the relay packet capture service process shown, the sample data set is only subscribed to the corresponding Redis message queue during the data processing stage after the sample data set passes the verification, thereby extracting the sample data set from the Redis message queue.

[0129] Specifically, the sample data set can be added to a task list. In response to task processing instructions, the sample data set can be retrieved from the task list and sent to a thread pool. In response to thread processing instructions, the sample data set can be extracted from the thread pool and validated. Validating the sample data set using a task list and thread pool approach enables asynchronous processing, resulting in higher validation efficiency.

[0130] Specifically, validating the sample dataset can involve concatenating multiple service modules called by the target program when processing business requests, thus obtaining a service call chain. This service call chain is then compared with a preset call chain to determine its completeness. As mentioned earlier, since the sample dataset includes service module names, concatenating multiple service modules yields the service call chain. The preset call chain represents the call chain when the target program's business request is processed normally. Therefore, by comparing the service call chain with the preset call chain, it can be determined whether any service calling modules are missing or incorrectly invoked, thereby confirming the completeness of the service call chain. Validating the completeness of the service call chain improves the reliability of subsequent test case generation based on the sample dataset and comparison of the test dataset with the sample dataset, thus enhancing the overall reliability of the testing method.

[0131] Once the service call chain is confirmed to be complete, the sample data set is stored in the benchmark database, and test cases are subsequently generated based on the sample data set.

[0132] Alternatively, validating the sample dataset can involve extracting a first tag to identify the sample dataset and multiple second tags from the benchmark database. Based on the matching relationships between the first tag and the multiple second tags, the storage status of the sample dataset in the benchmark database can be determined—either it is stored in the benchmark database or not. The sample dataset can also include the aforementioned first tag. Since the benchmark database can store multiple sample datasets, the tags for sample datasets already stored in the benchmark database are the second tags. By matching the first tag with the multiple second tags, it can be determined whether a sample dataset is already stored in the benchmark database. Specifically, when the first tag matches a certain second tag, it indicates that the sample dataset is already stored in the benchmark database, and therefore, it is considered old data and can be discarded. Besides directly using the first tag to identify the sample dataset, the first tag can also be used to identify request and response data from different service modules. The principle of subsequently comparing these with the second tags is similar and will not be elaborated further here. By using first and second labels, it is easy and quick to determine whether the sample data set is new data, thereby reducing the duplicate storage of sample data sets and the duplicate generation of test cases, reducing invalid processing, improving testing efficiency, and reducing the space occupied by the benchmark database.

[0133] Once the sample data set is determined to be new data, it is then stored in the benchmark database, and test cases are subsequently generated based on the sample data set.

[0134] It is understandable that the above two verification methods for sample data sets can be selected to be executed at least one, that is, only the integrity of the service call chain can be verified; or only whether the sample data set is new data can be verified; or both the integrity of the service call chain and whether the sample data set is new data can be verified.

[0135] The aforementioned validation of the sample dataset can be performed by a data extraction service. This data extraction service is a multi-threaded service that includes multiple service classes. The working process of the data extraction service in this embodiment of the invention is described below with reference to the accompanying drawings. (Refer to...) Figure 6 , Figure 6This is a schematic diagram of the data extraction service provided in an embodiment of the present invention. The data extraction service includes multiple service classes, specifically startup service, process service, message center service, task management service, and task processing service. Specifically, the process can begin by initializing the logger and the validation rules for the sample data set (the aforementioned service call chain integrity and the storage status of the sample data set) through the startup service. Then, the process service is started, which initializes the message center service and the task processing service. Next, the process service initializes the Redis service, registers Redis channels and message listeners, starts a message processing thread to process Redis subscription messages, and adds the sample data set to the task list. The message center service receives Redis subscription messages. Then, the process service starts a task processing thread, which, in response to task processing instructions, retrieves tasks (sample data sets) from the task list and places them into a thread pool. The task management service maintains the task list and manages the tasks. Finally, in response to thread processing instructions, the task processing service extracts the sample data set from the thread pool, validates it against validation rules to determine if it is new data, and verifies the integrity of the service call chain. The validated sample data sets are then used as the objects for generating new test cases and added to the task list, awaiting the generation of the next test cases. By using multiple service classes to validate the sample data set, asynchronous processing of sample data validation can be achieved, which can improve the efficiency of sample data validation.

[0136] In actual testing, considering that some types of target programs may have highly random or specific response data from certain service modules during runtime, for example, a target application has an automatic email sending function that automatically retrieves emails from the server and sends them every day at 5 pm, when testing this target program, it is necessary to wait until 5 pm to test whether the target application can call the corresponding service module in the server on time and successfully retrieve and send emails, which significantly reduces testing efficiency.

[0137] Based on this, in step 302, test cases are generated according to the sample data set. Specifically, the response data of each service module in the service call chain can be merged to obtain result simulation data, and result simulation identifiers corresponding to the result simulation data are generated. Key-value pairs are generated according to the result simulation data and the result simulation identifiers, and the key-value pairs are added to the test cases.

[0138] Among them, the result simulation data is used to simulate the response data of the corresponding service module during testing. When the response data of a certain service module has high randomness or specificity, the result simulation data can be used to replace the response data of the corresponding service module during the running of test cases, thereby improving testing efficiency.

[0139] Specifically, key-value pairs can be used to represent the correspondence between the simulated result data and the simulated result identifier. The simulated result identifier can be used as the "key," and the simulated result data can be used as the "value." The simulated result identifier can be included in the test request, and the simulated result data can be stored in the benchmark database. When the simulated result identifier is known, the corresponding simulated result data can be retrieved from the benchmark database according to the correspondence between the simulated result identifier and the simulated result data.

[0140] Accordingly, based on the above-mentioned simulation data processing, in step 303, the target program after the version change is triggered to run test cases. Specifically, the target program after the version change is triggered to generate test requests according to the test cases, determine the first service module currently called when processing the test request, obtain the simulation status of the second service module bypassed by the first service module, and when the simulation status indicates that the response data returned by the second service module to the first service module needs to be simulated, the result simulation data is obtained according to the result simulation identifier, and the result simulation data is used as the response data returned by the second service module to the first service module.

[0141] The simulation status indicates whether the response data returned by the second service module to the first service module needs to be simulated or not. A second service module bypassed by the first service module indicates that the second service module is a downstream service module of the first service module, or, in some cases, it can also indicate that the second service module is a parallel service module of the first service module. Since the test cases include key-value pairs generated from result simulation data and result simulation identifiers, during the execution of the test cases, when it is determined that the response data of a certain service module needs to be simulated, the result simulation identifier can be extracted from the test request, and the corresponding result simulation data can be obtained based on the result simulation identifier to replace the response data of that service module.

[0142] For example, taking the above service call chain S1-S2-S3-S4 as an example, assuming that the first service module currently being called is service module S3, then the second service module is service module S4. When the response data of service module S4 needs to be simulated, the result simulation identifier corresponding to service module S4 is extracted from the test request, and the corresponding result simulation data is obtained from the benchmark database according to the result simulation identifier, and returned to service module S3 as the response data of service module S4.

[0143] The working process of the use case generation service in this embodiment of the invention will be described below with reference to the accompanying drawings. (Refer to...) Figure 7 , Figure 7 This diagram illustrates the processing flow of the test case generation service provided in this embodiment of the invention. The test case generation service may also include multiple service classes, specifically including a test case generation method service, a test case generation control service, and a simulated data generation service. The test case generation method service exposes test case generation methods and simulated data generation methods. The test case generation control service calls the test case generation method, receives the sample data set transmitted by the data extraction service, generates test cases according to a preset test case template, and stores the test cases according to the link identifier. The simulated data generation service calls the simulated data generation method, obtains the response data of all bypass service modules during the processing of the business request according to the link identifier, saves these response data as result simulated data, and generates corresponding result simulated identifiers as key-value pairs in the test case. When running test cases subsequently, if a service module needs to simulate response data, the result simulated identifier in the test request is extracted through the simulation service (not shown in the attached diagram), and the corresponding result simulated data is obtained as response data based on the result simulated identifier.

[0144] The testing method provided in this invention can be implemented based on a continuous integration platform to improve the automation level of the testing method. When the updated code of the target program is submitted to the continuous integration platform, the platform triggers the execution of test cases in the target program after the version change. The continuous integration platform can have a test case library, which can store multiple test cases. When the target program's code is updated, the continuous integration platform can automatically retrieve and run test cases from the library, achieving automated testing. Based on this, before triggering the execution of test cases in the target program after the version change, test cases can be run in the continuous integration platform at a preset frequency, and the number of successful executions of the test cases in the continuous integration platform can be accumulated. When the number of successful executions is greater than or equal to a preset threshold, the test case is added to the test case library of the continuous integration platform.

[0145] The test cases are run in the continuous integration platform at a preset frequency, which can be achieved through daily builds. When the number of successful runs of a test case is greater than or equal to a preset threshold, it indicates that the test case is running stably and can be added to the test case library for subsequent testing. The preset threshold can be set according to actual conditions, such as 5 times, 10 times, etc., and this embodiment of the invention does not limit it.

[0146] Additionally, the historical execution times of test cases successfully run in the continuous integration platform can be obtained. Based on these historical execution times, a target execution time can be determined. A baseline data set obtained from running the test cases at the target execution time can then be acquired. This baseline data set is then used to update the sample data set in the baseline database. The target execution time can be the time when the test case was last successfully run in the continuous integration platform. For example, if the current time is 12:00 on January 5th, and the historical execution times are 15:00 on January 1st, 22:00 on January 2nd, and 10:00 on January 4th, then the target execution time is 10:00 on January 4th. The test data set obtained from running the test case at 10:00 on January 4th becomes the baseline data set. This baseline data set replaces the original corresponding sample data set in the baseline database, thus maintaining the data activity of the baseline database and improving the reliability of the tests. It is understood that the above-mentioned update of the sample dataset in the baseline database using the baseline data set can be performed once a day or once a week, depending on actual needs. This embodiment of the invention does not impose any limitations on this.

[0147] The following section, with reference to the accompanying diagrams, details the implementation of automatic test case execution and addition to the test case library. (Refer to...) Figure 8 , Figure 8 This is a partial schematic diagram of a continuous integration pipeline provided in an embodiment of the present invention. A continuous integration pipeline can be understood as a process formed by the steps of different stages of continuous integration. Figure 8 The example demonstrates two phases in a continuous integration pipeline: the build trigger node and the build environment phase.

[0148] First, during the build triggering phase, you can set up manual triggering of the continuous integration platform for continuous integration, or trigger the continuous integration platform for continuous integration through daily builds (e.g., at 6 AM every day).

[0149] During the environment setup phase, the continuous integration script code is first pulled to initialize the continuous integration environment. Then, the updated code of the target program is obtained. If the pull of the updated code of the target program fails, a cleanup operation is performed and the process is repeated to improve the stability of obtaining the updated code of the target program.

[0150] Then, the target program with the updated code is compiled and deployed, and stable test cases are automatically added to the test case library.

[0151] Then, test cases are automatically retrieved from the test case library and run to perform unit testing and integration testing. Unit testing is the testing work that verifies the correctness of the program modules of the target program. Integration testing is based on unit testing and performs orderly and incremental testing on all program modules. By performing unit testing and integration testing, the target program can be tested at different fine-grained levels, which helps to improve the reliability of the test.

[0152] Then, the data in the test case library is restored. Since some dirty data may be generated during the testing process, affecting the accuracy of the test cases in the test case library, restoring the data in the test case library helps to improve the reliability of the test case library.

[0153] Then, collect the code coverage and incremental coverage of this test;

[0154] Finally, the test results are automatically output and displayed. The test results may include, but are not limited to, the code coverage, incremental coverage, specific data content of the sample dataset, service call chain, test case parameters, etc.

[0155] In one possible implementation, before comparing the test data set with the sample data set in step 303, data to be removed can be filtered from the test data set and the sample data set according to preset filtering rules, thereby performing noise processing to reduce noise and improve the accuracy of the comparison between the test data set and the sample data set.

[0156] For example, data with random numbers as its attribute can be used as data to be removed from both the test dataset and the sample dataset. Specifically, since the data with random numbers as its attribute will vary in different tests, the sample dataset in the benchmark database will also change accordingly. For example, data such as object identifiers, timestamps, or tokens can be used as data to be removed. Directly comparing these data would affect the accuracy of the comparison between the test dataset and the sample dataset. By using data with random numbers as the data to be removed, the impact of comparing data with random numbers as its attribute on accuracy can be reduced.

[0157] For example, wildcards can be used to filter out data to be removed from the test dataset and sample dataset. Specifically, a wildcard can be used to replace a complete character; for example, in "A?", the "?" is the wildcard, and "A?" represents all characters starting with "A". Filtering out data using wildcards allows for a simple and quick selection of data to be removed based on actual needs. The specific data to be removed can be determined according to the actual situation, which will not be elaborated further in this embodiment. Determining the data to be removed using wildcards can reduce the impact of comparing specific data on accuracy.

[0158] For example, variable strings can be extracted from the target data type and used as the data to be removed. This allows for the selection of data to be removed from both the test dataset and the sample dataset. Specifically, the target data type can be a list, JSON, or URL, which contains variable strings. Taking URL data as an example, the format is typically "http: / / www.abc.com / XX", where "XX" is the URL suffix. When accessing the same webpage through this URL, the suffix "XX" is variable, and in this case, "XX" is the aforementioned variable string, which may affect the test results. By using variable strings as the data to be removed, the impact of comparing the target data type on accuracy can be reduced.

[0159] It is understood that one or more of the above three filtering methods can be selected to filter the data to be removed, and the embodiments of the present invention do not limit this.

[0160] The working process of the automated execution module is explained in detail below with reference to the accompanying diagrams. (Refer to...) Figure 9 , Figure 9 This is a schematic diagram illustrating the processing flow of the automated execution module provided in this embodiment of the invention. First, in response to a code update in the target program, the automated execution module initiates a continuous integration task. The test case execution service retrieves test cases from the test case library, runs the test cases, obtains a test data set, and stores the test data set in the test database. The data comparison service calls the data reading service, passing the link identifier to the data reading service. The data reading service reads the test data set (request data + response data) from the test database using the link identifier of the test data set, and reads the sample data set (request data + response data) from the benchmark database using the link identifier corresponding to the test case. Then, the test data set and the sample data set are passed to the data comparison service, which compares the test data set with the sample data set and outputs the comparison results. Then, the report generation service can generate a visualized test report based on the comparison results. Additionally, preset filtering rules can be configured in the configuration center to handle noise when the data comparison service compares the test data set with the sample data set. Furthermore, the sample data set can be updated using the test data set obtained from the continuous integration of the main code that was successfully run last time. Alternatively, if the code update of the target program is confirmed to have passed according to the test report, an updated sample data set can be obtained based on the updated target program, thereby maintaining the liveness of the sample data set.

[0161] Specifically, refer to Figure 10 , Figure 10This is a schematic diagram of the test case statistics interface provided in an embodiment of the present invention. The test report can include the aforementioned test case statistics, specifically displaying the total number of test cases in the continuous integration platform's test case library and the total number of service call links involved. It can also further refine the display by showing the number of test cases added to the continuous integration platform and the number of service call links belonging to short links. Furthermore, a first jump button 1001 is provided in the display area for the total number of test cases. Clicking this first jump button 1001 displays the total number of test cases automatically generated during testing for different versions of the target program. A second jump button 1002 is provided in the display area for the total number of service call links. Clicking this second jump button 1002 displays the total number of service modules, thus achieving a refined display of statistical data. In addition, detailed parameters of the test cases can be displayed, such as the test case identifier, test case name, link identifier corresponding to the test case, first tag of the sample data set, service module corresponding to the test case, continuous integration version number, time added to the continuous integration platform's test case library, continuous integration version number at the time of successful first run, and test case details. Visually displaying the relevant information of the test cases facilitates the maintenance and management of test cases, which helps improve the reliability of testing.

[0162] It should be noted that, in addition to the test case statistics mentioned above, the test report may also include automatically drawn service call chains, thereby visually demonstrating the service modules called by the target program.

[0163] The overall working principle of the testing system provided in the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. (Refer to...) Figure 11 , Figure 11 This is a service interaction diagram of a testing system provided in an embodiment of the present invention. The testing system mainly includes a test case generation phase and a test case execution phase during operation.

[0164] When generating test cases, the packet capture service acquires processing data from multiple service modules in the test environment, generates a sample data set, and then passes the sample data set to the data extraction service, storing it in the benchmark database. The data extraction service validates the sample data set, and the validated sample data set is passed to the test case generation service. The test case generation control service in the test case generation service generates test cases based on the sample data set and passes the link identifier corresponding to the sample data set to the simulation data generation service. The simulation data generation service retrieves the corresponding response data from the benchmark database based on the link identifier and generates a result simulation identifier based on the response data. The key-value pair of the result simulation identifier and the result simulation data is passed to the test case generation control service. The test case generation control service adds the result simulation identifier to the test case, adds the test case to the test case library, and identifies the test case using the test case identifier.

[0165] When running test cases, the test case execution service first generates a test case execution schedule table based on the link identifier, and then automatically runs the test cases according to the test case execution schedule table to obtain a test data set. The test case execution service passes the link identifier of the test data set to the data comparison service. The data comparison service calls the data reading service and passes the link identifier to the data reading service. The data reading service reads the test data set from the test database through the link identifier of the test data set, and reads the sample data set from the benchmark database through the link identifier corresponding to the test case. Then, it passes the test data set and the sample data set to the data comparison service. The data comparison service compares the test data set with the sample data set according to the preset filtering rules, and passes the comparison results to the report generation service. Finally, a test report is generated.

[0166] Figure 11 The testing system shown can automatically capture data, generate test cases, run tests, and output test results, enabling fast, end-to-end automated testing. This helps improve the accuracy and reliability of test results and enhances the stability and robustness of the target program.

[0167] Additionally, refer to Figure 12 , Figure 12 This is a complete flowchart of the testing method provided in the embodiments of the present invention. The testing method includes, but is not limited to, the following steps 1201 to 1211:

[0168] Step 1201: Trigger the continuous integration pipeline via daily builds;

[0169] Step 1202: Compile the target program, obtain the request and response data of multiple service modules called during the compilation process, generate a sample data set based on the request and response data of multiple service modules, and publish the sample data set to the message queue;

[0170] Step 1203: Obtain a sample data set from the message queue, and obtain the service call chain based on the sample data set;

[0171] Step 1204: Determine if the service call chain is complete. If yes, proceed to step 1205; otherwise, proceed to step 1201.

[0172] Step 1205: Determine whether the sample data set is new data. If yes, proceed to step 1206; otherwise, proceed to step 1201.

[0173] Step 1206: Store the sample data set in the benchmark database;

[0174] Step 1207: Generate test cases based on the sample dataset;

[0175] Step 1208: Determine if the version of the target program has changed. If yes, proceed to step 1209; otherwise, end the process.

[0176] Step 1209: Run the test cases to obtain the test data set, and store the test data set in the test database;

[0177] Step 1210: Obtain the sample data set from the benchmark database according to the link identifier corresponding to the sample data set, and obtain the test data set from the test database according to the link identifier corresponding to the test data set;

[0178] Step 1211: Compare the sample data set and the test data set according to the preset screening rules, and generate a test report based on the comparison results.

[0179] The specific methods for generating sample data sets based on request and response data from multiple service modules, determining the completeness of service call chains, determining whether the sample data set is new data, generating test cases based on the sample data set, and comparing the sample data set and test data set according to preset filtering rules in steps 1201 to 1211 above have been described in detail above and will not be repeated here. Based on the continuous integration platform, rapid, end-to-end automated testing can be achieved, resulting in refined testing and improving the accuracy and reliability of test results, as well as the stability and robustness of the target program.

[0180] It is understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this embodiment, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0181] The following example illustrates the principle of the testing method provided in this embodiment of the invention. The target program can be an instant messaging client, the business request can be a request to obtain promotional information, the promotional information can be an advertisement, and the specific form of the advertisement can be one or more of text and images. After obtaining the advertisement, it can be pushed and displayed in the chat scenario and information browsing scenario of the instant messaging client.

[0182] The target program's code is compiled daily through builds, generating business requests and sending them to the server. Figure 13 , Figure 13This is a schematic diagram illustrating an example of a service call chain provided in an embodiment of the present invention. The server first calls the ad retrieval entry module based on the business request. The ad retrieval entry module has three downstream modules: an ad retrieval logic layer module, a whitelist permission acquisition module, and an application information acquisition module. The ad retrieval logic layer module has two downstream modules: an ad unit status acquisition module and an ad engine fine-ranking module. The ad engine fine-ranking module has six downstream modules: a user object freshness acquisition module, a user object profile information acquisition module, an ad coarse ranking result acquisition module, an ad key-value pair data batch acquisition module, an ad ranking score acquisition module, and an ad key-value pair data acquisition module. The user object profile information acquisition module has one downstream module: a dynamic profile information acquisition module. The ad engine fine-ranking module combines a preset number (e.g., 100) of ads obtained by the ad coarse-ranking result acquisition module, the user freshness obtained by the user freshness module, the user profile information obtained by the dynamic profile information acquisition module, the ad key-value pairs obtained by the ad key-value pair data batch acquisition module or the ad key-value pair data acquisition module, and the ad ranking score obtained by the ad ranking score acquisition module. From this preset number of ads, it determines the most suitable target ad and returns it to the ad retrieval entry module according to the service call chain. The server then sends the target ad to the target application for display. In addition, during the processing of the above business requests, the whitelist permission acquisition module can determine whether the user has permission to retrieve ads, the application information acquisition module can determine the relevant information of the target application (e.g., name, version, etc.), and the ad unit status acquisition module can determine whether the ad unit is working properly.

[0183] The request and response data between the aforementioned service modules are stored in a benchmark database and used for the automatic generation of test cases. During the maintenance of the target program, the code of the functional modules corresponding to the aforementioned business requests needs to be updated. After the updated code is submitted to the continuous integration platform, the platform will run the aforementioned test cases to obtain the request and response data generated when the target program calls the aforementioned service modules based on the updated code. This data is then compared with the request and response data in the benchmark database to obtain the test results. The test results can display... Figure 13 The automatically generated service call chain is shown.

[0184] Reference Figure 14 , Figure 14 This is a schematic diagram of the structure of a testing device provided in an embodiment of the present invention. The testing device includes:

[0185] The data generation module 1401 is used to generate a sample data set and store the sample data set in the benchmark database. The sample data set includes the processing data of multiple service modules called when processing the business requests of the target program. The processing data of each service module includes a link identifier, which is used to identify the service call link formed by multiple service modules.

[0186] The test case generation module 1402 is used to generate test cases based on the sample data set.

[0187] The data comparison module 1403 is used to trigger the execution of test cases in the target program after the version change when the version of the target program changes, obtain the test data set, determine the link identifier corresponding to the test case, obtain the corresponding sample data set from the benchmark database according to the link identifier, compare the test data set with the sample data set, and obtain the test results.

[0188] Furthermore, the aforementioned data generation module 1401 is specifically used for:

[0189] Obtain the request data of each service module in the service call chain, and forward the request data to the corresponding service module;

[0190] Obtain the response data returned based on the request data, and forward the response data to the corresponding service module;

[0191] The request and response data of each service module are merged according to the preset data type to obtain the processing data of each service module.

[0192] The processing data from multiple service modules are merged to obtain a sample data set.

[0193] Furthermore, the aforementioned data generation module 1401 is specifically used for:

[0194] The target function is called to serialize the sample data set, resulting in the target data set;

[0195] Obtain the target file based on the preset file format for compilation, and convert the target data set into the target file format according to the target file;

[0196] The target data set, converted into the target file format, is stored in the benchmark database.

[0197] Furthermore, the aforementioned testing apparatus also includes a data extraction module 1404, which is used for:

[0198] Add the sample dataset to the task list;

[0199] In response to task processing instructions, retrieve the sample data set from the task list and send the sample data set to the thread pool;

[0200] In response to thread processing instructions, a sample data set is extracted from the thread pool and the sample data set is validated.

[0201] Furthermore, the aforementioned data extraction module 1404 is specifically used for at least one of the following:

[0202] Based on the sample dataset, multiple service modules called by the target program when processing business requests are connected in series to obtain the service call chain. The service call chain is then compared with the preset call chain to determine the completeness of the service call chain.

[0203] or,

[0204] Extract a first label from the sample dataset to identify the sample dataset, extract multiple second labels from the benchmark database, and determine the storage status of the sample dataset in the benchmark database based on the matching relationship between the first label and the multiple second labels. The storage status is either stored in the benchmark database or not stored in the benchmark database.

[0205] Furthermore, the aforementioned use case generation module 1402 is specifically used for:

[0206] Based on the sample dataset, we obtain the request and response data of each service module in the service call chain, as well as the chain identifier corresponding to the service call chain;

[0207] Obtain a preset test case template, add the request data and response data to the corresponding positions in the preset test case template, and obtain the test cases;

[0208] Test cases are identified based on link identifiers.

[0209] Furthermore, the aforementioned use case generation module 1402 is also used for:

[0210] The response data of each service module in the service call chain is merged to obtain the simulated result data;

[0211] The result simulation identifier corresponding to the generated result simulation data;

[0212] Generate key-value pairs based on the simulated data and the simulated identifier, and add the key-value pairs to the test cases.

[0213] Furthermore, the aforementioned data comparison module 1403 is also used for:

[0214] The target program, after triggering a version change, generates test requests based on the test cases.

[0215] Determine the first service module currently invoked when processing the test request, and obtain the simulation status of the second service module bypassed by the first service module. The simulation status is used to characterize whether the response data returned by the second service module to the first module needs to be simulated or not.

[0216] When the simulation status indicates that the response data returned by the second service module to the first service module needs to be simulated, the result simulation data is obtained according to the result simulation identifier, and the result simulation data is used as the response data returned by the second service module to the first service module.

[0217] Furthermore, the aforementioned data comparison module 1403 is also used for:

[0218] Run test cases in the continuous integration platform according to a preset frequency;

[0219] The cumulative number of times test cases have successfully run on the continuous integration platform;

[0220] When the number of successful runs is greater than or equal to the preset threshold, the test case will be added to the test case library of the continuous integration platform.

[0221] Furthermore, the aforementioned data comparison module 1403 is also used for:

[0222] Obtain the historical execution time of test cases successfully running in the continuous integration platform, and determine the target execution time based on the historical execution time;

[0223] Obtain the baseline data set obtained by running test cases at the target runtime, and use the baseline data set to update the sample data set in the baseline database.

[0224] Furthermore, the aforementioned data comparison module 1403 is also used for:

[0225] Compare the request data of the same service module in the test dataset with the request data of the sample dataset;

[0226] or,

[0227] The test dataset is compared with the response data of the corresponding service module in the sample dataset.

[0228] Furthermore, the aforementioned data comparison module 1403 is also used for:

[0229] According to the preset filtering rules, the data to be removed is filtered from the test data set and the sample data set, and then the data to be removed is removed.

[0230] The data to be removed is selected from the test dataset and the sample dataset according to preset filtering rules, including at least one of the following:

[0231] Data with random numbers as its attribute is selected as the data to be removed from the test data set and the sample data set.

[0232] or,

[0233] Use wildcards to filter out data to be removed from the test dataset and the sample dataset;

[0234] or,

[0235] Extract variable strings from the target type of data, use these variable strings as data to be removed, and filter out the data to be removed from the test data set and the sample data set.

[0236] Furthermore, the target program generates business requests in the first operating environment and runs test cases in the second operating environment. The aforementioned data generation module 1401 is also used for:

[0237] Determine the request type for the business request;

[0238] The risk level of the request type is determined based on the preset business classification rules;

[0239] When the risk level is less than or equal to the preset risk level threshold, the sample data set is determined as the data set to be generated.

[0240] The aforementioned testing apparatus 1400 and testing method are based on the same inventive concept. The data generation module 1401 generates a sample data set and stores it in a benchmark database. The test case generation module 1402 generates test cases based on the sample data set. Since the sample data set includes processing data from multiple service modules called when processing business requests of the target program, the data comparison module 1403, when the version of the target program changes, triggers the execution of test cases in the target program after the version change, enabling testing of the target program throughout the entire service call chain, thereby obtaining a test data set. Next, the link identifier corresponding to the test case is determined. Since the processing data of each service module includes a link identifier, and the link identifier is used to identify the service call chain formed by multiple service modules, the corresponding sample data set can be quickly obtained from the benchmark database based on the link identifier, facilitating comparison between the test data set and the sample data set to obtain test results. Therefore, the testing apparatus provided by this embodiment of the invention can achieve rapid, end-to-end automated testing, achieving refined testing effects, which is beneficial for improving the accuracy and reliability of test results and enhancing the stability and robustness of the target program.

[0241] In this embodiment of the invention, the electronic device used to execute the test method can be a server, as shown in the reference. Figure 15 , Figure 15This is a partial structural block diagram of a server 1500 provided in an embodiment of the present invention. The server 1500 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 1522 (e.g., one or more processors) and a memory 1532, and one or more storage media 1530 (e.g., one or more mass storage devices) for storing application programs 1542 or data 1544. The memory 1532 and storage media 1530 can be temporary or persistent storage. The program stored in the storage media 1530 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 1522 may be configured to communicate with the storage media 1530 and execute the series of instruction operations in the storage media 1530 on the server 1500.

[0242] Server 1500 may also include one or more power supplies 1526, one or more wired or wireless network interfaces 1550, one or more input / output interfaces 1558, and / or one or more operating systems 1541, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0243] The processor in the server can be used to execute test methods.

[0244] This invention also provides a computer-readable storage medium for storing program code, which is used to execute the execution test methods of the foregoing embodiments.

[0245] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the execution test methods of the foregoing embodiments.

[0246] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0247] It should be understood that in this invention, "at least one (item)" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0248] It should be understood that in the description of the embodiments of the present invention, multiple (or more) means two or more, greater than, less than, and exceeding are understood to exclude the number itself, while above, below, and within are understood to include the number itself.

[0249] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0250] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0251] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0252] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0253] It should also be understood that the various implementation methods provided in the embodiments of the present invention can be combined arbitrarily to achieve different technical effects.

[0254] The above provides a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A testing method, characterized in that, include: Generate a sample data set, which includes processing data of multiple service modules called when processing business requests of the target program. The processing data of each service module includes a link identifier, which is used to identify the service call link formed by multiple service modules. Add the sample data set to the task list; In response to a task processing instruction, the sample data set is retrieved from the task list and sent to the thread pool; In response to a thread processing instruction, the sample data set is extracted from the thread pool; Based on the sample data set, the multiple service modules called by the target program when processing the business request are connected in series to obtain the service call chain. The service call chain is then compared with a preset call chain to determine the integrity of the service call chain. A first tag for identifying the sample data set is extracted from the sample data set, and multiple second tags are extracted from the benchmark database. Based on the matching relationship between the first tag and the multiple second tags, the storage status of the sample data set in the benchmark database is determined, wherein the storage status is either stored in the benchmark database or not stored in the benchmark database. Provided that the service call chain is complete and the sample data set is determined to be new data based on the matching relationship between the first tag and multiple second tags, the sample data set is stored in the benchmark database, which is used to store data as a test comparison benchmark. Test cases are generated based on the sample data set, and the test cases are identified based on the link identifier; When the version of the target program changes, the target program with the changed version is triggered to run the test cases, obtain a test data set, determine the link identifier corresponding to the test cases, obtain the corresponding sample data set from the benchmark database based on the link identifier, compare the test data set with the sample data set, and obtain the test results. The step of comparing the test data set with the sample data set includes: The test data set is compared with the request data of the same service module in the sample data set; The test dataset is compared with the response data of the corresponding service module in the sample dataset.

2. The test method according to claim 1, characterized in that, The generated sample data set includes: Obtain the request data of each service module in the service call chain, and forward the request data to the corresponding service module; Obtain the response data returned based on the request data, and forward the response data to the corresponding service module; The request data and response data of each service module are merged according to a preset data type to obtain the processing data of each service module; The sample data set is obtained by merging the processing data from multiple service modules.

3. The test method according to claim 1, characterized in that, The step of storing the sample data set into the benchmark database includes: The target function is invoked to serialize the sample data set, resulting in the target data set. Obtain the target file compiled based on the preset file format, and convert the target data set into the target file format according to the target file; The target data set, converted into the target file format, is stored in the benchmark database.

4. The test method according to claim 1, characterized in that, The step of generating test cases based on the sample data set includes: Based on the sample data set, the request data and response data of each service module in the service call chain, as well as the link identifier corresponding to the service call chain, are obtained; Obtain a preset test case template, and add the request data and the response data to the corresponding positions in the preset test case template to obtain the test case.

5. The test method according to claim 4, characterized in that, The step of generating test cases based on the sample data set further includes: The response data of each service module in the service call chain are merged to obtain the result simulation data; Generate the result simulation identifier corresponding to the result simulation data; Generate key-value pairs based on the simulated data and the simulated identifier, and add the key-value pairs to the test case.

6. The test method according to claim 5, characterized in that, The target program that triggers the version change runs the test cases, including: The target program, after triggering the version change, generates a test request based on the test cases; Determine the first service module currently invoked when processing the test request, and obtain the simulation status of the second service module bypassed by the first service module. The simulation status is used to characterize whether the response data returned by the second service module to the first service module needs to be simulated or not. When the simulation state indicates that the response data returned by the second service module to the first service module needs to be simulated, the result simulation data is obtained according to the result simulation identifier, and the result simulation data is used as the response data returned by the second service module to the first service module.

7. The test method according to claim 1, characterized in that, Before the target program, after triggering the version change, runs the test case, the testing method further includes: The test cases are run in the continuous integration platform at a preset frequency. The total number of times the test cases were successfully run in the continuous integration platform; When the number of successful runs is greater than or equal to a preset threshold, the test case is added to the test case library of the continuous integration platform.

8. The test method according to claim 7, characterized in that, The testing method also includes: Obtain the historical runtime of the test case when it is successfully run in the continuous integration platform, and determine the target runtime based on the historical runtime. Obtain the benchmark data set obtained by running the test cases at the target runtime, and use the benchmark data set to update the sample data set in the benchmark database.

9. The test method according to claim 1, characterized in that, Before comparing the test data set with the sample data set, the testing method further includes: According to the preset filtering rules, data to be removed is filtered from the test data set and the sample data set, and the data to be removed is then removed. The step of selecting data to be removed from the test data set and the sample data set according to preset filtering rules includes at least one of the following: Data with random numbers as its data attribute is selected as the data to be removed from the test data set and the sample data set. or, The data to be removed is selected from the test data set and the sample data set using wildcards; or, Variable strings are extracted from the target type of data, and the variable strings are used as the data to be removed. The data to be removed is then selected from the test data set and the sample data set.

10. The test method according to claim 1, characterized in that, The target program generates the business request in a first operating environment, and the target program runs the test cases in a second operating environment. Before generating the sample data set, the testing method further includes: Determine the request type of the business request; The risk level of the request type is determined according to the preset business classification rules; When the risk level is less than or equal to a preset level threshold, the sample data set is determined as the data set to be generated.

11. A testing apparatus, characterized in that, include: A data generation module is used to generate a sample data set, which includes processing data of multiple service modules called when processing business requests of a target program. The processing data of each service module includes a link identifier, which is used to identify a service call link formed by multiple service modules. Add the sample data set to the task list; In response to a task processing instruction, the sample data set is retrieved from the task list and sent to the thread pool; In response to a thread processing instruction, the sample data set is extracted from the thread pool; Based on the sample data set, the multiple service modules called by the target program when processing the business request are connected in series to obtain the service call chain. The service call chain is then compared with a preset call chain to determine the integrity of the service call chain. A first tag for identifying the sample data set is extracted from the sample data set, and multiple second tags are extracted from the benchmark database. Based on the matching relationship between the first tag and the multiple second tags, the storage status of the sample data set in the benchmark database is determined, wherein the storage status is either stored in the benchmark database or not stored in the benchmark database. Provided that the service call chain is complete and the sample data set is determined to be new data based on the matching relationship between the first tag and multiple second tags, the sample data set is stored in the benchmark database, which is used to store data as a test comparison benchmark. The test case generation module is used to generate test cases based on the sample data set and to identify the test cases based on the link identifier. The data comparison module is used to trigger the target program after the version change to run the test cases when the version of the target program changes, obtain the test data set, determine the link identifier corresponding to the test cases, obtain the corresponding sample data set from the benchmark database according to the link identifier, compare the test data set with the sample data set, and obtain the test results. The step of comparing the test data set with the sample data set includes: The test data set is compared with the request data of the same service module in the sample data set; The test dataset is compared with the response data of the corresponding service module in the sample dataset.

12. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the test method according to any one of claims 1 to 10.

13. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the test method according to any one of claims 1 to 10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the test method according to any one of claims 1 to 10.

Images