System testing method, device, equipment and storage medium based on production traffic

Abstract

The embodiments of the present application provide a system testing method, apparatus, device and storage medium based on production traffic. The method divides the production traffic file into multiple traffic file blocks, determines test cases based on the traffic file blocks, inputs the test cases into the system to be tested, and outputs test results. If the test results do not meet the preset conditions, key fields are determined based on the traffic file blocks, the test results, preset test targets and each key field are input into a trained feedback model, and the confidence of each traffic file block is output through the trained feedback model. The test cases are determined based on the traffic file blocks and the confidence until the test results meet the preset conditions, and the corresponding traffic file blocks are obtained, thereby achieving accurate evaluation of the confidence of the traffic file blocks. This eliminates the need to re-collect the production traffic files, thereby improving the efficiency of system testing based on production traffic.

Description

Technical Field

[0001] The present application relates to the field of software testing technology, and in particular to a system testing method, apparatus, device and storage medium based on production traffic. Background Art

[0002] Production traffic is data from previous system (software) runs. Usually, after a system (software) update, production traffic is used to re-execute the system to verify whether the execution results are consistent with the previous results. This is used to determine whether the updated system (software) is usable. Therefore, production traffic is mostly used in test systems.

[0003] Existing testing methods based on production traffic require recollecting production traffic data when the test results are not ideal, and performing performance testing based on the recollected production traffic data. However, the collection of production traffic is time-consuming, making the testing process cumbersome and time-consuming, which reduces the testing efficiency of the system.

[0004] Application Contents

[0005] The embodiments of the present application provide a system testing method, apparatus, device and storage medium based on production traffic to solve the technical problem that the testing process is complicated and tedious due to multiple production traffic collection, which reduces the efficiency of system testing.

[0006] In one aspect, the present application provides a system testing method based on production traffic, the method comprising:

[0007] Split the collected production traffic files into multiple traffic file blocks;

[0008] Determining a test case based on the plurality of traffic file blocks;

[0009] Input the test case into the system to be tested and output the test result;

[0010] If the test result does not meet the preset condition, parsing the plurality of traffic file blocks to obtain corresponding plurality of key fields;

[0011] Inputting the test results, the preset test objectives, and each of the key fields into a trained feedback model, and outputting the confidence of each of the traffic file blocks through the trained feedback model, wherein the confidence of the traffic file block is used to characterize the credibility of each traffic file block as test data;

[0012] According to the traffic file blocks and the corresponding confidence levels, a test case is determined, and the steps of inputting the test case into the system to be tested and outputting the test result are continued until the test result meets the preset conditions, and the traffic file block is determined as the target production traffic of the system to be tested.

[0013] In one aspect, the present application provides a system testing device based on production flow, comprising:

[0014] The segmentation module is used to segment the collected production traffic files into multiple traffic file blocks;

[0015] A determination module, configured to determine a test case based on the plurality of traffic file blocks;

[0016] The first test module is used to input the test case into the system to be tested and output the test result;

[0017] A parsing module, configured to parse the plurality of traffic file blocks to obtain corresponding plurality of key fields if the test result does not meet a preset condition;

[0018] A second testing module is configured to input the test results, the preset test objectives, and each of the key fields into a trained feedback model, and output the confidence of each of the traffic file blocks through the trained feedback model, where the confidence of the traffic file block is used to characterize the credibility of each traffic file block as test data;

[0019] The third test module is used to determine the test case based on the traffic file blocks and the corresponding confidence levels, and continue to execute the steps of inputting the test case into the system to be tested and outputting the test results until the test results meet the preset conditions, and then determine the traffic file block as the target production traffic of the system to be tested.

[0020] On the one hand, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps in the above-mentioned system testing method based on production traffic.

[0021] On the one hand, the present application provides a computer-readable medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps in the above-mentioned system testing method based on production traffic.

[0022] The embodiment of the present application provides a system testing method based on production traffic, which divides the collected production traffic files into multiple traffic file blocks, determines test cases based on the multiple traffic file blocks, inputs the test cases into the system to be tested, outputs test results, and when the test results do not meet the preset conditions, parses the multiple traffic file blocks to obtain corresponding multiple key fields, inputs the test results, preset test targets and each key field into a trained feedback model, and outputs the confidence of each traffic file block through the trained feedback model. The confidence of the traffic file block is used to characterize the credibility of each traffic file block as test data. File blocks and their corresponding confidence levels are used to determine test cases, and the steps of inputting the test cases into the system to be tested and outputting the test results are continued until the test results meet the preset conditions. The traffic file blocks are then determined as the target production traffic of the system to be tested, thereby achieving an accurate assessment of the confidence level of the traffic file blocks. At the same time, the traffic file blocks when the test results meet the preset conditions are obtained and used as the target production traffic, thereby ensuring the accuracy of the target production traffic. This eliminates the need to re-collect the production traffic files and multiple tests, thereby achieving accurate testing of the system to be tested. This greatly improves the efficiency of obtaining production traffic, and also improves the efficiency of system testing based on production traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] in:

[0025] Figure 1 A schematic diagram of an application scenario of a system testing method based on production traffic in one embodiment;

[0026] Figure 2 is a flow chart of a system testing method based on production traffic in one embodiment;

[0027] Figure 3 is a structural block diagram of a system testing device based on production traffic in one embodiment;

[0028] Figure 4 FIG. 1 is a structural block diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0029] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0030] The system testing method based on production flow provided in this application can be applied in Figure 1 In an application environment, a terminal device communicates with a server via a network. The terminal device may include, but is not limited to, various personal computers, laptops, smartphones, tablet computers, and portable wearable devices. The server may be implemented as a standalone server or a server cluster consisting of multiple servers.

[0031] System framework 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network is used to provide a medium for communication links between terminal devices and servers. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables.

[0032] Users can use terminal devices to interact with the server through the network to receive or send messages, etc.

[0033] Terminal devices 101, 102, and 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, e-book readers, MP3 players (Moving Picture Eperts Group Audio Layer III), MP4 players (Moving Picture Eperts Group Audio Layer IV), laptop computers, desktop computers, etc.

[0034] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0035] It should be noted that the system testing method based on production traffic provided in the embodiment of the present invention is executed by a server. Accordingly, the system testing device based on production traffic is provided in the server.

[0036] It should be understood that Figure 1The number of terminal devices, networks and servers is only illustrative. According to implementation requirements, there may be any number of terminal devices, networks and servers. The terminal devices in the embodiment of the present invention may specifically correspond to application systems in actual production.

[0037] like Figure 2 As shown, in one embodiment, a system testing method based on production traffic is provided. The system testing method based on production traffic can be applied to both terminals and servers. This embodiment uses the application to a server as an example. The system testing method based on production traffic specifically includes the following steps:

[0038] Step 201: Segment the collected production traffic file to obtain multiple traffic file blocks.

[0039] Among them, the production traffic file refers to a file composed of a series of production traffic, that is, the production traffic file contains multiple production traffics, which are used as test data for the test system. The production traffic can be obtained from the operation log of the system to be tested before the update, or the system to be tested before the update can be operated in advance, and the operation data can be stored in the server after acquisition. The production traffic can be obtained from the service, or the pre-stored production traffic can be recorded to form the production traffic.

[0040] Segmentation refers to a method of classifying and splitting production traffic files. Specifically, segmentation can be performed based on the data type of the production traffic files, such as character type, binary type, etc., or based on the amount of production traffic in the production traffic files. Segmentation can also be performed based on a combination of the amount of production traffic and the data type of the production traffic files, thereby obtaining a traffic file block of one or more production traffic combinations. As a preferred method in this embodiment, segmentation based on the data type of the production traffic files is selected to achieve a finer-grained division of the traffic file blocks, improve the accuracy of the matching of the data type of the production traffic files with the system to be tested, and help improve the accuracy of subsequent tests.

[0041] Step 202: Determine a test case based on multiple traffic file blocks.

[0042] A test case is a test module that contains test logic and a corresponding test environment. More specifically, a test case can be a code block or a function library. Specifically, at least one of multiple traffic file blocks can be used as test data, and a test template of the system to be tested can be obtained and rendered to generate a corresponding test case. Alternatively, a test scenario can be obtained, and the traffic file block corresponding to the test scenario can be selected as test data. This test data can then be added to the test scenario to form a test case.

[0043] Step 203: Input the test case into the system to be tested and output the test result.

[0044] The system under test (SUT) refers to the software system running on the server that needs to be tested. The test result is the output after the SUT is tested, reflecting the performance of the SUT, such as a test success or test failure. Specifically, the server inputs the test case into the SUT, performs the test, and outputs the test result for subsequent processing based on the test result.

[0045] Step 204: If the test result does not meet the preset conditions, parse the multiple traffic file blocks to obtain corresponding multiple key fields.

[0046] Key fields refer to fields that reflect test-related attributes within a traffic file block. Specifically, key fields within a traffic file block can be parsed based on the semantics of the traffic file block or by comparing against a pre-defined test semantics library containing multiple standard test key fields. It is worth noting that to improve the efficiency of subsequent test case generation, key fields can be formatted, for example, to conform to a test case format.

[0047] A preset condition refers to a pre-set critical value of the test result used to measure the test effectiveness of the system under test. For example, the preset condition may be a test result with a success probability of 80%, 85%, or 90%. When the test result does not meet the preset condition, it indicates that the test effect has not met the requirements, that is, the test accuracy is lower than the critical value of the test result. To improve test efficiency, in this embodiment, multiple traffic file blocks are parsed to obtain corresponding key fields. This allows for further refined processing of the traffic file blocks, avoiding the tedious and time-consuming process of re-collecting production traffic files.

[0048] In step 205, the test results, preset test targets and key fields are input into the trained feedback model, and the confidence of each traffic file block is output through the trained feedback model. The confidence of the traffic file block is used to characterize the credibility of each traffic file block as test data.

[0049] The confidence of the traffic file block is used to characterize the credibility of each traffic file block as test data.

[0050] The trained feedback model is a pre-trained deep learning model for determining the confidence of the test results, preset test targets and traffic file blocks corresponding to each key field, and is used to feedback the accuracy of the test of the file traffic block in the system to be tested. The feedback model of this embodiment is used to learn the test results, preset test targets and each key field, and feedback the accuracy of the test of the file traffic block in the system to be tested. Specifically, at least one recurrent neural network model (Recurrent Neural Network) can be used for classification, wherein the recurrent neural network model can be at least one long short-term memory network LSTM. As part of the training of such a neural network model, the training input is various test results, preset test targets and key fields corresponding to each traffic file block, and a classifier is established for the test results, preset test targets and key fields corresponding to each traffic file block and the confidence of the traffic file block. This enables the feedback model to have the function of evaluating the confidence of the traffic file block.

[0051] It can be understood that by determining the test results through the trained feedback model, and determining the confidence of each traffic file block through preset test objectives and key fields, the credibility of the traffic file block as test data is analyzed, and then the automatic selection of the traffic file block is realized, avoiding the tedious operation of collecting traffic file blocks multiple times, which is conducive to improving the efficiency of subsequent tests.

[0052] In step 206, a test case is determined based on the traffic file blocks and their corresponding confidence levels, and the steps of inputting the test case into the system to be tested and outputting the test results are continued until the test results meet the preset conditions, and the traffic file blocks are determined as the target production traffic of the system to be tested.

[0053] Specifically, according to the traffic file blocks and their corresponding confidence levels, test cases are determined, and step 203 is repeated. When the test results meet the preset conditions, the traffic file blocks are determined as the target production traffic of the system to be tested. By using the trained feedback model to determine the confidence levels of the various traffic file blocks, accurate evaluation of the confidence levels of the traffic file blocks is achieved. At the same time, the traffic file blocks when the test results meet the preset conditions are obtained and used as the target production traffic, ensuring the accuracy of the target production traffic. This eliminates the need to re-collect the production traffic files and multiple tests, thereby enabling accurate testing of the system to be tested. This greatly improves the efficiency of obtaining production traffic, and also improves the efficiency of system testing based on production traffic.

[0054] The above-mentioned system testing method based on production traffic divides the collected production traffic files into multiple traffic file blocks, determines test cases based on the multiple traffic file blocks, inputs the test cases into the system to be tested, outputs test results, and when the test results do not meet the preset conditions, parses the multiple traffic file blocks to obtain corresponding multiple key fields, inputs the test results, preset test targets and each key field into the trained feedback model, and outputs the confidence of each traffic file block through the trained feedback model. The confidence of the traffic file block is used to characterize the credibility of each traffic file block as test data. Based on the corresponding confidence level, the test case is determined, and the steps of inputting the test case into the system to be tested and outputting the test result are continued. When the test result meets the preset conditions, the traffic file block is determined as the target production traffic of the system to be tested, thereby achieving an accurate assessment of the confidence level of the traffic file block. At the same time, the traffic file block when the test result meets the preset conditions is obtained and used as the target production traffic, thereby ensuring the accuracy of the target production traffic. Therefore, there is no need to re-collect the production traffic file and there is no need to conduct multiple tests, thereby achieving accurate testing of the system to be tested, greatly improving the efficiency of obtaining production traffic, and also improving the efficiency of system testing based on production traffic.

[0055] In one embodiment, the collected production traffic files are segmented to obtain multiple traffic file blocks, including: performing differential analysis on the production traffic files, selecting differential traffic files with a difference rate greater than a preset threshold from the production traffic files; segmenting the differential traffic files according to data type to obtain traffic file blocks corresponding to each data type.

[0056] Among them, differentiation analysis refers to the semantic similarity between each production flow in the production flow file. The lower the semantic similarity, the greater the differentiation rate.

[0057] Specifically, differential traffic files with a difference rate greater than a preset threshold are selected from the production traffic files, that is, production traffic with higher semantic similarity is screened and processed to avoid redundancy of production traffic with higher semantic similarity. Then, the differential traffic files are segmented according to the data type to obtain traffic file blocks corresponding to each data type, thereby realizing the segmentation of production traffic with different semantics in data types, realizing a more fine-grained division of production traffic files, improving the fine division of production traffic file flows, and helping to improve the accuracy of traffic file blocks.

[0058] In one embodiment, a test case is determined based on multiple traffic file blocks, including: obtaining a test interface of the system to be tested and a function library required to be loaded for running the test interface, the function library including multiple test functions; determining a target data type based on the data type of the input parameter in each test function; determining a traffic file block whose data type is the target data type as the first traffic block; and generating a test case based on each test function and the corresponding first traffic block.

[0059] Specifically, it is possible to obtain the dependent applications of the system to be tested, find the test interface of the dependent applications, and then determine the function library that needs to be loaded for its operation based on the test interface. The function library includes multiple test functions, and each test function declares the corresponding input parameter data type. The input parameter data type in each test function is used as the data type of the test function, that is, the input parameter data type is the target data type. A traffic block with the target data type is selected from multiple traffic file blocks and determined as the first traffic block. The test template of the system to be tested is obtained, and the first traffic block is used as test data. The test template is rendered according to the test data and the test function to generate a test case. In this embodiment, test cases are generated according to the test functions and test modules of the system to be tested through traffic file blocks of different data types, thereby improving the coupling degree between the test case and the traffic file block.

[0060] In one embodiment, when the test results do not meet the preset conditions, multiple traffic file blocks are parsed to obtain corresponding multiple key fields, including: generating a test report based on the test results that do not meet the preset conditions; constructing a test field attribute table of the test report, recording multiple test fields in the test field attribute table; calculating the semantic similarity between the fields in the traffic file block and the test fields; and determining the fields whose semantic similarity is greater than the preset similarity threshold as key fields.

[0061] Specifically, a test field attribute table of the test report is constructed, in which multiple test fields are recorded. Then, the semantic similarity between the fields in the traffic file block and the test fields is calculated. Specifically, the first feature vector of the field in the traffic file block and the second feature vector of the test field are extracted respectively, and the distance between the first feature vector and the second feature vector is calculated. The semantic similarity is determined according to the distance, and then the fields whose semantic similarity is greater than the preset similarity threshold are determined as key fields, thereby realizing the parsing of the key fields of the traffic file block. Since the key fields are finer-grained than the traffic file blocks, the accuracy of the production traffic can be improved based on the key fields in the future.

[0062] In one embodiment, a test case is determined based on traffic file blocks and their corresponding confidence levels, including: based on the confidence levels, filtering out traffic file blocks with confidence levels greater than a preset confidence threshold and determining them as second traffic blocks; and determining a test case based on the second traffic block and a test function.

[0063] Among them, the preset confidence threshold refers to a preset critical value for judging the confidence of a traffic file block as test data. For example, the preset confidence threshold can be 85%, 90% or 95%. Specifically, a traffic file block with a confidence greater than the preset confidence threshold is screened out from multiple confidence levels and determined as a second traffic block. The second traffic block is used as test data, and the test template is rendered according to the test data and the test function to generate a test case. Since the second traffic block is a more accurate traffic file block, the test case is determined according to the second traffic block and the test function, thereby optimizing the test case and further improving the accuracy of the test case.

[0064] In one embodiment, before inputting the test results, preset test targets and each key field into the trained feedback model and outputting the confidence of each traffic file block through the trained feedback model, the method also includes: obtaining a sample set, the sample set including the sample test results, the sample test targets, key fields and sample confidence corresponding to the sample test results; constructing multiple mapping matrices of the sample test results, key fields, sample test targets and corresponding sample confidences; inputting the mapping matrices and the corresponding sample confidences into a recursive neural network for training to obtain a trained feedback model.

[0065] Among them, recursive neural network is an unsupervised learning method. It uses recursive neural network for training and automatically learns the characteristics of traffic file blocks from the mapping matrix constructed by sample test results, key fields, sample test targets and corresponding sample confidence, which is conducive to improving the training efficiency of the feedback model.

[0066] Specifically, the sample test results, the sample test targets, key fields and sample confidence corresponding to the sample test results can more accurately and comprehensively evaluate the confidence of the traffic file block as test data for testing. Therefore, multiple mapping matrices are constructed based on the sample test results, key fields, sample test targets and corresponding sample confidence. The construction process of the mapping matrix is ​​as follows: the sample test results, key fields, sample test targets and corresponding sample confidence are constructed as an initial matrix, the sample test results, key fields, sample test targets and corresponding sample confidence are converted into a tree structure, the tree structure is constructed into a conversion matrix, and the initial matrix and the conversion matrix are merged into a mapping matrix, so that the confidence of the traffic file block as test data for testing can be learned more efficiently. The mapping matrix and the corresponding sample confidence are used as inputs of the recursive neural network for training to obtain a trained feedback model.

[0067] In one embodiment, after the test result satisfies a preset condition and the traffic file block is determined as the target production traffic of the system to be tested, the method further includes: replaying or recording the target production traffic.

[0068] The target production traffic playback refers to a test method in which the business traffic in the production environment is recorded, or the recorded business traffic is used for playback in the production environment or test environment.

[0069] Specifically, in this embodiment, since the target production flow is a more accurate production flow, replaying or recording the target production flow can further improve the efficiency of the system test.

[0070] like Figure 3 As shown, in one embodiment, a system testing device based on production flow is proposed, comprising:

[0071] A segmentation module 301 is used to segment the collected production traffic file to obtain multiple traffic file blocks;

[0072] A determination module 302 is configured to determine a test case based on the plurality of traffic file blocks;

[0073] The first testing module 303 is used to input the test case into the system to be tested and output the test result;

[0074] The parsing module 304 is configured to parse the plurality of traffic file blocks to obtain corresponding key fields if the test result does not meet the preset conditions;

[0075] A second testing module 305 is configured to input the test results, the preset test objectives, and each of the key fields into a trained feedback model, and output the confidence level of each of the traffic file blocks through the trained feedback model, where the confidence level of each of the traffic file blocks is used to characterize the credibility of each of the traffic file blocks as test data;

[0076] The third test module 306 is used to determine the test case based on the traffic file block and the corresponding confidence level, and continue to execute the steps of inputting the test case into the system to be tested and outputting the test result until the test result meets the preset condition, and then determine the traffic file block as the target production traffic of the system to be tested.

[0077] In one embodiment, the segmentation module includes:

[0078] an analyzing unit, configured to perform differential analysis on the production flow files, and select differential flow files having a differential rate greater than a preset threshold from the production flow files;

[0079] The segmentation unit is used to segment the differential traffic file according to data type to obtain the traffic file blocks corresponding to each data type.

[0080] In one embodiment, the determining module includes:

[0081] An acquiring unit, configured to acquire a test interface of the system to be tested and a function library required to be loaded for running the test interface, wherein the function library includes a plurality of test functions;

[0082] A first determining unit, configured to determine a target data type according to the data type of an input parameter in each of the test functions;

[0083] A second determining unit, configured to determine the traffic file block whose data type is the target data type as a first traffic block;

[0084] The first generating unit is configured to generate the test case according to each of the test functions and the corresponding first traffic block.

[0085] In one embodiment, the parsing module includes:

[0086] A second generating unit, configured to generate a test report according to the test results that do not meet the preset conditions;

[0087] A construction unit, configured to construct a test field attribute table of the test report, wherein the test field attribute table records a plurality of test fields;

[0088] a calculation unit, configured to calculate the semantic similarity between the field in the traffic file block and the test field;

[0089] The third determining unit is configured to determine the fields whose semantic similarity is greater than a preset similarity threshold as the key fields.

[0090] In one embodiment, the third test module includes:

[0091] a screening unit, configured to screen out, based on the confidence level, a flow file block having a confidence level greater than a preset confidence threshold and determine the block as a second flow file block;

[0092] A fourth determining unit is configured to determine the test case according to the second traffic block and the test function.

[0093] In one embodiment, the production flow-based system testing device further includes:

[0094] A first acquisition module is used to acquire a sample set, wherein the sample set includes a sample test result, a sample test target, a key field, and a sample confidence corresponding to the sample test result;

[0095] A construction module, configured to construct a plurality of mapping matrices using the sample test results, the key fields, the sample test targets, and the corresponding sample confidences;

[0096] A training module is used to input the mapping matrix and the corresponding sample confidence into a recursive neural network for training to obtain a trained feedback model.

[0097] In one embodiment, the system testing device based on production traffic further includes: a processing module, configured to replay or record the target production traffic.

[0098] Figure 4 FIG1 shows an internal structure diagram of a computer device in an embodiment. The computer device may be a server, including but not limited to a high-performance computer and a high-performance computer cluster. Figure 4 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement a system testing method based on production traffic. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can implement a system testing method based on production traffic. Those skilled in the art will understand that Figure 4The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0099] In one embodiment, the system testing method based on production traffic provided by the present application can be implemented in the form of a computer program. Figure 4 The computer device shown in FIG. The computer device's memory may store various program templates that make up the production traffic-based system testing apparatus, such as a segmentation module 301 , a determination module 302 , a first test module 303 , a parsing module 304 , a second test module 305 , and a third test module 306 .

[0100] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the following production traffic-based system testing method are implemented:

[0101] Split the collected production traffic files into multiple traffic file blocks;

[0102] Determining a test case based on the plurality of traffic file blocks;

[0103] Input the test case into the system to be tested and output the test result;

[0104] If the test result does not meet the preset condition, parsing the plurality of traffic file blocks to obtain corresponding plurality of key fields;

[0105] Inputting the test results, the preset test objectives, and each of the key fields into a trained feedback model, and outputting the confidence of each of the traffic file blocks through the trained feedback model, wherein the confidence of the traffic file block is used to characterize the credibility of each traffic file block as test data;

[0106] According to the traffic file blocks and the corresponding confidence levels, a test case is determined, and the steps of inputting the test case into the system to be tested and outputting the test result are continued until the test result meets the preset conditions, and the traffic file block is determined as the target production traffic of the system to be tested.

[0107] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps in a system testing method based on production traffic: segmenting a collected production traffic file to obtain a plurality of traffic file blocks;

[0108] Determining a test case based on the plurality of traffic file blocks;

[0109] Input the test case into the system to be tested and output the test result;

[0110] If the test result does not meet the preset condition, parsing the plurality of traffic file blocks to obtain corresponding plurality of key fields;

[0111] Inputting the test results, the preset test objectives, and each of the key fields into a trained feedback model, and outputting the confidence of each of the traffic file blocks through the trained feedback model, wherein the confidence of the traffic file block is used to characterize the credibility of each traffic file block as test data;

[0112] According to the traffic file blocks and the corresponding confidence levels, a test case is determined, and the steps of inputting the test case into the system to be tested and outputting the test result are continued until the test result meets the preset conditions, and the traffic file block is determined as the target production traffic of the system to be tested.

[0113] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0114] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A system testing method based on production traffic, characterized in that: The method comprises: Split the collected production traffic files into multiple traffic file blocks; Determining a test case based on the plurality of traffic file blocks; Input the test case into the system to be tested and output the test result; If the test result does not meet the preset condition, parsing the plurality of traffic file blocks to obtain corresponding plurality of key fields; Inputting the test results, the preset test objectives, and each of the key fields into a trained feedback model, and outputting the confidence of each of the traffic file blocks through the trained feedback model, wherein the confidence of the traffic file block is used to characterize the credibility of each traffic file block as test data; Determining a test case based on the traffic file blocks and the corresponding confidence levels, and continuing to perform the steps of inputting the test case into the system to be tested and outputting the test result until the test result satisfies the preset condition, and determining the traffic file block as the target production traffic of the system to be tested; The step of determining a test case based on the plurality of traffic file blocks includes: Acquire a test interface of the system to be tested and a function library required to be loaded for running the test interface, wherein the function library includes a plurality of test functions; Determine the target data type based on the data type of the input parameter in each of the test functions; Determine the traffic file block whose data type is the target data type as the first traffic block; The test case is generated according to each of the test functions and the corresponding first traffic block.

2. The system testing method based on production traffic according to claim 1, characterized in that: The collected production traffic files are segmented to obtain multiple traffic file blocks, including: Performing differential analysis on the production flow files, and selecting differential flow files having a differential rate greater than a preset threshold from the production flow files; The differential traffic files are segmented according to data types to obtain the traffic file blocks corresponding to each data type.

3. The system testing method based on production traffic according to claim 1, characterized in that: When the test result does not meet the preset conditions, the plurality of traffic file blocks are parsed to obtain corresponding plurality of key fields, including: Generating a test report based on the test results that do not meet the preset conditions; Constructing a test field attribute table of the test report, wherein the test field attribute table records a plurality of test fields; Calculating semantic similarity between a field in the traffic file block and the test field; The fields whose semantic similarity is greater than a preset similarity threshold are determined as the key fields.

4. The system testing method based on production traffic according to claim 1, characterized in that: Determining a test case according to the traffic file blocks and the corresponding confidence levels includes: According to the confidence level, a flow file block having a confidence level greater than a preset confidence threshold is screened out and determined as a second flow block; The test case is determined according to the second traffic block and the test function.

5. The system testing method based on production traffic according to claim 1, characterized in that: Before inputting the test results, the preset test targets, and each of the key fields into the trained feedback model and outputting the confidence of each of the traffic file blocks through the trained feedback model, the method further includes: Obtaining a sample set, the sample set including a sample test result, a sample test target, a key field, and a sample confidence corresponding to the sample test result; Constructing a plurality of mapping matrices using the sample test results, the key fields, the sample test targets and the corresponding sample confidences; The mapping matrix and the corresponding sample confidence are input into a recursive neural network for training to obtain a trained feedback model.

6. The system testing method based on production traffic according to claim 1, characterized in that: After determining the traffic file block as the target production traffic of the system to be tested when the test result satisfies the preset condition, the method further includes: The target production traffic is replayed or recorded.

7. A system testing device based on production flow, characterized in that: The system testing device based on production flow includes: The segmentation module is used to segment the collected production traffic files into multiple traffic file blocks; A determination module, configured to determine a test case based on the plurality of traffic file blocks; The first test module is used to input the test case into the system to be tested and output the test result; A parsing module, configured to parse the plurality of traffic file blocks to obtain corresponding plurality of key fields if the test result does not meet a preset condition; A second testing module is configured to input the test results, the preset test objectives, and each of the key fields into a trained feedback model, and output the confidence of each of the traffic file blocks through the trained feedback model, where the confidence of the traffic file block is used to characterize the credibility of each traffic file block as test data; a third testing module, configured to determine a test case based on the traffic file blocks and the corresponding confidence levels, and continue to execute the steps of inputting the test case into the system to be tested and outputting the test result until the test result satisfies the preset condition, thereby determining the traffic file block as the target production traffic of the system to be tested; Among them, the determination module is specifically used to obtain the test interface of the system to be tested and the function library required to be loaded for the operation of the test interface, and the function library includes multiple test functions; determines the target data type according to the data type of the input parameters in each of the test functions; determines the traffic file block with the data type of the target data type as the first traffic block; generates the test case according to each of the test functions and the corresponding first traffic block.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the system testing method based on production traffic are implemented as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the system testing method based on production traffic according to any one of claims 1 to 6 are implemented.

Images