Methods, apparatuses, and systems for generating test samples

Abstract

The application discloses a method for generating a test sample, comprising: receiving a generation request for generating a test sample, the generation request comprising a test sample generation strategy for a target application under test, the target application under test having a downstream application; in response to receiving the generation request, obtaining a to-be-varied sample of the target application under test, the to-be-varied sample comprising historical request data for the target application under test and historical response data responded by the downstream application to the response data of the target application under test, the response data of the target application under test being response data of the target application under test to the historical request data; and generating the test sample according to the test sample generation strategy and the to-be-varied sample for testing the target application under test.

Description

Technical Field

[0001] This application relates to the field of testing, and in particular to a method, apparatus and system for generating test samples, computing devices, computer-readable storage media and computer program products. Background Technology

[0002] Fuzzing is a common vulnerability discovery technique that involves providing unexpected input to a target application and observing the unusual results to uncover software vulnerabilities. Therefore, obtaining input capable of detecting problems in the target application is crucial in fuzzing.

[0003] In a standalone environment, test samples are typically generated by randomly mutating existing samples, and these samples are then used as input to test the target application. In this case, the mechanism for generating test samples is pre-defined for the standalone environment. Fuzzing mechanisms can only generate test samples according to a pre-defined strategy and cannot flexibly configure the mutation strategy, thus failing to meet the needs of testing different target applications. Furthermore, to quickly obtain samples, historical request data of the target application is often used directly. However, the coverage of historical request data is limited, so some special types of samples are easily overlooked, failing to meet the need for comprehensive testing of the target application. In addition, the computing power and data access capabilities of devices in a standalone environment are limited, making it impossible to meet the needs of some high-intensity tests. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus, system, computing device, computer-readable storage medium, and computer program product for generating test samples, which is intended to alleviate or overcome some or all of the defects mentioned above, as well as other possible defects.

[0005] According to one aspect of this application, a method for generating test samples is provided, comprising: receiving a generation request for generating test samples, the generation request including generating a test sample generation strategy for a target application under test, the target application under test having downstream applications; in response to receiving the generation request, obtaining a sample to be mutated for the target application under test, the sample to be mutated including historical request data for the target application under test and historical response data of the downstream applications responding to response data of the target application under test, the response data of the target application under test being the response data of the target application under test to the historical request data; and generating a test sample according to the test sample generation strategy and the sample to be mutated, the test sample being used to test the target application under test.

[0006] In some embodiments, the sample to be mutated is stored in the form of structured data, the structured data having a corresponding data format; and in response to receiving the generation request, obtaining the sample to be mutated of the target application under test includes: in response to receiving the generation request, obtaining the stored structured data of the target application under test; parsing the structured data according to the data format of the structured data to obtain the sample to be mutated.

[0007] In some embodiments, the data format corresponding to the structural data includes a compression format and a file format of the structural data; and wherein parsing the structural data according to the data format of the structural data to obtain the sample to be mutated includes: decompressing the structural data according to the compression format of the structural data to obtain at least one data file; and parsing the at least one data file according to the file format to obtain the sample to be mutated.

[0008] In some embodiments, the test sample generation strategy includes information indicating random mutation and a preset degree of mutation, the degree of mutation representing the extent to which the sample to be mutated is randomly mutated; and wherein generating a test sample according to the test sample generation strategy and the sample to be mutated includes: randomly mutating the sample to be mutated according to the preset degree of mutation to obtain the test sample.

[0009] In some embodiments, randomly mutating the sample to be mutated according to a preset degree of variability includes: determining at least one field of the sample to be mutated, wherein each field in the at least one field has a corresponding field type, and each field type has a corresponding random mutation method; for each corresponding field in the at least one field, randomly mutating the data at the corresponding field according to the preset degree of variability and the random mutation method corresponding to the field type of the corresponding field to obtain the test sample.

[0010] In some embodiments, randomly mutating the sample to be mutated according to a preset degree of mutability includes: randomly mutating the binary code representing the sample to be mutated according to the preset degree of mutability to obtain the test sample.

[0011] In some embodiments, the test sample generation strategy includes information indicating precise mutation and a corpus on which the precise mutation depends; and wherein generating a test sample based on the test sample generation strategy and the sample to be mutated includes: determining at least one field of the sample to be mutated, wherein each of the at least one field has a corresponding field type; and for each corresponding field of the at least one field, selecting corpus data with the same field type as the corresponding field from the corpus to replace the data of the corresponding field to obtain the test sample.

[0012] In some embodiments, the test sample generation strategy includes information indicating precise mutation, the data structure type targeted by the precise mutation, and at least one mutant; and wherein generating a test sample according to the test sample generation strategy and the sample to be mutated includes: deconstructing the sample to be mutated according to the data structure type to obtain a deconstructed sample to be mutated; and using the at least one mutant to mutate the deconstructed sample to be mutated to obtain the test sample.

[0013] In some embodiments, each of the at least one mutant has a preset priority; and wherein the step of using the at least one mutant to mutate the deconstructed sample to be mutated to obtain the test sample includes: sequentially using the at least one mutant to mutate the deconstructed sample to be mutated in order of priority of the at least one mutant to obtain the test sample.

[0014] According to another aspect of this application, an apparatus for generating test samples is provided, comprising: a scheduling module configured to receive a generation request for generating test samples, the generation request including generating a test sample generation strategy for a target application under test, the target application under test having downstream applications; an acquisition module configured to, in response to receiving the generation request, acquire a sample to be mutated for the target application under test, the sample to be mutated including historical request data for the target application under test and historical response data of the downstream applications responding to response data of the target application under test, the response data of the target application under test being the response data of the target application under test to the historical request data; and a processing module configured to generate test samples according to the test sample generation strategy and the sample to be mutated, the test samples being used to test the target application under test.

[0015] According to another aspect of this application, a system for generating test samples is provided, comprising: a memory configured to store a sample to be mutated of a target application under test in the form of structured data, the structured data having a corresponding data format; and an apparatus for generating test samples according to some embodiments of this application; wherein an acquisition module is further configured to acquire the sample to be mutated of the target application under test from the memory.

[0016] According to another aspect of this application, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of a method for generating test samples according to some embodiments of this application.

[0017] According to another aspect of this application, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, causes the processor to perform the steps of a method for generating test samples according to some embodiments of this application.

[0018] According to another aspect of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements a method for generating test samples according to some embodiments of this application.

[0019] The method, apparatus, and system for generating test samples claimed in this application utilize historical request data for the target application under test and historical response data from downstream applications responding to the target application under test. This allows for comprehensive mining of potential mutation samples of the target application under test, greatly enriching the dimensionality and diversity of the data structure of these samples and facilitating the comprehensive detection of defects in the target application under test. Furthermore, test sample generation strategies can be flexibly customized according to the characteristics of the target application under test, and then the data to be mutated is mutated according to the customized strategy to generate test samples. This overcomes the limitations of fixed strategies, and the generated test samples conform to the characteristics of the target application under test, thereby improving testing efficiency and enabling the rapid discovery of defects in the target application under test. Moreover, because the mutation strategy can be flexibly configured, the needs for testing different target applications under test are met.

[0020] These and other advantages of this application will become clear from the embodiments described below, and will be illustrated with reference to the embodiments described below. Attached Figure Description

[0021] Embodiments of this application will now be described in more detail with reference to the accompanying drawings, wherein: Figure 1 A flowchart illustrating the implementation of fuzz testing is shown. Figure 2 Exemplary application scenarios of methods for generating test samples according to some embodiments of this application are illustrated; Figure 3 An exemplary flowchart of a method for generating test samples according to some embodiments of this application is shown; Figure 4 A schematic diagram illustrating the process of generating test samples according to some embodiments of this application is shown; Figure 5 A schematic diagram illustrating the process of analyzing the incremental fields of the target application being tested according to some embodiments of this application; Figure 6 A schematic diagram illustrating the functionality of a mutation engine for generating test samples according to some embodiments of this application is shown; Figure 7 A schematic schematic diagram of the principle architecture for generating test samples according to some embodiments of this application is shown; Figure 8 An exemplary structural block diagram of an apparatus for generating test samples according to some embodiments of this application is shown; and Figure 9 An example system is shown, which includes an example computing device representing one or more systems and / or devices that can implement the various methods described herein. Detailed Implementation

[0022] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0023] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0024] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0025] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0026] It should be understood that although the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of this application. As used herein, the terms "and / or" and similar terms include all combinations of any, multiple, and all of the associated listed items.

[0027] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily essential for implementing this application, and therefore cannot be used to limit the scope of protection of this application.

[0028] Before detailing the embodiments of this application, the concepts of fuzz testing and related technologies will first be explained: Fuzz testing (or fuzzing) is a software testing technique. Its core idea is to input automatically or semi-automatically generated random data into a program and detect program anomalies, such as crashes or assertion failures, to discover potential program errors. The test samples generated according to this application can be used for fuzz testing.

[0029] Spark computing framework: Spark is an open-source cluster computing framework specifically designed for real-time data processing. In-memory cluster computing is Spark's most prominent feature, effectively improving application processing speed.

[0030] RecordIO file format: RecordIO is a file storage format with the file extension .rec. It compactly packages data for efficient reading and writing from distributed file systems. When testing with large datasets, loading and retrieving data becomes a bottleneck in shortening test time. The deep learning framework MXNet provides this data storage format, RecordIO, which allows for multi-process data loading, significantly improving access efficiency.

[0031] Protobuffer: Protobuffer (Google Protocol Buffers) is a language-independent, platform-independent, and scalable method for serializing structured data, provided by Google. It can be used for communication protocols, data storage, and more. Protobuffer supports C++, Java, and Python. Files encoded with protobuffer can have the .proto extension. Data structures can be defined within it, and then specially generated source code can be easily used to write and read data structures in various languages ​​across different data streams. Data structures can even be updated without breaking deployed programs compiled from older data structures. The content of a .proto file defines at least one message entity, which consists of multiple message fields. Message fields include: ① Field type: Integer (int32), string, and float, etc., where each data type can almost be mapped to basic data types in languages ​​like C++ and Java; ② Field name: The naming convention is basically the same as that of variables in languages ​​like C, C++, and Java.

[0032] HDFS (Hadoop Distributed File System) is a distributed file system that runs on commodity hardware. In practical storage, HDFS can split target data into multiple data blocks and store them in parallel across multiple storage devices. This distributed file system has three main characteristics: ① It can store very large files, where "very large" refers to hundreds of megabytes (MB), gigabytes (GB), or even terabytes (TB); in real-world applications, many distributed file system clusters store data at the petabyte (PB) level; ② It supports streaming data access; ③ It runs on commercial hardware, meaning HDFS does not require particularly expensive or reliable machines and can run on ordinary commercial machines. HDFS is a highly fault-tolerant system that provides high-throughput data access, making it suitable for large-scale datasets.

[0033] A database is a place to store electronic files, where users can perform operations such as adding, querying, updating, and deleting data. A "database" is a collection of data stored together in a certain way, shared by multiple users, with minimal redundancy, and independent of application programs. The Distributed File System (HDFS) is a specific type of database.

[0034] Figure 1 A schematic flowchart for implementing fuzz testing is shown.

[0035] During fuzz testing, the sample to be mutated 101 and configuration file 102 are first sent to the mutation engine 103. The sample to be mutated 101 is the historical input data of the target application under test. These samples are typically routine data that will not cause failures; therefore, they need to be mutated to obtain testable data. The configuration file 102 records the data structure, data block settings, etc., of the sample to be mutated 101, and is usually an XML file. Typically, the mutation engine 103 has a preset test sample generation strategy for the application to mutate the sample to be mutated 101.

[0036] After receiving the sample to be mutated 101 and the configuration file 102, the mutation engine 103 mutates the input data according to a preset test sample generation strategy and the configuration of the sample to be mutated 101 recorded in the configuration file 102, thereby obtaining a mutated sample. At the same time, the log recording unit 105 records the mutation status. After the mutation is completed, the generated mutated sample is transmitted to the executor 104.

[0037] The executor 104 performs tests on the target application under test in the test environment 106 based on the generated mutation-based samples. During and after execution, any failures that occur (such as application crashes or assertion failures) are recorded by the logging unit 105, with the samples causing the failures becoming valid samples. Finally, testers analyze the specific causes of the failures and repair the target application under test to reduce its defects.

[0038] exist Figure 1In the fuzzing implementation process shown, if the goal is to find defects in the target application under test, a mutation engine 103 with a specific strategy needs to be selected. This mutation engine 103 is then used to generate a large number of test samples under the specific strategy, and these large number of test samples are then used for testing. This mutation method lacks flexibility. Furthermore, using only the historical input data of the target application under test as the samples to be mutated results in insufficient coverage. In addition, in some internet applications, many applications (such as internet advertising-related applications) require frequent testing. For example, a rating application for advertising effectiveness needs to be tested every 3 hours. How to quickly generate a large number of test samples is also a problem that urgently needs to be solved.

[0039] Figure 2 An exemplary application scenario 200 of a method for generating test samples according to some embodiments of this application is illustrated. Application scenario 200 may include a database 201, a server 202, a network 203, and a terminal device 204, with the server 202 communicatively coupled to the terminal device 204 via the network 203.

[0040] In application scenario 200, server 202 receives a request to generate test samples. This request includes a test sample generation strategy for a target application under test, which has downstream applications. The generation request can be sent to server 202 in any manner; for example, it can be written according to the HTTP protocol and sent to server 202. Other communication protocols are also permitted, and this application does not limit them. The test sample generation strategy in the request can be a generation strategy defined according to the characteristics of the target application under test and as needed. Server 202 will complete subsequent mutation operations based on this test sample generation strategy.

[0041] After receiving a request to generate test samples, server 202 retrieves the target application to be mutated from database 201. This target application to be mutated includes historical request data for the target application and historical response data from downstream applications responding to the target application's response data. The target application's response data is the response data of the target application to the historical request data. Since the request to generate test samples is for the target application, server 202 can, for example, obtain the storage address of the target application's ...

[0042] After obtaining the mutation sample of the target application under test, server 202 generates a test sample according to the test sample generation strategy and the mutation sample. This test sample is used to test the target application under test. The testing process can be executed on server 202 or in another computing environment, optionally in a distributed computing system.

[0043] During and after the mutation process of the mutated samples according to the test sample generation strategy, the server 202 can transmit the statistics on the mutation status to the terminal device 204 via the network 203 in a wired or wireless manner. The terminal device 204 can receive and display the mutation statistics. The network 203 can be, for example, a wide area network (WAN), a local area network (LAN), a wireless network, a public telephone network, an intranet, or any other type of network well known to those skilled in the art.

[0044] It should be noted that database 201 can be a medium and / or device capable of persistently storing information, and / or a tangible storage device, or a memory capable of processing streaming data, such as a TDbank memory. Computer-readable storage media refers to non-signal-bearing media. Computer-readable storage media include hardware such as volatile and non-volatile, removable and non-removable media and / or storage devices implemented with methods or techniques suitable for storing information (such as computer-readable instructions, data structures, program modules, logic elements / circuits, or other data). As understood by those skilled in the art, instances of server 202 can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminals and servers can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0045] Terminal device 204 can be any type of mobile computing device, including mobile computers (e.g., personal digital assistants (PDAs), laptops, notebooks, tablets, netbooks, etc.), mobile phones (e.g., cellular phones, smartphones, etc.), wearable computing devices (e.g., smartwatches, head-mounted devices, including smart glasses, etc.) or other types of mobile devices. In some embodiments, terminal device 150 can also be a fixed computing device, such as a desktop computer, game console, smart TV, etc. Furthermore, when application scenario 200 includes multiple terminal devices 204, these multiple terminal devices 204 can be the same or different types of computing devices.

[0046] Figure 3 An exemplary flowchart of a method for generating test samples according to some embodiments of this application is shown. The method 300 shown can be performed on the server side (e.g., at...). Figure 2 The server 202 shown is implemented. Optionally, method 300 can be implemented in a cluster based on a distributed system. Figure 3 As shown, the method for generating test samples according to some embodiments of this application may include steps S301-S303.

[0047] In step S301, a request to generate test samples is received. This request includes a test sample generation strategy for a target application under test, which has downstream applications. The test sample generation strategy indicates the specific method of mutating the sample to be mutated, and can be specified by the tester, for example, before issuing the request to generate test samples. In this way, the characteristics of the target application under test (e.g., the field types of the input data of the target application under test, the file format of the sample to be mutated when storing it, etc.) can be fully considered to specifically set the test sample generation request. Downstream applications of the target application under test refer to applications that can receive data from the target application under test. For example, the target application under test can send its output data to a downstream application for processing. After processing, the downstream application can generate response data for the aforementioned output data. As an example, the response data can be fed back to the target application under test as input to the target application under test.

[0048] In step S302, in response to receiving a generation request, a mutation sample of the target application under test is obtained. This mutation sample includes historical request data for the target application under test and historical response data from downstream applications responding to the response data of the target application under test. The response data of the target application under test is the response data of the target application under test to the historical request data. The mutation sample is the basis for subsequent mutation operations and is the mutation object targeted by the test sample generation strategy. It can include any sample suitable for mutation. There are multiple ways to obtain the mutation sample of the target application under test, and this application does not limit this. For example, the generation request may include the storage address of the mutation sample of the target application under test, and the mutation sample of the target application under test can be obtained from the storage address; or the correspondence between the storage address of the mutation sample and the target application under test can be stored locally. After receiving the generation request, the storage address of the mutation sample of the target application under test is determined locally, and then the mutation sample of the target application under test is obtained from the storage address.

[0049] Furthermore, request data and response data typically differ significantly, often having entirely different data structures and field values. Request data includes the input data to be processed by the target application under test, also known as sample data. For example, when the target application is an image editor, the request data could be image data. Response data, in addition to image data, may also include logical data, etc. Therefore, historical response data from downstream applications greatly enriches the dimensions and diversity of the data structure of the sample to be mutated, helping to comprehensively detect defects in the target application under test. For example, the historical request data of ad generation application A might only include string data, such as ad slogans, while the response data of downstream applications of ad generation application A (such as rating applications) could include integer (int32) and floating-point (float) data. Including the response data of downstream applications for ad generation application A means that the sample to be mutated includes samples of string, integer, and floating-point data.

[0050] In step S303, a test sample is generated according to the test sample generation strategy and the sample to be mutated. This test sample is used to test the target application being tested.

[0051] exist Figure 3 In the method 300 shown, by utilizing historical request data for the target application under test and historical response data from downstream applications responding to the target application under test, the potential mutation samples of the target application under test can be comprehensively mined. This greatly enriches the dimensionality and diversity of the data structure of the potential mutation samples, which helps to comprehensively detect defects in the target application under test. Furthermore, mutation strategies can be customized according to the characteristics of the target application under test, improving the flexibility of the mutation strategies, generating valuable test samples, and thus improving the testing efficiency of the target application under test.

[0052] Figure 4 A schematic diagram illustrating an exemplary process for generating test samples according to some embodiments of this application is shown. Figure 4 As shown, the historical request data of the target application under test is first routed to the sandbox environment. The sandbox environment is a simulated environment in which the target application under test and its downstream applications are deployed.

[0053] In the sandbox environment, historical request data for the target application under test will be received. This historical request data will be input into the target application under test to obtain its response data. Then, the response data will be input into the downstream application of the target application under test to obtain its historical response data. The historical request data of the target application and the historical response data of the downstream application will be stored together as a sample to be mutated in HDFS.

[0054] Upon receiving a generation request, the mutation engine extracts the target application to be mutated from HDFS and performs mutations to obtain test samples. These test samples are then sent to the executor to test for defects in the target application in the test environment.

[0055] In this embodiment, historical request data of the target application under test is input into the downstream application to obtain historical response data of the downstream application. This historical response data of the downstream application greatly enriches the dimension and diversity of the data structure of the sample to be mutated, which helps to comprehensively detect the defects of the target application under test, without affecting the normal operation of the target application under test.

[0056] Since this environment is a simulation environment, no matter what problems or failures occur in the target application under test in this environment, it will not affect the normal operation of the target application under test outside the simulation environment, which can ensure the stability of relevant business scenarios.

[0057] To detect defects in the target application under test, a large number of test samples need to be generated from a large number of samples to be mutated. Therefore, the entire test relies on the processing of massive amounts of data. This raises the question of how to achieve fast access and fast input / output (I / O) of this massive amount of data. To address this issue, in some embodiments, efficient structured data formats can be used to store the data, such as the RecordIO file format described above, to store the samples to be mutated from the target application under test. Optionally, multi-process access can be used to access the data, which can greatly improve access efficiency.

[0058] In some embodiments, to accelerate storage speed and fully utilize storage space, the sample to be mutated is pre-stored in the form of structured data with a corresponding data format. In this case, when a generation request is received to obtain the sample to be mutated from the target application under test, the stored structured data of the target application under test can be obtained in response to the generation request; then, the structured data is parsed according to its data format to obtain the sample to be mutated. For example, the sample to be mutated can be pre-stored in a comma-separated value file format (CSV, sometimes also called character-separated values), for example, it can be stored in a database. Therefore, after receiving a generation request, the structured data in the CSV file format can be obtained from, for example, a database, and the structured data can be parsed according to the CSV file format to obtain the sample to be mutated from the target application under test. Alternatively, when storing the structured data, the data format can be pre-recorded locally, for example, stored together with the storage address of the sample to be mutated from the target application under test. It is worth noting that this application does not limit the data format of the structured data.

[0059] In some embodiments, the data format corresponding to the structured data may include a compressed format and a file format. In this case, when parsing the structured data according to its data format, the structured data can be decompressed according to its compressed format to obtain at least one data file; then, according to the file format, at least one data file can be parsed to obtain the sample to be mutated. In some storage methods, there is a nested relationship between data formats. For example, for a file in the RecordIO file format, its storage unit is a record, and the user can define the record as needed. For example, a record can be defined as a byte stream encoded by protobuffer, in which case a record can be composed of a proto file (.proto). This data access method can realize fast data access and input / output, greatly shortening the time for generating test samples. Optionally, this embodiment can be implemented in a distributed system including multiple computing devices (such as... Figure 2 The server 202 can provide multi-threaded concurrent access to massive amounts of data, thereby greatly shortening the testing time.

[0060] To better explain this embodiment, the data access method provided by this embodiment will be described below in conjunction with a specific implementation scenario.

[0061] In the HDFS distributed file system, 10 users input their personal information into a statistics application. This application, used to collect user information, is the target test application. The personal information of these 10 users is saved as historical request data for the statistics application in .proto file format, forming 10 .proto files. These 10 files are then saved to a single RecordIO file and immediately stored in HDFS as streaming data, meaning they flow into HDFS in a short period of time. One example of a .proto file is shown below: message student{ required int32 age=10; optional string name="wang"; }

[0062] When the server receives a request to generate test samples for the statistical application, it uses the identifier in the request as an index to retrieve the storage address and data format of the samples to be mutated from the local configuration file. Then, it retrieves the RecordIO file for the statistical application from HDFS and parses it into 10 .proto files according to the RecordIO file format parsing method. Finally, it parses these 10 .proto files according to the protobuffer protocol rules to obtain the samples to be mutated for the statistical application. For example, the .proto file parsed above might yield a message body and related data for two fields, age and name; this data constitutes the samples to be mutated.

[0063] In some embodiments, the test sample generation strategy includes information indicating random mutation and a preset degree of mutation, where the degree of mutation represents the extent to which the sample to be mutated is randomly mutated; and wherein generating a test sample according to the test sample generation strategy and the sample to be mutated includes: randomly mutating the sample to be mutated according to the preset degree of mutation to obtain a test sample. The information indicating random mutation indicates that the sample to be mutated is randomly mutated. Random mutation means randomly selecting one mutation method from all mutation methods to mutate the sample to be mutated. Common mutation methods include adding, deleting, and modifying the sample to be mutated, which reflect simulations of different extreme scenarios, as shown in Table 1: Table 1 operate Variation Override fields Scene Description Modify modify_rate(0%-10%) All basic field types The data in the simulated field was incorrectly modified. Delete delete_rate(0%-10%) All basic field types Data was missing from the simulated field. Add add_rate(0%-10%) All basic field types Simulation protocol content upgrade .

[0064] The aforementioned random mutations can be used to simulate corresponding extreme scenarios and detect the robustness of the target application under test.

[0065] The degree of variability allows testers to flexibly control the extent of random mutation. A smaller degree of variability reduces the computational load and shortens the mutation time. For example, when performing random mutation on the byte stream 0110101101, if the degree of variability is preset to 20%, then mutating will be performed on 2 random bits out of these 10 bits, such as the first and last two bits, resulting in the randomly mutated byte stream 1110101100.

[0066] In some embodiments, the degree of mutability can also be represented by the probability of mutating the sample to be mutated. During the generation of test samples, random mutation can be performed on each field individually (called exhaustive random mutation), or random mutation can be performed on a subset of fields. If this subset of fields includes all field types in the sample to be mutated, then the testing effect of random mutation on a subset of fields is almost the same as that of exhaustive random mutation, and the testing time is significantly reduced due to the significantly reduced computational load. Implementing random mutation on a subset of fields involves mutating the sample to be mutated based on the degree of mutability. In an optional embodiment, the degree of mutability can be set to 5%. Practice shows that setting the degree of mutability to 5% can achieve 100% field type coverage. Therefore, in a test process that only needs to consider field type coverage, there is no need to perform exhaustive random mutation on all fields; only 5% of the fields need to be randomly mutated, which can significantly shorten the testing time and improve testing efficiency.

[0067] In some embodiments, random mutation of the sample to be mutated according to a preset degree of variability includes: determining at least one field of the sample to be mutated, wherein each field in the at least one field has a corresponding field type, and each field type has a corresponding random mutation method; for each corresponding field in the at least one field, according to the preset degree of variability and the random mutation method corresponding to the field type of the corresponding field, randomly mutating the data at the corresponding field to obtain a test sample. The correspondence between field type and random mutation method can be, for example, that the value of an integer type object can only be modified to an integer, and the value of a character type object can only be modified to a character; the value of an integer type object cannot be modified to a character.

[0068] Various programming languages ​​involve different fields, each represented by a specific field name indicating its type. Based on historical testing experience, different specific test samples exist for different field types. These specific test samples are highly likely to detect defects in the target application under test. For example, null is a common test sample for various fields. For each field type, modifying the value of an object within that field to null can be used as a test sample. Therefore, randomly modifying the data in each field to generate specific test samples can quickly and efficiently detect defects in the target application under test.

[0069] In some embodiments, the data in each field can be randomly mutated according to Table 2 to obtain test samples.

[0070] Table 2 Field type Test samples string "", null, garbled characters, long strings float MAX_VALUE, MIN_VALUE, 1, 0, -1, null long MAX_VALUE, MIN_VALUE, 1, 0, -1, null int MAX_VALUE, MIN_VALUE, 1, 0, -1, null double MAX_VALUE, MIN_VALUE, 1, 0, -1, null boolean true, false, null Enum Exhaustive search, null byte byte random, null .

[0071] In some embodiments, random mutation of the sample to be mutated is performed according to a preset degree of mutability, including: randomly mutating the binary code representing the sample to be mutated according to the preset degree of mutability to obtain a test sample. In this embodiment, non-perceptual random mutation is performed on the sample to be mutated. The sample to be mutated exists as a message body during transmission. This message body is a byte stream encoded according to local serialization encoding rules, such as the byte stream "...011010...". These byte streams embody the various data structures of the sample to be mutated, such as structures and fields. In this embodiment, the object of random mutation is the byte stream "...011010...", for example, mutated to "...100010...", rather than the data of the fields in the sample to be mutated. This non-perceptual mutation eliminates the parsing process, has a faster mutation execution speed, and is not limited by any encoding rules. It can randomly mutate the data at any field, thereby making it more likely to generate diverse test samples, which helps to detect defects in the target application under test.

[0072] In the case of exhaustive random mutation, assuming there are 3 groups of samples to be mutated, each group has 4000 files, each file has 5000 fields, the target application under test has 10 downstream applications, each field has an average of 5 possible values, and there are 3 mutation dimensions: addition, deletion, and modification, then the mutation amount is 3*4000*5000*10*3*5=9 billion, and the mutation time is 9 billion / 100 (qps) / 3600 (s) = 25000h. The unit QPS is queries per second, which is a measure of the amount of traffic processed by a specific query server within a specified time.

[0073] In the case of non-perceptual mutation, since random mutation is performed directly on the file, the mutation amount is 3*4000=12000, and the mutation time is 12000 / 100 (qps) / 60 (s) = 2 min. Therefore, non-perceptual mutation has a faster mutation speed and can effectively shorten the test time.

[0074] In some embodiments, the test sample generation strategy includes information indicating precise mutation and a corpus on which precise mutation depends; and wherein generating a test sample according to the test sample generation strategy and the sample to be mutated includes: determining at least one field of the sample to be mutated, wherein each field in the at least one field has a corresponding field type; for each corresponding field in the at least one field, selecting corpus data with the same field type as the corresponding field from the corpus to replace the data in the corresponding field to obtain the test sample. In long-term testing operations, valuable corpus data can be extracted and accumulated from each testing operation. This valuable corpus data has broad applicability and efficient testing capabilities, and can quickly detect defects in the target application under test in subsequent testing operations. The location where this valuable corpus data is stored is called a corpus, and each piece of valuable corpus data is called a corpus data. When the test sample generation strategy indicates precise mutation, precise mutation can be implemented based on the content of the corpus. By replacing the data at the fields of the sample to be mutated in the corpus data, the corpus can be utilized, where the field type acts as a mapping relationship.

[0075] In some embodiments, for each corresponding field in at least one field, all corpus data with the same field type as the corresponding field are selected from the corpus and the data of the corresponding field is replaced respectively, thereby generating multiple corresponding fields. The data of each corresponding field is one of the corpus data mentioned above. For example, if there are two integer data entries in the corpus, and the sample to be mutated has an integer field, then for the integer field in the sample to be mutated, the data of the two corpus data entries mentioned above are selected from the corpus and replaced, resulting in two integer fields. The data of these two integer fields are one of the two corpus data entries mentioned above.

[0076] In some embodiments, the test sample generation strategy includes information indicating precise mutation, the data structure type targeted by the precise mutation, and at least one mutant; and wherein generating test samples according to the test sample generation strategy and the sample to be mutated includes: deconstructing the sample to be mutated according to the data structure type to obtain a deconstructed sample to be mutated; and using at least one mutant to mutate the deconstructed sample to be mutated to obtain the test sample. The sample to be mutated is written in a certain programming language. Different programming languages ​​have different types of data structures, so the sample to be mutated can be deconstructed according to the data structure to achieve the purpose of "decomposing" the sample to be mutated. This allows the sample to be mutated from a file structure into a finer-grained data structure, such as structs and fields. Testers can determine the desired data structure based on the characteristics of the target application being tested, and uniformly apply mutants to the desired data structure.

[0077] In this embodiment, the types of mutants include: field duplication mutants, oversized string mutants, field addition mutants, field deletion mutants, field swap mutants, fixed value mutants, structure disruption mutants, and field order reversal mutants, among others. Oversized string mutants can replace the string field data of the sample to be mutated with an excessively long string; field duplication mutants can repeat the data of a field in the sample to be mutated several times and assign it to that field; field addition mutants can add a field to the sample to be mutated; field deletion mutants can modify the data of a field to null; field swap mutants can swap the data between different fields; fixed value mutants can replace the data of a field in the sample to be mutated with a fixed value; structure disruption mutants can assign data to a field that does not conform to the field type; and field order reversal mutants can reverse the order of different fields, for example, to reverse the order.

[0078] In some embodiments, each of the at least one mutant has a preset priority. In this case, when using the at least one mutant to mutate the deconstructed sample to be mutated to obtain the test sample, the at least one mutant can be used sequentially according to its priority to mutate the deconstructed sample to be mutated to obtain the test sample. The mutant priority indicates the order in which the mutants are used; the mutant with the highest priority will be used first, and the mutant with the lowest priority will be used last. The order in which the mutants are implemented may lead to different mutation results. For example, when a field repetition mutant and a fixed value mutant are selected, if the field repetition mutant is applied first and then the fixed value mutant, the field value will be a fixed value; if the fixed value mutant is applied first and then the field repetition mutant, the field value will be two fixed values. Therefore, assigning different priorities to the mutants helps to generate the desired test data.

[0079] Before performing precise mutation, to specifically detect defects in the updated code (i.e., incremental code) of the target application under test, a static analysis can be performed on the target application under test's code. This involves analyzing the syntax tree to determine the incremental code of the target application under test and identifying incremental fields from the incremental code. To detect defects in the incremental code, a test sample generation strategy should be configured for the incremental fields, such that the generated test samples include test samples whose fields have the same field type as the fields in the incremental fields. Therefore, in embodiments where the test sample generation strategy includes information indicating precise mutation, the incremental code and incremental fields of the target application under test can be automatically analyzed before implementing the test sample generation strategy.

[0080] Figure 5A schematic diagram illustrating the process of analyzing the incremental fields of the target application being tested according to some embodiments of this application is shown. Figure 5 As shown, in this embodiment, the developers first perform an update operation on the target application under test, such as uploading the local code version to a remote server for merging via a remote update operation. This remote server hosts the officially running target application under test, resulting in the updated code. Subsequently, the remote server side (e.g., Figure 2 Server 202 receives the updated code and performs static code analysis, analyzing the syntax tree of the updated code to obtain incremental code, and further determining incremental fields from the incremental code. This step can be performed, for example, in a... Figure 4 The mutation engine shown performs the mutation steps before proceeding. The remote server can then implement precise mutation methods described in some of the embodiments above for these incremental fields, generating test samples. Finally, these test samples are sent to the fuzz test executor to perform tests and detect defects in the target application being tested.

[0081] This application discloses several embodiments of methods for generating test samples. The features, functions, structures, and steps of these embodiments can be combined with each other, and this application does not limit them. Figures 6-8 Examples of methods for generating test samples by combining some of the embodiments described above are described respectively.

[0082] Figure 6 A functional schematic diagram of a mutation engine for generating test samples according to some embodiments of this application is shown. For example... Figure 6 As shown, the mutation engine 600 has two functional layers: an interaction layer 610 and a computation layer 620. The mutation engine, especially the computation layer 620, can be implemented in the Spark computing framework, for example.

[0083] The function of the interaction layer 610 is to provide the interaction information and interaction sequence required in the preparation operation of the sample to be mutated and the mutation operation of the sample to be mutated. Specific functions include one or more of the following: Spark task scheduling 611, data source configuration management 612, mutation result statistics 613, data structure analysis 614, and test sample display 615.

[0084] Spark Task Scheduler 611's function is, for example, to use an asynchronous message queue to receive test sample generation requests via an HTTP mutation interface, and to be responsible for the management and scheduling of mutation tasks.

[0085] The function of data source configuration management 612 is to manage the storage address of the samples to be mutated for each target application under test, and the data format when storing structured data. The samples to be mutated include historical request data for the target application under test and historical response data of downstream applications responding to the response data of the target application under test. The response data of the target application under test is the response data of the target application under test to the historical request data.

[0086] The function of Mutation Result Statistics 613 is to provide information such as the execution information of the mutation task, the number of mutated fields, the number and degree of mutation samples, the number of mutation log entries, and the differences between the samples before and after mutation.

[0087] The function of Data Structure Analysis 614 is to analyze the data structure of the sample to be mutated and deconstruct the sample to be mutated into a preset data structure.

[0088] The function of test sample display 615 is to process the generated test samples, each of which has a data structure. Processing the generated test samples may include selecting one or more test samples from each data structure as display data for display, and optionally, transmitting the display data to a terminal device for display.

[0089] The computation layer 620 is responsible for acquiring the sample to be mutated and generating test samples based on the test sample generation strategy and the sample to be mutated. Specific functions include data access 621, data parsing 622, random mutation 623, and precise mutation 624.

[0090] The data access 621 function, for example, decompresses the structured data according to the compression format in the structured data format to obtain at least one data file. The compression formats it can decompress include RecordIO, PowerBuilder (a development tool developed by Sybase in the United States), CSV, etc.

[0091] Data Parser 622's function is to parse at least one data file according to the file format in the structured data to obtain the sample to be mutated. The file formats it can parse include protobuffer, flatbuffer (a cross-platform serialization library provided by Google), json (JS object abbreviation), etc.

[0092] The function of random mutation 623 is to perform random mutations on the sample to be mutated. It can include three sub-functions: mutation degree control 6231, non-perceptual mutation 6232, and mutation operation statistics 6233. The function of mutation degree control 6231 is to control the degree of random mutation of the sample to be mutated. The function of non-perceptual mutation 6232 is to directly mutate the byte stream that exists in the form of a message body. The function of mutation operation statistics 6233 is to count the mutation operations in random mutation and send them to the mutation result statistics 613 in the interaction layer 610.

[0093] The function of Precise Mutation 624 is to perform precise mutation on the sample to be mutated. It can include six sub-functions: Corpus Loading 6241, Fission Mode 6242, Random Mode 6243, Mutator Loading 6244, Mutator Scheduling 6245, and Mutation Operation Statistics 6246. Corpus Loading 6241 loads the corpus and then calls the corpus data. Fission Mode 6242, under the condition of loading the corpus, selects all corpus data with the same field type as the corresponding field in each corresponding field of at least one field of the sample to be mutated, and replaces the data in the corresponding field, thereby generating multiple corresponding fields. The data in each corresponding field is one corpus data from all the aforementioned corpus data. Random Mode 6243, under the condition of loading the corpus, selects one corpus data with the same field type as the corresponding field in each corresponding field of at least one field of the sample to be mutated, and replaces the data in that corresponding field to obtain the test sample. The function of the mutant loading 6244 is to load a mutant and call the loaded mutant to perform precise mutation on the sample to be mutated. The function of the mutant scheduling 6245 is, for example, to use at least one mutant in order of priority to mutate the deconstructed sample to be mutated to obtain the test sample. The function of the mutation operation statistics 6246 is to statistically analyze the mutation operations in the precise mutation and transmit the results to the mutation result statistics 613 in the interaction layer 610.

[0094] Mutagen 630 refers to the mutants that can be loaded by the mutant loading sub-function 6244 in the precise mutation 624 of the computation layer 620. These include basic field outlier mutants, field duplication mutants, threshold mutants, boundary fluctuation mutants, field addition mutants, field swap mutants, field order reversal mutants, and field deletion mutants. These mutants 630 can sequentially mutate the deconstructed sample to be mutated according to the mutant scheduling 6245.

[0095] If conventional fuzzing tools are used (which typically employ pre-defined strategies to generate test samples), mutating 500,000 samples to be mutated can take up to 11 hours, while... Figure 6 The functional framework shown can reduce the time to mutate 500,000 samples to be mutated to 10 minutes, significantly improving computational efficiency. Figure 6 The mutation engine 600 described in the illustrated embodiment has a high return on investment, and the time required to implement this embodiment is much less than the time saved in the process of generating test samples.

[0096] Figure 7 A schematic schematic diagram of the principle architecture for generating test samples according to some embodiments of this application is shown. Figure 6 The mutant engine 600 shown can be implemented using this architecture 700. For example... Figure 7 As shown, it includes three main parts: scheduling layer 710, data calculation layer 720, and mutation display layer 730.

[0097] The scheduling layer 710 can receive test sample generation requests, which include generating a test sample generation strategy for a target application under test, which has downstream applications. If there are multiple test sample generation requests, the scheduling layer 710 can store these multiple test sample generation requests in the form of a queue, for example, in local memory (DB). Furthermore, the scheduling layer 710 can update the test sample generation strategy, for example, by updating the test sample generation request corresponding to the strategy.

[0098] When executing one of the test sample generation requests, the data computation layer 720 can create a computation task based on the task configuration file. This can be achieved by... Figure 6 This is implemented using the Spark task scheduling function 611 in the interaction layer 610. The task configuration file includes the test sample generation strategy and basic mutation information. The basic mutation information includes the address where the test samples are stored. Additionally, based on the data configuration file, the mutation samples to be mutated for the target application can be obtained. These mutation samples include historical request data for the target application and historical response data from downstream applications responding to the target application's response data. The target application's response data is the response data from the historical request data. The data configuration file includes data format information and the storage address of the mutation samples for each target application. The functionality of the data configuration file can be achieved by... Figure 6 The data source configuration management 612 shown is used to implement this. Furthermore, during the execution of one of the test sample generation requests, the address for storing the test sample can be updated. For example, an update request containing the address for storing the test sample can be written in the scheduling layer 710 and sent to the task configuration file in the data computation layer 720 to complete the update. This update request can be written according to the HTTP protocol.

[0099] When executing one of the test sample generation requests, the data computation layer 720 can retrieve the target application's mutated sample from the storage 740 according to the storage address of the mutated sample of the target application recorded in the data source configuration management 612. This mutated sample includes historical request data for the target application and historical response data from downstream applications responding to the target application's response data. The target application's response data is the response data of the target application to the historical request data. The storage 740 can be a distributed file system that can store batch data, streaming data, and structured data in various formats, including but not limited to RecordIO, Power Builder, CSV, flatbuffer, and JSON data formats.

[0100] When the sample to be mutated is stored in the form of structured data, it can be derived from... Figure 6 The data access function 621 in the computation layer 620 decompresses the structural data according to the compression format of the structural data to obtain at least one data file. Then, the data parsing function 622 of the computation layer 620 parses the at least one data file according to the file format to obtain the sample to be mutated.

[0101] After obtaining the sample to be mutated, random or precise mutation is performed in the data computation layer 720 according to the definition of the test sample generation strategy. This can be achieved... Figure 6 This is achieved through the random mutation 623 or precise mutation 624 shown. The mutation display layer 730 can display mutation statistics. This can be achieved by... Figure 6 The variation results statistics shown in 613 and the test sample display in 615 are achieved together.

[0102] Figure 8An exemplary structural block diagram of an apparatus 800 for generating test samples according to some embodiments of this application is shown. The apparatus 800 includes a scheduling module 801, an acquisition module 802, and a processing module 803. The scheduling module 801 is configured to receive a generation request for generating test samples, the generation request including a test sample generation strategy for a target application under test, the target application under test having downstream applications. The acquisition module 802 is configured to, in response to receiving the generation request, acquire a sample to be mutated for the target application under test, the sample to be mutated including historical request data for the target application under test and historical response data from the downstream applications responding to response data for the target application under test, the response data being the response data of the target application under test to the historical request data. The processing module 803 is configured to generate test samples based on the test sample generation strategy and the sample to be mutated, the test samples being used to test the target application under test.

[0103] It should be noted that the various modules described above can be implemented in software, hardware, or a combination of both. Multiple different modules can be implemented within the same software or hardware architecture, or a single module can be implemented by multiple different software or hardware architectures. As an example, the scheduling module 801 can be implemented in... Figure 7 In the scheduling layer 710 of the architecture shown, the acquisition module 802 and the processing module 803 can be implemented in Figure 7 The data computation layer 720 of the architecture shown.

[0104] In the apparatus for generating test samples claimed in this application, by utilizing historical request data for the target application under test and historical response data from downstream applications responding to the target application under test, the potential mutation samples of the target application under test can be comprehensively mined. This greatly enriches the dimensionality and diversity of the data structure of the potential mutation samples, facilitating the comprehensive detection of defects in the target application under test. Furthermore, the test sample generation strategy can be flexibly customized according to the characteristics of the target application under test, and then the potential mutation data is mutated according to the customized test sample generation strategy to generate test samples. This overcomes the limitations of fixed strategies, and the generated test samples conform to the characteristics of the target application under test, thereby improving testing efficiency and enabling the rapid discovery of defects in the target application under test. Moreover, because the mutation strategy can be flexibly configured, the needs for testing different target applications under test are met.

[0105] This application provides a system for generating test samples, including: a memory configured to store a sample to be mutated of a target application under test in the form of structured data, the structured data having a corresponding data format; and the apparatus for generating test samples disclosed in this application; wherein, an acquisition module is further configured to acquire the sample to be mutated of the target application under test from the memory.

[0106] Figure 9 An example system 900 is illustrated, which includes an example computing device 910 representing one or more systems and / or devices that can implement the various methods described herein. The computing device 910 can be, for example, a server of a service provider, a device associated with a server, a system-on-a-chip, and / or any other suitable computing device or computing system. (Refer to above) Figure 8 The apparatus 800 for generating test samples described herein may take the form of a computing device 910. Alternatively, the apparatus 800 for generating test samples may be implemented as a computer program in the form of a software application 916.

[0107] The example computing device 910 shown includes a processing system 911 communicatively coupled to each other, one or more computer-readable media 912, and one or more I / O interfaces 913. Although not shown, the computing device 910 may also include a system bus or other data and command transfer system that couples the various components to each other. The system bus may include any or a combination of different bus architectures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus utilizing any of the various bus architectures. Various other examples, such as control and data lines, are also conceived.

[0108] Processing system 911 represents the functionality of performing one or more operations using hardware. Therefore, processing system 911 is illustrated as including hardware elements 914 that can be configured as processors, function blocks, etc. This may include other logic devices implemented in hardware as application-specific integrated circuits (ASICs) or formed using one or more semiconductors. Hardware element 914 is not limited by the materials in which it is formed or the processing mechanism employed therein. For example, a processor may consist of semiconductors and / or transistors (e.g., integrated circuits (ICs)). In such a context, processor-executable instructions may be electronically executable instructions.

[0109] Computer-readable medium 912 is illustrated as including memory / storage device 915. Memory / storage device 915 represents a memory / storage capacity associated with one or more computer-readable media. Memory / storage device 915 may include volatile media (such as random access memory (RAM)) and / or non-volatile media (such as read-only memory (ROM), flash memory, optical disk, magnetic disk, etc.). Memory / storage device 915 may include fixed media (e.g., RAM, ROM, fixed hard disk drive, etc.) and removable media (e.g., flash memory, removable hard disk drive, optical disk, etc.). Computer-readable medium 912 may be configured in various other ways as further described below.

[0110] One or more I / O interfaces 913 represent the functionality to allow users to input commands and information to the computing device 910 using various input devices and optionally also to present information to the user and / or other components or devices using various output devices. Examples of input devices include keyboards, cursor control devices (e.g., mice), microphones (e.g., for voice input), scanners, touch functionality (e.g., capacitive or other sensors configured to detect physical touch), cameras (e.g., capable of detecting non-touch-related motion as gestures using visible or invisible wavelengths (such as infrared frequencies), etc. Examples of output devices include display devices (e.g., monitors or projectors), speakers, printers, network interface cards, haptic-responsive devices, etc. Therefore, the computing device 910 can be configured to support user interaction in various ways as further described below.

[0111] The computing device 910 also includes a software application 916. The software application 916 may be, for example, a software instance of the means 800 for generating test samples, and may implement the techniques described herein in combination with other elements in the computing device 910.

[0112] This application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computing device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computing device to perform the method for generating test samples provided in the various optional implementations described above.

[0113] This document describes various technologies within the general context of software and hardware components or program modules. Generally, these modules include routines, programs, objects, elements, components, data structures, etc., that perform specific tasks or implement specific abstract data types. As used herein, the terms "module," "function," and "component" generally refer to software, firmware, hardware, or a combination thereof. The technologies described herein are characterized as platform-independent, meaning that these technologies can be implemented on a variety of computing platforms with various processors.

[0114] Implementations of the described modules and technologies may be stored on or transmitted across some form of computer-readable medium. The computer-readable medium may include a variety of media accessible by the computing device 910. By way of example and not limitation, the computer-readable medium may include "computer-readable storage media" and "computer-readable signal media".

[0115] In contrast to simple signal transmission, carrier waves, or signals themselves, a "computer-readable storage medium" refers to a medium and / or device capable of persistently storing information, and / or a tangible storage device. Therefore, a computer-readable storage medium refers to a non-signal-bearing medium. Computer-readable storage media include hardware such as volatile and non-volatile, removable and non-removable media and / or storage devices implemented using methods or techniques suitable for storing information (such as computer-readable instructions, data structures, program modules, logic elements / circuits, or other data). Examples of computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage devices, hard disks, cassette tapes, magnetic tapes, disk storage devices or other magnetic storage devices, or other storage devices, tangible media, or articles of art suitable for storing desired information and accessible by a computer.

[0116] "Computer-readable signal medium" refers to a signal-bearing medium configured to transmit instructions, such as via a network, to hardware of computing device 910. A signal medium typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, data signal, or other transmission mechanism. Signal media also include any information transmission medium. The term "modulated data signal" refers to a signal in which one or more of its characteristics are set or altered to encode information. By way of example and not limitation, communication media include wired media such as wired networks or direct connections, and wireless media such as acoustic, RF, infrared, and other wireless media.

[0117] As previously stated, hardware element 914 and computer-readable medium 912 represent instructions, modules, programmable device logic, and / or fixed device logic implemented in hardware, which in some embodiments may be used to implement at least some aspects of the techniques described herein. Hardware elements may include components of integrated circuits or systems-on-a-chip, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and other implementations or other hardware devices in silicon. In this context, hardware elements may serve as processing devices for executing program tasks defined by instructions, modules, and / or logic embodied by the hardware element, and as hardware devices for storing instructions for execution, such as the previously described computer-readable storage medium.

[0118] The foregoing combinations can also be used to implement the various techniques and modules described herein. Therefore, software, hardware, or program modules and other program modules can be implemented as one or more instructions and / or logic embodied on some form of computer-readable storage medium and / or by one or more hardware elements 914. The computing device 910 can be configured to implement specific instructions and / or functions corresponding to the software and / or hardware modules. Thus, for example, by using the computer-readable storage medium and / or hardware elements 914 of a processing system, modules can be implemented at least partially in hardware as modules executable as software by the computing device 910. Instructions and / or functions can be executable / operable by one or more articles of art (e.g., one or more computing devices 910 and / or processing systems 911) to implement the techniques, modules, and examples described herein.

[0119] In various embodiments, the computing device 910 can be configured in various ways. For example, the computing device 910 can be implemented as a computer-type device, including personal computers, desktop computers, multi-screen computers, laptop computers, netbooks, etc. The computing device 910 can also be implemented as a mobile device, including mobile devices such as mobile phones, portable music players, portable gaming devices, tablet computers, multi-screen computers, etc. The computing device 910 can also be implemented as a television-type device, including devices with or connected to a generally large screen in a leisure viewing environment. These devices include televisions, set-top boxes, game consoles, etc.

[0120] The techniques described herein can be supported by these various configurations of computing device 910, and are not limited to specific examples of the techniques described herein. Functionality can also be implemented, wholly or partially, on the “cloud” 920 using distributed systems, such as through platform 922 as described below.

[0121] Cloud 920 includes and / or represents platform 922 for resource 924. Platform 922 abstracts the underlying functionality of the hardware (e.g., server) and software resources of cloud 920. Resource 924 may include applications and / or data that can be used when performing computer processing on a server located remotely from computing device 910. Resource 924 may also include services provided via the Internet and / or via subscriber networks such as cellular or Wi-Fi networks.

[0122] Platform 922 can abstract resources and functions to connect computing device 910 to other computing devices. Platform 922 can also be used to abstract resource hierarchy to provide a corresponding level of hierarchy for any encountered needs for resource 924 implemented via platform 922. Therefore, in interconnect device embodiments, the implementation of the functions described herein can be distributed throughout system 900. For example, functions can be implemented partly on computing device 910 and partly through platform 922, which abstracts the functions of cloud 920.

[0123] It should be understood that, for clarity, embodiments of this application have been described with reference to different functional units. However, it will be apparent that, without departing from this application, the functionality of each functional unit may be implemented in a single unit, in multiple units, or as part of other functional units. For example, functionality described as being performed by a single unit may be performed by multiple different units. Therefore, references to specific functional units are considered merely as references to the appropriate units used to provide the described functionality, and not as indicating a strict logical or physical structure or organization. Thus, this application may be implemented in a single unit, or may be physically and functionally distributed among different units and circuits.

[0124] Although this application has been described in conjunction with some embodiments, it is not intended to be limited to the specific forms set forth herein. Rather, the scope of this application is limited only by the appended claims. Additionally, although individual features may be included in different claims, these may be advantageously combined, and inclusion in different claims does not imply that such a combination of features is not feasible and / or advantageous. The order of features in the claims does not imply that the features must be in any particular order of their operation. Furthermore, in the claims, the word "comprising" does not exclude other elements, and the terms "a" or "an" do not exclude a plurality. Reference numerals in the claims are provided only as explicit examples and should not be construed as limiting the scope of the claims in any way.

[0125] It is understood that the specific embodiments of this application involve data such as user information, information related to the user's historical ad targeting behavior, and audience profiles. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of the samples to be mutated must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

Claims

1. A method for generating test samples, comprising: Receive a request to generate test samples, the request including a strategy for generating test samples for a target application under test, the target application under test having downstream applications, the downstream applications of the target application under test being applications that receive data from the target application under test, the test sample generation strategy including a generation strategy defined according to the characteristics of the target application under test and as needed. In response to receiving the generation request, a mutation sample of the target application under test is obtained. The mutation sample includes historical request data for the target application under test and historical response data of the downstream application responding to the response data of the target application under test. The response data of the target application under test is the response data of the target application under test to the historical request data. The historical response data is obtained by inputting the response data of the target application under test into the downstream application of the target application under test. The mutation sample is the mutation object targeted by the test sample generation strategy. When the test sample generation strategy indicates to perform precise mutation, precise mutation is achieved based on the content of the corpus. Test samples are generated based on the test sample generation strategy and the sample to be mutated, and the test samples are used to test the target application under test.

2. The method according to claim 1, wherein, The sample to be mutated is stored in the form of structural data, and the structural data has a corresponding data format; as well as The step of obtaining the mutation sample of the target application under test in response to receiving the generation request includes: In response to receiving the generation request, the stored structure data of the target application under test is obtained; The structural data is parsed according to the data format of the structural data to obtain the sample to be mutated.

3. The method according to claim 2, wherein, The data format corresponding to the structural data includes the compression format and file format of the structural data; as well as The process of parsing the structural data according to its data format to obtain the sample to be mutated includes: According to the compression format of the structure data, the structure data is decompressed to obtain at least one data file; According to the file format, the at least one data file is parsed to obtain the sample to be mutated.

4. The method according to claim 1, wherein, The test sample generation strategy includes information indicating random mutation and a preset degree of mutation, wherein the degree of mutation represents the extent to which the sample to be mutated is randomly mutated. as well as The step of generating test samples based on the test sample generation strategy and the sample to be mutated includes: The sample to be mutated is randomly mutated according to a preset variability to obtain the test sample.

5. The method according to claim 4, wherein randomly mutating the sample to be mutated according to a preset degree of variability includes: Determine at least one field of the sample to be mutated, wherein each of the at least one fields has a corresponding field type, and each field type has a corresponding random mutation method; For each corresponding field in the at least one field, the data at the corresponding field is randomly mutated according to the preset variability and the random mutation method corresponding to the field type of the corresponding field to obtain the test sample.

6. The method according to claim 4, wherein, The step of randomly mutating the sample to be mutated according to a preset degree of variability includes: The binary code representing the sample to be mutated is randomly mutated according to the preset variability to obtain the test sample.

7. The method according to claim 1, wherein, The test sample generation strategy includes information indicating precise mutations and the corpus on which precise mutations depend. as well as The step of generating test samples based on the test sample generation strategy and the sample to be mutated includes: Determine at least one field of the sample to be mutated, wherein each of the at least one field has a corresponding field type; For each corresponding field in the at least one field, select corpus data with the same field type as the corresponding field from the corpus to replace the data in the corresponding field, so as to obtain the test sample.

8. The method according to claim 1, wherein, The test sample generation strategy includes information indicating precise mutation, the data structure type targeted by precise mutation, and at least one mutant. as well as The step of generating test samples based on the test sample generation strategy and the sample to be mutated includes: The sample to be mutated is deconstructed according to the data structure type to obtain the deconstructed sample to be mutated. The at least one mutant is used to mutate the deconstructed sample to be mutated to obtain the test sample.

9. The method according to claim 8, wherein, Each of the at least one mutant has a preset priority; as well as The step of using the at least one mutant to mutate the deconstructed sample to be mutated to obtain the test sample includes: The at least one mutant is used sequentially according to its priority to mutate the deconstructed sample to be mutated, so as to obtain the test sample.

10. An apparatus for generating test samples, comprising: The scheduling module is configured to receive a request to generate test samples. The request includes a strategy for generating test samples for a target application under test. The target application under test has downstream applications, which are applications that receive data from the target application under test. The test sample generation strategy includes a generation strategy defined according to the characteristics of the target application under test and as needed. The acquisition module is configured to acquire, in response to receiving the generation request, a mutation sample of the target application under test. The mutation sample includes historical request data for the target application under test and historical response data of the downstream application responding to the response data of the target application under test. The response data of the target application under test is the response data of the target application under test to the historical request data. The historical response data is obtained by inputting the response data of the target application under test into the downstream application of the target application under test. The mutation sample is the mutation object targeted by the test sample generation strategy. When the test sample generation strategy indicates precise mutation, precise mutation is achieved based on the content of the corpus. The processing module is configured to generate test samples based on the test sample generation strategy and the sample to be mutated, the test samples being used to test the target application under test.

11. A system for generating test samples, comprising: The memory is configured to store the target test application's mutation samples in the form of structured data, the structured data having a corresponding data format; The apparatus for generating test samples according to claim 10; The acquisition module is further configured to acquire the mutation sample of the target application to be tested from the memory.

12. A computing device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1-9.

13. A computer-readable storage medium storing a computer program thereon, which, when executed by a processor, causes the processor to perform the steps of the method according to any one of claims 1-9.

14. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 9.

Images