Method, device and computer program product for updating a test case library
Patent Information
- Application Number
- CN202510571272.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-30
- Publication Date
- 2026-10-30
Smart Images
Figure CN123011680A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of software testing, and more specifically to methods, apparatus, and computer program products for updating a library of test cases. Background Technology
[0002] A test case library is a database or collection of tools used to store, manage, and maintain test cases. It is a core asset in the software testing process, helping testing teams execute tests efficiently, reuse test cases, and track test results. By increasing the diversity of test cases, potential defects can be discovered more comprehensively, testing blind spots can be reduced, and software testing coverage and efficiency can be ensured.
[0003] Fuzzing involves running a program with a seed input, mutating the seed input to improve branch coverage, and iteratively executing the input mutation and target program execution process. This technique can be applied to real-world scenarios such as web browsers and databases. Summary of the Invention
[0004] In a first aspect of the embodiments of this disclosure, a method for updating a test case library is provided. The method includes obtaining candidate test cases; extracting feature representations of the candidate test cases using a language model; determining the similarity between the feature representations of the candidate test cases and feature representations in a corresponding feature representation library of the test case library; and adding the candidate test cases to the test case library in response to the similarity meeting a predetermined condition.
[0005] In a second aspect of the embodiments of this disclosure, an electronic device is provided. The electronic device includes one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement a method for updating a test case library. The method includes obtaining candidate test cases; extracting feature representations of the candidate test cases using a language model; determining the similarity between the feature representations of the candidate test cases and feature representations in a corresponding feature representation library of the test case library; and adding the candidate test cases to the test case library in response to the similarity meeting a predetermined condition.
[0006] In a third aspect of the embodiments of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements a method for updating a test case library. The method includes obtaining candidate test cases; extracting feature representations of the candidate test cases using a language model; determining the similarity between the feature representations of the candidate test cases and feature representations in a corresponding feature representation library of the test case library; and adding the candidate test cases to the test case library in response to the similarity meeting a predetermined condition.
[0007] It should be understood that the description in the Summary of the Invention section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0008] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:
[0009] Figure 1 A schematic diagram of an example environment in which several embodiments of the present disclosure may be implemented is shown;
[0010] Figure 2 A schematic diagram of a method for updating a test case library based on a language model according to some embodiments of the present disclosure is shown;
[0011] Figure 3 A flowchart illustrating the determination of whether to add a candidate test case to the test case library according to some embodiments of this disclosure is shown;
[0012] Figure 4 A schematic diagram of a preprocessing method according to some embodiments of the present disclosure is shown;
[0013] Figure 5 A flowchart is shown illustrating a method for updating a seed corpus for fuzz testing according to some embodiments of this disclosure;
[0014] Figure 6 A schematic diagram of a method for mutating fuzz test seeds according to some embodiments of the present disclosure is shown; and
[0015] Figure 7 A block diagram of a device that can implement several embodiments of the present disclosure is shown. Detailed Implementation
[0016] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0017] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.
[0018] Software testing tasks, such as Application Programming Interface (API) testing, often involve a large number of configurable parameters, such as file paths, filenames, and permissions. Software testing needs to comprehensively cover various scenarios, including correct parameter format, parameter values outside the valid range, and incorrect parameter format. Furthermore, parameter combination testing is indispensable because some defects only manifest under specific parameter combinations; for example, an exception is only triggered when parameter A and parameter B are set simultaneously. In such tests, the number of input parameter combinations can grow exponentially; for example, with 10 optional parameters, the number of combinations can reach 1024.
[0019] In practical software testing tasks, manually creating test cases to cover all the aforementioned scenarios is highly impractical due to the sheer number of test cases. Creating new test cases randomly, on the other hand, struggles to guarantee the relevance and effectiveness of the tests. For example, fuzzing explores program behavior by generating random inputs. During product development, after comprehensive functional testing, most common errors have been fixed, reducing the probability of encountering unexpected behavior in fuzzing and lengthening the time required to discover new problems. Furthermore, checking code coverage is extremely difficult in most real-world scenarios. For large software products with millions of lines of code, checking code coverage for each test case individually is virtually impossible; a single code coverage analysis is extremely time-consuming, and even if it could be completed, its efficiency would be very low. Therefore, to improve testing efficiency, it is necessary to select valuable test cases and add them to a test case library as input for software testing.
[0020] To address this, embodiments of this disclosure propose a scheme for updating a test case library. In these embodiments, the test case library has a corresponding feature representation library. Before adding one or more candidate test cases to the test case library, feature representations of the candidate test cases can be extracted using a language model. The similarity between the feature representation of the candidate test case and each feature representation in the feature representation library can also be calculated. Then, based on these similarities, it can be determined whether to add the candidate test case to the library. Thus, when updating the test case library, candidate test cases with low similarity to existing test cases can be added, while candidate test cases with high similarity to existing test cases can be discarded.
[0021] In this way, candidate test cases with lower similarity to existing test cases can be selected as valuable test cases and added to the test case library, thereby improving the diversity of the test case library and reducing the unnecessary extension of testing time due to excessively high similarity of test cases.
[0022] Figure 1 A schematic diagram of an example environment 100 in which several embodiments of the present disclosure may be implemented is shown. Figure 1 This may include candidate test cases 102, a language model 104, feature representations of candidate test cases 106, a test case library 108, a feature representation library 110, and a computing device 112. It should be understood that... Figure 1 The types and number of models, data transmission processes, and arrangements shown are merely illustrative. Example environment 100 may include different numbers and arrangements of models, data transmission processes, and various additional elements.
[0023] like Figure 1 As shown, in example environment 100, the language model 104 can be used to extract the candidate test cases 102 to obtain the feature representation 106 of the candidate test cases. For example, the candidate test cases 102 can be encoded into feature vectors using the language model 104.
[0024] like Figure 1 As shown, in example environment 100, feature representation library 110 is obtained by operating on the test cases in test case library 108. For example, each test case in test case library 108 is input into language model 104, feature representation is extracted, and the feature representation is stored in database to obtain feature representation library 110.
[0025] like Figure 1 As shown, in example environment 100, similarity calculation is performed between the feature representation 106 of the candidate test case and the feature representation library 110 based on computing device 112. Furthermore, computing device 112 can feed back the similarity calculation result to test case library 108 to determine whether to add candidate test case 102 to test case library 108. Additionally, for candidate test case 102 added to test case library 108, the feature representation 106 of that candidate test case can be added to feature representation library 110.
[0026] In example environment 100, feature representations can be quickly extracted using language model 104. Furthermore, computing device 112 can perform similarity calculations between the feature representations 106 of candidate test cases and the feature representations in feature representation library 110, and determine whether to add candidate test case 102 to test case library 108 based on this similarity. This method allows for the rapid filtering of candidate test cases with low similarity to existing test cases and their addition to the test case library, improving test case diversity and testing efficiency.
[0027] Figure 2 A schematic diagram of a method 200 for updating a test case library based on a language model according to some embodiments of the present disclosure is shown. Figure 2 The methods in the middle can be updated as follows: Figure 1 The test case library shown is 108. (As shown...) Figure 2 As shown in box 202, method 200 obtains candidate test cases to be added to the test case library. Candidate test cases can be obtained by mutating test cases in the test case library, or they can be directly generated by manual design or automated tools, etc. Method 200 does not impose any restrictions on this.
[0028] In box 204, feature representations of candidate test cases are extracted using a language model. In some embodiments, preprocessing of candidate test cases is performed according to a uniform procedure before inputting the language model. The example preprocessing method will be discussed in the section on... Figure 4 The description will provide a detailed explanation.
[0029] In box 206, the similarity between the feature representations of candidate test cases and the feature representations in the feature representation library is determined. In some embodiments, the feature representation library is obtained by processing test cases in the test case library, and the feature representations in the feature representation library correspond one-to-one with the test cases in the test case library.
[0030] In box 208, in response to the similarity meeting a predetermined condition, candidate test cases are added to the test case library. It should be noted that candidate test cases can be selected from a candidate set. In this case, it can be determined whether multiple candidate test cases in the same batch in the candidate set meet the predetermined condition, and candidate test cases in the candidate set whose similarity meets the predetermined condition are added to the test case library.
[0031] In this way, method 200 can filter out candidate test cases that are less similar to existing test cases and add them as valuable test cases to the test case library, thereby improving the diversity of the test case library and improving the efficiency of software testing.
[0032] Figure 3A flowchart of a method 300 for updating a test case library according to some embodiments of this disclosure is shown. Figure 3 As shown, candidate test case 302 and test case 304 from the test case library are input into box 306 for preprocessing. Box 306 can perform preprocessing on the input test cases according to a unified standard. The following section will combine... Figure 4 This describes a preprocessing method according to some embodiments of the present disclosure. Figure 4 A schematic diagram of a preprocessing method 400 according to some embodiments of the present disclosure is shown.
[0033] In block 402, method 400 selects at least one of the test cases in the test case library and candidate test cases as test cases to be processed, and determines the parameter set for each test case to be processed. In some embodiments, the parameter set for each test case to be processed includes at least one parameter, and the data corresponding to each parameter is key-value pair data, wherein the key in the key-value pair corresponds to the parameter name, and the value in the key-value pair corresponds to the parameter value.
[0034] In software testing, changing the parameter value of a test case from one random string to another will generally not affect the results. (Only when the change reaches boundary conditions, such as the string length exceeding a certain range, will it affect the results. However, such cases are usually covered by functional testing.) In box 404, method 400 removes the parameter value for each pending test case. For example, for the following exemplary test case:
[0035]
[0036]
[0037] By using the above methods, Method 400 ensures that the process of feature extraction of test cases through language models is not affected by the parameter values of the test cases, thereby avoiding the impact of parameter value differences on the similarity assessment between candidate test cases and existing test cases in the library, and further ensuring the diversity of the test case library.
[0038] For a test case, changing the order of its internal parameter names will not affect the test results. In block 406, method 400 sorts the multiple parameter names in the test case after processing in block 404 according to a unified preset sorting strategy. In some embodiments, the sorting strategy can be to arrange these parameter names in ascending or descending alphabetical order. This disclosure does not limit the specific sorting strategy, as long as it can sort the multiple parameter names in all test cases according to a unified standard. In this way, method 400 can avoid adding test cases with the same parameter names (the difference is only in the internal order of the parameter names) to the test case library repeatedly, thereby further ensuring the diversity of the test case library.
[0039] return Figure 3 In box 308, method 300 extracts features from at least one of the preprocessed candidate test cases and test cases in the test case library using the language model 104 (e.g., a sentencetransformer) of example 100. In some embodiments, method 300 converts the test cases in the test case library (e.g., a seed corpus) 108 into feature representations (e.g., vectors) using the language model 104 of example 100, and generates a feature representation library (e.g., storing the feature representations in a vector library Fuss).
[0040] In box 310, method 300 can calculate the similarity between the feature representations of candidate test cases and existing test cases in the library using the computing device 112 of example 100. In some embodiments, the feature representations can be vectors, and the similarity between candidate test cases and existing test cases in the library can be achieved by calculating the cosine similarity of the corresponding vectors. For example, for a vector a and a vector b, the cosine similarity calculation formula is as follows:
[0041] a*b / |a| / |b|
[0042] In this case, cosine similarity is a floating-point number ranging from 0 to 1.
[0043] In box 312, the similarity between the feature representation of candidate test case 302 and each feature representation in the feature representation library is compared with a uniform preset threshold. If the similarity between the feature representation of the candidate test case and each feature representation in the feature representation library is less than the preset threshold, proceed to box 318 and add candidate test case 302 to the test case library; otherwise, proceed to box 316 to determine whether test case 302 meets other criteria. In some embodiments, the preset threshold can be dynamically adjusted according to the needs of the actual testing task. For example, when the testing task requires high efficiency, a smaller preset threshold can be selected, thereby reducing the number of candidate test cases added to the test case library and ensuring that the candidate test cases added to the test case library have low similarity to existing test cases in the library.
[0044] Alternatively or additionally, in box 314, candidate test case 302 is selected from the candidate set. The similarity of candidate test case 302 is compared with the similarity of other test cases in the candidate set. If the similarity of candidate test case 302 is less than the similarity of other test cases in the candidate set that account for a preset threshold, proceed to box 318 and add candidate test case 302 to the test case library; otherwise, proceed to box 316 to determine whether test case 302 meets other criteria. In some embodiments, the similarity between a candidate test case and each test case in the test case library is calculated, and the maximum value is selected as the similarity of the candidate test case, which is then compared with the similarity of other test cases in the candidate set. In some embodiments, the preset threshold can be dynamically adjusted according to the needs of the actual testing task. For example, if more test cases are needed to ensure test comprehensiveness, the preset threshold can be lowered so that more test cases in the candidate set can meet the conditions and be added to the test case library. In some embodiments, after performing preprocessing 306, feature extraction 308, and similarity determination 301 on all test cases in the candidate set, the candidate test cases with the lowest similarity in the candidate set (e.g., the test cases with the lowest similarity ranking) are selected and added to the test case library.
[0045] In box 316, it is determined whether a candidate test case not added to the test case library in the above steps meets any of the other criteria (such as the candidate test case causing a program error, covering a new scenario, etc.). If the candidate test case meets any of the other criteria, it is added to the test case library. If the candidate test case does not meet all other criteria, proceed to box 320 and discard the candidate test case. It should be noted that in some embodiments, box 316 can be executed independently of other steps in this disclosure. That is, when a candidate test case meets other criteria (such as causing a program error), it is not necessary to execute other steps in this disclosure; proceed directly to box 318 and add the candidate test case to the test case library. In other embodiments, the process shown in box 316 can be omitted. In this case, if the candidate test case does not meet the judgment conditions of boxes 312 and 314, proceed directly to box 320 and discard the candidate test case.
[0046] In box 318, method 300 adds candidate test cases to the test case library. In some embodiments, when a candidate test case is added to the test case library, the feature representation corresponding to the candidate test case is also added to the feature representation library accordingly, thereby dynamically updating the similarity judgment benchmark of subsequent candidate test cases and ensuring the diversity among all test cases in the test case library.
[0047] Figure 5 A flowchart of a method 500 for updating a seed corpus for fuzz testing according to some embodiments of the present disclosure is shown. In fuzz testing, test cases are randomly generated, and compared to other testing methods, fuzz testing requires a significant amount of time to detect program problems. In method 500, the seed corpus is used as an example of a test case library, and mutated seeds are used as examples of candidate test cases. By determining the similarity between newly generated mutated seeds and existing seeds in the seed corpus in real time during the fuzzing process, seeds with lower similarity are added to the seed corpus, thereby increasing the diversity of the seed corpus and reducing the time consumption of fuzz testing. Figure 5 As shown in box 502, method 500 prepares the initial seed corpus. In some embodiments, the seed corpus can be prepared using methods found in existing fuzzing techniques, for example, Figure 6 The original seed 602 is an example seed for creating storage units in the data domain.
[0048] In box 504, the seeds in the seed corpus are encoded using a language model to obtain feature vectors. In some embodiments, a uniform preprocessing operation can be performed on the seeds in the seed corpus before the encoding operation. For example, according to... Figure 4The preprocessing method 400 shown can determine the parameter set of each seed in the seed corpus. The parameter set of each seed includes at least one parameter. The data corresponding to each parameter belongs to key-value pair data, where the key in the key-value pair corresponds to the parameter name and the value in the key-value pair corresponds to the parameter value. It can also remove the parameter value of each seed in the seed corpus.
[0049] For example, in fuzzing, changes in parameter values are insufficient to make a mutation seed valuable. For instance, for a string parameter such as a filename, changing the filename value from a random string to another will, in most cases, not change the test results. While changes in parameter values reaching boundary conditions, such as when the string length exceeds a certain range, will affect the test results, these tests are typically included in boundary value testing within functional testing and are not part of the primary task of fuzzing. By employing the methods described above, we ensure that the selection of mutation seeds is not influenced by changes in parameter values.
[0050] After removing parameter values from the seed, the different parameters in the seed are sorted according to a unified preset sorting strategy to obtain the preprocessed seed. In some embodiments, the different parameter names in a seed are sorted in ascending or descending alphabetical order.
[0051] In some embodiments, the feature vectors obtained in block 504 are stored in a vector library such as Fuss, thereby obtaining a vector library corresponding to the seed corpus. In some embodiments, the vectors of newly added seeds in the seed corpus are stored in the vector library in real time to achieve dynamic updates.
[0052] In box 506, a seed from the seed corpus is selected and mutated to obtain a mutated seed. In some embodiments, the mutation of the seed includes one or more operations such as increasing seed parameters, decreasing seed parameters, or changing seed parameters. The following is in conjunction with... Figure 6 This invention describes a method 600 for mutating fuzzy test seeds according to some embodiments of the present disclosure. According to some embodiments of the present disclosure, in method 600, a mutated seed 604 can be obtained by deleting parameters (e.g., deleting hard_limit:2048 from the original seed 602), or a mutated seed 606 can be obtained by adding parameters (e.g., adding stream_limit_config:{write_stream_soft_limit:32} to the original seed 602).
[0053] In box 508, the mutated seed is encoded using a language model to obtain the vector corresponding to the mutated seed. In some embodiments, before performing the encoding operation, the mutated seed and the seeds in the seed corpus can be preprocessed uniformly according to the preprocessing method 400 described above. In some embodiments, the same language model is used to uniformly perform the encoding operation on the mutated seed and the seeds in the seed corpus.
[0054] In box 510, the similarity between the mutated seed and the vector corresponding to the seed in the corpus is calculated. In some embodiments, the cosine similarity between the mutated seed and the vector corresponding to the seed in the corpus can be calculated; for example, the cosine similarity is 0.5 for vector a(1,0,1) and vector b(1,1,0). In some embodiments, one or more methods, including but not limited to GPU acceleration, multi-core CPU parallel processing, and distributed computing, can be used to accelerate the calculation of vector similarity.
[0055] In box 512, it is determined whether the similarity between the mutated seed and the vector corresponding to the seed in the corpus is less than a preset threshold. In some embodiments, if the similarity between the mutated seed and all seeds in the corpus is lower than the threshold (e.g., 0.7), the mutated seed is deemed valuable and added to the seed corpus. In other embodiments, the similarity between multiple mutated seeds and all seeds in the corpus is calculated separately. If the similarity between the mutated seed and other mutated seeds is less than a certain percentage (e.g., 80%), it is deemed valuable and added to the seed corpus.
[0056] If the similarity between the mutated seed and the vector corresponding to the seed in the corpus is less than a preset threshold, proceed to box 516 and add the mutated seed to the seed corpus; otherwise, proceed to box 514 to determine whether the mutated seed meets other criteria. If the mutated seed meets other criteria 514, proceed to box 516 and add the mutated seed to the seed corpus; otherwise, proceed to box 518 and discard the mutated seed. It should be noted that in some embodiments, box 514 can be executed independently of other steps in this disclosure. That is, when a candidate test case meets other criteria (such as causing program errors or covering new scenarios), it is not necessary to execute other steps in this disclosure, and the candidate test case can be directly added to the test case library. In other embodiments, the process shown in box 514 can be omitted.
[0057] In some embodiments, if a mutant seed is added to the seed corpus, its corresponding vector is also added to the vector library, thus using the newly added seed as a benchmark for judging the similarity of subsequent seeds. For example, the seed corpus has N original seeds, and the vector library has N vectors that correspond one-to-one with the original seeds. If the similarity of a newly generated mutant seed with the other N original seeds meets a threshold condition, the newly generated mutant seed is added to the seed corpus, and its corresponding vector is added to the vector library. For the next mutant seed, it is necessary to determine the similarity between its corresponding vector and the N+1 vectors in the vector library. In this way, method 500 can reduce the similarity between multiple mutant seeds added to the seed corpus.
[0058] In box 520, determine whether the fuzzing test has been completed. If the fuzzing test has not yet been completed, method 500 can return from box 520 to box 506 to regenerate the mutation seed, thereby continuously generating mutation seeds during the fuzzing process and determining whether to add them to the seed corpus. If the fuzzing test has been completed, proceed to box 522 and end the process of method 500.
[0059] This method allows for the determination of the similarity between mutated seeds and existing seeds in the seed corpus. By comparing the similarity with a preset threshold, mutated seeds with low similarity to existing seeds can be identified and added to the seed corpus. Therefore, it ensures the diversity of seeds in the corpus and avoids the inefficiency of fuzz testing caused by blindly adding mutated seeds.
[0060] Figure 7 A schematic block diagram of an example device 700 that can be used to implement embodiments of the present disclosure is shown. As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 702 or loaded from storage unit 707 into random access memory (RAM) 703. Various programs and data required for the operation of device 700 may also be stored in RAM 703. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.
[0061] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 707, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0062] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as method 200. For example, in some embodiments, method 200 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 707. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of method 200 described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform method 200 by any other suitable means (e.g., by means of firmware).
[0063] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload programmable logic devices (CPLDs), and so on.
[0064] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0065] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. Furthermore, although operations are depicted in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.
[0066] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.
Claims
1. A method for updating a test case library, comprising: Obtain candidate test cases; The feature representations of the candidate test cases are extracted using a language model; Determine the similarity between the feature representations of the candidate test cases and the feature representations in the corresponding feature representation library of the test case library; and In response to the similarity meeting a predetermined condition, the candidate test cases are added to the test case library.
2. The method according to claim 1, further comprising: The feature representation library corresponding to the test case library is generated by extracting the feature representation of each test case in the test case library through a language model.
3. The method according to claim 2, further comprising: Determine a parameter set for at least one of the test cases in the test case library and the candidate test cases. The parameter set includes at least one parameter, and the data corresponding to each parameter belongs to key-value pair data, wherein the key in the key-value pair corresponds to the parameter name of the parameter, and the value in the key-value pair corresponds to the parameter value of the parameter. Based on the parameter set, remove the parameter values of at least one of the test cases in the test case library and the candidate test cases to obtain test cases containing parameter names from the test case library and at least one of the candidate test cases. as well as According to a preset sorting strategy, the parameter names of at least one of the test cases in the test case library and the candidate test cases are sorted to obtain the preprocessed test cases in the test case library and at least one of the candidate test cases.
4. The method according to claim 1, wherein the test case library is a seed corpus for fuzz testing, and the candidate test cases are candidate seeds for fuzz testing.
5. The method of claim 4, wherein obtaining candidate test cases comprises performing a mutation operation on the seeds in the seed corpus by one or more of the following operations: Increase the parameters of the seed; Delete the parameters of the seed; Change the parameters of the seed.
6. The method according to claim 4, wherein the feature representation of the candidate seed is a vector, the feature representation library is a vector library, and determining the similarity between the vector of the candidate seed and the vector in the vector library includes: Determine the cosine similarity between the vectors of the candidate seeds and the vectors in the vector library.
7. The method of claim 1, wherein adding the candidate test cases to the test case library in response to the similarity satisfying a predetermined condition comprises: In response to the fact that the similarity between the feature representation of the candidate test case and each feature representation in the feature representation library is less than a preset threshold, the candidate test case is added to the test case library.
8. The method of claim 1, wherein the candidate test cases are selected from a candidate set, and adding the candidate test cases to the test case library in response to the similarity meeting a predetermined condition comprises: In response to the similarity of the candidate test case being less than the similarity of other test cases in the candidate set that account for a percentage of a preset threshold, the candidate test case is added to the test case library.
9. The method according to claim 1, further comprising: In response to the candidate test case causing a program error, the candidate test case is added to the test case library.
10. The method according to claim 1, further comprising: In response to the similarity meeting a predetermined condition, the feature representation of the candidate test case is added to the feature representation library.
11. An electronic device, comprising: At least one processor; as well as Coupled to the at least one processor and having instructions stored thereon, the instructions, when executed by the at least one processor, cause the electronic device to perform actions, the actions including: Obtain candidate test cases; The feature representations of the candidate test cases are extracted using a language model; Determine the similarity between the feature representations of the candidate test cases and the feature representations in the corresponding feature representation library of the test case library; and In response to the similarity meeting a predetermined condition, the candidate test cases are added to the test case library.
12. The device according to claim 11, wherein the action further includes: The feature representation library corresponding to the test case library is generated by extracting the feature representation of each test case in the test case library through a language model.
13. The device according to claim 12, wherein the action further includes: Determine a parameter set for at least one of the test cases in the test case library and the candidate test cases. The parameter set includes at least one parameter, and the data corresponding to each parameter belongs to key-value pair data, wherein the key in the key-value pair corresponds to the parameter name of the parameter, and the value in the key-value pair corresponds to the parameter value of the parameter. Based on the parameter set, remove the parameter values of at least one of the test cases in the test case library and the candidate test cases to obtain test cases containing parameter names from the test case library and at least one of the candidate test cases. as well as According to a preset sorting strategy, the parameter names of at least one of the test cases in the test case library and the candidate test cases are sorted to obtain the preprocessed test cases in the test case library and at least one of the candidate test cases.
14. The device of claim 11, wherein the test case library is a seed corpus for fuzz testing, and the candidate test cases are candidate seeds for fuzz testing.
15. The device of claim 14, wherein obtaining candidate test cases comprises performing a mutation operation on the seeds in the seed corpus by one or more of the following operations: Increase the parameters of the seed; Delete the parameters of the seed; Change the parameters of the seed.
16. The device of claim 14, wherein the feature representation of the candidate seed is a vector, the feature representation library is a vector library, and determining the similarity between the vector of the candidate seed and the vector in the vector library comprises: Determine the cosine similarity between the vectors of the candidate seeds and the vectors in the vector library.
17. The device of claim 11, wherein adding the candidate test case to the test case library in response to the similarity satisfying a predetermined condition comprises: In response to the fact that the similarity between the feature representation of the candidate test case and each feature representation in the feature representation library is less than a preset threshold, the candidate test case is added to the test case library.
18. The device of claim 11, wherein the candidate test cases are selected from a candidate set, and adding the candidate test cases to the test case library in response to the similarity meeting a predetermined condition comprises: In response to the similarity of the candidate test case being less than the similarity of other test cases in the candidate set that account for a percentage of a preset threshold, the candidate test case is added to the test case library.
19. The device according to claim 11, wherein the action further includes: In response to the similarity meeting a predetermined condition, the feature representation of the candidate test case is added to the feature representation library.
20. A computer program product tangibly stored on a non-volatile computer-readable medium and comprising machine-executable instructions that, when executed, cause a machine to perform actions, the actions comprising: Obtain candidate test cases; The feature representations of the candidate test cases are extracted using a language model; Determine the similarity between the feature representations of the candidate test cases and the feature representations in the corresponding feature representation library of the test case library; and In response to the similarity meeting a predetermined condition, the candidate test cases are added to the test case library.