Training methods, test case generation methods, and storage media for test case generation models

Abstract

This application relates to the field of terminal technology and provides a training method for a test case generation model. The method generates test code containing at least one known vulnerability using a current test code generation model. By inputting the test code into the current test case generation model, test cases are generated, and fuzz testing is performed on the test code based on the test cases to obtain test results. Through bidirectional optimization of the test case generation model and the test code generation model, the effectiveness of the test cases and the difficulty and / or depth of vulnerability triggering in the test code can be continuously improved. On the one hand, this improves the vulnerability discovery capability of the test cases and the ability to discover vulnerabilities of different depths, thereby improving the testing efficiency of fuzz testing. On the other hand, by continuously updating the test code, the vulnerability discovery and vulnerability mining capabilities of the test case generation model are improved when facing different code types. Therefore, the test case generation model has high applicability.

Description

Technical Field

[0001] This application belongs to the field of terminal technology, and in particular relates to a training method for a test case generation model, a test case generation method, and a storage medium. Background Technology

[0002] With the rapid development of electronic technology, numerous software applications (APPs) are available for users in fields such as reading, travel, learning, consulting, and daily life, providing users with a wealth of services. The stability and security of software applications running on electronic devices are crucial factors affecting user experience and protecting user privacy.

[0003] Before launching electronic devices to the market, or when providing system updates for electronic devices, terminal manufacturers need to test the stability and security of the system services and / or software applications running on these system services. Currently, testing is usually conducted using fuzzing to randomly generate test cases. However, fuzzing can detect a limited number of vulnerabilities, which can easily lead to many undiscovered vulnerabilities remaining in the electronic device after fuzzing, resulting in poor testing effectiveness and affecting the stability and security of the software running on the electronic device. Summary of the Invention

[0004] In view of this, embodiments of this application provide a training method for a test case generation model, a test case generation method, and a storage medium to solve the problem that existing fuzzing tests detect a limited number of vulnerabilities, which can easily lead to electronic devices still having many undiscovered vulnerabilities after fuzzing, resulting in poor test performance.

[0005] The first aspect of this application provides a method for training a use case generation model, including:

[0006] Input the source code of the sample process and the known vulnerability information into the current test code generation model to generate test code containing at least one known vulnerability;

[0007] Input the test code into the current test case generation model to generate test cases;

[0008] Perform fuzz testing on the test code based on the test cases, and obtain the test results;

[0009] If the current test case generation model does not meet the preset optimization goal, and the test result does not meet the preset test goal, then after optimizing the current test case generation model, return to the step of inputting the test code into the current test case generation model and subsequent steps;

[0010] If the current test case generation model does not meet the preset optimization target, and the test result meets the preset test target, optimize the current test code generation model, regenerate the test code, and return to execute the step of inputting the test code into the current test case generation model and subsequent steps;

[0011] If the current test case generation model reaches the preset optimization target, then the current test case generation model will be output as the trained test case generation model.

[0012] The test code generation model and the test case generation model are built based on a neural network model.

[0013] The first aspect of this application provides a training method for a test case generation model. A test code generation model can generate test code containing at least one known vulnerability. By inputting the test code into the current test case generation model, test cases can be generated. Fuzz testing is then performed on the test code based on the test cases to obtain test results. By bidirectionally optimizing the test case generation model and the test code generation model, the effectiveness of the test cases and the difficulty and / or depth of vulnerability triggering in the test code can be continuously improved. On the one hand, this improves the vulnerability discovery capability of the test cases and the ability to discover vulnerabilities of different depths, thereby improving the effectiveness of the test cases generated by the test case generation model and thus improving the testing efficiency of fuzz testing. On the other hand, by continuously updating the test code, the vulnerability discovery capability and vulnerability discovery capability of the test case generation model when facing different code are improved, making the test case generation model highly applicable.

[0014] A second aspect of this application provides a test case generation method, including:

[0015] The source code of the process under test is input into the test case generation model to obtain test cases, which are used to perform fuzz testing.

[0016] The use case generation model is trained based on the training method provided in the first aspect of the embodiments of this application.

[0017] The second aspect of this application provides a test case generation method. By inputting the source code of the process under test into the pre-trained test case generation model provided in the above embodiments, at least one test case can be obtained. The at least one test case has the advantages of strong vulnerability discovery capability and strong ability to discover vulnerabilities of different depths. When performing fuzz testing, it can effectively discover vulnerabilities of different depths in the source code of the process under test, thereby improving testing efficiency.

[0018] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the training method for the use case generation model provided in the first aspect of this application or the test case generation method provided in the second aspect.

[0019] It is understood that the beneficial effects of the third aspect mentioned above can be found in the relevant descriptions in the first or second aspects mentioned above, and will not be repeated here. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application;

[0022] Figure 2 This is a schematic diagram of the first type of training method provided in the embodiments of this application;

[0023] Figure 3 This is an interaction sequence diagram between the current test code generation model, the current test case generation model, and the fuzz testing module provided in the embodiments of this application;

[0024] Figure 4 This is a schematic diagram of the second flowchart of the training method provided in the embodiments of this application;

[0025] Figure 5 This is a schematic diagram of the third training method provided in the embodiments of this application;

[0026] Figure 6 This is a schematic diagram of the fourth training method provided in the embodiments of this application;

[0027] Figure 7 This is a fifth flowchart illustrating the training method provided in the embodiments of this application;

[0028] Figure 8 This is a schematic diagram of the first type of test case generation method provided in the embodiments of this application;

[0029] Figure 9 This is a schematic diagram of the second process of the test case generation method provided in the embodiments of this application. Detailed Implementation

[0030] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0031] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0032] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0033] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0034] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0035] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0036] In applications, testing is currently usually conducted by randomly generating test cases through fuzzing. However, the number of vulnerabilities detected by fuzzing is limited, which can easily lead to many undiscovered vulnerabilities remaining in electronic devices after fuzzing. This results in poor testing effectiveness and affects the stability and security of the software running on electronic devices.

[0037] To address the aforementioned technical problems, this application provides a training method for a test case generation model. The current test code generation model can generate test code containing at least one known vulnerability. By inputting the test code into the current test case generation model, test cases can be generated. Fuzz testing is then performed on the test code based on the test cases to obtain test results. Through bidirectional optimization of the test case generation model and the test code generation model, the effectiveness of the test cases and the difficulty and / or depth of vulnerability triggering in the test code can be continuously improved. On the one hand, this enhances the vulnerability discovery capability of the test cases and the ability to discover vulnerabilities of varying depths, thereby improving the effectiveness of the test cases generated by the test case generation model and thus increasing the efficiency of fuzz testing. On the other hand, by continuously updating the test code, the vulnerability discovery and vulnerability mining capabilities of the test case generation model are improved when facing different code types, making the test case generation model highly applicable.

[0038] The training method for the use case generation model provided in this application can be applied to terminal devices. Terminal devices can be mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. This application does not impose any restrictions on the specific type of terminal device.

[0039] Figure 1An exemplary structural diagram of terminal device 1 is shown. Terminal device 1 may include a processor 10, a memory 20, a power module 30, an audio module 40, a camera module 50, a sensor module 60, an input module 70, a display module 80, and a wireless communication module 90, etc. The audio module 40 may include a speaker 41 and a microphone 42, etc.; the camera module 50 may include a short-focus camera 51, a long-focus camera 52, and a flash 53, etc.; the sensor module 60 may include an infrared sensor 61, an accelerometer 62, a position sensor 63, a fingerprint sensor 64, and an iris sensor 65, etc.; the input module 70 may include a touch panel 71 and an external input unit 72, etc.; and the wireless communication module 90 may include wireless communication units such as Bluetooth, ZigBee, Optical Wireless, Wireless Local Area Network (WLAN), and Near Field Communication (NFC).

[0040] In applications, processor 10 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0041] In applications, memory 20 may be an internal storage unit of the terminal device in some embodiments, such as a hard drive or memory of the terminal device. In other embodiments, memory 20 may be an external storage device of the terminal device, such as a plug-in hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., provided on the terminal device. Furthermore, memory 20 may include both internal and external storage units of the terminal device. Memory 20 is used to store operating system 21, application program 22, computer program 23, and data, etc. Memory 20 may also be used to temporarily store data that has been output or will be output.

[0042] It is understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on terminal device 1. In other embodiments of this application, terminal device 1 may include more or fewer components than illustrated, or combine certain components, or different components, such as input / output devices, network access devices, etc. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0043] like Figure 2 As shown, the training method for the use case generation model provided in this application embodiment includes the following steps S201 to S206:

[0044] Step S201: Input the source code of the sample process and the known vulnerability information into the current test code generation model to generate test code containing at least one known vulnerability.

[0045] In application, the test code generation model can be built based on a neural network model. This model can be continuously trained and iterated, with the current model representing the model obtained after the previous training or iteration. Specifically, the test code generation model can be built based on one or more different types of neural network models, such as Convolutional Neural Networks (CNN), Region-Convolutional Neural Networks (R-CNN), Fully Convolutional Networks (FCN), Region-Fully Convolutional Networks (R-FCN), and Feature Pyramid Networks (FPN). This application embodiment does not impose any restrictions on the specific network structure of the image recognition model.

[0046] In applications, the source code of the sample process can be written in C, C++, or Python, among other programming languages. A pre-written process can be obtained, or a process excerpted from a pre-written program can be used as a sample process. The source code of the sample process must meet the condition of being executable after compilation, and the source code of the sample process can contain any number of unknown vulnerabilities. This application does not impose any limitations on the language type, code content, or number of lines of code in the source code of the sample process.

[0047] In applications, known vulnerability information can include publicly disclosed vulnerability information and source code vulnerability information. Publicly disclosed / source code vulnerability information includes at least one known vulnerability, which may be an out-of-bounds array element, an unsigned parameter type, or a null pointer, etc. Publicly disclosed vulnerability information can be obtained by reading open-source vulnerability repositories, while source code vulnerability information can be obtained by reading the project files or bug logs of sample processes. This application does not limit the types and categories of vulnerabilities included in the known vulnerability information.

[0048] In the application, by inputting the source code of the sample process and known vulnerability information into the current test code generation model, the model can obtain the syntactic structure of the sample process's source code. Specifically, it can convert the sample process's source code into an Abstract Syntax Tree (AST), filter available known vulnerabilities based on the syntactic structure, and determine the injection points of these vulnerabilities. The syntactic type of the available known vulnerabilities must belong to at least one of the syntactic types described in the syntactic structure. The model then extracts source code segments from the sample process's source code that have the same syntactic type as the available known vulnerabilities and injects at least one available known vulnerability into the corresponding source code segment, generating test code containing at least one known vulnerability and a native vulnerability. The native vulnerability is a vulnerability carried by the sample process's source code.

[0049] Step S202: Input the test code into the current test case generation model to generate test cases.

[0050] In application, the network structure of the use case generation model can refer to the network structure of the code generation model described above. The use case generation model can be continuously trained and iterated; the current use case generation model represents the use case generation model obtained after the previous round of training or iteration.

[0051] In the application, after obtaining the test code, it can be input into the current test case generation model. The current test case generation model can be used to obtain the code attributes of the input interface of the test process corresponding to the test code. The code attributes can include the interface name, the function type defined by the interface, the function parameters or annotations defined by the interface, etc. Test cases are generated based on the code attributes of the input interface. Specifically, test cases can be generated based on the interface name / annotation, and the correspondence can be preset. Alternatively, the corresponding target data type can be obtained based on the function type defined by the interface, and test cases can be generated based on the target data type. Furthermore, test cases can be generated based on the function parameters defined by the interface and the target data type to improve the effectiveness of the test cases, thereby improving testing efficiency.

[0052] It should be noted that the current test case generation model can generate multiple test cases, which can be used to test the test code through multiple test cases. This can increase the probability that the test cases will trigger vulnerabilities and cause the test process to crash, thereby improving testing efficiency.

[0053] In one embodiment, the method further includes the following steps prior to step S202:

[0054] Get the number of optimizations for the current test case generation model;

[0055] When the preset number of optimization attempts is reached, the current test case generation model is determined to have reached the preset optimization target.

[0056] Alternatively, obtain the number of times the test results meet the preset test objectives;

[0057] When the number of times the target is met reaches the preset number of times the target is met, it is determined that the current test case generation modulus has reached the preset optimization target.

[0058] In application, before generating test cases by inputting test code into the current test case generation model, it can be determined whether the current test case generation model has achieved the preset optimization goal. The following explains two methods for determining whether the preset optimization goal has been achieved:

[0059] In the application, the number of optimizations of the current use case generation model can be obtained; when the number of optimizations reaches a preset number, it is determined that the current use case generation model has reached the preset optimization goal. The preset number of optimizations can be 50, 100, or 200 times, and this application embodiment does not impose any restrictions on the specific value of the preset number of optimizations.

[0060] In the application, the number of times the test results meet the preset test target can be obtained; when the number of times the target is met reaches the preset number, it is determined that the current test case generation model has met the preset optimization target. The preset number of times the target is met can be 10, 20, or 30 times; this application embodiment does not impose any limitation on the specific value of the preset number of times the target is met. The optimization method, test results, and preset test target of the current test case generation model can be referred to the relevant descriptions in the following embodiments.

[0061] Step S203: Perform fuzz testing on the test code according to the test cases and obtain the test results.

[0062] In the application, test results are obtained by inputting multiple test cases one by one into the input interface of the test process corresponding to the test code. Test results can be determined based on whether the test process responds to the test cases and receives output data. Specifically, for all test cases generated by the current test case generation model, if the test process responds to the test cases and receives output data (i.e., the test cases do not trigger vulnerabilities causing the test process to crash), the test result is determined to have not met the preset test objective. If the test process responds to any test case but does not receive output data (i.e., any of the aforementioned test cases trigger vulnerabilities causing the test process to crash), the test result is determined to have met the preset test objective. Test results can also be determined based on whether the test process is interrupted or reports an error. Specifically, if the test process is not interrupted and does not report an error (i.e., the test cases do not trigger vulnerabilities causing the test process to crash), the test result is determined to have not met the preset test objective. If the test process terminates or reports an error (i.e., at least one test case triggers a vulnerability causing the test process to crash), the test result is determined to have met the preset test objective. This application's embodiments do not impose any limitations on the method for determining test results.

[0063] Step S204: If the current test case generation model does not meet the preset optimization goal, and the test results do not meet the preset test goal, then optimize the current test case generation model and return to step S202 and subsequent steps.

[0064] In the application, if the current test case generation model does not reach the preset optimization goal and the test results do not reach the preset test goal, it means that none of the multiple test cases generated by the current test case generation model have triggered the vulnerability and caused the test process to crash. The current test case generation model needs to be optimized so that the test cases have the ability to trigger the vulnerability. After optimization, the process returns to step S202 to perform loop testing until the current test case generation model reaches the preset optimization goal or the test results reach the preset test goal.

[0065] In application, the optimization method of the current test case generation model can be achieved by changing the test case generation strategy in one or more ways, such as bitwise flipping, arithmetic addition and subtraction, replacement, insertion or splicing, so as to realize the variation of the test case generation model and improve the diversity of test cases.

[0066] Step S205: If the current test case generation model does not meet the preset optimization target, but the test result meets the preset test target, optimize the current test code generation model, regenerate the test code, and return to execute step S202 and subsequent steps.

[0067] In the application, if the current test case generation model does not reach the preset optimization goal, but the test result reaches the preset test goal, it means that at least one test case generated by the current test case generation model can trigger a vulnerability that causes the test process to crash. The current test code generation model can be optimized to change the difficulty and / or depth of the vulnerability that the test code triggers. After optimization, the process returns to step S202 to perform loop testing until the current test case generation model reaches the preset optimization goal or the test result reaches the preset test goal again.

[0068] In application, the current test code generation model can be optimized by extending the vulnerability triggering path. Specifically, the condition judgment code or the functions that need to be called can be adjusted before the vulnerability is triggered to change the difficulty and / or depth of the vulnerability triggered by the test code. This allows the current test case generation model to learn more about vulnerabilities with different triggering conditions, increasing the probability of test cases triggering vulnerabilities when facing test code of different difficulty and / or depth. This improves the vulnerability discovery and deep vulnerability mining capabilities of the test cases generated by the test case generation model, thereby improving the testing efficiency of fuzzing.

[0069] In applications, when optimizing the current test case generation model and the current test code generation model, Generative Adversarial Networks (GANs) can be used as the optimization framework. Specifically, the current test code generation model can be used as the generative model in the GAN, and the current test case generation model can be used as the discriminative model. By learning from each other through game theory in the GAN, the performance of the current test code generation model and the current test case generation model can be improved in both directions, thereby enhancing the optimization effect.

[0070] In one embodiment, step S205 further includes:

[0071] When regenerating the test code, update the types and / or number of known vulnerabilities contained in the regenerated test code.

[0072] In the application, each time test code is regenerated, the current test code generation model can update the types and / or number of known vulnerabilities contained in the regenerated test code to improve the diversity of the types and / or number of vulnerabilities in the test code, thereby improving the vulnerability discovery capability of test cases when facing different types and / or different numbers of vulnerabilities, and thus improving the testing efficiency of fuzz testing.

[0073] Step S206: If the current test case generation model reaches the preset optimization target, then output the current test case generation model as the trained test case generation model.

[0074] Among them, the test code generation model and the test case generation model are built based on neural network models.

[0075] In application, when the current test case model reaches the preset optimization goal, it means that the test cases generated by the current test case generation model have a sufficiently high vulnerability discovery capability and deep vulnerability mining capability, and the current test case generation model can be output as a trained test case generation model.

[0076] In application, the current test code generation model can inject known vulnerability information into the source code of the sample process, ensuring that the generated test code includes at least one known vulnerability and a native vulnerability in the source code. The test code is then input into the current test case generation model to generate test cases. These test cases are used to test the code, and the test results are obtained. If the current test case generation model does not meet the preset optimization goal, and the test results do not meet the preset test goal, the current test case generation model is optimized until the test cases generated by the current model can trigger the vulnerability in the test code. If the current test case generation model does not meet the preset optimization goal, and the test results meet the preset test goal... During the testing process, the current test code generation model is optimized to change the difficulty and / or depth at which vulnerabilities in the test code are triggered. By bidirectionally optimizing the test case generation model and the test code generation model, the effectiveness of test cases and the difficulty and / or depth at which vulnerabilities in the test code are triggered can be continuously improved. This continuously improves the vulnerability discovery capability of test cases and the ability to discover vulnerabilities at different depths, thereby improving the effectiveness of test cases generated by the test case generation model and thus improving the efficiency of fuzz testing. Furthermore, by continuously updating the test code, the vulnerability discovery and vulnerability discovery capabilities of the test case generation model when faced with different code are improved, thereby improving the applicability of the test case generation model.

[0077] Figure 3 An exemplary sequence diagram of the interaction between the current test code generation model, the current test case generation model, and the fuzzing module is shown, wherein the fuzzing module is used to execute the training method from steps S203 to S206.

[0078] like Figure 4 As shown, in one embodiment, based on Figure 2 The corresponding embodiment includes the following steps S401 to S408:

[0079] Step S401: Input the source code of the sample process and the known vulnerability information into the current test code generation model to generate test code containing at least one known vulnerability.

[0080] In application, the training method provided in step S401 is the same as the training method provided in step S201 above, and will not be repeated here.

[0081] Step S402: Determine whether the current use case generation model has reached the preset optimization goal. If yes, proceed to step S407; otherwise, proceed to step S403.

[0082] In application, the judgment method of step S402 can refer to the relevant description in step S202 above, and will not be repeated here.

[0083] Step S403: Obtain the target data types supported by the input interface of the test code;

[0084] Step S404: Input the test code and target data type into the current test case generation model to generate test cases.

[0085] In application, if the current test case generation model does not reach the preset optimization goal, the current test case generation model needs to continue generating test cases and optimizing until the preset optimization goal is reached. Before generating test cases, the target data types supported by the input interface of the test code can be obtained. The method for obtaining the target data types can be referred to the relevant description in step S202 above, and will not be repeated here.

[0086] In application, the target data type can be input into the current test case generation model to generate test cases whose data type belongs to the target data type. It should be noted that the target data type can include multiple data types, and the data type of the test cases can include one or more of the target data types. Furthermore, test code can also be input into the current test case generation model, allowing the model to acquire code characteristics such as execution logic or syntax structure, thereby improving the effectiveness of inputting test code into test cases.

[0087] Step S405: Perform fuzz testing on the test code according to the test cases and obtain the test results;

[0088] Step S406: If the current test case generation model does not meet the preset optimization target and the test results do not meet the preset test target, then optimize the current test case generation model and return to execute step S402 and subsequent steps.

[0089] Step S407: If the current test case generation model does not meet the preset optimization target, but the test result meets the preset test target, optimize the current test code generation model, regenerate the test code, and return to execute step S402 and subsequent steps.

[0090] Step S408: If the current test case generation model reaches the preset optimization target, then output the current test case generation model as the trained test case generation model.

[0091] Among them, the test code generation model and the test case generation model are built based on neural network models.

[0092] In application, the training methods provided in steps S405 to S408 are the same as those provided in steps S203 to S206 above, and will not be repeated here. The difference is that after step S203, the process returns to step S202: inputting the test code into the current test case generation model to generate test cases; after step S406, the process returns to step S402: determining whether the current test case generation model has reached the preset optimization goal.

[0093] like Figure 5 As shown, in one embodiment, based on Figure 4 The corresponding embodiment includes the following steps S501 to S510:

[0094] Step S501: Input the source code of the sample process and the known vulnerability information into the current test code generation model to generate test code containing at least one known vulnerability.

[0095] In application, the training method provided in step S501 is the same as the training method provided in step S201 above, and will not be repeated here.

[0096] Step S502: Compile the test code and obtain the test progress;

[0097] Step S503: If a runtime error occurs during the test process, regenerate the test code and return to execute step S502 and subsequent steps.

[0098] Step S504: If the test process runs normally, proceed to step S505 and subsequent steps.

[0099] In the application, after generating test code containing at least one known vulnerability using the current test code generation mode, the test code can be tested to ensure its effectiveness. The testing methods for the test code are explained below:

[0100] In the application, the test code is first compiled into an executable test process, which is then run to obtain the results. If an error occurs, it indicates a bug in the test code, rendering it unusable. The test code needs to be regenerated, and the process returns to step S502. If the test process runs normally, the test code is usable, and subsequent steps can be executed. It should be noted that inputting sensitive data corresponding to known vulnerabilities into the test process can cause the known vulnerability to crash. When performing validity testing on the test code, commonly used data can be input into the test process to reduce the probability of false positives caused by sensitive data triggering known vulnerabilities.

[0101] In applications, before generating test cases to perform fuzz testing on the test code, the validity of the test code can be determined to ensure that the test code can run normally, avoid inaccurate test results due to the test code not running normally, and improve the accuracy of fuzz testing.

[0102] Step S505: Determine whether the current use case generation model has reached the preset optimization target. If yes, proceed to step S511; otherwise, proceed to step S506.

[0103] Step S506: Obtain the target data types supported by the input interface of the test code;

[0104] Step S507: Input the test code and target data type into the current test case generation model to generate test cases;

[0105] Step S508: Perform fuzz testing on the test code according to the test cases and obtain the test results;

[0106] Step S509: If the current test case generation model does not meet the preset optimization target and the test results do not meet the preset test target, then optimize the current test case generation model and return to execute step S505 and subsequent steps.

[0107] Step S510: If the current test case generation model does not meet the preset optimization target, but the test result meets the preset test target, optimize the current test code generation model, regenerate the test code, and return to execute step S505 and subsequent steps.

[0108] Step S511: If the current test case generation model reaches the preset optimization target, then output the current test case generation model as the trained test case generation model.

[0109] Among them, the test code generation model and the test case generation model are built based on neural network models.

[0110] In application, the training methods provided in steps S505 to S511 can refer to the training methods provided in steps S402 to S408 above, and will not be repeated here.

[0111] like Figure 6 As shown, in one embodiment, based on Figure 5 The corresponding embodiment includes the following steps S601 to S613:

[0112] Step S601: Input the source code of the sample process and the known vulnerability information into the current test code generation model to generate test code containing at least one known vulnerability.

[0113] In application, the training method provided in step S601 is the same as the training method provided in step S201 above, and will not be repeated here.

[0114] Step S602: Compile the test code and obtain the test progress;

[0115] Step S603: If a runtime error occurs during the test process, regenerate the test code and return to execute step S602 and subsequent steps.

[0116] Step S604: If the test process runs normally, proceed to step S605 and subsequent steps;

[0117] Step S605: Determine whether the current use case generation model has reached the preset optimization target. If yes, proceed to step S613; otherwise, proceed to step S606.

[0118] Step S606: Obtain the target data types supported by the input interface of the test code;

[0119] Step S607: Input the test code and target data type into the current test case generation model to generate test cases;

[0120] In application, the training methods provided in steps S601 to S607 are the same as those provided in steps S501 to S507 above, and will not be repeated here.

[0121] Step S608: Perform fuzz testing on the test process corresponding to the test code based on the test cases to obtain the running status of the test process.

[0122] In applications, test results can include the running status of the test process. The input of fuzz testing is multiple test cases, and the output is the running status of the test process. Specifically, multiple test cases can be input one by one into the input interface of the test process corresponding to the test code, and the running status of the test process can be monitored.

[0123] Step S609: If the test process runs normally, it is determined that the test result has not reached the preset test target;

[0124] Step S610: If the test process crashes, determine that the test result has reached the preset test target.

[0125] In the application, for all test cases, after the test case inputs the test code to the input interface of the corresponding test process, if the test process runs normally, it means that the test case did not trigger the vulnerability and cause the test process to crash, and the test result is determined to have not achieved the preset test goal; if any test case causes the test process to crash, it means that any of the above test cases triggered the vulnerability and caused the test process to crash, and the test result is determined to have achieved the preset test goal.

[0126] Step S611: If the current test case generation model does not meet the preset optimization goal and the test result does not meet the preset test goal, then optimize the current test case generation model and return to execute the step of inputting the test code into the current test case generation model and subsequent steps.

[0127] Step S612: If the current test case generation model does not meet the preset optimization target, but the test result meets the preset test target, optimize the current test code generation model, regenerate the test code, and return to execute the step of inputting the test code into the current test case generation model and subsequent steps.

[0128] Step S613: If the current test case generation model reaches the preset optimization target, then output the current test case generation model as the trained test case generation model.

[0129] Among them, the test code generation model and the test case generation model are built based on neural network models.

[0130] In application, the training methods provided in steps S611 to S613 are the same as those provided in steps S509 to S511 above, and will not be repeated here.

[0131] like Figure 7 As shown, in one embodiment, based on Figure 6 The corresponding embodiment includes the following steps S701 to S714:

[0132] Step S701: Input the source code of the sample process and the known vulnerability information into the current test code generation model to generate test code containing at least one known vulnerability.

[0133] Step S702: Obtain the target data types supported by the input interface of the test code;

[0134] Step S703: Compile the test code and obtain the test progress;

[0135] Step S704: If the test process runs normally, proceed to step S707 and subsequent steps;

[0136] Step S705: If a runtime error occurs during the test process, regenerate the test code and return to execute step S703 and subsequent steps;

[0137] Step S706: Determine whether the current use case generation model has reached the preset optimization target. If yes, proceed to step S713; otherwise, proceed to step S707.

[0138] Step S707: Input the test code and target data type into the current test case generation model to generate test cases.

[0139] In application, the training methods provided in steps S701 to S707 are the same as those provided in steps S501 to S507 above, and will not be repeated here.

[0140] Step S708: Perform fuzz testing on the test process corresponding to the test code based on the test cases to obtain the running status and test time of the test process.

[0141] In application, test results can include the running status and test time of the test process. Fuzz testing takes multiple test cases as input and outputs the running status and test time of the test process. Specifically, multiple test cases can be input one by one into the input interface of the test process corresponding to the test code, monitoring the running status and test time of the test process. The start time of the test time is the input point of the test process corresponding to the first test case input, and the end time is the crash of the test process.

[0142] Step S709: If the test process runs normally, it is determined that the test result has not reached the preset test target;

[0143] Step S710: If the test process crashes and the test time is not less than the preset test time, it is determined that the test result has not reached the preset test target.

[0144] Step S711: If the test process crashes and the test time is less than the preset test time, it is determined that the test result has reached the preset test target.

[0145] In the application, for all test cases, after inputting the test code into the input interface of the corresponding test process, if the test process runs normally, it indicates that the test case did not trigger the vulnerability and cause the test process to crash, and the test result is determined to have failed to meet the preset test objective. If the test process crashes and the test time is not less than the preset test time, it indicates that a test case caused the test process to crash, but the test time is too long, and the speed at which the test case triggers the vulnerability is too slow, meaning the effectiveness of the test case is poor, and the test result is determined to have failed to meet the preset test objective. If the test process crashes and the test time is less than the preset test time, it indicates that a test case caused the test process to crash, and the test time is less than the preset test time, meaning the effectiveness of the test case is good, and the test result is determined to have met the preset test objective. The specific duration of the preset test time can be set according to the actual test scenario.

[0146] In application, by simultaneously detecting the running status of the test process and the test time during fuzz testing, it is possible to determine whether the test cases generated by the current test case generation model can cause the test process to crash. It is also possible to quantify the proportion of effective test cases generated by the current test case generation model, thereby achieving a comprehensive analysis of the performance of the current test case generation model and improving the optimization effect of the current test case generation model.

[0147] Step S712: If the current test case generation model does not meet the preset optimization goal, and the test results do not meet the preset test goal, then optimize the current test case generation model and return to execute the step of inputting the test code into the current test case generation model and subsequent steps.

[0148] Step S713: If the current test case generation model does not meet the preset optimization target, but the test result meets the preset test target, optimize the current test code generation model, regenerate the test code, and return to execute the step of inputting the test code into the current test case generation model and subsequent steps.

[0149] Step S714: If the current test case generation model reaches the preset optimization target, then output the current test case generation model as the trained test case generation model.

[0150] Among them, the test code generation model and the test case generation model are built based on neural network models.

[0151] In application, the training methods provided in steps S712 to S714 are the same as those provided in steps S508 to S510 above, and will not be repeated here.

[0152] like Figure 8 As shown, the test case generation method provided in this application embodiment includes the following step S801:

[0153] Step S801: Input the source code of the process under test into the test case generation model to obtain test cases. The test cases are used to perform fuzz testing.

[0154] The use case generation model is trained based on the training method provided in the above embodiments.

[0155] In the application, the process under test and the sample process can be the same process or different processes; specifically, they can be one or more processes within a native Android service. After obtaining the trained test case generation model, the source code of the process under test can be input into the test case generation model according to the actual testing needs to obtain at least one test case for fuzz testing.

[0156] In application, by inputting the source code of the process under test into the pre-trained test case generation model provided in the above embodiments, at least one test case can be obtained. The above at least one test case has the advantages of strong vulnerability discovery capability and strong ability to discover vulnerabilities of different depths. When performing fuzz testing, it can effectively discover vulnerabilities of different depths in the source code of the process under test, thereby improving testing efficiency.

[0157] like Figure 9 As shown, in one embodiment, based on Figure 8 The corresponding embodiment includes the following steps S901 to S903:

[0158] Step S901: Input the source code of the process to be tested into the test case generation model to obtain test cases. The test cases are used to perform fuzz testing.

[0159] In application, the test case generation method provided in step S901 can refer to the relevant description in step S801 above, and will not be repeated here.

[0160] Step S902: Perform fuzz testing on the source code of the process under test according to the test cases, and obtain the test results;

[0161] Step S903: When the process under test crashes, obtain the corresponding crash code segment and the valid test cases that cause the process under test to crash. The process under test is obtained by compiling the source code of the process under test.

[0162] In application, fuzz testing can be performed on the source code of the process under test according to the test cases to obtain the test results. The method for obtaining the test results can refer to the relevant descriptions in steps S608 or S708 above, and will not be repeated here.

[0163] In applications, test results can include the running status of the test process. When a crash is detected in the process under test, the crash code segment that caused the crash can be obtained, as well as the valid test cases that caused the crash. This allows testers to analyze the source code of the process under test based on the crash code segment and valid test cases, thereby improving the test results.

[0164] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0165] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the training method or test case generation method embodiments of the above-described use case generation models.

[0166] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0167] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] In the embodiments provided in this application, it should be understood that the disclosed terminal devices and methods can be implemented in other ways. For example, the terminal device embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0169] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A training method for a use case generation model, characterized in that, include: Input the source code of the sample process and the known vulnerability information into the current test code generation model to generate test code containing at least one known vulnerability; Input the test code into the current test case generation model to generate test cases; Perform fuzz testing on the test code based on the test cases, and obtain the test results; If the current test case generation model does not meet the preset optimization goal, and the test result does not meet the preset test goal, then after optimizing the current test case generation model, return to the step of inputting the test code into the current test case generation model and subsequent steps; If the current test case generation model does not meet the preset optimization target, and the test result meets the preset test target, the current test code generation model is optimized, the test code is regenerated, and the process returns to the step of inputting the test code into the current test case generation model and subsequent steps; when regenerating the test code, the types and / or number of known vulnerabilities contained in the regenerated test code are updated; If the current test case generation model reaches the preset optimization target, then the current test case generation model will be output as the trained test case generation model. Among them, the test code generation model and the test case generation model are built based on neural network models.

2. The training method as described in claim 1, characterized in that, The step of inputting the test code into the current test case generation model includes: Obtain the target data types supported by the input interface of the test code; Input the test code and the target data type into the current test case generation model.

3. The training method as described in claim 1, characterized in that, Before inputting the test code into the current test case generation model, the process also includes: Compile the test code and obtain the test progress; If a runtime error occurs in the test process, the test code is regenerated, and the process of compiling the test code, obtaining the test process, and subsequent steps are returned to be executed. If the test process runs normally, execute the step of inputting the test code into the current test case generation model and subsequent steps.

4. The training method as described in claim 1, characterized in that, Before inputting the test code into the current test case generation model to generate test cases, the process also includes: Obtain the number of optimizations for the current use case generation model; When the number of optimization attempts reaches a preset number of optimization attempts, it is determined that the current use case generation model has reached the preset optimization target; Alternatively, obtain the number of times the test results meet the preset test objectives; When the number of times the target is met reaches the preset number of times the target is met, it is determined that the current use case generation modulus has reached the preset optimization target.

5. The training method as described in claim 1, characterized in that, The test results include the running status of the test process; the step of performing fuzz testing on the test code based on the test cases and obtaining the test results includes: Perform fuzz testing on the test process corresponding to the test code based on the test cases to obtain the running status of the test process; If the test process runs normally, it is determined that the test result has not met the preset test target; If the testing process crashes, the test result is determined to have met the preset test target.

6. The training method as described in claim 5, characterized in that, The test results also include the test time of the test process. The step of performing fuzz testing on the test code based on the test cases to obtain test results includes: Perform fuzz testing on the test process corresponding to the test code based on the test cases to obtain the running status and test time of the test process; If the test process crashes and the test time is not less than the preset test time, the test result is determined to have failed to meet the preset test target. The statement that if the test process crashes, the test result is determined to have reached the preset test target is as follows: if the test process crashes and the test time is less than the preset test time, the test result is determined to have reached the preset test target.

7. A test case generation method, characterized in that, include: The source code of the process under test is input into the test case generation model to obtain test cases, which are used to perform fuzz testing. The use case generation model is trained based on the training method described in any one of claims 1 to 6.

8. The test case generation method as described in claim 7, characterized in that, After inputting the source code of the process under test into the test case generation model to obtain test cases, the process further includes: Perform fuzz testing on the source code of the process under test according to the test cases, and obtain the test results; When the process under test crashes, the corresponding crash code segment and the valid use cases that cause the process under test to crash are obtained. The process under test is obtained by compiling the source code of the process under test.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the training method as described in any one of claims 1 to 6, or the test case generation method as described in claim 7 or 8.

Images