LLM-based API Penetration Testing Method, Device, Medium and Product

Abstract

This application relates to the field of penetration testing technology, and in particular, to an API penetration testing method, device, medium, and product based on LLM. The method includes identifying test identifiers included in the API to be tested, and determining a corresponding target test case framework based on the test identifiers. The target test case framework is composed of the test requirements of the API to be tested and the LLM model. The target test case framework contains multiple functional nodes and the target test cases corresponding to each functional node; using the target test case framework to perform penetration testing on the API to be tested; when the penetration testing corresponding to all functional nodes is completed, counting the vulnerability penetration results corresponding to all vulnerable functional nodes, and feeding back the statistical results. A vulnerable functional node is a functional node with vulnerabilities in the penetration results. This application can improve the efficiency of performing penetration testing on APIs.

Description

Technical Field

[0001] This application relates to the field of penetration testing technology, and in particular, to an API penetration testing method, device, medium, and product based on LLM. Background Art

[0002] API (Application Programming Interface) is an application programming interface that can serve as a communication bridge between different systems or services. Through the API, different systems or services can cooperate with each other, transfer data and instructions. Therefore, the security of the API directly affects the security of the entire application system or service. If there are security vulnerabilities in the API, hackers or related malicious websites will attack the API based on the security vulnerabilities, thereby stealing sensitive data in the system or service, and may even cause significant losses to the system or service.

[0003] In related technologies, manual testing is generally used to perform penetration testing on the API to timely discover hidden security vulnerabilities in the API. However, since the API contains a large number of interfaces and functions at different levels, if all interfaces and functions need to be comprehensively covered, it may lead to a high time-consuming penetration testing process. Moreover, the error rate of the test results obtained through manual testing is also relatively high, and the penetration testing efficiency is low. Summary of the Invention

[0004] In order to improve the efficiency of penetration testing on the API, this application provides an API penetration testing method, device, medium, and product based on LLM.

[0005] In a first aspect, this application provides an API penetration testing method based on LLM, adopting the following technical solution:

[0006] An API penetration testing method based on LLM includes:

[0007] Identifying test identifiers included in the API to be tested, and determining a corresponding target test case framework based on the test identifiers. The target test case framework is composed of the test requirements of the API to be tested and the LLM model, and the target test case framework contains multiple function nodes and target test cases corresponding to each function node;

[0008] Performing penetration testing on the API to be tested by using the target test case framework;

[0009] When the penetration testing corresponding to all function nodes is completed, counting the penetration results of all vulnerable function nodes, and feeding back the statistical results. The vulnerable function nodes are function nodes with penetration results showing vulnerabilities.

[0010] By adopting the above technical solution, a target test case framework corresponding to each API to be tested is constructed through the LLM, which can cover various functions and boundary conditions in the API to be tested to a large extent. Moreover, this automatic identification of test identifiers and framework acquisition method facilitates avoiding the cumbersome process of manual search and screening, thereby improving the efficiency in the penetration testing preparation process. Additionally, during the penetration testing process, the vulnerability conditions corresponding to each functional node are monitored and analyzed in real time, and whether there are vulnerable functional nodes is monitored. This automatic statistical method can not only improve the efficiency in the penetration testing process but also reduce the error probability that may be brought by manual statistics. By automatically identifying test identifiers, obtaining test case frameworks, conducting penetration testing, and statistically analyzing and feeding back results, it is convenient to solve problems such as high time consumption and high error rate in the penetration testing process of the API to be tested, thereby improving the efficiency in the penetration testing process.

[0011] In a possible implementation manner, when performing penetration testing on the API to be tested by using the target test case framework, when a vulnerable functional node is detected, it includes:

[0012] Obtain the vulnerability penetration result corresponding to the vulnerable functional node and the associated test case corresponding to the associated functional node, where the vulnerable functional node is a functional node with a vulnerability in the penetration result;

[0013] Identify the vulnerability features included in the vulnerability penetration result and the associated preconditions of the associated test case corresponding to the associated functional node, and based on the vulnerability features and the associated preconditions, determine the target preconditions;

[0014] Determine updated test cases from a preset test case library based on the target preconditions, and perform penetration testing on the API to be tested based on the updated test cases and the target test case framework.

[0015] By adopting the above technical solution, since there may be an association relationship between each functional node in the API to be tested, once an exception occurs in the current functional node, it may affect the pre-input of the associated functional node, etc. By monitoring and analyzing the vulnerability situation corresponding to each functional node in real time, and adjusting the pre-input of the associated functional node in a timely manner to determine a new test case, rather than using the original test case after a vulnerability appears. Since the updated test case is closer to the actual security requirements, by updating the test case of the associated functional node in a timely manner, the security status of the API to be tested can be more accurately reflected, thereby facilitating the reduction of the error rate of the penetration test results. In addition, when a vulnerability appears, it is corrected in a timely manner and the updated test case is used to continue the penetration test, without the need for manual supervision and manual generation of test cases, which is convenient for reducing the duration of the entire penetration test process, thereby facilitating the improvement of the efficiency of the penetration test on the API to be tested.

[0016] In a possible implementation manner, the process of determining whether the penetration test corresponding to all functional nodes is completed includes:

[0017] Identify the position node of the current functional node in the target test case framework, and record the penetration times of the API to be tested in the target test case framework;

[0018] If the position node is a leaf node and the penetration times are consistent with the number of functional nodes included in the target test case framework, it is determined that the penetration test of the API to be tested is completed.

[0019] By adopting the above technical solution, by identifying the position node corresponding to the current functional node and the penetration times experienced by the API to be tested, jointly determine whether the penetration test of the API to be tested is completed, which is convenient for improving the accuracy of the judgment result. In addition, by using an automated identification method to judge whether the penetration test of the API to be tested is completed, it is convenient for improving the judgment efficiency, and can also reduce the possibility of manual intervention and misjudgment, thereby improving the reliability of the judgment result.

[0020] In a possible implementation manner, the formation process of the target test case framework includes:

[0021] Obtain the test requirements of the API to be tested, identify the requirement identifier corresponding to each test requirement, and perform functional division on the test requirements according to each requirement identifier to obtain at least one functional requirement group;

[0022] Import each functional requirement group into the trained LLM model to obtain the test case corresponding to each functional requirement group;

[0023] Each functional requirement group is determined as a functional node, and the association relationship between each functional node is determined based on the test requirements included in each functional requirement group;

[0024] Based on the association relationship between each functional node and the test cases corresponding to each functional node, the target test case framework is obtained.

[0025] By adopting the above technical solution, by identifying the requirement identifier corresponding to each test requirement and performing functional partitioning on the test requirements, the scattered test requirements can be organized into structured functional requirement groups, which helps to more clearly understand all aspects of the API to be tested, so as to ensure that the penetration testing work is more systematic and organized. Since the test case framework is automatically generated according to the functional requirement groups and takes into account the association relationship between functional nodes, when using the test case framework to perform penetration testing on the API to be tested, test omissions can be reduced, thereby improving the accuracy of the penetration testing results.

[0026] In a possible implementation manner, the determining the target precondition based on the vulnerability feature and the associated precondition includes:

[0027] Identify the combined position corresponding to the vulnerability feature from the associated precondition, and clear the combined position;

[0028] According to the vulnerability feature and the preset mapping relationship, determine the replacement character corresponding to the vulnerability feature, where the preset mapping relationship is the corresponding relationship between the vulnerability feature and the replacement character;

[0029] Write the replacement character into the cleared combined position to obtain the target precondition.

[0030] By adopting the above technical solution, by identifying the combined position and updating the precondition by character update at the combined position instead of regenerating a new precondition, it is convenient to improve the rate when determining the target precondition. In addition, by determining the corresponding replacement character through the vulnerability feature, it is convenient to improve the association degree between the target precondition and the associated precondition.

[0031] In a possible implementation manner, after performing penetration testing on the API to be tested based on the updated test case and the target test case framework, it further includes:

[0032] Store the updated test case and the original test case side by side in the target test case framework, and record the usage frequencies of the updated test case and the original test case within a preset time period;

[0033] If the usage frequency of the updated test case within the preset time period is higher than that of the original test case within the preset time period, update the target test case framework according to the updated test case.

[0034] By adopting the above technical solution, by storing the updated test case and the original test case side by side, it is convenient to handle different penetration test situations. After analyzing and observing the usage frequencies of the two test cases over a period of time, duplicate removal is performed on the target test case framework according to the analysis and observation results, which is convenient for improving the processing speed when using the target test case framework.

[0035] In a possible implementation manner, the method further includes:

[0036] When an access request is detected, feedback the progress display framework, where the formation process of the progress display framework includes:

[0037] Identify the function nodes that are in the penetration test stage at the current moment, and mark them in the target test case framework according to the first flag, which is used to represent that the penetration test is being performed based on the function nodes;

[0038] When it is detected that the penetration test of the function node is completed, change the first flag to the second flag, indicating that the penetration test based on the function node has been completed;

[0039] When it is detected that the function node is a vulnerable function node, determine the simulated repair duration of the vulnerable function node based on the simulated test data, and replace the first flag with the third flag, indicating that there is a vulnerability in the penetration test based on the function node, and superimpose the simulated repair duration at the third flag.

[0040] By adopting the above technical solution, through the recording and marking functions of the progress display framework, every step of the penetration test process is clearly recorded, making the penetration test process more transparent and traceable, which is helpful for subsequent problem troubleshooting and auditing work. In addition, by superimposing the simulated repair duration, it helps relevant staff to better evaluate the impact and priority of the vulnerability, so as to formulate more effective repair strategies.

[0041] In a second aspect, the present application provides an electronic device, adopting the following technical solution:

[0042] An electronic device, the electronic device includes:

[0043] At least one processor;

[0044] A memory;

[0045] At least one application program, wherein the at least one application program is stored in a memory and configured to be executed by at least one processor, and the at least one application program is configured to: execute the above-mentioned LLM-based API penetration testing method.

[0046] In a third aspect, the present application provides a computer-readable storage medium, adopting the following technical solution:

[0047] A computer-readable storage medium, comprising: a computer program stored therein that can be loaded and executed by a processor to execute the above-mentioned LLM-based API penetration testing method.

[0048] In a fourth aspect, the present application provides a computer program product, adopting the following technical solution:

[0049] A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, it implements the above-mentioned LLM-based API penetration testing method.

[0050] In summary, the present application includes at least one of the following beneficial technical effects:

[0051] By constructing a target test case framework corresponding to each API to be tested through LLM, it can cover various functions and boundary conditions in the API to be tested to a large extent, and this automated method of identifying test identifiers and obtaining frameworks facilitates avoiding the cumbersome process of manual search and screening, thus facilitating improving the efficiency in the penetration testing preparation process. Additionally, during the penetration testing process, it also monitors and analyzes the vulnerability conditions corresponding to each functional node in real time and monitors whether there are vulnerable functional nodes. This automated statistical method can not only improve the efficiency during the penetration testing process but also reduce the error probability that may be brought by manual statistics. By automatically identifying test identifiers, obtaining test case frameworks, conducting penetration testing, and statistically analyzing and feeding back results, it facilitates solving problems such as high time consumption and high error rate during the penetration testing process of the API to be tested, thus facilitating improving the efficiency during the penetration testing process.

[0052] Since there may be correlation relationships among the various functional nodes in the API to be tested, once an exception occurs in the current functional node, it may affect the pre-inputs of the associated functional nodes, etc. By monitoring and analyzing the vulnerability situation corresponding to each functional node in real time, and adjusting the pre-inputs of the associated functional nodes in a timely manner to determine new test cases, rather than using the original test cases after vulnerabilities occur. Since the updated test cases are closer to the actual security requirements, by updating the test cases of the associated functional nodes in a timely manner, the security status of the API to be tested can be more accurately reflected, thus facilitating the reduction of the error rate of the penetration test results. In addition, when a vulnerability occurs, it is corrected in a timely manner and the updated test cases are used to continue the penetration test, without the need for manual supervision and manual generation of test cases, which is convenient for reducing the duration of the entire penetration test process, thus facilitating the improvement of the efficiency of the penetration test for the API to be tested. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a schematic flowchart of a method for API penetration testing based on LLM in an embodiment of the present application;

[0054] Figure 2 is a schematic flowchart of a method for updating test cases in an embodiment of the present application;

[0055] Figure 3 is a schematic structural diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] The following further describes the present application in detail Figures 1-3 in conjunction with the accompanying drawings.

[0057] Those skilled in the art can make modifications to this embodiment without creative contributions according to their needs after reading this specification, but as long as they are within the scope of the claims of the present application, they are protected by the Patent Law.

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0059] Specifically, the embodiment of the present application provides an LLM-based API penetration testing method, which is executed by an electronic device. The electronic device can be a server or a terminal device. Among them, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected through wired or wireless communication methods, and the embodiment of the present application does not make any restrictions here.

[0060] Reference Figure 1 , Figure 1 FIG. is a schematic flowchart of an LLM-based API penetration testing method in an embodiment of the present application. The method includes steps S110-S130, where:

[0061] Step S110: Identify the test identifiers included in the API to be tested, and determine the corresponding target test case framework based on the test identifiers. The target test case framework is composed of the test requirements of the API to be tested and the LLM model. The target test case framework includes multiple function nodes and the target test cases corresponding to each function node.

[0062] Specifically, the test identifier can be a character or a code. The specific identifier is not specifically limited in the embodiment of the present application and can be set by relevant technical personnel. Since the content and functions to be tested for different APIs to be tested may be different, in the embodiment of the present application, a corresponding target test case framework is set for each API to be tested to cope with the penetration testing situations that different APIs to be tested may encounter. The test case framework includes multiple function nodes, and each function node corresponds to a test case for performing penetration testing on the API to be tested regarding the corresponding function node. The test case is a set of data composed of elements such as test input, execution conditions, and expected results to complete a specific requirement or target test.

[0063] The LLM (Large Language Model) model, also known as a large language model or a large model, is a language model constructed using artificial neural networks. Through self-supervised learning or semi-supervised learning, it can train the requirement text to obtain test cases corresponding to the requirement text, which can greatly reduce the time for manually writing test cases and thus facilitate improving the efficiency of penetration testing preparation work.

[0064] Furthermore, the formation process of the target test case framework can specifically include:

[0065] Obtain the test requirements for the API to be tested, identify the requirement identifier corresponding to each test requirement, and divide the test requirements according to each requirement identifier to obtain at least one functional requirement group, where each functional requirement group contains at least two test requirements; import each functional requirement group into the trained LLM model to obtain the test cases corresponding to each functional requirement group; determine each functional requirement group as a functional node, and determine the association relationship between each functional node based on the test requirements included in each functional requirement group; obtain the target test case framework according to the association relationship between each functional node and the test cases corresponding to each functional node.

[0066] Specifically, the test requirements for the API to be tested are the directions for penetration testing of the API to be tested. The test requirements may include: functional testing, that is, verifying the requests and responses of the API to be tested to ensure that the returned data format and structure are consistent with the expectations, and verifying whether the business logic of the API to be tested is correct; performance testing, that is, testing the response time, throughput, concurrency, etc. of the API to be tested under different loads and conditions to ensure that the API to be tested can meet the performance requirements of the actual application scenario; security testing, that is, testing the potential security risks of the API to be tested, such as input validation, authorization, access control, etc. to ensure that the API to be tested can resist common security attacks, such as SQL injection, cross-site scripting attack (XSS), etc.; boundary value testing, that is, testing the boundary value situation of the parameters accepted by the API to be tested. For example, when the passed parameter is large enough or negative, whether the interface can be processed normally. The test requirements corresponding to different APIs to be tested may be different. For example, the test requirements may also include stability testing, compatibility testing, document consistency testing, etc. The specific content of the test requirements is not specifically limited in the embodiments of the present application. Different test requirements correspond to a requirement identifier, and the corresponding test requirements can be located according to the requirement identifier corresponding to the API to be tested. The same API to be tested may contain multiple requirement identifiers, and the specific number of requirement identifiers is not specifically limited in the embodiments of the present application.

[0067] Requirements with different identifiers may belong to the same type of function. For example, fault tolerance processing tests and compatibility tests can be classified as the same type of function, such as quality assurance functions. Requirements identifiers belonging to the same type of function have the same identifying characters. By identifying the identifying characters in different requirement identifiers, multiple test requirements can be functionally classified to obtain at least one functional requirement group. A functional requirement group may contain one test requirement or multiple test requirements. The number of test requirements contained in a functional requirement group is not specifically limited in the embodiments of the present application. All test requirements contained in a functional requirement group are imported as a whole requirement into a trained LLM model to obtain corresponding test cases. A test case is a set of elements consisting of test inputs, execution conditions, expected results, etc., for data to complete the penetration test of a specific requirement or target. A test case can describe the penetration test steps of the API to be tested based on the functional requirement group in the form of a document, so as to perform a penetration test on a certain function in the API to be tested to determine whether a certain function in the API to be tested meets a specific requirement. The LLM model is obtained by training with a large amount of sample data, and the large amount of sample data contains a large number of test requirements and manually generated test cases.

[0068] Each functional requirement group corresponds to the same or the same type of function. There may be an association relationship between the functions corresponding to different functional requirement groups. For example, functional requirement group a needs to perform a penetration test on the security of the login process, and functional requirement group b needs to perform a penetration test on the security of the data area accessible by the logged-in user. Since the premise for a logged-in user to access data is successful login, and there is a corresponding relationship between the login identity and the access permission, it can be determined that there is an association relationship between functional requirement group a and functional requirement group b. It is possible to determine whether there is an association relationship between a functional requirement group and other functional requirement groups by identifying whether the test requirements included in each functional requirement group contain an association identifier. If an association identifier is included, other functional requirement groups with an association relationship can be determined based on the association mapping relationship. The number of functional requirement groups having an association relationship with the same requirement functional group may be 0, 1, or multiple. Each functional requirement group is determined as a functional node, and the functional nodes with an association relationship are connected, and finally the target test case framework corresponding to the API to be tested can be obtained.

[0069] By identifying the requirement identifier corresponding to each test requirement and functionally classifying the test requirements, the scattered test requirements can be organized into structured functional requirement groups, which helps to more clearly understand all aspects of the API to be tested to ensure that the penetration test work is more systematic and well-organized.

[0070] Step S120: Use the target test case framework to perform a penetration test on the API to be tested.

[0071] Specifically, locate the starting function node from the target test case framework, obtain the starting test case corresponding to the starting function node, and perform penetration testing on the API to be tested based on the starting function node using this starting test case. After the penetration testing based on the starting function node is completed, it is possible to automatically transfer to the next function node according to the association relationship between the function nodes and execute the penetration testing based on the next function node. For example, if the current function node is the penetration testing of the login process, the corresponding test case may be to imitate the attack methods of hackers or malicious websites, bypass the defense measures of the API to be tested for the login process, directly bypass the identity authentication of the API to be tested, and forcibly access the API to be tested. If the response result given by the API to be tested is the same as the expected result corresponding to the test case, it indicates that the penetration testing of the current function node has been completed.

[0072] Step S130: When the penetration testing corresponding to all function nodes is completed, count the penetration results of the vulnerable function nodes corresponding to all vulnerabilities and provide feedback on the statistical results. A vulnerable function node is a function node with vulnerabilities in the penetration results.

[0073] Specifically, if the response result given by the API to be tested is different from the expected result corresponding to the test case, it indicates that there are vulnerabilities in the current function node, and the current function node is determined to be a vulnerable function node. Since there is an association relationship between function nodes, and the preconditions included in the test cases corresponding to the next function node may be related to the penetration results of the current function node, if there are vulnerable function nodes, it may affect the penetration testing effect of the next function node. To reduce the error rate of penetration results, when a vulnerable function node is detected, steps S1 - S3 are also included, as Figure 2 shown, where:

[0074] Step S1: Obtain the penetration results of the vulnerable function node corresponding to the vulnerability and the associated test cases corresponding to the associated function nodes. A vulnerable function node is a function node with vulnerabilities in the penetration results.

[0075] Specifically, the penetration results of the vulnerable function node contain specific vulnerability situations. For example, when the vulnerable function node is the penetration testing of the login process, the specific vulnerability situation can be the identity authentication defense vulnerability, that is, hackers or malicious websites can bypass the identity authentication defense operation of the API to be tested and forcibly access the API to be tested. At this time, the API to be tested may face the situation of sensitive data being stolen or tampered with.

[0076] Step S2: Identify the vulnerability characteristics included in the penetration results of the vulnerability and the associated preconditions of the associated test cases corresponding to the associated function nodes, and determine the target preconditions based on the vulnerability characteristics and the associated preconditions.

[0077] Step S3: Determine updated test cases from a preset test case library based on the target preconditions, and perform penetration testing on the API to be tested based on the updated test cases and the target test case framework.

[0078] Specifically, the vulnerability characteristics corresponding to the specific vulnerability situation can be identified from the vulnerability penetration results. According to the vulnerability characteristics, the associated preconditions corresponding to the associated function nodes can be updated. The updated associated preconditions are more adaptable to the actual penetration testing situation. After updating the associated preconditions, the associated test cases are updated.

[0079] After a vulnerability appears, it is corrected in a timely manner and the penetration testing continues with the updated test cases, without the need for manual supervision and manual generation of test cases, which helps to reduce the time taken for the entire penetration testing process, thereby improving the efficiency of penetration testing the API to be tested.

[0080] If, during the penetration testing of the current function node, the response result given by the API to be tested is the same as the expected result corresponding to the test case, it indicates that the penetration testing of the current function node has been completed; if, during the penetration testing of the current function node, the response result given by the API to be tested is different from the expected result corresponding to the test case, it also indicates that the penetration testing of the current function node has been completed, and at the same time, the associated test cases of the associated function nodes need to be updated based on the vulnerability penetration results of the current function node.

[0081] After the penetration testing corresponding to all function nodes is completed, the vulnerability penetration results of all vulnerable function nodes are integrated and fed back. The number of vulnerable function nodes in the feedback content is not specifically limited in the embodiments of the present application.

[0082] By using an LLM to construct the target test case framework for each API to be tested, it is possible to cover various functions and boundary conditions in the API to be tested to a large extent. Moreover, this automated method of identifying test identifiers and obtaining frameworks helps to avoid the cumbersome process of manual search and screening, thereby improving the efficiency in the penetration testing preparation process. Additionally, during the penetration testing process, the vulnerability situation corresponding to each function node is monitored and analyzed in real time, and whether there are vulnerable function nodes is monitored. This automated statistical method can not only improve the efficiency in the penetration testing process but also reduce the error probability caused by manual statistics. By automatically identifying test identifiers, obtaining test case frameworks, performing penetration testing, and statistically analyzing and feeding back results, it helps to solve problems such as high time consumption and high error rates in the penetration testing process of the API to be tested, thereby improving the efficiency in the penetration testing process.

[0083] Furthermore, to improve the accuracy of the judgment results, the process of determining whether the penetration testing corresponding to all function nodes has been completed may specifically include:

[0084] Identify the location node of the current functional node in the target test case framework, and record the penetration times of the API to be tested in the target test case framework; if the location node is a leaf node and the penetration times are the same as the number of functional nodes included in the target test case framework, it is determined that the API to be tested has completed the penetration test.

[0085] Specifically, the target test case framework includes starting functional nodes and leaf functional nodes. Penetration testing can be performed based on multiple functional nodes in the target test case framework simultaneously, or one by one. The specific penetration testing order is not specifically limited in the embodiments of the present application. Therefore, simply determining whether the current functional node is a leaf node cannot determine whether the API to be tested has been penetration-tested based on all functional nodes in the target test case framework.

[0086] In the embodiments of the present application, by identifying whether the current functional node is a leaf node and recording the penetration times of the API to be tested based on the target test case framework, and jointly judging whether the API to be tested has completed the penetration test according to the location node corresponding to the current functional node and the penetration times experienced by the API to be tested, it is convenient to improve the accuracy of the judgment result.

[0087] Furthermore, in order to improve the rate of determining the target preconditions, based on the vulnerability characteristics and associated preconditions, the target preconditions are determined, which may specifically include:

[0088] Identify the combined position corresponding to the vulnerability characteristics from the associated preconditions and clear the combined position; determine the replacement character corresponding to the vulnerability characteristics according to the vulnerability characteristics and the preset mapping relationship, where the preset mapping relationship is the corresponding relationship between the vulnerability characteristics and the replacement characters; write the replacement character into the cleared combined position to obtain the target preconditions.

[0089] Specifically, the content of the preconditions varies depending on the different penetration testing scenarios of the API to be tested. However, it usually involves a description of the conditions that must be met before performing a certain penetration testing content, or a combination of conditions that must be satisfied. For example, before performing the penetration testing corresponding to functional node b, the combination of conditions that need to be satisfied can be "xxx, meet condition 1, meet condition 2", where "xxx" can be the penetration test result of the previous functional node a. When the previous functional node a is a vulnerable functional node, functional node b can be regarded as an associated functional node of functional node a, and the "preconditions xxx, meet condition 1, meet condition 2" corresponding to functional node b can be used as the associated preconditions. At this time, the associated preconditions need to be adjusted. Through the vulnerability characteristics, it can be determined that the combination position that needs to be adjusted is the first position, that is, the position of "xxx" in the associated preconditions. There is a corresponding position identifier for each combination position in the associated preconditions. By matching the position identifier with the vulnerability characteristics, the combination position of the vulnerability characteristics in the associated pre - combination can be determined. The corresponding relationship between the vulnerability characteristics and the position identifiers can be determined by relevant staff according to historical penetration testing experience and uploaded to the electronic device in advance. The specific content of this corresponding relationship is not specifically limited in the implementation of this application and can be adjusted and modified by relevant staff according to actual penetration testing experience.

[0090] There is also a corresponding relationship between the vulnerability characteristics and the replacement characters. According to the vulnerability characteristics, the corresponding replacement characters can be uniquely identified. The specific content of the replacement characters is not specifically limited in the application embodiments. As long as it can represent that the precondition content at the corresponding combination position of the vulnerability characteristics is abnormal or has a vulnerability, writing the replacement characters into the corresponding combination position of the vulnerability characteristics can update the associated preconditions. The combination position can be cleared first and then the replacement characters can be written, or the replacement characters can directly overwrite the content corresponding to the original combination position. The specific method is not specifically limited in the application embodiments. By identifying the combination position and updating the preconditions by character update at the combination position instead of regenerating new preconditions, it is convenient to improve the rate of determining the target preconditions.

[0091] Furthermore, in order to improve the processing rate when using the target test case framework, in the embodiments of this application, after performing penetration testing on the API to be tested based on the updated test case and the target test case framework, it further includes:

[0092] Storing the updated test case and the original test case side by side in the target test case framework and recording the usage frequencies of the updated test case and the original test case within a preset time period; if the usage frequency of the updated test case within the preset time period is higher than the usage frequency of the original test case within the preset time period, then update the target test case framework according to the updated test case.

[0093] Specifically, the updated test case is the test case updated based on the vulnerability feature. The original test case is the test case containing the vulnerability feature content. The data storage location of the original test case can be identified and confirmed in the target test case, and a parallel storage area is added based on this data storage location. The updated test case is written into the parallel storage area, rather than emptying the test case after using the updated test case or directly replacing the original test case with the updated test case. Instead, the deletion item is determined based on the usage conditions of the updated test case and the original test case in a future period of time.

[0094] Since the same API to be tested may need to be penetration-tested repeatedly for multiple times, when the penetration requirements remain unchanged, the same target test case can be used for penetration testing. The preset time period is the early stage of the corresponding time period when the API to be tested is penetration-tested repeatedly. For example, when the corresponding time period for the API to be tested to be penetration-tested repeatedly is 10:00 - 10:30, the preset time period can be 10:00 - 10:08. The penetration testing requirements and methods for different APIs to be tested may be different. Therefore, the preset time periods corresponding to different APIs to be tested are also different, and the specific duration of the preset time period is not specifically limited in the embodiments of the present application.

[0095] Statistically analyze the usage frequency a of the updated test case and the usage frequency b of the original test case within the preset time period. When the usage frequency a is higher than the usage frequency b, it indicates that the updated test case is more adaptable to the penetration testing situation in a future period of time. At this time, the original test case can be directly removed from the target test case framework, and the updated test case is retained. After analyzing and observing the usage frequencies of the two test cases for a period of time, duplicate removal is performed on the target test case framework according to the analysis and observation results, which is convenient for improving the processing speed when using the target test case framework.

[0096] Furthermore, in order to make the penetration testing process more transparent and traceable, the method provided in the embodiments of the present application further includes:

[0097] When an access request is detected, feedback is given to the progress display framework. The formation process of the progress display framework includes:

[0098] Identify the function nodes that are currently in the penetration testing stage at the current moment, and mark them in the target test case framework according to the first flag, which is used to indicate that penetration testing is being performed based on the function nodes; when it is detected that the penetration testing of the function node is completed, the first flag is replaced with the second flag, indicating that the penetration testing based on the function node has been completed; when it is detected that the function node is a vulnerability function node, determine the simulated repair duration of the vulnerability function node based on the simulated test data, and replace the first flag with the third flag, indicating that there is a vulnerability in the penetration testing based on the function node, and the simulated repair duration is superimposed on the third flag.

[0099] Specifically, the specific first flag, second flag, and third flag are not specifically limited in the embodiments of the present application. The first flag is used to indicate that a penetration test is being performed based on a certain function node at the current moment, and a yellow identifier can be used to mark the position of the function node in the target test case framework in yellow; the second flag is used to indicate that the penetration test based on a certain function node has been completed and there are no vulnerabilities in the penetration process, and a green identifier can be used to change the position of the function node from yellow to green in the target test case framework; the third flag is used to indicate that the penetration test based on a certain function node has been completed, but there are vulnerabilities in the penetration process, and a red identifier can be used to change the position of the function node from yellow to red in the target test case framework, so as to obtain a progress display framework, and the progress display framework is the target test case framework including the first flag, the second flag, and the third flag. By observing different flags in the progress display framework, it is possible to intuitively view the test stage of the API to be tested during the penetration test, and it is also possible to intuitively view the penetration results when performing penetration tests based on different function nodes.

[0100] The simulated test data includes the time required for simulating the repair of vulnerabilities when different vulnerable function nodes appear. When a vulnerable function node appears, the simulated repair duration for repairing the current vulnerable function node is determined based on the simulated test data, and the simulated repair duration is superimposed on the current vulnerable function node based on AR technology, which helps relevant staff better evaluate the impact and priority of the vulnerabilities, so as to formulate more effective repair strategies. The display form of the simulated repair duration can be "There is a vulnerability, and the repair duration is xx seconds", and the specific display form is not specifically limited in the embodiments of the present application. If the vulnerability repair is not completed within the simulated repair duration, a maintenance prompt message will be generated according to the vulnerable function node, and the generated maintenance prompt message will be sent to the maintenance queue for maintenance queuing. The maintenance queue is a storage space for storing maintenance prompt messages, and the messages are sorted in the maintenance queue according to the duration between the end time of the repair duration and the current time. For example, the repair duration of maintenance prompt message 1 ends at 10:00, and the repair duration of maintenance prompt message 2 ends at 10:20. In the maintenance queue, maintenance prompt message 1 is located before maintenance prompt message 2. When the vulnerable function node corresponding to the maintenance prompt message is a key function node, queuing jump processing can be performed according to the importance level of the function node, that is, the vulnerability repair of the vulnerable function node with a higher importance level is preferentially performed.

[0101] Since penetration testing can be performed simultaneously based on multiple functional nodes in the target test case framework at the same time, that is, penetration testing in multiple functional directions can be performed on the API to be tested simultaneously, there may be multiple functional nodes identified with the first flag in the target test case framework at the same time. When there are multiple vulnerability functional nodes identified with the third flag in the target test case framework, the node positions of each vulnerability functional node in the target test case framework can be identified, and based on each node position, the abnormal area can be determined to judge whether multiple vulnerability functional nodes are at the same level. If they are at the same level, an abnormal warning instruction is generated. The target test case framework can be regarded as a node tree, and this node tree is a set with a hierarchical relationship composed of n (n>0) finite functional nodes. In this set, each functional node has a specific level, and the level is determined according to the relative position between each functional node and the starting functional node. The starting functional node is located in the first layer, and the child nodes of the starting functional node are located in the second layer, and so on. When multiple vulnerability functional nodes are all at the same level, the penetration results generated after penetration testing based on multiple vulnerability functional nodes may have a concentrated and significant impact on the next-level nodes. This concentrated impact may cause the penetration testing process of the API to be tested to be severely restricted or the test to be abnormal. Therefore, when it is detected that multiple vulnerability functional nodes are at the same level, an abnormal warning signal needs to be generated immediately to remind relevant staff to adjust the penetration testing process in time.

[0102] An electronic device is provided in an embodiment of the present application, such as Figure 3 shown Figure 3 The electronic device 300 shown includes: a processor 301 and a memory 303. Among them, the processor 301 and the memory 303 are connected, such as connected through a bus 302. Optionally, the electronic device 300 may further include a transceiver 304. It should be noted that in practical applications, the transceiver 304 is not limited to one, and the structure of the electronic device 300 does not constitute a limitation to the embodiment of the present application.

[0103] The processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules, and circuits described in connection with the disclosure of this application. The processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0104] The bus 302 may include a path for transmitting information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 only one line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0105] The memory 303 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or it may also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0106] The memory 303 is used to store the application program code for executing the solution of this application, and is controlled by the processor 301 to execute. The processor 301 is used to execute the application program code stored in the memory 303 to implement the content shown in the foregoing method embodiments.

[0107] Among them, the electronic device includes but is not limited to: mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Figure 3 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of this application.

[0108] The embodiments of this application provide a computer-readable storage medium, on which a computer program is stored. When it runs on a computer, it enables the computer to execute the corresponding content in the foregoing method embodiments.

[0109] The embodiments of this application provide a computer program product, which includes a computer program that implements the method in any of the above embodiments when executed by a processor.

[0110] It should be understood that although the steps in the flowchart of the accompanying drawings are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0111] The above are only some embodiments of this application. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of this application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of this application.

Claims

1. An API penetration testing method based on LLM, characterized in that, Including: Identifying the test identifiers included in the API to be tested, and determining the corresponding target test case framework based on the test identifiers. The target test case framework is composed of the test requirements of the API to be tested and the LLM model, and the target test case framework includes multiple function nodes and the target test cases corresponding to each function node; Performing penetration testing on the API to be tested using the target test case framework; When the penetration testing corresponding to all function nodes is completed, counting the vulnerability penetration results corresponding to all vulnerable function nodes and feeding back the statistical results. The vulnerable function nodes are the function nodes with vulnerabilities in the penetration results; Among them, when performing penetration testing on the API to be tested using the target test case framework, when a vulnerable function node is detected, it includes: Obtaining the vulnerability penetration result corresponding to the vulnerable function node and the associated test case corresponding to the associated function node. The vulnerable function node is the function node with a vulnerability in the penetration result; Identifying the vulnerability characteristics included in the vulnerability penetration result, and identifying the associated preconditions of the associated test case corresponding to the associated function node, and determining the target preconditions based on the vulnerability characteristics and the associated preconditions; Determining the updated test case corresponding to the associated function node from the preset test case library based on the target preconditions, and performing penetration testing on the API to be tested based on the updated test case and the target test case framework; Among them, the formation process of the target test case framework includes: Obtaining the test requirements of the API to be tested, identifying the requirement identifiers corresponding to each test requirement, and functionally dividing the test requirements according to each requirement identifier to obtain at least one function requirement group; Importing each function requirement group into the trained LLM model to obtain the test cases corresponding to each function requirement group; Determining each function requirement group as a function node, and determining the association relationship between each function node based on the test requirements included in each function requirement group. Judging whether the test requirements included in each function requirement group contain an association identifier. If an association identifier is included, other function requirement groups associated with the function requirement group can be determined based on the association mapping relationship; Obtaining the target test case framework according to the association relationship between each function node and the test cases corresponding to each function node; Among them, determining the target preconditions based on the vulnerability characteristics and the associated preconditions includes: Identifying the combined position corresponding to the vulnerability characteristics from the associated preconditions and clearing the combined position; Determining the replacement character corresponding to the vulnerability characteristics according to the vulnerability characteristics and the preset mapping relationship. The preset mapping relationship is the corresponding relationship between the vulnerability characteristics and the replacement characters; Writing the replacement character into the cleared combined position to obtain the target preconditions.

2. The API penetration testing method based on LLM according to claim 1, wherein, The process of judging whether the penetration testing corresponding to all function nodes is completed includes: Identifying the position node of the current function node in the target test case framework and recording the penetration times of the API to be tested in the target test case framework; If the position node is a leaf node and the number of penetration times is the same as the number of function nodes included in the target test case framework, it is determined that the API to be tested has completed the penetration test.

3. The API penetration testing method based on LLM according to claim 1, wherein It further includes: When an access request is detected, feedback is provided to the progress display framework, where the formation process of the progress display framework includes: Identifying the function nodes that are currently in the penetration test stage and marking them in the target test case framework according to the first flag, which is used to represent that the penetration test is being performed based on the function nodes; When it is detected that the function node has completed the penetration test, the first flag is replaced with a second flag, representing that the penetration test based on the function node has been completed; When it is detected that the function node is a vulnerable function node, the simulated repair duration of the vulnerable function node is determined based on the simulated test data, and the first flag is replaced with a third flag, representing that there is a vulnerability in the penetration test based on the function node, and the simulated repair duration is added to the third flag.

4. The API penetration testing method based on LLM according to claim 2, characterized in that, After performing the penetration test on the API to be tested based on the updated test case and the target test case framework, it further includes: Storing the updated test case and the original test case side by side in the target test case framework and recording the usage frequencies of the updated test case and the original test case within a preset time period; If the usage frequency of the updated test case within the preset time period is higher than the usage frequency of the original test case within the preset time period, the target test case framework is updated according to the updated test case.

5. An electronic device, characterized in that, The electronic device includes: At least one processor; A memory; At least one application program, where the at least one application program is stored in the memory and is configured to be executed by the at least one processor, and the at least one application program is configured to: execute an API penetration test method according to any one of claims 1-4 based on LLM.

6. A computer-readable storage medium, characterized in that, It includes: A computer program stored that can be loaded and executed by a processor to perform an API penetration test method according to any one of claims 1-4 based on LLM.

7. A computer program product, characterized in that, It includes a computer program, and when the computer program is executed by a processor, it implements the steps of an API penetration test method according to any one of claims 1-4 based on LLM.

Images