Automated penetration testing method and system based on network target range

Abstract

The present invention provides an automated penetration testing method and system based on a network range, the method comprising the following steps: obtaining the running configuration information of the service to be tested, and using the recognition decision tree to classify it, sending the running configuration information of different categories to the corresponding vulnerability mining module for processing according to the recognition result, mining potential vulnerabilities and building a basic range; constructing a test plan based on the mined potential vulnerabilities and the acquired running configuration information; executing the test plan, and judging whether the test passes. The present invention saves time for repeated mining of repeated vulnerabilities through automated vulnerability mining and network range technology, and can conduct multiple groups of tests in different environments in parallel, uninterruptedly, and for a long time, and quickly discover security issues.

Description

Technical Field

[0001] The present invention relates to automated penetration testing, and in particular to an automated penetration testing method and system based on a network target range. Background Art

[0002] Security testing is always required before a service goes online, and the testing involves many aspects. Testing by network security personnel alone is very dependent on the experience and ability of the personnel, and usually cannot fully discover problems. At the same time, the development of technology and modularization has made the technology stack used by application services very complicated. Languages, middleware, containers, operating systems, service management methods, etc. will be different, and of course there will be many differences in their combination methods. Therefore, it will become increasingly difficult for manual testing to conduct comprehensive analysis and testing. Vulnerability mining, including code auditing, has developed to the point where we have accumulated technical experience for automation, and the development of hardware performance is sufficient to support the construction and execution of comprehensive automated penetration testing solutions.

[0003] Currently, automated penetration testing solutions require personnel to have sufficient security skills, and testing in a real machine operating environment may cause services to gradually become unavailable due to test data and various penetration operations.

[0004] In order to solve the above problems, people have been seeking an ideal technical solution. Summary of the invention

[0005] The purpose of the present invention is to address the deficiencies in the prior art and thus provide an automated penetration testing method and system based on a network target range.

[0006] In order to achieve the above object, the present invention provides a first aspect of an automated penetration testing method based on a network range, comprising the following steps:

[0007] Obtain the running configuration information of the service to be tested, use the recognition decision tree to identify and classify the running configuration information, and send different categories of running configuration information to the corresponding vulnerability mining module for processing, potential vulnerabilities and building a basic target range according to the recognition results;

[0008] Constructing a test plan based on the mined potential vulnerabilities, manually input operation configuration information of the service to be tested, and detected operation configuration information of the service to be tested, wherein the test plan includes a test script and a script execution environment construction plan;

[0009] Generate the required target range test scenario according to the script execution environment construction plan, upload the test script to the target test scenario node in the network target range, execute the test script, and input the script execution result into the recursive neural network to determine whether the test passes.

[0010] A second aspect of the present invention provides an automated penetration testing system based on a network range, comprising:

[0011] The information acquisition module is used to obtain the running configuration information of the service to be tested;

[0012] The recognition and classification module uses the recognition decision tree to recognize and classify the operation configuration information, and sends different categories of operation configuration information to the corresponding vulnerability mining module for processing according to the recognition results to mine potential vulnerabilities;

[0013] A test plan construction module is used to construct a test plan based on the potential vulnerabilities discovered, the manually input operation configuration information of the service to be tested, and the detected operation configuration information of the service to be tested, wherein the test plan includes a test script and a script execution environment construction plan;

[0014] The network range generation module is used to generate the basic range based on the operation configuration information and generate the required range test scenarios according to the script execution environment construction plan;

[0015] The test execution module uploads the test script to the target test scenario node in the network target range and executes the test script;

[0016] The result analysis module inputs the script execution results into the recursive neural network to determine whether the test passes.

[0017] Based on the above, the vulnerability mining module includes:

[0018] The event-based vulnerability mining module is used to match event-based vulnerabilities with test cases and / or interface documents and mine possible event-based vulnerabilities.

[0019] The general vulnerability matching module is used to match the technology stack information with general vulnerabilities based on the vulnerability library and discover possible general vulnerabilities.

[0020] Code audit module: used to reverse engineer the program package to obtain the program code and source code for auditing, to find out program errors, loopholes and violations of program specification codes;

[0021] The port scanning module is used to perform port scanning on the established basic target range, detect whether the final application service environment is consistent with the service and network configuration of the acquired application program, and supplement the running configuration information of the service to be tested that has not been manually entered.

[0022] The present invention has outstanding substantive features and significant progress compared to the prior art, specifically:

[0023] The present invention classifies the running configuration information of the service to be tested by identifying the classification tree, and calls the vulnerability library, code audit and port scanning modules according to the classification results to automatically mine vulnerabilities, which can not only provide a certain guarantee for the comprehensiveness of the automated penetration test, but also save the time for repeated mining of repeated vulnerabilities and save the consumption of human resources;

[0024] Introducing network target range technology, building a basic target range, and generating test scripts and corresponding target range test scenarios based on different vulnerability mining results, and conducting tests in multiple groups of different environments in parallel, uninterruptedly, and for a long time will help discover more security issues more quickly.

[0025] The detection module supplements the manually input data, so that even when personnel do not have sufficient security technology accumulation, they can still conduct more complete security tests on the developed applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is further described in detail below through specific implementation methods.

[0028] Example 1

[0029] like Figure 1 As shown, the present invention proposes an automated penetration testing method based on a network target range, comprising the following steps:

[0030] Obtain the running configuration information of the service to be tested, use the recognition decision tree to identify and classify the running configuration information, and send different categories of running configuration information to the corresponding vulnerability mining module for processing based on the recognition results, build a basic target range, and mine potential vulnerabilities;

[0031] Constructing a test plan based on the potential vulnerabilities discovered and the operation configuration information obtained, wherein the test plan includes a test script and a script execution environment construction plan;

[0032] Generate the required target range test scenario according to the script execution environment construction plan, upload the test script to the target test scenario node in the network target range, execute the test script, and input the script execution result into the recursive neural network to determine whether the test passes.

[0033] In specific implementation, the script execution results obtained after the script is executed include but are not limited to response data, console output, logs, files, and network data packets; after obtaining the script execution results, a recursive neural network is used to classify the script execution results, which can detect the test pass status and provide the service's vulnerabilities and potential vulnerability sets.

[0034] Specifically, the running configuration information of the service to be tested includes the technology stack information used, the service and configuration information of the application, existing test cases, source code, and program packages; among them, the technology stack information includes but is not limited to programming languages, middleware, databases, operating systems, etc.; the service and configuration information of the application includes but is not limited to operating system images, open ports, program paths, network description information, middleware deployment strategies, security strategies, interface documents, file path sets that should be exposed, service lists, etc.

[0035] In the specific implementation, the running configuration information of the service to be tested can be input by the user, or the detection module can detect the running configuration information that is not explicitly stated in the service to be tested, such as technology stack information, open ports, etc., based on the running configuration information input by the user, thereby supplementing and improving the manual input, so that when personnel do not have sufficient security technology accumulation, they can also perform more complete security testing on the developed application.

[0036] In this embodiment, the running configuration information of the service to be tested includes technology stack information, program packages, source code, test cases, interface documents, open ports, service lists and network description information. According to the identification results, different types of running configuration information are sent to the corresponding vulnerability mining module for processing. The specific steps of mining potential vulnerabilities are as follows:

[0037] Reverse engineer the identified program packages to obtain program codes, and send the program codes to the code audit module for auditing to find program errors, vulnerabilities, and violations of program specifications;

[0038] Send the source code to the code audit module for auditing to find out program errors, vulnerabilities and violations of program specifications;

[0039] A basic target range is established based on the identified network description information and service list, wherein the basic target range includes basic service nodes and middleware nodes required for the service, and no additional network environment is configured; a port scan is performed on the basic target range according to the network description information and service list to detect the security of the service environment; specifically, whether the open port of the final application service environment is consistent with the open port in the running configuration information of the service to be tested, if not, the service environment is insecure;

[0040] Send the identified technology stack information to the general vulnerability matching module, and mine possible general vulnerabilities based on the vulnerability library;

[0041] The identified test cases and interface documents are sent to the event-type vulnerability matching module to mine possible event-type vulnerabilities.

[0042] The present invention classifies the running configuration information of the service to be tested by identifying a classification tree, and calls the vulnerability library, code auditing and port scanning modules to perform automatic vulnerability mining according to the classification results. This can not only provide a certain guarantee for the comprehensiveness of automated penetration testing, but also save time for repeated mining of repeated vulnerabilities, while saving the consumption of human resources.

[0043] It is understandable that when the running configuration information of the service to be tested does not include an open port, the service and configuration information of the incomplete application program can be supplemented according to the port scanning result.

[0044] After vulnerability mining, a test plan is constructed based on the mined potential vulnerabilities, the manually input running configuration information of the service to be tested, and the detected running configuration information of the service to be tested. The test plan includes a test script and a script execution environment construction plan; wherein the test script generation steps are as follows:

[0045] Generate source code vulnerability test scripts based on the service list, network description information, and the audit results of the code audit module;

[0046] Generate common vulnerability testing scripts based on discovered common vulnerabilities;

[0047] Generate different types of event-type test scripts for all interfaces based on the discovered event-type vulnerabilities;

[0048] Generate attack test scripts based on port scanning results and process monitoring results.

[0049] It can be understood that the content of event-type test scripts includes but is not limited to initiating requests to the application interface; the content of source code vulnerability test scripts includes but is not limited to initiating requests to the application code; the content of general vulnerability test scripts includes but is not limited to initiating requests to services; the content of attack test scripts includes but is not limited to using different protocols to request open ports of the service system and packet capture and parsing.

[0050] The script execution environment construction scheme includes the attacking machine and the defending machine included in the test scenario, and the network structure design between them.

[0051] Furthermore, generating the required range test scenarios according to the script execution environment construction plan is specifically as follows: determining the number of virtual computers included in each range test scenario, the deployment method of the computer network, and the process services and open ports included in each virtual computer according to the running configuration information of the service to be tested and the script execution environment construction plan, and using virtual technology to generate the range test scenarios; wherein the script execution environment construction plan includes an operating system, a service management type, a virtualization type, and a service running path, etc.

[0052] Specifically, the target range test scenario includes an attack test scenario and a service test scenario. The attack test scenario is a scenario that is independent of the service environment being tested but not isolated from the service scenario network being tested. In this scenario, an attack test script is executed to conduct security attacks such as DDoS on the ports open to the service to verify whether the service has the ability to respond to attacks; the service test scenario is used to execute source code vulnerability test scripts, general vulnerability test scripts, and event-based test scripts.

[0053] Introducing network target range technology, building a basic target range, and generating test scripts and corresponding target range test scenarios based on different vulnerability mining results, and conducting tests in multiple groups of different environments in parallel, uninterruptedly, and for a long time will help discover more security issues more quickly.

[0054] Example 2

[0055] The difference between this embodiment and embodiment 1 is that the running configuration information of the service to be tested includes technology stack information, program packages, source code, operating system image and interface documents. At this time, according to the classification results, the running configuration information of different categories is sent to the corresponding vulnerability mining module for processing. The specific steps of mining potential vulnerabilities are as follows:

[0056] Reverse engineer the identified program packages to obtain program codes, and send the program codes to the code audit module for auditing to find program errors, vulnerabilities, and violations of program specifications;

[0057] Send the source code to the code audit module for auditing to find out program errors, vulnerabilities and violations of program specifications;

[0058] A basic target range is established based on the identified operating system image, and a process scan is performed on the basic target range to detect the system service list. A port scanning tool is used to perform port scanning on the nodes in the scene to detect open ports and protocols.

[0059] The identified interface documents are sent to the event-type vulnerability matching module to mine possible event-type vulnerabilities.

[0060] Furthermore, a test plan is constructed based on the potential vulnerabilities discovered, the manually input operation configuration information of the service to be tested, and the detected operation configuration information of the service to be tested. The test plan includes a test script and a script execution environment construction plan; wherein the test script generation steps are as follows:

[0061] Generate source code vulnerability test scripts based on the audit results of the code audit module;

[0062] Generate common vulnerability testing scripts based on discovered common vulnerabilities;

[0063] Generate different types of event-type test scripts for all interfaces based on the discovered event-type vulnerabilities;

[0064] Generate attack test scripts based on process scanning results and port scanning results.

[0065] Example 3

[0066] The difference between this embodiment and Embodiment 1 is that after constructing the shooting range test scenario, it is also necessary to monitor the shooting range test scenario in real time, monitor the service availability of the shooting range test scenario according to the running configuration information of the service to be tested, and automatically recycle and rebuild the shooting range test scenario when the service is unavailable.

[0067] Among them, recycling refers to the separate destruction of unavailable parts in the target range test scenario or recovery using snapshots, and reconstruction refers to the regeneration of individually destroyed virtual machine nodes.

[0068] Example 4

[0069] The difference between this embodiment and embodiment 1 is that: log information is monitored and collected during script execution, the collected log information is screened and analyzed in combination with the technology stack information, and valid information is screened out based on abnormal keywords; the screened out valid information and the script execution results are classified using a recursive neural network to detect the test pass status and provide a set of service vulnerabilities and potential vulnerabilities.

[0070] Example 5

[0071] This embodiment provides an automated penetration testing system based on a network range, including:

[0072] The information acquisition module is used to obtain the running configuration information of the service to be tested;

[0073] Specifically, the information acquisition module includes:

[0074] The input module is used for manually inputting the customized running configuration information of the service to be tested;

[0075] Detection module: used to detect the manually input operation configuration information, detect the operation configuration information that is not explicitly stated in the service to be tested, and supplement and improve the manually input operation configuration information;

[0076] The setting of the detection module can perform a relatively complete security test on the developed application even if the tester does not have sufficient security technology accumulation;

[0077] The recognition and classification module uses the recognition decision tree to recognize and classify the operation configuration information, and sends different categories of operation configuration information to the corresponding vulnerability mining module for processing according to the recognition results to mine potential vulnerabilities;

[0078] A test plan construction module is used to construct a test plan based on the potential vulnerabilities discovered and the acquired operation configuration information, wherein the test plan includes a test script and a script execution environment construction plan;

[0079] The network range generation module is used to generate the basic range based on the operation configuration information and generate the required range test scenarios according to the script execution environment construction plan;

[0080] The test execution module uploads the test script to the target test scenario node in the network target range and executes the test script;

[0081] The result analysis module inputs the script execution results into the recursive neural network to determine whether the test passes.

[0082] In specific implementation, the vulnerability mining module includes:

[0083] The event-based vulnerability mining module is used to match event-based vulnerabilities with test cases and / or interface documents and mine possible event-based vulnerabilities.

[0084] The general vulnerability matching module is used to match the technology stack information with general vulnerabilities based on the vulnerability library and discover possible general vulnerabilities.

[0085] Code audit module: used to reverse engineer the program package to obtain the program code and source code for auditing, to find out program errors, loopholes and violations of program specification codes;

[0086] The port scanning module is used to perform port scanning on the established basic target range, detect whether the final application service environment is consistent with the service and network configuration of the acquired application program, and supplement and improve the manually input operation configuration information.

[0087] This embodiment classifies the running configuration information of the service to be tested by identifying the classification tree, and calls the vulnerability library, code audit and port scanning modules to automatically mine vulnerabilities according to the classification results. It can not only provide a certain guarantee for the comprehensiveness of the automated penetration test, but also save the time for repeated mining of repeated vulnerabilities, and save the consumption of human resources; introduce network target range technology, build a network target range, and conduct multiple groups of tests in different environments in parallel, uninterruptedly and for a long time, which is conducive to discovering more security issues more quickly; supplement the manually input data through the detection module, so that when the personnel do not have sufficient security technology accumulation, they can also perform a relatively complete security test on the developed application.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or some technical features can be replaced by equivalents without departing from the spirit of the technical solution of the present invention, which should be included in the scope of the technical solution for protection of the present invention.

Claims

1. An automated penetration testing method based on a network range, characterized in that: The following steps are involved: Obtain the running configuration information of the service to be tested, use the recognition decision tree to identify and classify the running configuration information, and send different categories of running configuration information to the corresponding vulnerability mining module for processing according to the recognition results, to mine potential vulnerabilities and build a basic target range; Constructing a test plan based on the potential vulnerabilities discovered and the operation configuration information obtained, wherein the test plan includes a test script and a script execution environment construction plan; Generate the required range test scenario according to the script execution environment construction plan, upload the test script to the target test scenario node in the network range, execute the test script, and input the script execution result into the recursive neural network to determine whether the test passes; Among them, the required range test scenarios generated according to the script execution environment construction plan include: According to the running configuration information of the service to be tested and the script execution environment construction plan, the number of virtual computers included in each target range test scenario, the deployment method of the computer network, and the process services and open ports included in each virtual computer are determined, and the target range test scenario is generated using virtual technology.

2. According to claim 1, an automated penetration testing method based on a network range is characterized in that: Obtaining the running configuration information of the service to be tested includes the following steps: manually inputting the customized running configuration information of the service to be tested, detecting the manually input running configuration information through a detection module, detecting the running configuration information not explicitly stated in the service to be tested, and supplementing and improving the manually input running configuration information.

3. The method for automated penetration testing based on a network range according to claim 1, characterized in that: When the running configuration information of the service to be tested includes technology stack information, program packages, source code, test cases, as well as interface documents, open ports, service lists, and network description information, According to the identification results, different types of running configuration information are sent to the corresponding vulnerability mining module for processing. The specific steps of mining potential vulnerabilities are as follows: Reverse engineer the identified program packages to obtain program codes, and send the program codes to the code audit module for auditing to find program errors, vulnerabilities, and violations of program specifications; Send the source code to the code audit module for auditing to find out program errors, vulnerabilities and violations of program specifications; Establish a basic target range based on the identified network description information and service list, and perform port scanning and process scanning on the basic target range to detect the security of the final application service environment; Send the identified technology stack information to the general vulnerability matching module, and mine possible general vulnerabilities based on the vulnerability library; The identified test cases and interface documents are sent to the event-type vulnerability matching module, and possible event-type vulnerabilities are mined based on the vulnerability library.

4. The method of automated penetration testing based on a network range according to claim 1, characterized in that: When the running configuration information of the service to be tested includes technology stack information, program packages, source code, operating system images, and interface documents, The specific steps of sending different types of operation configuration information to the corresponding vulnerability mining module for processing according to the identification results are as follows: Reverse engineer the identified program packages to obtain program codes, and send the program codes to the code audit module for auditing to find program errors, vulnerabilities, and violations of program specifications; Send the source code to the code audit module for auditing to find out program errors, vulnerabilities and violations of program specifications; A basic target range is established based on the identified operating system image, and a process scan is performed on the basic target range to detect the system service list. A port scanning tool is used to perform port scanning on the nodes in the scene to detect open ports and protocols. The identified interface documents are sent to the event-type vulnerability matching module to mine possible event-type vulnerabilities.

5. The automated penetration testing method based on a network range according to claim 1 is characterized in that: Send the script execution results to the recurrent neural network to determine whether the test passes: During the script execution process, log information is monitored and collected, and the collected log information is filtered and analyzed based on the technology stack information, and valid information is filtered out based on abnormal keywords; The filtered valid information and script execution results are classified using a recursive neural network to detect whether the test passes and provide a set of service vulnerabilities and potential vulnerabilities.

6. The automated penetration testing method based on a network range according to claim 1, characterized in that: After constructing the shooting range test scenario, it is also necessary to monitor the shooting range test scenario in real time, monitor the service availability of the shooting range test scenario according to the running configuration information of the service to be tested, and automatically recycle and rebuild the shooting range test scenario when the service is unavailable.

7. An automated penetration testing system based on a network range, characterized in that: The information acquisition module is used to obtain the running configuration information of the service to be tested; The recognition and classification module uses the recognition decision tree to recognize and classify the operation configuration information, and sends different categories of operation configuration information to the corresponding vulnerability mining module for processing according to the recognition results to mine potential vulnerabilities; A test plan construction module is used to construct a test plan based on the potential vulnerabilities discovered and the acquired operation configuration information, wherein the test plan includes a test script and a script execution environment construction plan; The network range generation module is used to generate a basic range based on the operation configuration information and generate the required range test scenario according to the script execution environment construction plan; wherein, generating the required range test scenario according to the script execution environment construction plan includes: Determine the number of virtual computers included in each range test scenario, the deployment method of the computer network, and the process services and open ports included in each virtual computer based on the running configuration information of the service to be tested and the script execution environment construction plan, and use virtual technology to generate the range test scenario; The test execution module uploads the test script to the target test scenario node in the network target range and executes the test script; The result analysis module inputs the script execution results into the recursive neural network to determine whether the test passes.

8. The automated penetration testing system based on a network range according to claim 7, characterized in that: The information acquisition module includes: The input module is used for manually inputting the customized running configuration information of the service to be tested; Detection module: used to detect the manually input operation configuration information, detect the operation configuration information that is not explicitly stated in the service to be tested, and supplement and improve the manually input operation configuration information.

9. The automated penetration testing system based on a network range according to claim 7, characterized in that: The vulnerability mining module includes: The event-type vulnerability matching module is used to match event-type vulnerabilities against test cases and / or interface documents based on the vulnerability library to mine possible event-type vulnerabilities; The general vulnerability matching module is used to match the technology stack information with general vulnerabilities based on the vulnerability library, and to mine possible general vulnerabilities based on the vulnerability library; Code audit module: used to reverse engineer the program package to obtain the program code and source code for auditing, to find out program errors, loopholes and violations of program specification codes; The port scanning module is used to perform port scanning on the established basic target range, detect whether the final application service environment is consistent with the service and network configuration of the acquired application program, and supplement and improve the manually input operation configuration information.

Images