A method for security detection and effectiveness verification of penetration testing tools

Abstract

The present invention discloses a method for security detection and effectiveness verification of a penetration testing tool, the method comprising: performing security detection on the penetration testing tool to obtain a security detection result of the penetration testing tool; obtaining penetration testing historical data information; the penetration testing historical data information comprises network flow information, log information and memory data information; processing the penetration testing historical data information to obtain a penetration testing tool effectiveness verification result. The present invention solves the problems of single security detection method, lack of comprehensiveness and lagging tool effectiveness verification analysis ideas in the usability analysis of penetration testing tools under dynamic network changes through comprehensive security detection, targeted execution effectiveness analysis and other means, and realizes comprehensive tool security detection at the three levels of features, behaviors and flows and effectiveness detection of diversified penetration testing capabilities.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a method for security detection and effectiveness verification of a penetration testing tool. Background Art

[0002] In a complex target network environment, its network topology, traffic analysis, defense strategy, etc. are in a state of constant dynamic change, which in turn leads to changes in the overall situation of the target network. In the face of such a dynamic network environment, it is difficult to ensure the effective operation of various penetration testing tools and to effectively support the smooth execution of penetration testing tasks. Therefore, starting from the availability of penetration testing tools, it is important to conduct comprehensive execution security testing and execution performance verification, realize the efficient execution of diversified penetration testing tools in the target network environment, and successfully complete various penetration testing tasks. It is an important research point for comprehensive mapping and analysis of the target and feedback to improve the level of network protection.

[0003] The key to the smooth execution of penetration testing tools is to conduct security detection of various dynamic and static features from multiple aspects such as the source code layer, binary code layer, and traffic layer after assembling and generating the penetration testing tools, and to verify and analyze the execution effect, so as to achieve defect optimization of the tools. At present, there is a lack of comprehensive and public theoretical research results and commercial products in the security detection and performance verification of penetration testing tools. On the one hand, in the research of tool security detection, it mainly includes dynamic detection, static detection and hybrid detection in the traditional non-visual direction, as well as visual detection direction, which is mainly based on single research methods such as features, behaviors, and traffic, and studies single contents including vulnerabilities, source code defects, binary code similarities, and supply chain pollution. The security detection of diversified penetration testing tools lacks comprehensiveness and strong actual relevance; on the other hand, there is a lack of mature theoretical results that have not yet been published in terms of tool performance verification. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method for security detection and effectiveness verification of penetration testing tools, focusing on the demand for efficient execution of penetration testing tools in complex target network environments. On the one hand, security detection is performed on the homology, dynamic and static characteristics, and network characteristics of source code and binary code from the three levels of characteristics, behavior, and traffic, to ensure that the security mechanism can be circumvented and the penetration testing tool can be executed smoothly in the target network environment; on the other hand, the effectiveness of different penetration testing capabilities such as mapping and penetration of the assembled tool is analyzed to provide data support for the optimization and improvement of the penetration testing tool, thereby ensuring the smooth execution of the penetration testing task and providing targeted protection improvement ideas and suggestions for the target network facing the test.

[0005] In order to solve the above technical problems, an embodiment of the present invention discloses a method for security detection and effectiveness verification of a penetration testing tool, the method comprising:

[0006] S1, perform security testing on the penetration testing tool to obtain the security testing result of the penetration testing tool;

[0007] S2, obtaining penetration test historical data information; the penetration test historical data information includes network traffic information, log information and memory data information;

[0008] S3, processing the penetration test historical data information to obtain the penetration test tool effectiveness verification result.

[0009] As an optional implementation manner, in the embodiment of the present invention, the security detection of the penetration testing tool to obtain the security detection result of the penetration testing tool includes:

[0010] S11, performing source code homology detection on the penetration testing tool to obtain source code homology detection information;

[0011] S12, performing binary homology detection on the penetration testing tool to obtain binary homology detection information;

[0012] S13, performing sensitive character processing on the penetration testing tool to obtain sensitive character feature information;

[0013] S14, performing a static test on the penetration testing tool to obtain static test feature information;

[0014] S15, dynamically testing the penetration testing tool to obtain dynamic testing feature information;

[0015] S16, performing network feature analysis on the penetration testing tool to obtain network feature information;

[0016] S17, integrating the source code homology detection information, the binary homology detection information, the sensitive character feature information, the static test feature information, the dynamic test feature information and the network feature information to obtain a security detection result of a penetration testing tool.

[0017] As an optional implementation manner, in the embodiment of the present invention, the source code homology detection is performed on the penetration testing tool to obtain the source code homology detection information, including:

[0018] S111, preprocessing the source code of the penetration testing tool to obtain preprocessed source code information;

[0019] S112, performing a metric-based similarity analysis on the preprocessed source code information to obtain metric similarity information;

[0020] S113, performing text-based similarity analysis on the preprocessed source code information to obtain text similarity information;

[0021] S114, performing a structure-based similarity analysis on the preprocessed source code information to obtain structure similarity information;

[0022] S115, performing semantic-based similarity analysis on the preprocessed source code information to obtain semantic similarity information;

[0023] S116, processing the metric similarity information, the text similarity information, the structural similarity information and the semantic similarity information to obtain source code homology detection information.

[0024] As an optional implementation manner, in an embodiment of the present invention, the performing a metric-based similarity analysis on the preprocessed source code information to obtain metric similarity information includes:

[0025] S1121, dividing the preprocessed source code information to obtain N source code segments;

[0026] S1122, processing the N source code fragments according to a preset metric standard to obtain a metric value vector;

[0027] S1123, performing similarity analysis on the metric value vector to obtain metric similarity information.

[0028] As an optional implementation manner, in an embodiment of the present invention, the performing text-based similarity analysis on the preprocessed source code information to obtain text similarity information includes:

[0029] S1131, decomposing the preprocessed source code information to obtain M code fragments;

[0030] S1132, performing hash processing on the M code snippets to obtain M hash values;

[0031] S1133, processing the M hash values ​​to obtain text similarity information.

[0032] As an optional implementation manner, in the embodiment of the present invention, the binary homology detection is performed on the penetration testing tool to obtain the binary homology detection information, including:

[0033] S121, performing binary static feature homology analysis on the penetration testing tool to obtain static homology feature information;

[0034] S122, performing binary code slice homology analysis on the penetration testing tool to obtain code homology information;

[0035] S123, processing the static homology feature information and the code homology information to obtain binary homology detection information.

[0036] As an optional implementation manner, in an embodiment of the present invention, the sensitive character processing of the penetration testing tool to obtain sensitive character feature information includes:

[0037] S131, processing the source code of the penetration testing tool to obtain a character string containing Chinese characters;

[0038] S132, performing text processing on the character string containing Chinese characters to obtain pre-processed text information;

[0039] S133, performing feature extraction on the preprocessed text information to obtain text feature information;

[0040] S134, using the text feature information, training a preset sensitive character detection model to obtain an optimized sensitive character detection model;

[0041] S135, using the optimized sensitive character detection model, performing sensitive character processing on the penetration testing tool to be processed to obtain sensitive character feature information.

[0042] As an optional implementation manner, in an embodiment of the present invention, the processing of the penetration test historical data information to obtain the penetration test tool effectiveness verification result includes:

[0043] S31, processing the network traffic information to obtain a validity verification result based on network traffic monitoring;

[0044] S32, processing the log information to obtain the validity verification result based on the target machine error log monitoring;

[0045] S33, processing the memory data information to obtain a validity verification result based on the target machine memory data monitoring;

[0046] S34, integrating the validity verification result based on network traffic monitoring, the validity verification result based on target machine error log monitoring and the validity verification result based on target machine memory data monitoring to obtain the penetration testing tool effectiveness verification result.

[0047] As an optional implementation manner, in the embodiment of the present invention, the processing of network traffic information to obtain a validity verification result based on network traffic monitoring includes:

[0048] S311, using a network monitoring tool to capture network traffic and obtain network traffic information; the network traffic information includes the traffic size, connection frequency, communication frequency, data transmission volume, protocol, data packet content, timestamp, protocol type and port number of the network traffic;

[0049] S312, processing the flow size of the network flow and the connection frequency to obtain a flow analysis result;

[0050] S313, processing the communication frequency, the data transmission volume and the protocol to obtain a behavior analysis result;

[0051] S314, analyzing the data packet content to obtain vulnerability exploit component detection results;

[0052] S315, performing communication detection on the network traffic information to obtain a communication data detection result;

[0053] S316, performing authority maintenance detection on the network traffic information to obtain an authority maintenance stability analysis result;

[0054] S317, processing the timestamp, the protocol type and the port number to obtain a tool concealment analysis result;

[0055] S318, integrating the traffic analysis results, the behavior analysis results, the vulnerability exploitation component detection results, the communication data detection results, the permission maintenance stability analysis results and the tool concealment analysis results to obtain a validity verification result based on network traffic monitoring.

[0056] As an optional implementation, in an embodiment of the present invention, the processing of log information to obtain a validity verification result based on target machine error log monitoring includes:

[0057] S321, parsing the log information to obtain log parsing information;

[0058] S322, extracting keywords from the log parsing information to obtain log keyword information;

[0059] S323, performing anomaly detection on the log keyword information to obtain an anomaly detection result;

[0060] S324, performing payload analysis on the log parsing information to obtain a structural analysis result;

[0061] S325, performing vulnerability exploitation analysis on the log parsing information to obtain a vulnerability exploitation test identification result;

[0062] S326, performing behavior analysis on the log parsing information to obtain a behavior analysis result;

[0063] S327, integrating the anomaly detection result, the structure analysis result, the vulnerability exploitation test identification result and the behavior analysis result to obtain a validity verification result based on target machine error log monitoring.

[0064] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0065] The present invention takes the real target network environment with irregular dynamic changes as the background, focuses on the efficient execution and comprehensive availability detection requirements of penetration testing tools in complex target network environments, and proposes a tool security detection and effectiveness verification method for comprehensive availability. Mainly through comprehensive security detection, targeted execution performance analysis and other means, it solves the problems of single security detection method, lack of comprehensiveness and lagging tool effectiveness verification analysis ideas in the availability analysis of penetration testing tools under dynamic network changes, realizes comprehensive tool security detection at the three levels of characteristics, behavior and traffic, and effectiveness detection of diversified penetration testing capabilities, supports the smooth execution of penetration testing tasks, and then provides improvement ideas for subsequent comprehensive mapping and analysis of cooperative target networks, and improvement of protection levels; in addition, it can also provide a reference for the assembly and generation process of penetration testing tools, and discover and avoid abnormal execution situations such as target crash, loss of connection of penetration testing tools, and out-of-control execution in advance. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0067] Figure 1 It is a flow chart of a method for security detection and effectiveness verification of a penetration testing tool disclosed in an embodiment of the present invention;

[0068] Figure 2 It is a flow chart of another method for security detection and effectiveness verification of a penetration testing tool disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0071] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0072] The present invention discloses a method for security detection and effectiveness verification of a penetration testing tool, the method comprising: performing security detection on the penetration testing tool to obtain a security detection result of the penetration testing tool; obtaining penetration testing historical data information; the penetration testing historical data information comprises network flow information, log information and memory data information; processing the penetration testing historical data information to obtain a penetration testing tool effectiveness verification result. The present invention solves the problems of single security detection method, lack of comprehensiveness and lagging tool effectiveness verification analysis ideas in the usability analysis of penetration testing tools under dynamic network changes by means of comprehensive security detection and targeted execution effectiveness analysis, and realizes comprehensive tool security detection at the three levels of features, behaviors and flows and effectiveness detection of diversified penetration testing capabilities. The following are detailed descriptions respectively.

[0073] Embodiment 1

[0074] See also Figure 1 , Figure 1 This is a flow chart of a method for security detection and effectiveness verification of a penetration testing tool disclosed in an embodiment of the present invention. Figure 1The described penetration testing tool security detection and effectiveness verification method is applied to the field of network security technology. On the one hand, it performs security detection on the homology, dynamic and static characteristics, and network characteristics of source code and binary code from the three levels of characteristics, behavior, and traffic, to ensure that the penetration testing tool can circumvent the security mechanism in the target network environment and execute smoothly; on the other hand, it performs effectiveness analysis on the different penetration testing capabilities of the assembled tool, such as mapping and penetration, to provide data support for the optimization and improvement of the penetration testing tool, thereby ensuring the smooth execution of the penetration testing task and providing targeted protection improvement ideas and suggestions for the target network being tested. Figure 1 As shown, the penetration testing tool security detection and effectiveness verification method may include the following operations:

[0075] S1, perform security testing on the penetration testing tool to obtain the security testing result of the penetration testing tool;

[0076] S2, obtaining penetration test historical data information; the penetration test historical data information includes network traffic information, log information and memory data information;

[0077] S3, processing the penetration test historical data information to obtain the penetration test tool effectiveness verification result.

[0078] Optionally, performing security detection on the penetration testing tool to obtain the security detection result of the penetration testing tool includes:

[0079] S11, performing source code homology detection on the penetration testing tool to obtain source code homology detection information;

[0080] S12, performing binary homology detection on the penetration testing tool to obtain binary homology detection information;

[0081] S13, performing sensitive character processing on the penetration testing tool to obtain sensitive character feature information;

[0082] S14, performing a static test on the penetration testing tool to obtain static test feature information;

[0083] Combine decryption, reverse engineering and other technologies to remove code obfuscation, shelling and other information, extract and compare program codes to identify known penetration test codes and obtain static test feature information;

[0084] S15, dynamically testing the penetration testing tool to obtain dynamic testing feature information;

[0085] Monitor and analyze the behavior of the program at runtime based on the virtual environment, and combine deep learning algorithms to identify known and suspected abnormal penetration test codes to obtain dynamic test feature information;

[0086] S16, performing network feature analysis on the penetration testing tool to obtain network feature information;

[0087] Identify penetration test behaviors based on network features such as DNS records, HTTP header information, port information, and certificate information to obtain network feature information;

[0088] S17, integrating the source code homology detection information, the binary homology detection information, the sensitive character feature information, the static test feature information, the dynamic test feature information and the network feature information to obtain the security detection result of the penetration test tool. Using the formula:

[0089] q=aλ1+pλ2+bλ3+cλ4+dλ5+eλ6

[0090] Among them, q is the security detection result of the penetration testing tool, a is the source code homology detection information, p is the binary homology detection information, b is the sensitive character feature information, c is the static test feature information, d is the dynamic test feature information, e is the network feature information, λ1 is the source code homology detection information weight, λ2 is the binary homology detection information weight, λ3 is the sensitive character feature information weight, λ4 is the static test feature information weight, λ5 is the dynamic test feature information weight, λ6 is the network feature information weight, λ1+λ2+λ3+λ4+λ5+λ6=1.

[0091] Optionally, performing source code homology detection on the penetration testing tool to obtain source code homology detection information includes:

[0092] S111, preprocessing the source code of the penetration testing tool to obtain preprocessed source code information;

[0093] Includes: Cleaning the source code of the penetration testing tool, removing comments, blanks, metadata, etc., highlighting the actual structure and logic of the code. Standardizing the code style: Unifying indentation, naming conventions and coding standards to reduce misjudgments caused by style differences.

[0094] S112, performing a metric-based similarity analysis on the preprocessed source code information to obtain metric similarity information;

[0095] S113, performing text-based similarity analysis on the preprocessed source code information to obtain text similarity information;

[0096] S114, performing a structure-based similarity analysis on the preprocessed source code information to obtain structure similarity information;

[0097] Constructing an AST (Abstract Syntax Tree) of the code for the preprocessed source code information, and comparing the similarity of the abstract syntax tree structures, evaluating the execution path and data dependency of the code, and obtaining structural similarity information based on the evaluation results;

[0098] S115, performing semantic-based similarity analysis on the preprocessed source code information to obtain semantic similarity information;

[0099] Evaluate similarity by analyzing the actual functional and behavioral semantics of the code, executing code snippets to understand their semantics, and comparing the output or behavior, evaluating the obtained results, and determining semantic similarity information based on the evaluation results

[0100] S116, processing the metric similarity information, the text similarity information, the structural similarity information and the semantic similarity information to obtain source code homology detection information.

[0101] The machine learning model is trained using the historical data of the obtained measurement similarity information, text similarity information, structural similarity information and semantic similarity information to obtain an optimized machine learning model;

[0102] The source code information to be processed is processed using an optimized machine learning model to obtain source code homology detection information.

[0103] Optionally, performing a metric-based similarity analysis on the preprocessed source code information to obtain metric similarity information includes:

[0104] S1121, dividing the preprocessed source code information to obtain N source code segments; the division method is not limited;

[0105] S1122, processing the N source code fragments according to a preset metric standard to obtain a metric value vector;

[0106] Preset metrics like cyclomatic complexity, lines of code, naming conventions, etc.

[0107] The metric vector is constructed by constructing each source code snippet into a vector or multi-dimensional data point.

[0108] S1123, performing similarity analysis on the metric value vector to obtain metric similarity information.

[0109] Using a similarity measurement method (such as cosine similarity, Jaccard similarity, Euclidean distance, etc.), similarity analysis is performed on the metric value vector to obtain metric similarity information.

[0110] Optionally, performing text-based similarity analysis on the preprocessed source code information to obtain text similarity information includes:

[0111] S1131, decomposing the preprocessed source code information to obtain M code fragments; the decomposition method is not limited;

[0112] Preprocessing includes cleaning the code: removing comments, blank lines, metadata and other non-code text.

[0113] Standardize code: Ensure consistent code style, such as unified indentation, naming conventions, etc.

[0114] Ignore compiler dependencies: remove compiler-specific generated code, such as debug information, etc.

[0115] S1132, performing hash processing on the M code snippets to obtain M hash values;

[0116] Hash processing is implemented using hash functions such as MD5, SHA-1, and SHA-256;

[0117] S1133, processing the M hash values ​​to obtain text similarity information.

[0118] Create a database to store the hash value of each code file or snippet and its corresponding file name or location.

[0119] For a new code snippet, calculate its hash value and compare it with the hash value in the database.

[0120] Direct comparison: If an exact match of hash values ​​is found, it indicates that the code snippet is identical or highly similar to a code snippet in the database.

[0121] Approximate comparison: If the hash values ​​are not exactly the same, but have some similarity (for example, most of the bits of the hash values ​​are the same), the similarity of the codes can be further evaluated using edit distance or other similarity metrics.

[0122] The text similarity information is scored according to the degree of similarity, which is a value between 0 and 1. The larger the value, the higher the similarity.

[0123] Optionally, performing binary homology detection on the penetration testing tool to obtain binary homology detection information includes:

[0124] S121, performing binary static feature homology analysis on the penetration testing tool to obtain static homology feature information;

[0125] Compare static features of executable files such as string features, file compilation features, digital certificate features, and file description features to detect whether the files have the same source code or the same functions, and obtain static homology feature information.

[0126] S122, performing binary code slice homology analysis on the penetration testing tool to obtain code homology information;

[0127] Extract code slices with key API calls as entry points in the program, compare the similarity between code slices to detect whether the files have the same source code or the same function, and obtain code homology information;

[0128] Both static homology feature information and code homology information are normalized based on the results of detection or comparison, and the resulting values ​​are as large as possible, indicating a higher degree of similarity.

[0129] S123, processing the static homology feature information and the code homology information to obtain binary homology detection information. Using the formula:

[0130] p=p1w1+p2w2

[0131] Among them, p is the binary homology detection information, p1 is the static homology feature information, p2 is the code homology information, w1 is the static homology feature information weight, w2 is the code homology information weight, w1+w2=1.

[0132] Optionally, the sensitive character processing of the penetration testing tool to obtain sensitive character feature information includes:

[0133] S131, processing the source code of the penetration testing tool to obtain a character string containing Chinese characters;

[0134] Use regular expressions or other string matching algorithms to detect the source code of the penetration testing tool, retain the strings containing Chinese characters, and remove other content.

[0135] S132, performing text processing on the character string containing Chinese characters to obtain pre-processed text information;

[0136] This includes segmenting and tagging the character string containing Chinese characters, and further analyzing and judging based on the location and context of specific characters to determine whether there are sensitive character features and obtain pre-processed text information;

[0137] S133, performing feature extraction on the preprocessed text information to obtain text feature information;

[0138] Using a bag-of-words model to extract features from the preprocessed text information to obtain text feature information;

[0139] S134, using the text feature information, training a preset sensitive character detection model to obtain an optimized sensitive character detection model;

[0140] The sensitive character detection model preset in this embodiment is the YOLOv5 model;

[0141] Optionally, the preset sensitive character detection model is a CTPN model;

[0142] The CTPN model includes a VGG16 convolutional model, a bidirectional LSTM model, and a fully connected layer;

[0143] Inputting the text feature information into the VGG16 convolutional model; the VGG16 network includes 13 convolutional layers, 5 maximum pooling layers and 3 fully connected layers;

[0144] In the convolution layer, a two-dimensional convolution kernel w∈R is used. 3×3 Extract the feature information of the text to obtain the feature matrix C n ;

[0145]

[0146] Among them, n represents the number of convolution operations, m represents the number of convolution kernels, and p i represents the i-th feature matrix obtained, f represents the nonlinear activation function, · represents the corresponding operation of the shared weight of the convolution kernel and the feature matrix, w represents the weight of the convolution kernel, b represents the bias value, R 3×3 represents a 3×3 real matrix;

[0147] The maximum pooling method is used in the pooling layer, and the feature extraction formula is expressed as follows:

[0148] p u =Max 2×2 [C n ]

[0149] Among them, u represents the number of pooling times, Max 2×2 represents the maximum pooling operation method of a 2×2 matrix, p u is the extracted feature; after multiple convolution and pooling operations, the Reshape processed data stream is input into the bidirectional LSTM model to obtain a feature vector with time series attributes, which is expressed as follows:

[0150] o t =g(V st +V s ' T′ )

[0151]

[0152] Among them, s t represents the output of the forward sequence at time t, s′ t Represents the output of the reverse sequence at time t, U Xt Represents the initial input of the forward sequence, U′ Xt Represents the initial input of the reverse sequence, W St-1 Represents the input at the previous moment in the forward sequence. The input of the next moment in reverse time sequence, o t Represents the output at time t. After feature extraction by the time series model, the spatial + sequence feature vector is input into the RPN network and passes through two feature extraction networks at the same time. One of them obtains positive feedback and negative feedback classification through the softmax classification feature vector set, and the other is used to calculate the boundary value regression offset for the feature vector set to obtain accurate detection results.

[0153] S135, using the optimized sensitive character detection model, performing sensitive character processing on the penetration testing tool to be processed to obtain sensitive character feature information.

[0154] Optionally, the processing of the penetration test historical data information to obtain a penetration test tool effectiveness verification result includes:

[0155] S31, processing the network traffic information to obtain a validity verification result based on network traffic monitoring;

[0156] S32, processing the log information to obtain the validity verification result based on the target machine error log monitoring;

[0157] S33, processing the memory data information to obtain a validity verification result based on the target machine memory data monitoring;

[0158] Through memory capture, memory analysis, code detection and other means, we analyze the running of Shellcode loading, post-loading execution and other tools in memory, evaluate the running, encryption and anti-killing of Shellcode, accurately locate the problems in Shellcode construction and execution, and obtain the effectiveness verification results based on the target machine memory data monitoring;

[0159] S34, integrating the validity verification results based on network traffic monitoring, the validity verification results based on target machine error log monitoring, and the validity verification results based on target machine memory data monitoring to obtain the effectiveness verification results of the penetration testing tool. Using the formula:

[0160] m=m1k1+m2k2+m3k3

[0161] Among them, m is the effectiveness verification result of the penetration testing tool, m1 is the effectiveness verification result based on network traffic monitoring, m2 is the effectiveness verification result based on target machine error log monitoring, m3 is the effectiveness verification result based on target machine memory data monitoring, k1 is the weight of the effectiveness verification result based on network traffic monitoring, k2 is the weight of the effectiveness verification result based on target machine error log monitoring, k3 is the weight of the effectiveness verification result based on target machine memory data monitoring, k1+k2+k3=1.

[0162] Optionally, the processing of the network traffic information to obtain a validity verification result based on network traffic monitoring includes:

[0163] S311, using a network monitoring tool to capture network traffic and obtain network traffic information; the network traffic information includes the traffic size, connection frequency, communication frequency, data transmission volume, protocol, data packet content, timestamp, protocol type and port number of the network traffic;

[0164] S312, processing the flow size of the network flow and the connection frequency to obtain a flow analysis result;

[0165] S313, processing the communication frequency, the data transmission volume and the protocol to obtain a behavior analysis result;

[0166] S314, analyzing the content of the data packet to obtain a vulnerability exploit component detection result;

[0167] S315, performing communication detection on the network traffic information to obtain a communication data detection result;

[0168] S316, performing authority maintenance detection on the network traffic information to obtain an authority maintenance stability analysis result;

[0169] S317, processing the timestamp, the protocol type and the port number to obtain a tool concealment analysis result;

[0170] S318, integrating the traffic analysis results, the behavior analysis results, the vulnerability exploitation component detection results, the communication data detection results, the permission maintenance stability analysis results and the tool concealment analysis results to obtain the effectiveness verification results based on network traffic monitoring. Using the formula: n = n1l1 + n2l2 + n3l3 + n4l4 + n5l5 + n6l6

[0171] Among them, n is the effectiveness verification result based on network traffic monitoring, n1 is the traffic analysis result, n2 is the behavior analysis result, n3 is the vulnerability exploitation component detection result, n4 is the communication data detection result, n5 is the permission maintenance stability analysis result, n6 is the tool concealment analysis result, l1 is the traffic analysis result weight, l2 is the behavior analysis result weight, l3 is the vulnerability exploitation component detection result weight, l4 is the communication data detection result weight, l5 is the permission maintenance stability analysis result weight, l6 is the tool concealment analysis result weight, l1+l2+l3+l4+l5+l6=1.

[0172] Optionally, the processing of the log information to obtain a validity verification result based on the target machine error log monitoring includes:

[0173] S321, parsing the log information to obtain log parsing information;

[0174] S322, extracting keywords from the log parsing information to obtain log keyword information;

[0175] Optionally, a method for extracting keywords from the log parsing information is:

[0176] The log parsing information is subjected to a lattice network construction, which is a prior art in the art. The set of nodes for constructing the lattice network is N = {n0, n1, ...}, the set of arcs is E = {e0, e1, ...}, and each node n i ∈N contains a time stamp t(n i ), arc e u→v represents the arc connecting node u to node v. Let the nodes in the keyword network be NS = {N0, N1, ...}, Represents the node N in the keyword network i Connect to node N j A set of arcs.

[0177] 1. Any n i ∈N i , n j ∈N i , by t(n i )<t(n j ) we can get i≤j.

[0178] 2. Any n i ∈N i , n j ∈N i , by t(n i )=t(n j ) we can get i=j.

[0179] 3. Any eu→v ∈E, if for u∈N i and v∈N j , e u→v Equivalent to a group Then i≤m≤n≤j. (Here n=m+1, e u→v The two endpoints belong to N m and N n . )

[0180] The word grid network is converted into a keyword network, and the generation algorithm is:

[0181] 1. Convert keywords into corresponding syllable strings: K1…K M (M is the number of syllables of the keyword). Traverse all nodes N in the Lattice network and find the node n that matches K1. k , set to

[0182] 2. As the end point, search for the nearest node n in reverse time order. k-1 If n k-1 and There is no arc between them, then n k-1 Also belongs to Continue Search k-1 Otherwise, the search stops.

[0183] 3. As the starting point, search for similar nodes n backward in chronological order k+1 If n k+1 and There is no arc between them, then n k+1 belong Continue Search k+1 Otherwise, the search stops.

[0184] 4. For e k→k+1 belong n k+1 Belong to N k2 .

[0185] 5. All included in The arc e k→k+1 Form a keyword confusion network.

[0186] 6. For each arc If u belongs to N s , n i Belong to N t When t=s+1, belong otherwise, belong

[0187]

[0188] Where w(l) and w(e) are the words corresponding to arcs l and e, sim(·,·) refers to the similarity between two words. It refers to the arc and e overlap in normalized time. In addition, each node n in the Lattice i ∈N contains the time stamp t(n i ) will be used as a constraint to determine the termination of the search.

[0189] The data normalization method is used to confirm the keywords. First, the linguistic probability likelihood score of each candidate is normalized:

[0190]

[0191] Among them, Value is the set of linguistic score values ​​in the keyword network; minValue and maxValue are the minimum and maximum values ​​in the set respectively; x is the score before normalization; y is the score after normalization. After that, the sum of the scores is calculated and marked as the final score of the arc in the keyword network.

[0192] In each keyword network, select the arc candidate with the highest score and determine whether it is a keyword. If it is, output the keyword and time tag; if not, skip and enter the next keyword network. In this way, the keyword detection is completed.

[0193] S323, performing anomaly detection on the log keyword information to obtain an anomaly detection result;

[0194] S324, performing payload analysis on the log parsing information to obtain a structural analysis result;

[0195] S325, performing vulnerability exploitation analysis on the log parsing information to obtain a vulnerability exploitation test identification result;

[0196] S326, performing behavior analysis on the log parsing information to obtain a behavior analysis result;

[0197] S327, integrating the anomaly detection result, the structure analysis result, the vulnerability exploitation test identification result and the behavior analysis result to obtain the validity verification result based on the target machine error log monitoring. Using the formula:

[0198] y=z1r1+z2r2+z3r3+z4r4

[0199] Among them, y is the effectiveness verification result based on the target machine error log monitoring, z1 is the anomaly detection result, z2 is the construction analysis result, z3 is the vulnerability exploitation test identification result, z4 is the behavior analysis result, r1 is the anomaly detection result weight, r2 is the construction analysis result weight, r3 is the vulnerability exploitation test identification result weight, r4 is the behavior analysis result weight, r1+r2+r3+r4=1.

[0200] The weights involved in this embodiment are all set by experiments, depending on the specific situation, and this embodiment does not impose any restrictions.

[0201] It can be seen that the present invention takes the real target network environment with irregular dynamic changes as the background, focuses on the efficient execution and comprehensive availability detection requirements of penetration testing tools in complex target network environments, and proposes a tool security detection and effectiveness verification method for comprehensive availability. Mainly through comprehensive security detection, targeted execution performance analysis and other means, it solves the problems of single security detection method, lack of comprehensiveness and lagging tool effectiveness verification analysis ideas in the availability analysis of penetration testing tools under dynamic network changes, and realizes comprehensive tool security detection at the three levels of characteristics, behavior and traffic, and effectiveness detection of diversified penetration testing capabilities, supports the smooth execution of penetration testing tasks, and then provides improvement ideas for subsequent comprehensive mapping and analysis of cooperative target networks, and improvement of protection levels; in addition, it can also provide a reference for the assembly and generation process of penetration testing tools, and discover and avoid abnormal execution situations such as target crash, loss of connection of penetration testing tools, and out-of-control execution in advance.

[0202] Embodiment 2

[0203] In order to reduce the possibility of penetration testing actions being associated with analysis, this embodiment mainly implements six types of security detection, including source code homology, binary homology, sensitive character features, static test features, dynamic test features, and network features of different tool components; in addition, based on the analysis of network traffic, monitoring logs, memory data and other information, the effectiveness of the penetration testing tool's reconnaissance test, breakthrough test, control and defense test capabilities is verified. The overall principle diagram is as follows Figure 2 shown.

[0204] 1. Source code homology detection: Combine multiple technologies and strategies to evaluate the similarity between codes from different perspectives and improve the accuracy and reliability of detection. Including:

[0205] 1) Preprocessing and standardization:

[0206] Clean the code: remove comments, whitespace, metadata, etc. to highlight the actual structure and logic of the code.

[0207] Standardized code style: Unified indentation, naming conventions, and coding standards to reduce misjudgments due to style differences.

[0208] 2) Similarity analysis based on metrics

[0209] Predefined Metrics Set: Choose appropriate code metrics such as cyclomatic complexity, lines of code, naming conventions, etc.

[0210] Metric calculation: Calculate the corresponding metric values ​​and construct the metric value of each code snippet into a vector or multi-dimensional data point.

[0211] Use similarity measurement methods (such as cosine similarity, Jaccard similarity, Euclidean distance, etc.) to calculate the similarity scores between code snippets, and judge the similarity of code snippets based on the threshold.

[0212] 3) Text-based similarity analysis:

[0213] Tokenization: Break the code into tokens and then compare token sequences.

[0214] Edit distance or LCS: Calculates the longest common subsequence or edit distance between code snippets.

[0215] Cosine similarity or Jaccard similarity: Convert the token sequence into vectors and calculate the similarity between them.

[0216] 4) Similarity analysis based on structure:

[0217] AST (Abstract Syntax Tree) comparison: Build ASTs of your code and compare the similarity of the tree structures.

[0218] Control flow and data flow analysis: Evaluate the execution paths and data dependencies of your code.

[0219] 5) Similarity analysis based on semantics:

[0220] Semantic Similarity: Evaluates similarity by analyzing the actual functionality and behavior of the code.

[0221] Symbolic execution or dynamic analysis: executing snippets of code to understand their semantics and comparing outputs or behaviors.

[0222] 6) Homology determination based on machine learning:

[0223] Train machine learning models: Use historical data to train models to identify code homology.

[0224] Cluster analysis: Cluster code snippets to discover potentially similar groups of code.

[0225] 2. Binary homology detection

[0226] 1) Static feature homology analysis: Compare static features such as string features, file compilation features, digital certificate features, and file description features of executable files to detect whether the files have the same source code or the same functions.

[0227] 2) Binary code slice homology analysis: Extract code slices with key API calls as entry points in the program, and compare the similarities between code slices to detect whether the files have the same source code or the same functions.

[0228] 3. Sensitive character detection: Perform string matching and analysis on the code file, use regular expressions or other string matching algorithms to find strings containing Chinese characters, and then further analyze and judge based on the location and context of specific characters to determine whether sensitive character features exist. At the same time, natural language processing technology can also be used to perform word segmentation and part-of-speech tagging on the text in the code file to improve the accuracy and efficiency of detection.

[0229] 4. Static feature detection: Combine decryption, reverse engineering and other technologies to remove code obfuscation, packing and other information, and determine known penetration test codes by extracting and matching program codes.

[0230] 5. Dynamic feature detection: Monitor and analyze the behavior of the program at runtime based on the virtual environment, and combine learning algorithms to identify known and suspected abnormal penetration test codes.

[0231] 6. Network feature detection: Identify penetration test behaviors based on network features such as DNS records, HTTP header information, port information, and certificate information.

[0232] 7. Validity verification: Generate a simulated target machine environment based on target system fingerprint information, target network return data packet characteristics, target port opening status, and other information collected from various historical penetration tests. Integrate network traffic monitoring system, error log monitoring system, and memory data monitoring system to comprehensively monitor the operation of each part of the simulated target machine, and verify the execution effect of the penetration test tool from three dimensions: network, local operation status, and memory data.

[0233] 1) Validation based on network traffic monitoring

[0234] Based on traffic capture, traffic analysis, traffic detection and behavioral analysis, it captures and analyzes network content such as vulnerability exploitation test components, Shellcode test communication data, permission maintenance test stability, concealment, etc.

[0235] 2) Verification of the effectiveness of target machine error log monitoring

[0236] Based on log collection, log parsing, data analysis and other means, we analyze system logs, program error logs, firewall logs and other information to accurately locate code-level problems such as Payload construction and vulnerability exploitation testing.

[0237] 3) Validation of target machine memory data monitoring

[0238] Through memory capture, memory analysis, code detection and other means, we analyze the running of Shellcode loading, post-loading execution and other tools in memory, judge the running, encryption and anti-killing of Shellcode, and accurately locate the problems in Shellcode construction and execution.

[0239] Embodiment 3

[0240] Optional source code homology detection includes:

[0241] 1. Metric-based source code homology detection

[0242] Select metrics: Choose appropriate code metrics based on the analysis objectives, such as cyclomatic complexity, number of lines of code, naming conventions, code density, number of operations and operators in the program, and potential execution times, as well as object-oriented design weight (Weighted Methods per Class), coupling between objects, inter-class coupling, responsiveness, etc.

[0243] Standardization: Standardize the code, including unified naming conventions, code style, indentation, etc.

[0244] Denoising: remove comments, blank lines, non-code text (such as documentation), etc.

[0245] Metric extraction: Apply metrics to the preprocessed code snippets and calculate the corresponding metric values.

[0246] Build metric vector: Build the metric value of each code snippet into a vector or multi-dimensional data point.

[0247] Similarity measurement: Select similarity measurement methods such as cosine similarity, Jaccard similarity, and Euclidean distance to calculate the similarity scores between code snippets.

[0248] Result analysis: Determine the threshold of the similarity score based on experience or experiments to judge the similarity of the code snippets.

[0249] 2. Source code homology detection based on hash text comparison

[0250] Choose a hash function: MD5, SHA-1, SHA-256, etc.

[0251] Clean code: remove comments, blank lines, metadata, and other non-code text.

[0252] Standardize code: Ensure consistent code style, such as unified indentation, naming conventions, etc.

[0253] Ignore compiler dependencies: remove compiler-specific generated code, such as debug information, etc.

[0254] Calculate hash values: Use file-level hashing, or chunk the code; then hash each code file or snippet to generate a unique hash value.

[0255] Build hash database: Create a database to store the hash value of each code file or fragment and its corresponding file name or location.

[0256] Hash value comparison: For a new code snippet, calculate its hash value and compare it with the hash value in the database.

[0257] Direct comparison: If an exact match of hash values ​​is found, it indicates that the code snippet is identical or highly similar to a code snippet in the database.

[0258] Approximate comparison: If the hash values ​​are not exactly the same, but have some similarity (for example, most of the bits of the hash values ​​are the same), the similarity of the codes can be further evaluated using edit distance or other similarity metrics.

[0259] Handling hash collisions: If different code snippets produce the same hash value, combine other homology detection methods to further analyze the code.

[0260] Result analysis: Based on similar code snippets and their similarity scores, the similarity between code snippets is determined.

[0261] 3. Homology detection based on large-scale abstract syntax tree AST comparison

[0262] Source code parsing: Parse the source code file through the compiler front end of the programming language or a specialized parsing tool, build a structured representation of the code, and generate the corresponding AST.

[0263] AST preprocessing: Clean AST: remove information that does not affect the program logic, such as comments, blank lines, unnecessary modifiers, etc.

[0264] Standardize AST: unify naming conventions, formatting styles, etc.

[0265] AST structuring: Convert AST into an intermediate representation (IR) suitable for semantic analysis, such as a control flow graph (CFG) and a program dependency graph (PDG).

[0266] Feature extraction: Extract comparative features such as node type, attributes, control flow structure, data flow structure, etc. from AST.

[0267] Similarity measurement: Quantify the similarity between ASTs through methods such as edit distance, subtree similarity, and node matching algorithms.

[0268] Similarity comparison: Compare and calculate AST or its feature similarity scores based on the defined similarity metric through algorithms such as longest common subsequence (LCS), pattern matching, and tree traversal, and filter out pairs of source code with the same source code based on the set threshold.

[0269] 4. Semantic-based source code homology detection

[0270] Code syntax and semantic analysis: Use the compiler front end or a specialized parsing tool to perform syntax and semantic analysis on the source code and generate an abstract syntax tree (AST).

[0271] AST transformation: Convert the AST into a control flow graph (CFG), program dependency graph (PDG), or an intermediate representation (IR) suitable for semantic analysis.

[0272] Semantic enhancement: By adding type information, data flow information, control flow information, etc., the IR is semantically enhanced to enrich the semantic content of the IR.

[0273] Feature extraction: Extract comparative features such as function calls, variable declarations, loop structures, and logical operations from the semantically enhanced IR.

[0274] Context analysis: Analyze the usage patterns, calling relationships, usage of library functions, etc. of the code to determine the context and usage of the code and further understand the semantics of the code.

[0275] Build semantic models: Create semantic models based on logical expressions, program slicing, or other semantic analysis to capture the semantic content and behavior of code snippets.

[0276] Semantic matching and comparison: Use semantic matching techniques such as pattern matching, graph matching, logical reasoning, and neural network models to compare the semantic models of code snippets and identify similar code segments.

[0277] Based on the results of the similarity measurement, similar code snippet pairs are filtered out.

[0278] 5. Code homology detection based on machine learning and data mining

[0279] Data preprocessing: Clean the data and remove irrelevant information, such as comments, spaces, etc.

[0280] Standardize code to conform to a consistent format and style.

[0281] Feature extraction: Extract features such as the AST structure, control flow graph, data flow graph, n-gram sequence, code word frequency, number of code lines, complexity measurement, etc. from the source code.

[0282] Use methods such as word embedding or code embedding to convert the code into a numerical vector representation.

[0283] Feature selection: Use correlation analysis, feature importance assessment, recursive feature elimination and other methods to select the most informative feature subset from the extracted features.

[0284] Dataset partitioning: Divide the dataset into training set, validation set, and test set.

[0285] Model selection: Select the random forest learning model and use the labeled training dataset to train the machine learning model so that it can predict the homology between code snippets based on the extracted features.

[0286] Model evaluation: Based on indicators such as accuracy, recall, and F1 score, the validation set is used to evaluate the performance of the model. The test set is used for cross-validation to evaluate the generalization ability and performance of the model.

[0287] Model optimization: Adjust the model structure, reselect features, adjust hyperparameters, and other optimizations based on the evaluation results.

[0288] Classification and prediction: Automatically classify new code snippets based on the trained model and predict the possibility that the code snippet belongs to the same category.

[0289] Embodiment 4

[0290] Optional, binary homology testing includes:

[0291] 1. Binary static feature homology analysis

[0292] Disassembly and Intermediate Representation (IR) Construction: Convert binary code into assembly code using disassembly tools such as IDAPro, Ghidra, etc. Convert assembly code into intermediate representation IR such as LLVM IR, CIL (C Intermediate Language), etc.

[0293] Control Flow Graph (CFG) and Data Flow Graph (DFG) Construction: Build CFG and DFG from disassembled code or IR that reveal the control flow and data flow structure of the program.

[0294] Static feature extraction: Extract static features from disassembled code, IR or CFG / DFG, including:

[0295] Function signatures and call graphs; instruction sequences and opcode statistics; structure of control flow graphs and data flow graphs

[0296] Register and memory usage patterns;

[0297] Constant and string features: frequency of occurrence, file path, network address, encryption algorithm, function name, and other information.

[0298] Compilation environment characteristics: compiler-specific information (such as optimization level and warning information), compiler version, compiler compilation options, and other information.

[0299] Code layout and structure (e.g., sections, segments)

[0300] Digital certificate characteristics: information about the signer, validity period of the certificate, etc.

[0301] File description features: file size, creation time, modification time, access time and other information.

[0302] Feature vectorization: Use methods such as histogram, TF-IDF, N-gram model, etc. to convert the extracted features into numerical vectors. Vectorization can be performed.

[0303] Similarity metrics: Similarity metrics such as cosine similarity, Jaccard similarity, and edit distance are used to quantify the similarity between binary files.

[0304] Machine Learning Model Training: Using pattern recognition techniques, neural network deep learning models identify and classify similarities between codes.

[0305] Homology Analysis: Use the extracted feature vectors and similarity measurement methods to perform homology analysis on binary files.

[0306] Use the trained deep learning model to classify unknown samples.

[0307] 2. Binary code slice homology analysis

[0308] Disassembly: Use a disassembly tool (such as IDA Pro, Ghidra, etc.) to convert binary files into assembly code.

[0309] Build Control Flow Graph (CFG): Build CFG from assembly code, identifying basic blocks, functions, loop structures and the control flow relationships between them.

[0310] Extract code slices:

[0311] a) Classification of penetration testing tools: Classify the input binary files into different code types such as reconnaissance test, breakthrough test, control and defense test, etc.

[0312] b) Extraction of key API calls: For different code types, extract key API calls such as network communication API and file operation API as entry points;

[0313] c) Code slice extraction: Starting from the entry point, according to CFG, static analysis tools such as IDA Pro and Binary Ninja are used to automatically extract binary code slices.

[0314] Feature extraction: Extract static features from each slice, including:

[0315] Instruction sequences and opcodes, control flow structures (e.g., loops, branches), data flow structures (e.g., data dependencies), register and memory usage patterns, frequency of constant and string occurrences, compiler-specific information (e.g., optimization flags),

[0316] Feature vectorization: Use histogram, TF-IDF, N-gram model and other technologies to convert the extracted features into numerical vectors.

[0317] Code slice comparison: Compare code slices of different samples of the same type and calculate the similarity between them using algorithms such as edit distance and cosine similarity.

[0318] Homology analysis: A code slicing analysis model based on file test types. For each sample, it is compared with other samples of the same type and their similarity scores are calculated.

[0319] Embodiment 5

[0320] Optional, sensitive character feature detection includes:

[0321] 1. Construction of sensitive word database: Establish a database including politically sensitive words, sensitive words for specific language and cultural backgrounds, and other characters.

[0322] 2. Text processing:

[0323] Word segmentation: perform word segmentation for Chinese, Japanese, Korean, etc.

[0324] Character encoding: Perform character encoding processing such as UTF-8 and support special characters and symbols in various languages.

[0325] Contextual Analysis: Considering the differences in context in different languages, language-specific Natural Language Processing (NLP) tools are used to analyze the context and semantics of the text.

[0326] Consider cultural differences: Different cultures have different definitions of sensitive content. Based on cultural differences, the sensitivity of certain words or expressions should be revised.

[0327] Preprocessing: remove punctuation marks from the input text, convert it to lowercase (for case-sensitive languages), etc.

[0328] 3. Feature extraction:

[0329] Extract features from the processed text, including:

[0330] Single word: directly matches the words in the sensitive word library.

[0331] Stem or Root: Matches the stem or root form of a word to identify inflections or derivatives.

[0332] Contextual features: Consider the context of words to reduce false positives.

[0333] 4. Pattern matching: Use regular expressions, Aho-Corasick algorithm and other pattern matching algorithms to quickly identify sensitive words in the text.

[0334] 5. Natural Language Processing (NLP):

[0335] Apply NLP technology to improve the accuracy of detection, including:

[0336] Part-of-speech tagging: Identifying the part of speech of a word to help understand the context.

[0337] Dependency Analysis: Analyze dependencies in sentences to better understand the purpose of vocabulary.

[0338] Entity Recognition: Identify and process entities such as proper nouns, places, organizations, etc.

[0339] 6. Machine Learning: Use machine learning algorithms such as classifiers and cluster analysis to train based on large amounts of labeled data to identify potentially sensitive content.

[0340] Embodiment 6

[0341] Optional, static test feature detection includes:

[0342] 1. Obtain program code: Obtain the binary code or source code of the program by disassembling or decompressing;

[0343] 2. Text preprocessing: remove punctuation marks, convert to lowercase (for case-sensitive languages), and perform word segmentation (for languages ​​that require word segmentation) on the input text.

[0344] 3. Extract features: Extract features of program code based on known penetration test code features such as string features, file compilation features, digital certificate features, and file description features.

[0345] Feature matching: Use regular expressions, feature hashing and other methods to compare and match the extracted features with known penetration test code features to determine whether the program contains penetration test code.

[0346] 4. Static detection optimization

[0347] 1) Natural Language Processing (NLP): including:

[0348] Part-of-speech tagging: Identifying the part of speech of a word to help understand the context.

[0349] Dependency Analysis: Analyze dependencies in sentences to better understand the purpose of vocabulary.

[0350] Entity Recognition: Identify and process entities such as proper nouns, places, organizations, etc.

[0351] 2) Heuristic analysis: Generate new heuristic rules based on the identified penetration test behavior patterns, and apply heuristic methods to identify penetration test behaviors such as unusual instruction sequences and abnormal control flow structures.

[0352] 3) Machine learning assistance: Use classification machine learning algorithms to train models based on a large number of labeled samples to achieve automated detection.

[0353] Embodiment 7

[0354] Optionally, dynamic test feature detection includes:

[0355] 1. Dynamic execution of programs: Load the program to be tested into a physical or virtual sandbox environment and start the execution of the program; generate input data randomly or based on policies, and drive the program to execute different paths.

[0356] 2. Execution tracking: records the execution process of the program, including system calls, API calls, file operations, network activities, process and thread creation, etc.

[0357] 3. Feature extraction

[0358] Behavior analysis: Use behavioral analysis tools to monitor and analyze the behavior of the program during runtime, and extract specific dynamic features such as network connections and file operations;

[0359] Code instrumentation: insert code into the program to be tested, record the program's runtime behavior, and extract specific dynamic features from it;

[0360] Signs include: unusual system call sequences, file system operations such as creating, modifying, or deleting files, network communications, especially with known suspicious servers, process injection or creation of hidden processes, attempts to modify system configuration or registry entries, unusual memory usage patterns

[0361] 4. Behavioral analysis: Compare and match network communication features, file operation features, process behavior features, etc. with known penetration test code features to determine whether the program contains penetration test code.

[0362] 5. Heuristic detection: Formulate and apply heuristic rules based on expert knowledge and historical data, and combine statistical methods to identify suspicious behaviors, atypical abnormal behaviors, etc.;

[0363] 6. Machine learning training: Based on a large amount of labeled data, it uses the Long Short-Term Memory (LSTM) algorithm to identify and classify penetration testing behaviors.

[0364] Embodiment 8

[0365] Optionally, network feature detection includes:

[0366] 1. Packet analysis: Decode and parse the captured network traffic packets to extract various protocol and application layer information such as IP address, port number, protocol type, header information, and payload content.

[0367] 2. Feature extraction: Extract key features such as specific byte sequences, patterns, behaviors, statistical properties such as traffic frequency, specific encoding or encryption methods, etc. from the parsed data.

[0368] 3. Establish a baseline: Based on time series analysis, establish a statistical model or network behavior baseline based on normal traffic data.

[0369] 4. Training model: Train the Long Short-Term Memory (LSTM) learning model to analyze traffic data to identify abnormal traffic behavior.

[0370] 5. Behavior analysis: Based on the model, analyze traffic behavior patterns such as communication frequency, duration, flow direction, and protocol usage; based on the model analysis results, identify abnormal traffic patterns such as uncontrolled propagation tests and data leakage tests that deviate from the normal baseline. The correlation between some specific information and penetration test behavior identification is as follows:

[0371]

[0372] Embodiment 9

[0373] Optionally, perform validity verification including:

[0374] 1. Effectiveness verification based on network traffic monitoring

[0375] 1) Traffic capture: Use network monitoring tools such as Wireshark, tcpdump, Bro / Zeek to capture network traffic and obtain data such as source and destination IP addresses, port numbers, protocol types, and packet contents.

[0376] 2) Traffic analysis: Use traffic analysis tools such as NetFlow, sFlow, and IPFIX to obtain statistical information such as network traffic size and connection frequency, and identify normal network communication patterns and abnormal behaviors.

[0377] 3) Traffic detection: Use specific network features such as IP addresses, domain names, hash values, etc. to identify suspicious traffic, and use signature-based detection methods to identify known behavior patterns and penetration testing software.

[0378] 4) Behavioral analysis: Analyze the behavior patterns of communication frequency, data transmission volume, protocol usage, etc. in network traffic to identify potential penetration testing activities, and use the Support Vector Machines (SVM) learning model to identify traffic that is significantly different from normal behavior patterns.

[0379] 5) Vulnerability exploit component detection: Identify specific HTTP request paths, parameters or header information and other network requests related to vulnerabilities, analyze data packet contents, and look for suspicious codes or commands that can be used for vulnerability exploitation testing.

[0380] 6) Shellcode communication data detection: detect non-standard port connections in network traffic, suspicious patterns in encrypted traffic, and other data packets that may contain shellcode, analyze the payload of the data packet, and determine abnormal instruction sequences, buffer overflow test patterns, and other shellcode execution traces.

[0381] 7) Analysis of the stability of authority maintenance: Monitor network traffic for account password cracking tests, backdoor program deployment, and other behaviors that can be used to maintain authority, analyze communication patterns and behaviors, and identify testing mechanisms used to maintain control over the target system.

[0382] 8) Tool concealment analysis: Analyze the traffic’s timestamp, protocol type, port number and other metadata to find traces of covert communications and identify hidden behaviors of penetration testing activities such as data segmentation, protocol obfuscation, and encrypted communications in network traffic.

[0383] 2. Verification of the effectiveness of target machine error log monitoring

[0384] 1) Log collection: Configure the system and firewall to record detailed log information such as timestamp, source and destination IP addresses, port numbers, protocol types, transport layer information, session status, etc.

[0385] 2) Log parsing: Define parsing rules for specific formats and structures of different types of logs, and use log parsing tools such as Logstash, Grok, or other custom scripts to extract key information from logs.

[0386] 3) Anomaly detection: Build abnormal behavior indicators based on known normal behavior patterns and security policies, and combine statistical analysis and decision tree algorithms to identify abnormal behaviors.

[0387] 5) Payload construction analysis: Detect abnormal HTTP requests, suspicious file uploads, encrypted or obfuscated data transmissions, and other traces indicating suspicious payload construction in the logs, analyze the request headers, parameters, loads, etc., and look for traces that can be used for code injection testing, SQL injection testing, or other penetration testing.

[0388] 6) Vulnerability Exploitation Test Identification: Monitor connection attempts from known specific IP addresses or IP addresses related to penetration testing activities, identify specific penetration test vectors for known vulnerabilities and other test patterns that may indicate vulnerability exploitation.

[0389] 7) Behavioral analysis: Analyze user and system behavior to identify unusual login attempts, unauthorized configuration changes, activities at non-standard times, and other behaviors that may indicate privilege escalation testing and lateral movement testing.

[0390] 3. Verification of the effectiveness of target machine memory data monitoring

[0391] 1) Memory capture: Use memory forensics tools such as Volatility, Rekall, and FTK Imager to capture and save system memory images.

[0392] 2) Memory analysis: Analyze the process list, network connections, open files, registry entries and other information in the memory image to identify abnormal processes, threads, module loading and memory allocation.

[0393] 3) Code detection: Use software analysis sandboxes such as Cuckoo Sandbox and Joe Sandbox to dynamically analyze suspicious behaviors in memory in combination with features such as specific byte sequences and API call patterns.

[0394] 4) Shellcode analysis: Identify shellcode loading points such as new modules and injected code segments in the process space in memory, and analyze the shellcode execution environment such as the process it is loaded into and the memory layout during execution.

[0395] 5) Execution status determination: Analyze the code execution path, system calls and API calls in memory to determine the execution status of shellcode and identify the triggering mechanism of shellcode such as specific events, time or user input.

[0396] 6) Runtime behavior analysis: monitor process behaviors such as thread creation, module loading, memory mapping, etc. in memory, identify suspicious activities, analyze communication and data transfer between processes, and look for interactions between shellcode and other penetration testing components.

[0397] 7) Encryption and anti-killing analysis: Identify encrypted shells or packed programs in memory, analyze their encryption and decryption mechanisms, and detect code obfuscation and anti-analysis techniques in memory such as anti-debugging and anti-sandbox strategies.

[0398] 8) Shellcode structure location: Use disassembly and reverse engineering tools such as IDA Pro, Ghidra, and Radare2 to analyze the structure and function of shellcode, and identify vulnerability exploitation tests, backdoor tests, and data theft tests in shellcode.

[0399] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0400] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0401] Finally, it should be noted that the security detection and performance verification method of a penetration testing tool disclosed in the embodiment of the present invention discloses only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for security detection and effectiveness verification of a penetration testing tool, characterized in that: The method comprises: S1, perform security testing on the penetration testing tool to obtain the security testing results of the penetration testing tool, including: S11, performing source code homology detection on the penetration testing tool to obtain source code homology detection information; S12, performing binary homology detection on the penetration testing tool to obtain binary homology detection information; S13, performing sensitive character processing on the penetration testing tool to obtain sensitive character feature information; S14, performing a static test on the penetration testing tool to obtain static test feature information; S15, dynamically testing the penetration testing tool to obtain dynamic testing feature information; S16, performing network feature analysis on the penetration testing tool to obtain network feature information; S17, integrating the source code homology detection information, the binary homology detection information, the sensitive character feature information, the static test feature information, the dynamic test feature information and the network feature information to obtain a penetration testing tool security detection result; S2, obtaining penetration test historical data information; the penetration test historical data information includes network traffic information, log information and memory data information; S3, processing the penetration test historical data information to obtain the penetration test tool effectiveness verification result, including: S31, processing the network traffic information to obtain a validity verification result based on network traffic monitoring; S32, processing the log information to obtain the validity verification result based on the target machine error log monitoring; S33, processing the memory data information to obtain a validity verification result based on the target machine memory data monitoring; S34, integrating the validity verification result based on network traffic monitoring, the validity verification result based on target machine error log monitoring and the validity verification result based on target machine memory data monitoring to obtain the penetration testing tool effectiveness verification result.

2. The method for security detection and effectiveness verification of penetration testing tools according to claim 1, characterized in that: The source code homology detection is performed on the penetration testing tool to obtain source code homology detection information, including: S111, preprocessing the source code of the penetration testing tool to obtain preprocessed source code information; S112, performing a metric-based similarity analysis on the preprocessed source code information to obtain metric similarity information; S113, performing text-based similarity analysis on the preprocessed source code information to obtain text similarity information; S114, performing a structure-based similarity analysis on the preprocessed source code information to obtain structure similarity information; S115, performing semantic-based similarity analysis on the preprocessed source code information to obtain semantic similarity information; S116, processing the metric similarity information, the text similarity information, the structural similarity information and the semantic similarity information to obtain source code homology detection information.

3. The method for security detection and effectiveness verification of penetration testing tools according to claim 2, characterized in that: The performing a metric-based similarity analysis on the preprocessed source code information to obtain metric similarity information includes: S1121, dividing the preprocessed source code information to obtain N source code segments, where N is a positive integer; S1122, processing the N source code fragments according to a preset metric standard to obtain a metric value vector; S1123, performing similarity analysis on the metric value vector to obtain metric similarity information.

4. The method for security detection and effectiveness verification of penetration testing tools according to claim 2, characterized in that: The performing text-based similarity analysis on the preprocessed source code information to obtain text similarity information includes: S1131, decomposing the preprocessed source code information to obtain M code fragments, where M is a positive integer; S1132, performing hash processing on the M code snippets to obtain M hash values; S1133, processing the M hash values ​​to obtain text similarity information.

5. The method for security detection and effectiveness verification of penetration testing tools according to claim 1, characterized in that: The binary homology detection is performed on the penetration testing tool to obtain binary homology detection information, including: S121, performing binary static feature homology analysis on the penetration testing tool to obtain static homology feature information; S122, performing binary code slice homology analysis on the penetration testing tool to obtain code homology information; S123, processing the static homology feature information and the code homology information to obtain binary homology detection information.

6. The method for security detection and effectiveness verification of penetration testing tools according to claim 1, characterized in that: The sensitive character processing of the penetration testing tool to obtain sensitive character feature information includes: S131, processing the source code of the penetration testing tool to obtain a character string containing Chinese characters; S132, performing text processing on the character string containing Chinese characters to obtain pre-processed text information; S133, performing feature extraction on the preprocessed text information to obtain text feature information; S134, using the text feature information, training a preset sensitive character detection model to obtain an optimized sensitive character detection model; S135, using the optimized sensitive character detection model, performing sensitive character processing on the penetration testing tool to be processed to obtain sensitive character feature information.

7. The method for security detection and effectiveness verification of penetration testing tools according to claim 1, characterized in that: The processing of the network traffic information to obtain the validity verification result based on the network traffic monitoring includes: S311, using a network monitoring tool to capture network traffic and obtain network traffic information; the network traffic information includes the traffic size, connection frequency, communication frequency, data transmission volume, protocol, data packet content, timestamp, protocol type and port number of the network traffic; S312, processing the flow size of the network flow and the connection frequency to obtain a flow analysis result; S313, processing the communication frequency, the data transmission volume and the protocol to obtain a behavior analysis result; S314, analyzing the data packet content to obtain vulnerability exploit component detection results; S315, performing communication detection on the network traffic information to obtain a communication data detection result; S316, performing authority maintenance detection on the network traffic information to obtain an authority maintenance stability analysis result; S317, processing the timestamp, the protocol type and the port number to obtain a tool concealment analysis result; S318, integrating the traffic analysis results, the behavior analysis results, the vulnerability exploitation component detection results, the communication data detection results, the permission maintenance stability analysis results and the tool concealment analysis results to obtain a validity verification result based on network traffic monitoring.

8. The method for security detection and effectiveness verification of penetration testing tools according to claim 1, characterized in that: The log information is processed to obtain the validity verification result based on the target machine error log monitoring, including: S321, parsing the log information to obtain log parsing information; S322, extracting keywords from the log parsing information to obtain log keyword information; S323, performing anomaly detection on the log keyword information to obtain an anomaly detection result; S324, performing payload analysis on the log parsing information to obtain a structural analysis result; S325, performing vulnerability exploitation analysis on the log parsing information to obtain a vulnerability exploitation test identification result; S326, performing behavior analysis on the log parsing information to obtain a behavior analysis result; S327, integrating the anomaly detection result, the structure analysis result, the vulnerability exploitation test identification result and the behavior analysis result to obtain a validity verification result based on target machine error log monitoring.

Images