A chain information collection and vulnerability troubleshooting method and related products
Through chain information collection and vulnerability investigation methods, combined with active and passive mode information collection technology, an asset fingerprint library is built and matched with the vulnerability database, which solves the problems of low vulnerability detection efficiency and insufficient coverage in the existing technology, and achieves the correct and correct Comprehensive and accurate vulnerability detection of complex systems improves the reliability and adaptability of detection.
Patent Information
- Application Number
- CN202411251768.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-09-09
AI Technical Summary
Existing vulnerability detection methods have low detection efficiency, insufficient coverage, and low accuracy in complex network environments and diversified system structures, making it difficult to fully identify security threats in the system.
Using chain information collection and vulnerability investigation methods, combining active mode and passive mode, generate abnormal HTTP messages, use custom port sequence scanning and active crawling technology to obtain system information, and analyze network traffic by combining traffic detection and deep packet analysis technology. Use fingerprint recognition technology to build an asset fingerprint library and match it with the vulnerability database for vulnerability detection.
It realizes comprehensive and accurate detection of security vulnerabilities in the system, improves detection efficiency and accuracy, ensures the reliability and effectiveness of detection results, adapts to complex system structures, and reduces the need for manual intervention.
Smart Images

Figure CN118764326B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information security technology, and in particular to a chain information collection and vulnerability troubleshooting method and related products. Background Art
[0002] With the rapid development of the Internet, the complexity and scale of information systems are increasing, and security vulnerabilities in the system have become an important factor that seriously threatens system security. Traditional vulnerability detection methods usually rely on a single scanning tool or manual inspection, which is inefficient and prone to missing critical vulnerabilities. In addition, existing vulnerability detection methods often have insufficient adaptability when dealing with diverse system structures and complex network environments, making it difficult to comprehensively and effectively detect various potential threats in the system.
[0003] In practical applications, vulnerability detection usually requires multiple steps, including information collection, protocol analysis, fingerprint recognition, and final vulnerability verification. Traditional detection methods are mostly focused on a certain link, lacking systematic and integrated solutions, resulting in incomplete detection results and difficulty in accurately identifying real threats in the system. Therefore, how to use fingerprint recognition technology to effectively integrate feature data from different sources based on information collection, and combine it with known vulnerability information to conduct comprehensive and accurate vulnerability detection has become an urgent problem to be solved in the current information security field.
[0004] To sum up, the vulnerability detection methods in the existing technology have shortcomings in detection efficiency, coverage and accuracy, and cannot adapt to the complex needs of modern information systems. There is an urgent need for a chain information collection and vulnerability investigation method that can integrate multiple detection methods to improve the comprehensiveness and accuracy of system vulnerability detection. Summary of the invention
[0005] The technical problem to be solved by the present invention is that the existing vulnerability detection methods have low detection efficiency, insufficient coverage and low accuracy when dealing with complex network environments and diversified system structures. The purpose is to provide a chain information collection and vulnerability troubleshooting method and related products, which realizes comprehensive and accurate identification and detection of security vulnerabilities in the system by integrating multiple information collection and analysis means, thereby improving the effectiveness and reliability of system security assessment.
[0006] The present invention is achieved through the following technical solutions:
[0007] A chain information collection and vulnerability troubleshooting method, comprising:
[0008] Information collection: Generate and send malformed HTTP messages in active mode, and use custom port sequence scanning and active crawler technology to obtain system service and page information; Use traffic detection and deep packet analysis technology in passive mode to analyze network traffic data;
[0009] Asset fingerprint identification: using fingerprint identification technology to match the fingerprint feature vector generated by the information collection results with the known fingerprint library, and combining the matching results to build an asset fingerprint library;
[0010] Vulnerability detection: By matching the asset fingerprint library with the known vulnerability information in the vulnerability database, vulnerability detection is performed to obtain the detection results;
[0011] Result output: output information collection results, asset fingerprint library and detection results.
[0012] Specifically, the malformed HTTP message includes:
[0013] Header information mutation messages: messages containing HTTP header fields in random order, messages that insert illegal fields or replace legal field names with random strings, and messages that insert overlong strings in certain fields;
[0014] URL mutation messages: messages with URLs that are too long and exceed the standard limit, messages with illegal characters inserted in the URL or using incorrect encoding, and messages with URL structure mutations generated by inserting invalid paths or parameters.
[0015] Main body mutation message: POST request message containing overlong data, message using non-standard format to encode main body data, message with characters inserted in main body data;
[0016] Mixed message: Combined header information mutation message, URL mutation message and body mutation message;
[0017] Specifically, the methods of using custom port sequence scanning in active mode include:
[0018] Obtain the IP address range of the target system through configuration files or network detection tools, generate a preliminary list of potential open ports, and initiate the scanning task;
[0019] Construct a priority function based on port usage frequency, port order and adjustment parameters, and calculate the priority of each port through the priority function to generate a set of custom port sequences arranged by priority;
[0020] Using the generated custom port sequence, each port of the target system is scanned one by one, and the response data of the port is obtained by interacting with each port through the underlying protocol; the response data includes communication delay, port version information, and protocol header information;
[0021] The response data of each scanned port is analyzed, and multi-dimensional response feature information is extracted to form a response feature vector set.
[0022] Specifically, methods using active crawler technology include:
[0023] Combine distributed breadth-first search and depth-first search to build a crawler strategy. Breadth-first search is used to preferentially collect shallow pages and subdomains of the target system, while depth-first search is used to deeply explore the nested links within each subdomain and page.
[0024] Extract relevant information about each crawled page, including page content, links, and subdomains;
[0025] Apply deduplication algorithms to deduplicate crawled pages and subdomain information. The deduplication algorithms include a combination of SimHash and MinHash. The uniqueness of the page is determined by calculating the SimHash and MinHash signatures of the page content.
[0026] Based on the deduplicated page data, a page feature vector set is generated. The page feature vector set includes page content features, link features, and subdomain features.
[0027] Specifically, the method of analyzing network traffic data using traffic detection and deep packet analysis technology includes:
[0028] Capture the target system's network communication data through the network device's traffic mirror port or traffic sampling technology. The network communication data includes the source and destination addresses of the data packet, the transmission protocol type, and the port characteristics.
[0029] Use deep packet analysis technology to perform multi-level analysis on captured data packets;
[0030] Parse the data link layer and extract the protocol information of the physical layer and data link layer, including the source MAC address, destination MAC address and protocol type;
[0031] Parse the network layer and extract IP protocol related information, including source IP address, destination IP address, TTL value, transport protocol type and total length of IP packet;
[0032] Parse the transport layer and extract relevant information of the transport layer protocol, including source port number, destination port number, TCP sequence number, confirmation number and flag bit;
[0033] Parse the application layer protocol and extract relevant information of the high-level protocol, including the application layer protocol name, version, protocol header information and the length of the payload data;
[0034] Based on the analysis results, multi-dimensional network traffic feature information is extracted, including protocol distribution features, packet sequence features, application layer data features, communication session time features, and abnormal behavior pattern features;
[0035] The extracted network traffic feature information is structured and processed to generate a set of network traffic feature vectors.
[0036] Specifically, the method of building an asset fingerprint library includes:
[0037] The response feature vector, page feature vector and network traffic feature vector are obtained from the response of the target system through the abnormal HTTP message collection, and a fingerprint feature vector set is generated by integrating multi-source data. The fingerprint feature vector set includes: basic feature vector, high-order feature vector and interactive feature vector;
[0038] The matching result is determined by calculating the similarity between the fingerprint feature vector of the target system and the feature template in the known fingerprint library;
[0039] The matching results are combined with the extracted fingerprint feature vectors to build the asset fingerprint library of the target system.
[0040] Specifically, the matching results include the identified service type characteristics, version information, and port characteristics;
[0041] Service type characteristics: Determine the service type characteristics running in the target system by matching with the service type template in the known fingerprint library;
[0042] Version information: Determine the version information corresponding to the service type by comparing the version features in the fingerprint feature vector with the version features in the known fingerprint library;
[0043] Port features: The port features corresponding to the service type are determined by comparing the open port features in the fingerprint feature vector with the port templates in the known fingerprint library.
[0044] Specifically, the vulnerability detection methods include:
[0045] Arrange the asset fingerprint features in the asset fingerprint library into a standardized data structure, wherein the data structure includes service type features, version information, port features and fingerprint features;
[0046] The data structure in the asset fingerprint library is matched with the vulnerability entries in the vulnerability database for similarity. First, the relevant vulnerability entries are filtered according to the service type characteristics and port characteristics, then the version information is checked to see if it is within the impact range defined by the vulnerability entry, and finally the matching degree is calculated;
[0047] If the matching degree is greater than the set value, it is determined that the corresponding vulnerability exists in the target system.
[0048] A computer program product includes a computer program / instruction, which implements the above-mentioned chain information collection and vulnerability troubleshooting method when executed by a processor.
[0049] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0050] The present invention provides a chain information collection and vulnerability troubleshooting method, comprising the steps of message generation, information collection, asset fingerprint identification, vulnerability detection and result output; firstly, a malformed HTTP message is generated and sent to a target system, and then the service, page and network flow information of the target system are collected through active mode and passive mode; then, a fingerprint feature vector set is generated by integrating the collected page feature vectors, network flow feature vectors and response feature vectors, and the fingerprint feature vector set is matched with a known fingerprint library to build an asset fingerprint library of the target system; finally, vulnerability detection is performed and the detection result is output by matching the asset fingerprint library with known vulnerability information in a vulnerability database.
[0051] The present invention realizes the comprehensive collection of target system information by combining multiple information collection methods, thereby improving the coverage and accuracy of information collection. Fingerprint recognition technology builds a detailed asset fingerprint library by integrating feature data from different sources, thereby improving the adaptability and recognition accuracy of complex system structures.
[0052] The vulnerability detection method achieves comprehensive detection of potential vulnerabilities in the system by matching with the known vulnerability database, ensuring the reliability and effectiveness of the detection results. In addition, the chain operation process simplifies the complex vulnerability troubleshooting process, reduces the need for manual intervention, and improves detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] The accompanying drawings illustrate exemplary embodiments of the present invention and, together with the description thereof, are used to explain the principles of the present invention. These drawings are included to provide a further understanding of the present invention, and the accompanying drawings are included in and constitute a part of this specification and do not constitute a limitation of the embodiments of the present invention.
[0054] Figure 1 It is a flowchart of a chain information collection and vulnerability troubleshooting method according to the present invention. DETAILED DESCRIPTION
[0055] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and implementation methods. It is understood that the specific implementation methods described herein are only used to explain the relevant content, rather than to limit the present invention.
[0056] It should also be noted that, for the convenience of description, only the parts related to the present invention are shown in the drawings.
[0057] In the absence of conflict, the embodiments and features of the embodiments of the present invention may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0058] Embodiment 1, as Figure 1 As shown, a chain information collection and vulnerability troubleshooting method includes:
[0059] Information collection: through active mode, generate and send malformed HTTP messages, and use custom port sequence scanning and active crawler technology to obtain system service and page information; through passive mode, use traffic detection and deep packet analysis technology to analyze network traffic data.
[0060] In this step, information about the target system is collected through active and passive modes. In active mode, malformed HTTP messages are first generated and sent. These malformed messages are used to trigger abnormal behavior of the target system, thereby revealing potential vulnerabilities in the system. Malformed HTTP messages include three types of mutated messages: header information mutated messages, URL mutated messages, and body mutated messages.
[0061] Header information mutation messages: messages containing HTTP header fields in random order, messages that insert illegal fields or replace legal field names with random strings, and messages that insert overlong strings in certain fields (such as User-Agent).
[0062] URL mutation message: a message with a URI that is too long and exceeds the standard limit, a message with illegal characters inserted in the URI or using incorrect encoding, or a message with a URI structure mutation generated by inserting invalid paths or parameters.
[0063] Body variation messages: messages containing POST requests with overlong data, messages that use non-standard formats (such as mixed encoding) to encode body data, and messages that insert characters into body data.
[0064] Mixed message: a message that combines header information mutation, URL mutation, and body mutation.
[0065] Active mode also includes using custom port sequence scanning and active crawler technology to obtain system service and page information. Custom port sequence scanning defines a specific port scanning sequence to scan the open ports of the target system one by one and obtain port-related service information. Active crawler technology builds an automated tool to traverse the target system's pages and extract service and structure information.
[0066] In passive mode, network traffic data is analyzed through traffic detection and deep packet analysis technology. Traffic detection refers to monitoring the flow of data packets in the network to identify possible attack patterns or abnormal behaviors. Deep packet analysis (DPI) technology further analyzes the content of the data packet and extracts detailed information at the protocol level to gain a deeper understanding of the target system.
[0067] Asset fingerprint identification uses fingerprint identification technology to match the fingerprint feature vector generated by the information collection results with the known fingerprint library, and builds the asset fingerprint library based on the matching results; using fingerprint identification technology, the various feature data obtained through information collection are used to generate fingerprint feature vectors. These feature vectors include the service type, protocol, port information, etc. of the target system. Then, these fingerprint feature vectors are matched with the known fingerprint library to identify the specific services and components in the target system, and finally build the asset fingerprint library.
[0068] Vulnerability detection, by matching the asset fingerprint library with the known vulnerability information in the vulnerability database, performs vulnerability detection, identifies possible security vulnerabilities in the target system, and generates corresponding detection results.
[0069] Output results: output information collection results, asset fingerprint library and detection results. Integrate and output the results of the entire information collection, asset identification and vulnerability detection to generate a final vulnerability detection report. The report includes detailed fingerprint information of the target system, detected vulnerabilities and corresponding security recommendations.
[0070] In the second embodiment, the method of using a custom port sequence scan in active mode includes:
[0071] The IP address range of the target system is obtained through configuration files or network detection tools. Configuration files usually contain known network information, while network detection tools identify active IP addresses in the system through active scanning or passive detection. After obtaining the IP address range, a preliminary list of potential open ports is generated to initialize the scanning task.
[0072] After generating a preliminary port list, a priority function based on port usage frequency, port order and adjustment parameters is constructed, and the priority of each port is calculated through the priority function to generate a set of custom port sequences arranged by priority.
[0073] For a custom port sequence collection, , For the ports, The total number of ports that need to be scanned.
[0074] The port usage frequency can be based on historical data or industry standards, and the port order is arranged according to security and usage importance. The adjustment parameters are used to control the priority differences of different ports. The priority value calculated by this function is used to measure the importance of the port, select the port with high priority for priority scanning, and define the priority function ,in, For the The historical usage frequency of each port (from database or big data statistics), is a tuning parameter used to control the steepness of the priority distribution, It is used to set the mean parameter of the scan concentration area; the priority of each port is calculated, and the port with a larger priority function value is selected for priority scanning.
[0075] Using the generated custom port sequence, each port of the target system is scanned one by one, and the response data of the port is obtained by interacting with each port through the underlying protocol; the response data includes communication delay, port version information, and protocol header information; scanning is achieved through parallel deep protocol analysis;
[0076] Define each port The response function of the eigenvector used to represent the port response , ,in, is the communication delay of the port, The port version information is identified by a specific protocol identifier. Protocol header information for the port;
[0077] Multivariate response function ,in, , and To control the weight coefficients of different response characteristics, Port version information Repositories with known services The similarity of Protocol header information With known protocol header library The similarity.
[0078] Analyze the response data of each scanned port, extract multi-dimensional response feature information, and form a response feature vector set , feature information can include port response time, protocol version, whether there is an abnormal response, etc. After the analysis is completed, these feature information are combined into a feature vector set. This feature vector set will serve as the basic data for subsequent fingerprint identification and vulnerability detection.
[0079] In the third embodiment, the method using active crawler technology includes:
[0080] The crawler strategy is constructed by combining distributed breadth-first search and depth-first search. Breadth-first search is used to preferentially collect shallow pages and subdomains of the target system, that is, to first visit pages and resources that are closer to the initial node in the network, so as to quickly obtain the global structure and basic information of the system. Depth-first search is used to deeply explore the nested links within each subdomain and page, so as to discover deeply hidden pages and resources. The combined crawler strategy enables the crawler to efficiently cover all hierarchical structures of the target system and collect as much information as possible.
[0081] In order to optimize crawler efficiency, a multi-level queue structure is designed: main queue ——Store the URLs to be crawled and process them according to the breadth-first principle. Auxiliary queue ——Store the links extracted from the current page and process them according to the depth-first principle. Queue processing flow:
[0082] from Extract the URL , and start processing. Extract All links in the corresponding page , and resolve subdomains . Add the newly extracted link to In. When When empty, from Take out the link and continue deep crawling.
[0083] Extract relevant information of each crawled page, including page content, links and subdomains, reflecting the content structure of the page, the relationship between pages and the overall topology of the target system.
[0084] De-duplication algorithms are applied to deduplicate crawled pages and subdomain information. The de-duplication algorithms include a combination of SimHash and MinHash. The uniqueness of the page is determined by calculating the SimHash and MinHash signatures of the page content. SimHash is an algorithm used to quickly detect document similarity. By calculating the hash signature of the page content, it can effectively identify similar pages. MinHash is an algorithm that deduplicates by calculating the similarity of document collections. By calculating the SimHash and MinHash signatures of each page content, the system can accurately determine whether different pages are duplicate pages, thereby retaining unique page data.
[0085] Based on the deduplicated page data, a page feature vector set is generated. The page feature vector set includes page content features (reflecting the text and structure information of the page), link features (the number of links contained in the page and the targets they point to), and subdomain features (the subdomain information to which the page belongs).
[0086] Embodiment 4: By analyzing the real-time traffic and captured data packets in the network, we can gain an in-depth understanding of the communication behavior, used protocols, service characteristics, and potential security threats of the target system.
[0087] Methods for analyzing network traffic data using traffic detection and deep packet analysis techniques include:
[0088] Capture the target system's network communication data through the traffic mirroring port of the network device (passively monitor network traffic through the mirroring port (SPAN) of the network switch or a network probe) or traffic sampling technology (sample from large-scale network traffic to ensure the representativeness and coverage of data packets). The network communication data includes the source and destination addresses of the data packets, the transmission protocol type, and the port characteristics.
[0089] Use deep packet analysis technology to perform multi-level analysis on captured data packets;
[0090] The link layer is the second layer of the OSI model and is responsible for data transmission between nodes. Parse the data link layer and extract the protocol information of the physical layer and the data link layer, including the source MAC address, destination MAC address and protocol type; ,in, and It is a data packet The source and destination MAC addresses, This is the Ethernet type field, which indicates the upper-layer protocol type (such as IPv4, ARP).
[0091] The network layer is the third layer of the OSI model and is responsible for routing and addressing across different networks. Parse the network layer and extract information related to the IP protocol, including the source IP address, destination IP address, TTL value, transport protocol type, and total length of the IP packet; ,in, and are the source and destination IP addresses of the data packet; is the time-to-live field, indicating the hop limit of the packet; Indicates the transport layer protocol type (such as TCP, UDP); Is the total length of the IP packet.
[0092] The transport layer is the fourth layer of the OSI model and is responsible for end-to-end communication management. Parse the transport layer and extract relevant information about the transport layer protocol, including the source port number, destination port number, TCP sequence number, confirmation number, and flag bit; ,in, and are the source and destination port numbers; and It is the TCP sequence number and confirmation number; It is the TCP flag bit, such as SYN, ACK, etc.
[0093] The application layer is the seventh layer of the OSI model and is responsible for data exchange between user-side applications. Parse the application layer protocol and extract relevant information of the high-level protocol, including the application layer protocol name, version, protocol header information, and the length of the payload data; ,in, It is the name of the application layer protocol, such as HTTP and DNS. Is the protocol version number. It is the protocol header information (such as HTTP header field). Is the payload length of the application layer data.
[0094] Based on the analysis results, multi-dimensional network traffic feature information is extracted, including protocol distribution characteristics (the proportion of different protocol types in network traffic), packet sequence characteristics (the order and arrangement pattern of data packets), application layer data characteristics (type, content and pattern of application layer data), time characteristics of communication sessions (duration of sessions, time interval between data packets), abnormal behavior pattern characteristics (detected abnormal traffic behaviors, such as sudden increase in traffic, duplicate packets, error packets, etc.);
[0095] The extracted network traffic feature information is structured and processed to generate a set of network traffic feature vectors.
[0096] In embodiment 5, a method for constructing an asset fingerprint library includes:
[0097] The response feature vector, page feature vector and network traffic feature vector are collected from the target system through the response of the malformed HTTP message, and a fingerprint feature vector set is generated by integrating multi-source data. The fingerprint feature vector set includes: basic feature vector, high-order feature vector and interactive feature vector; fingerprint feature matrix : ,in, is the basic eigenvector, is a high-order eigenvector, is the interaction feature vector.
[0098] First, we collect a variety of feature data from the target system. These data come from the response of the malformed HTTP message, including response feature vectors, page feature vectors, and network traffic feature vectors. Through multi-source data integration technology, we process these data from different sources to generate a comprehensive fingerprint feature vector set. This set includes three main feature vectors: basic feature vectors, high-order feature vectors, and interactive feature vectors.
[0099] The basic feature vector includes simple and common feature information, such as HTTP response code, port status, etc. That is, the basic attributes of the page are extracted from the page feature vector, the basic information of network communication is extracted from the network traffic feature vector, and the response characteristics of the system are extracted from the response feature vector. The extraction process does not involve complex calculations, only simple reading and sorting. Then the basic features of each source are integrated to form a unified basic feature vector.
[0100] Basic eigenvectors: ,in, is the HTTP response code (such as 200, 404, etc.), is the response time, is the message length, The matches of specific keywords appearing in the response.
[0101] High-order feature vectors are complex features obtained by further processing and analysis of basic features, such as pattern recognition, entropy of response behavior, etc. That is, they are usually obtained by further processing and analysis of basic features. For example, high-order features can be constructed by calculating the entropy of page content, analyzing the distribution of data packets in traffic, or identifying repeated patterns in responses. Use pattern recognition algorithms or statistical analysis methods to deeply process basic features and extract representative high-order feature vectors.
[0102] High-order eigenvectors ,in, The entropy value of the response content is used to measure the randomness and complexity of the data; for pattern recognition features based on the content of the response (e.g., sequential patterns or repetitive structures); To respond to the rate at which certain elements in the content change (such as how often a data field changes).
[0103] The interaction feature vector is generated by analyzing the interaction relationship between the various parts of the system, such as the timing relationship between request-response pairs, traffic patterns, etc. That is, the time series of network traffic and response characteristics are analyzed to identify the timing dependencies and interaction behaviors. For example, the time interval between request-response pairs is analyzed to identify the interaction patterns related to specific services or protocols. Through association analysis, the association relationship between pages, traffic, and responses is identified to generate an interaction feature vector. For example, the interaction characteristics can be derived by analyzing the link structure between multiple pages or by analyzing the correlation between multiple traffic packets.
[0104] Interaction feature vector ,in, is the correlation between different request-response pairs (e.g., the content similarity between adjacent responses); To capture the sequential dependency characteristics between request-response pairs, a sequence model (such as Markov chain) is used; It is the time-dependent characteristics between request-response pairs (such as the time interval pattern between responses).
[0105] In order to obtain the final fingerprint feature vector , the basic feature vector, high-order feature vector and interactive feature vector are fused together. Define the fused fingerprint feature vector ,in, , , It is the weight parameter in the fusion process, which is used to adjust the influence of various features.
[0106] The matching result is determined by calculating the similarity between the fingerprint feature vector of the target system and the feature template in the known fingerprint library; after the fingerprint feature vector set is generated, it is matched with the feature template in the known fingerprint library. The matching process is achieved by calculating the similarity between the fingerprint feature vector of the target system and the feature template in the fingerprint library. The higher the similarity, the more similar the characteristics of the target system are to the known template, thereby determining the specific services, protocols or components that the system may run.
[0107] Fingerprint matching function Used to determine the fingerprint of the target system, ,in, It is the first Fingerprint feature vector. is a similarity calculation function, such as cosine similarity or Euclidean distance. If Value exceeds a certain threshold , then the target system is considered Matches known fingerprint features.
[0108] Combine the matching results with the extracted fingerprint feature vector to build the asset fingerprint library of the target system. Based on the similarity matching results, combine the identified service type features, version information, and port features with the extracted fingerprint feature vector to finally build the asset fingerprint library of the target system. The asset fingerprint library is a detailed description of various types of assets in the target system, including the feature information of each service and component in the system.
[0109] The matching results include the identified service type characteristics, version information, and port characteristics;
[0110] Service type characteristics: Determine the service type characteristics running in the target system by matching with the service type template in the known fingerprint library; service type characteristics can include categories such as Web server, middleware, and database.
[0111] Version information: Determine the version information corresponding to the service type by comparing the version features in the fingerprint feature vector with the version features in the known fingerprint library; determine the version information corresponding to each service type. This step helps to further refine the identification and determine the specific version running in the system.
[0112] Port features: The port features corresponding to the service type are determined by comparing the open port features in the fingerprint feature vector with the port template in the known fingerprint library; the port features are used to identify the specific communication port of the service in the network and help locate the actual operating location of the service.
[0113] In embodiment 6, the method for performing vulnerability detection includes:
[0114] The asset fingerprint features in the asset fingerprint library are organized into a standardized data structure, which includes service type features, version information, port features and fingerprint features; the service type features describe the various services (such as Web servers, databases, etc.) running in the target system, and the version information identifies the specific version corresponding to each service. The port features are used to indicate the communication ports of these services in the network, and the fingerprint features are a summary of the overall system features, which are used in the subsequent matching process.
[0115] The data structure in the asset fingerprint library is matched with the vulnerability entries in the vulnerability database for similarity. The vulnerability database is a collection of pre-stored known vulnerability information. Each vulnerability entry contains the service type, version information, and related port information that may be affected. The matching process is divided into the following steps:
[0116] Preliminary screening: First, filter out possible related vulnerability entries based on service type characteristics and port characteristics. By comparing the service type and port information in the system with the entry characteristics in the vulnerability database, the scope of vulnerabilities that need further inspection is narrowed down.
[0117] Version information verification: After initially screening out relevant vulnerability entries, further check the affected version range of each vulnerability entry and verify whether the service version information of the target system falls within the range. If the service version is within the affected range of the vulnerability, the vulnerability entry will be considered highly relevant.
[0118] Calculate the matching degree: For the vulnerability entries that have passed the version verification, further calculate the matching degree between them and the fingerprint features of the target system. The matching degree is calculated by comparing the fingerprint features of the target system with the relevant features in the vulnerability entry. The matching degree directly reflects the degree of fit between the vulnerability and the target system.
[0119] By screening and matching the asset fingerprint features of the target system with the entries in the vulnerability database layer by layer, the system can identify potential security vulnerabilities. First, the potentially relevant vulnerability entries are screened out, and then the matching degree between the vulnerability and the target system is calculated through precise verification of version information and similarity matching of fingerprint features. Finally, when the matching degree exceeds the set value, it can be considered that the vulnerability exists in the system, and corresponding protective measures can be taken accordingly.
[0120] Embodiment 7, a chain information collection and vulnerability troubleshooting terminal, includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the above-mentioned chain information collection and vulnerability troubleshooting method when executing the computer program.
[0121] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing of the terminal by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an execution program required for at least one function, etc.
[0122] The data storage area can store data created according to the use of the terminal, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0123] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the chain information collection and vulnerability troubleshooting method as described above is implemented.
[0124] Without loss of generality, computer readable media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer readable instruction data structures, program modules or other data. Computer storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state storage technology, CD-ROM, DVD or other optical storage, cassettes, magnetic tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media are not limited to the above. The above-mentioned system memory and mass storage devices can be collectively referred to as memory.
[0125] A computer program product includes a computer program / instruction, which implements the chain information collection and vulnerability troubleshooting method as described above when executed by a processor.
[0126] A computer program product includes a computer program or set of instructions for performing specific tasks or implementing specific functions. These programs or instructions are designed to be executed by a processor to implement a series of predefined steps or operations. The program product may be stored in various forms of computer storage media, such as memory, hard disk, solid-state drive, optical disk or other forms of digital storage devices. It may exist in the form of compiled binary code or in the form of scripts or bytecodes that can be executed by an interpreter. The program product uses carefully designed algorithms and logical instructions to enable the processor to process data in a specific order and manner to complete various functions such as data analysis, user interaction, device control, etc.
[0127] It should be understood by those skilled in the art that the above embodiments are only for the purpose of clearly illustrating the present invention, and are not intended to limit the scope of the present invention. For those skilled in the art, other changes or modifications may be made based on the above invention, and these changes or modifications are still within the scope of the present invention.
Claims
1. A chain information collection and vulnerability troubleshooting method, characterized in that: include: Information collection, through active mode, generates and sends malformed HTTP messages, and uses custom port sequence scanning and active crawler technology to obtain system service and page information; Analyze network traffic data using traffic inspection and deep packet analysis technology in passive mode; Asset fingerprint identification: using fingerprint identification technology to match the fingerprint feature vector generated by the information collection results with the known fingerprint library, and combining the matching results to build an asset fingerprint library; Vulnerability detection: By matching the asset fingerprint library with the known vulnerability information in the vulnerability database, vulnerability detection is performed to obtain the detection results; Result output: output information collection results, asset fingerprint library and detection results; Methods for using custom port sequence scanning in active mode include: Obtain the IP address range of the target system through configuration files or network detection tools, generate a preliminary list of potential open ports, and initiate the scanning task; Construct a priority function based on port usage frequency, port order and adjustment parameters, and calculate the priority of each port through the priority function to generate a set of custom port sequences arranged by priority; For a custom port sequence collection, , For the ports, The total number of ports that need to be scanned; Defining a priority function ,in, For the The historical usage frequency of the ports, is a tuning parameter used to control the steepness of the priority distribution, It is used to set the mean value parameter of the scan concentration area; the priority of each port is calculated, and the port with the larger priority function value is selected for priority scanning; Using the generated custom port sequence, each port of the target system is scanned one by one, and the response data of the port is obtained by interacting with each port through the underlying protocol; the response data includes communication delay, port version information, and protocol header information; Define the response function of each port as a eigenvector representing the port response , ,in, is the communication delay of the port, The port version information is identified by a specific protocol identifier. Protocol header information of the port; multivariate response function ,in, , and To control the weight coefficients of different response characteristics, Port version information Repositories with known services The similarity of Protocol header information With known protocol header library similarity; Analyze the response data of each scanned port, extract multi-dimensional response feature information, and form a response feature vector set ; The method of building an asset fingerprint library includes: The response feature vector, page feature vector and network traffic feature vector are obtained from the response of the target system through the abnormal HTTP message collection, and a fingerprint feature vector set is generated by integrating multi-source data. The fingerprint feature vector set includes: basic feature vector, high-order feature vector and interactive feature vector; Fingerprint feature matrix : ,in, is the basic eigenvector, is a high-order eigenvector, is the interaction feature vector; ,in, is the HTTP response code, is the response time, is the message length, The matching of specific keywords in the response; ,in, is the entropy value of the response content, for pattern recognition features based on response content; To respond to the rate of change of certain elements in the content; ,in, is the correlation between different request-response pairs; is the sequential dependency characteristic between request-response pairs, is the temporal dependency characteristic between request-response pairs; Define the fused fingerprint feature vector ,in, , , is the weight parameter in the fusion process; After generating the fingerprint feature vector set, it is matched with the feature template in the known fingerprint library. The fingerprint matching function ,in, is the th fingerprint feature vector in the known fingerprint library, is the similarity calculation function; According to the results of similarity matching, the identified service type features, version information and port features are combined with the extracted fingerprint feature vector to finally build the asset fingerprint library of the target system.
2. A chain information collection and vulnerability troubleshooting method according to claim 1, characterized in that: The malformed HTTP message includes: Header information mutation messages: messages containing HTTP header fields in random order, messages that insert illegal fields or replace legal field names with random strings, and messages that insert overlong strings in certain fields; URL mutation messages: messages with URLs that are too long and exceed the standard limit, messages with illegal characters inserted into the URL or using incorrect encoding, and messages with URL structure mutations generated by inserting invalid paths or parameters; Main body mutation message: POST request message containing overlong data, message using non-standard format to encode main body data, message with characters inserted in main body data; Mixed message: a message that combines header information mutation, URL mutation, and body mutation.
3. A chain information collection and vulnerability troubleshooting method according to claim 1, characterized in that: Ways to use active crawling techniques include: Combine distributed breadth-first search and depth-first search to build a crawler strategy. Breadth-first search is used to preferentially collect shallow pages and subdomains of the target system, while depth-first search is used to deeply explore the nested links within each subdomain and page. Extract relevant information about each crawled page, including page content, links, and subdomains; Apply deduplication algorithms to deduplicate crawled pages and subdomain information. The deduplication algorithms include a combination of SimHash and MinHash. The uniqueness of the page is determined by calculating the SimHash and MinHash signatures of the page content. Based on the deduplicated page data, a page feature vector set is generated. The page feature vector set includes page content features, link features, and subdomain features.
4. A chain information collection and vulnerability troubleshooting method according to claim 1, characterized in that: Methods for analyzing network traffic data using traffic detection and deep packet analysis techniques include: Capture the target system's network communication data through the network device's traffic mirror port or traffic sampling technology. The network communication data includes the source and destination addresses of the data packet, the transmission protocol type, and the port characteristics. Use deep packet analysis technology to perform multi-level analysis on captured data packets; Parse the data link layer and extract the protocol information of the physical layer and data link layer, including the source MAC address, destination MAC address and protocol type; Parse the network layer and extract IP protocol related information, including source IP address, destination IP address, TTL value, transport protocol type and total length of IP packet; Parse the transport layer and extract relevant information of the transport layer protocol, including source port number, destination port number, TCP sequence number, confirmation number and flag bit; Parse the application layer protocol and extract relevant information of the high-level protocol, including the application layer protocol name, version, protocol header information and the length of the payload data; Based on the analysis results, multi-dimensional network traffic feature information is extracted, including protocol distribution features, packet sequence features, application layer data features, communication session time features, and abnormal behavior pattern features; The extracted network traffic feature information is structured and processed to generate a set of network traffic feature vectors.
5. A chain information collection and vulnerability troubleshooting method according to claim 1, characterized in that: The matching results include the identified service type characteristics, version information, and port characteristics; Service type characteristics: Determine the service type characteristics running in the target system by matching with the service type template in the known fingerprint library; Version information: Determine the version information corresponding to the service type by comparing the version features in the fingerprint feature vector with the version features in the known fingerprint library; Port features: The port features corresponding to the service type are determined by comparing the open port features in the fingerprint feature vector with the port templates in the known fingerprint library.
6. A chain information collection and vulnerability troubleshooting method according to claim 1, characterized in that: Methods for vulnerability detection include: Arrange the asset fingerprint features in the asset fingerprint library into a standardized data structure, wherein the data structure includes service type features, version information, port features and fingerprint features; The data structure in the asset fingerprint library is matched with the vulnerability entries in the vulnerability database for similarity. First, the relevant vulnerability entries are filtered according to the service type characteristics and port characteristics, then the version information is checked to see if it is within the impact range defined by the vulnerability entry, and finally the matching degree is calculated; If the matching degree is greater than the set value, it is determined that the corresponding vulnerability exists in the target system.
7. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by the processor, the chain information collection and vulnerability troubleshooting method as described in any one of claims 1 to 6 is implemented.
