A full node fast acquisition analysis system and method of an i2p network

Abstract

The application relates to a full-node fast collection and analysis system and method of an I2P network, wherein a task is triggered through a central control component, each detection node in a detection component starts the task in parallel after receiving the context of the task, and a speed-up node collects, collected node information is transmitted into a calculation component in real time to complete index calculation, calculation time is saved, and a data analysis component analyzes index information falling into a database after one detection task is completed, and finally obtains recommended nodes and abnormal nodes. Compared with the prior art, the application has the advantages of flexibility, effectiveness, time saving and the like.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a system and method for rapid data collection and analysis across all nodes in an I2P network. Background Technology

[0002] I2P is a scalable, self-organizing, and resilient packet-switched anonymity network layer that can run on any number of applications with varying degrees of anonymity or security awareness. When using an I2P network, users have the freedom to choose based on their needs regarding anonymity, reliability, bandwidth usage, and latency. Applications available in I2P include web browsing, chat, file sharing, email, blogging, and more, with new applications constantly being developed.

[0003] I2P anonymous networks face numerous problems and challenges. Attackers can launch external attacks through surveillance or exploit internal nodes to compromise the network. In multi-route-based anonymous communication systems, collusion attacks are among the most common. Attackers and malicious nodes control multiple nodes on the multi-route to infer user identities and communication information; anonymity is compromised when the proportion of malicious nodes exceeds a certain threshold. Furthermore, attackers can determine the sender and receiver identities by observing the synchronization patterns of communication messages. The longer an attacker observes synchronized communication, the higher the likelihood of node collusion.

[0004] Currently, solutions to collusion attacks on I2P node selection algorithms mainly focus on increasing the cost of nodes entering the hash table, such as using passive solutions like CAPTCHAs and community trust mechanisms. There is no solution that actively selects or avoids selecting certain nodes by collecting and analyzing information from all nodes to improve the security of I2P node selection. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a fast data acquisition and analysis system and method for all nodes in an I2P network.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] This invention provides a rapid data acquisition and analysis system for all nodes in an I2P network, the system comprising:

[0008] Central control component: manages all other components, obtains task information through interaction with the user, and triggers various tasks;

[0009] Probe component: After receiving the task context, it starts the task in parallel through various probe nodes to quickly collect information from all nodes;

[0010] Calculation components: Real-time node information generated by consumption and historical information in the database are used to calculate metrics;

[0011] Data analysis component: After a detection mission is completed, the indicator information that has fallen into the database is analyzed to obtain recommended nodes and abnormal nodes;

[0012] Communication between the central control component and the detection component, computing component and data analysis component is accomplished using a distributed application coordination service; the detection component and the computing component are connected using a message queue that allows for long-term connections.

[0013] Specifically, the central control component includes:

[0014] Task triggering module: Triggers node collection tasks, returns recommended node tasks, and returns abnormal node tasks through user interaction mode or timed triggering mode;

[0015] Task management module: Adopts a distributed file management architecture to uniformly manage all task contexts;

[0016] Task Results Module: This module handles the use of task results, including result display and processing of netDB used directly in the generated I2P.

[0017] Specifically, the detection component includes:

[0018] Node detection module: Generates detection tasks based on the task context to collect node information of the I2P network;

[0019] Progress saving module: Saves the task progress to prevent the task from having to start from the beginning if an error occurs and the task is restarted.

[0020] Information transmission module: rewrites information according to requirements and writes it to the message queue producer.

[0021] Specifically, the computing component includes:

[0022] Information input module: Connects to the message queue consumer to receive streaming data and connects to the database to receive batch data;

[0023] Information calculation module: Aggregates real-time node information and historical node information, and uses various operators to calculate indicators;

[0024] Information output module: stores the calculation results in the database for use by the central control components;

[0025] Information transmission module: rewrites information according to requirements and writes it to the message queue producer.

[0026] Specifically, the data analysis component includes:

[0027] Data storage module: Stores various information and metrics of nodes in the I2P network;

[0028] Data Analysis Module: Analyzes node information and metrics in the database of the data storage module to obtain recommended nodes and abnormal nodes.

[0029] In another aspect, this invention provides a method for rapid data acquisition and analysis across all nodes in an I2P network. This method, based on the aforementioned system, includes the following steps:

[0030] S1: The central control component obtains task information and creates task context through the front end and open interfaces, and retrieves a list of online probe instances through the distributed file system, selecting a probe instance. Specifically:

[0031] The central control component obtains task information through a front-end or open interface to create a task context. It then retrieves a list of online probe instances through a distributed file system, elects instances based on the number of probes, and writes the task number and task start status code to the corresponding instance path. The corresponding probe instance, upon receiving the task start status code, requests the task context from the central control component, constructs the task based on the task context, and writes the corresponding status code of the construction result back to the distributed file system. Simultaneously, the central control component writes the task number and task start status code to the distributed file system's computing component path. The main computing instance elects an instance to execute the task, requests the task context from the central control component, constructs the task based on the task context, and writes the corresponding status code of the construction result back to the distributed file system. The central control component also collects information on the task construction status of both components, decides whether to terminate or rerun the task, and begins continuous monitoring of the distributed file system to check for any abnormalities or successful completion of the task.

[0032] S2: The probe component obtains the task context for the selected probe instance, constructs and starts the task, and generates a status code indicating task completion. Specifically:

[0033] First, request connection resources from the connection pool. After requesting, construct a probe message based on rID and flood node information, and send the message. If the flood node successfully returns node information, send the node information to the message queue, determine the size relationship between rID and rID in the progress database. If it is greater than rID in the process database, acquire a read-write lock and write the data. If the request to the flood node times out and the retry times out, discard the request and release the connection pool resources.

[0034] When a process or instance restarts for some reason, the instance detects the task number that is running in the distributed pull, retrieves the task context from the central control component based on the number, retrieves the task progress from the progress database, and quickly continues the task.

[0035] S3: The computing component calculates the obtained task context, constructs and starts the task, and generates a task completion status code. Specifically:

[0036] Node information enters the computing component from the message queue consumer, and the task context is broadcast to each node of the computing component. In the aggregation node, the IP address in the real-time task information is obtained, and the node's historical information in the database is searched using the IP address as the keyword. The node's real-time information is copied, and the original node information flows out from the main stream and falls into the node information database. The copied real-time node information and historical node information are aggregated based on the amount of historical data in the historical summary information. The node information that has completed aggregation or has timed out enters the computing node, and the index calculation is completed according to the configured computing index. The index falls from the flow measurement into the node index database.

[0037] S4: After receiving the status codes indicating that the computing and detection components have completed their tasks, the central control component notifies the data analysis component to begin working. The data analysis component then identifies recommended and abnormal nodes. Specifically:

[0038] After receiving the status codes indicating that the computing and detection components have completed their tasks, the central control component notifies the data analysis component to begin working. First, it identifies nodes with excessively low levels of both uniform rID distribution and the proportion of nodes with online time exceeding 24 hours, classifying them as suspected anomalous nodes. For suspected anomalous nodes, it analyzes their historical online time and rID distribution, classifying those approaching the level of suspected anomalous nodes as anomalous nodes. Finally, it identifies nodes with excellent performance across multiple metrics, including average capacity, average bandwidth, average latency, and reachability, and classifies them as recommended nodes.

[0039] S5: The central control component writes the task completion status code to the distributed file system probe instance and computing component path, waits for each component to report the task release status code to clear the task path, writes the task information and logs to the database, and obtains the task results from the data analysis component to get the generated netDB file.

[0040] The I2P network full-node rapid data acquisition and analysis system and method provided by this invention have at least the following advantages compared to existing technologies:

[0041] 1) Existing solutions all add some simple indicators (such as node latency) to the local database of I2P nodes to select nodes for tunneling. The reason is that the node database of the I2P network is a distributed network database, and collecting all nodes is costly. If information on a large number of nodes cannot be collected, adding indicators will not only fail to effectively reduce the probability of selecting an attacking node, but may also result in no nodes being available. This invention adopts an I2P full node collection and analysis scheme to actively analyze recommended nodes and abnormal nodes. Full node collection increases the range of nodes that I2P nodes can choose to establish tunnels with, increases the cost for attackers to carry out collusive attacks, and provides more choices, allowing users to select nodes more flexibly.

[0042] 2) To enable the system to collect all nodes in the I2P network more quickly and maintain a working state at all times, this invention adopts a multi-instance detection, streaming computation, and state preservation technical solution. Multi-instance detection refers to the detection component consisting of multiple detection instances, managed uniformly by a central control component through a distributed file system, thus improving detection speed. Streaming computation refers to the real-time inflow of node information obtained by the detection component into the computation component for aggregation and metric calculation, allowing the detection and computation components to work in parallel and reducing computation time. State preservation refers to the state preservation modules of each component working together with the distributed file system, ensuring that if the entire task encounters an anomaly, it can be restored to the state and progress before the anomaly upon restart, without needing to re-execute the lengthy task of collecting all nodes. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the structure of the full-node fast acquisition and analysis system of the I2P network in the embodiment;

[0044] Figure 2 This is a flowchart illustrating the fast data acquisition and analysis method for all nodes in an I2P network as described in this embodiment. Detailed Implementation

[0045] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort should fall within the scope of protection of the present invention.

[0046] Example

[0047] This invention provides a rapid full-node data collection and analysis system for I2P networks. This system can collect full-node information and metrics of an I2P network and provide recommended and abnormal nodes. First, a central control component triggers a task. Upon receiving the task context, each probe node in the probe component begins its task in parallel, accelerating node collection. The collected node information is then fed into the computing component in real time to complete metric calculations, saving computation time. After each probe task is completed, the data analysis component analyzes the metric information stored in the database and ultimately obtains the recommended and abnormal nodes.

[0048] In this patent, an anomalous node refers to a node that may exhibit malicious behavior, characterized by prominent and overly similar anomalous indicators. A recommended node refers to a node whose performance indicators rank among the top of all nodes and which is not an anomalous node.

[0049] In detail, the system of this invention includes a central control component, a detection component, a computing component, and a data analysis component. The central control component is responsible for managing all other components, interacting with users, and triggering various tasks; the detection component, as a producer, is responsible for quickly collecting information from all nodes; the computing component is responsible for consuming real-time generated node information and historical information in the database, and calculating indicators; the database component is responsible for data storage and some non-streaming calculations. Communication between the central control component and other components should be coordinated using a distributed application service; the detection component and the computing component should be connected using a message queue capable of long-lived connections.

[0050] The central control component of the system of the present invention includes:

[0051] Task triggering module: Triggers tasks such as node collection, returning recommended nodes, and returning abnormal nodes through user interaction or timed triggering.

[0052] Task Management Module: Using a distributed file management architecture, it manages all task contexts in a unified manner, ensuring context consistency. It can also work with other components and related modules to enable tasks to recover, preventing long-running tasks from failing or taking too long due to interruptions or other anomalies.

[0053] Task Results Module: Handles the use of task results, including result display and generating netDB files that can be directly used by I2P (a header file specific to Unix and Linux, which mainly defines network-related structures, variable types, macros, functions, etc.).

[0054] Furthermore, the task triggering module should trigger different types of tasks through user interaction modes such as the front end or a timed trigger mode when the daily rID (a value calculated by the node number and date hash, representing the node's position in the hash ring) changes. When triggering a node collection task, the detection component, calculation component, and data analysis component need to be scheduled; when triggering a task to return a recommended node or an abnormal node, the data analysis component needs to be scheduled.

[0055] Furthermore, the task management module should maintain the task number and running status code in the distributed file system and write the entire task context into the database. When the central control component encounters an error or restarts, the context can be retrieved from the database to continue the task. When other components encounter an error or restart, the task number and running status can be retrieved from the distributed file system to request the context again to continue the task.

[0056] Based on the above, the interaction between the central control component, the computing component, and the detection component is as follows: The central control component writes the task number and task start status code to the corresponding path in the distributed file system. After the computing and detection components detect changes in the leaf nodes of the corresponding path, they request the task context from the central control component based on the task number, construct the task according to the context, and write back a running status code to notify the central control component that the task is running normally upon successful construction. Throughout the task, the central control component monitors the task path in the distributed file system and pays attention to the status codes. If other components write back abnormal status codes, it decides to rerun or terminate the task. When the computing component completes its calculation, it reports its status to the central control component, which then writes an end status code to the distributed file system to notify all components that the current task has ended. Each component divides its process at the task granularity, and each task generates different distributed file paths through different task numbers for inter-component communication, maintaining consistency and preventing interference between components.

[0057] Furthermore, the task result module refers to the ability of the central control component to produce various output results after interacting with data analysis, including but not limited to displaying on the front end, generating a CSV file containing the results, and generating a local netDB file that can be directly used by the I2P network.

[0058] The detection components of the system of this invention include:

[0059] Node detection module: Generates detection tasks based on the task context to collect node information of the I2P network;

[0060] Progress saving module: Saves the task progress to prevent the task from having to start from the beginning if an error occurs and the task is restarted.

[0061] Information transmission module: It can rewrite information as needed and write it to the producer end of the message queue.

[0062] Furthermore, the node detection module should retrieve the rID range of the nodes it is responsible for detecting from the task context and use a thread pool for parallel detection. The thread pool should be configured according to the actual configuration of the detection instance to ensure that there is neither memory accumulation nor memory overflow. After obtaining the rID, a single thread constructs an I2P I2NP exploration message and sends it to the flooded node list in the context to obtain node information.

[0063] Furthermore, the progress saving module should record the rID in the response to each exploration message and write it to the database. When the process restarts due to an exception such as memory overflow, the probe instance can find the number of the running task in the distributed file system and retrieve the task context based on the number. Then, the probe instance retrieves the rIDs that were detected before the task restarted from the database. The rIDs are the lower bound of the range of rIDs obtained from the context, which are incremented, thus avoiding duplicate probes. This is very important for such long-running tasks.

[0064] Each probe instance in the probe component of this invention should have an immutable unique ID. This ID, as part of the distributed file system path, ensures that each probe instance has a unique path. When the central control component issues tasks, it can elect probe instances existing in the path and send different contexts. After an instance process restarts, it can also obtain the task status from the corresponding path based on its immutable ID and continue executing the task. A complete probe instance's task path should be / task / instanceID / task ID / , and the contents of the files under that path should be the task's status code. The probe component has a corresponding set of status codes for different stages of the task and for different exceptions, which are reported through the status path in the distributed file system, so that the central control component can grasp the running status of each probe instance in the probe component.

[0065] Furthermore, the information transmission module is responsible for disassembling and organizing the node information in the returned response after the probe instance receives a response, and then writing it into the message queue. The message queue should be a peer-to-peer asynchronous message queue that can maintain long-term connections, with the probe component acting as the message producer and the computation component acting as the message consumer.

[0066] The computing components of the system of this invention include:

[0067] Information input module: Connects to the message queue consumer to receive streaming data and connects to the database to receive batch data;

[0068] Information calculation module: Aggregates real-time node information and historical node information, and then uses various operators to calculate indicators;

[0069] Information output module: Stores the calculation results in the database for use by the central control components.

[0070] The computing and detection components of this invention interact with the central control component in roughly the same way, obtaining task numbers from the distributed file system and requesting context from the central control component. The difference is that the computing component does not use instance numbers for distributed management; instead, it employs a master-slave management architecture of a distributed streaming data engine for internal management, eliminating the need to number instances in file paths.

[0071] Furthermore, the information input module will have two sources: a message queue and a database. The message queue provides streaming data of real-time node information, while the database provides batch data of historical node information. The historical node information summary and task context in the database are also distributed as input streams.

[0072] Furthermore, the information calculation module is primarily responsible for aggregating node information and calculating metrics. After real-time information flows into the aggregation node of the calculation module, historical information and historical information summaries simultaneously flow into the aggregation node. The real-time and historical information are aggregated based on the information summaries, which carry the information to be calculated for the metrics. After aggregation, the real-time information flow falls into the node information database, and the aggregated information flow flows from the measurement flow to the calculation node. After the calculation node completes the metric calculation, the information falls into the node metric database.

[0073] The data analysis component of this invention includes:

[0074] Data storage module: A database that stores various information and indicators of nodes in the I2P network, and a task database that includes task history information, I2P node information summary, and the progress of detection component tasks;

[0075] Data Analysis Module: Analyzes node information and metrics that fall into the database of the data storage module to identify recommended nodes and abnormal nodes.

[0076] In this invention, node information includes capacity, bandwidth, online time, reachability, and latency. Capacity refers to the number of tunnels a routing node successfully establishes within a given period. Bandwidth is the weighted average of a node's speed across different time periods. Online time refers to the amount of time a node is online. Reachability determines whether a node is eligible to be selected as a tunnel participant. Latency is a measurement of a node's latency and also reflects whether the node is online.

[0077] Metrics for I2P nodes include: average capacity, average bandwidth, average latency, number of times online for more than or less than 24 hours, proportion, significance of uniform rID distribution, online frequency, online time, and reachability. The average values, reachability, and online frequency reflect a node's long-term status and stability; the number of times online for more than or less than 24 hours, proportion, significance of uniform rID distribution, and online time primarily reflect whether a node exhibits abnormal behavior or whether nodes with potentially abnormal behavior are too close together.

[0078] The preferred embodiments of the present invention will be further described below.

[0079] like Figure 1 As shown, the I2P network's full-node rapid data acquisition and analysis system consists of a central control component, a detection component, a computing component, and a data analysis component. The central control component, acting as the command center of the entire distributed architecture, is responsible for scheduling other components, issuing task contexts, and displaying and generating task results. The detection component comprises multiple detection instances, uniformly elected and scheduled by the central control component, responsible for sending detection messages and obtaining node information. The computing component adopts a master-slave architecture, with election completed internally and scheduling handled by the central control unit scheduling the master computing instance, responsible for calculating node metrics in both stream and batch processing. The data analysis component also adopts a master-slave architecture, directly scheduled by the central control unit, responsible for analyzing node information and node metrics to identify recommended and abnormal nodes.

[0080] The entire distributed system requires the connection of a distributed file system and a message queue. The distributed file system ensures task consistency, connects the central control component with the computing and probing components, and carries information such as task numbers, status codes, and probing component election. The message queue ensures that information produced by the probing components can enter the computing components in a streaming manner, allowing node information to be consumed in an orderly and timely manner.

[0081] This invention also provides a method for rapid data acquisition and analysis of all nodes in an I2P network. This method applies the aforementioned rapid data acquisition and analysis system for all nodes in an I2P network, such as... Figure 2 As shown, the method includes the following five steps:

[0082] Step 1: The central control component obtains task information and creates task context through the front end, open interfaces, etc. The central control component obtains a list of online probe instances through the distributed file system, elects an instance based on the number of task probes, and writes the task number and task start status code to the corresponding instance path. The corresponding probe instance listens for the task start status code, requests the task context from the central control component, constructs the task based on the task context, and writes the corresponding status code of the construction result back to the distributed file system. The central control component writes the task number and task start status code to the distributed file system's compute component path. The main compute instance elects an instance to execute the task, requests the task context from the central control component, constructs the task based on the task context, and writes the corresponding status code of the construction result back to the distributed file system. The central control component collects the task construction status of both components, decides whether to terminate or rerun the task, and begins continuous monitoring of the distributed file system to monitor whether the task terminates abnormally or completes correctly.

[0083] Task information includes:

[0084] Task number;

[0085] Detection range: determined by the I2P distributed database rID, the entire range is divided by the number of detection instances;

[0086] Flooding Node List: All obtained flooding nodes are assigned to different probe instances to prevent a single flooding node from having too high a QPS, which would slow down the response.

[0087] Probe timeout: The timeout period for each probe message to prevent the entire task from running too slowly;

[0088] Probe retry count: Whether the probe instance retryes after failing to receive a response, and the number of retries;

[0089] Detect connection pool count: Detect the size of the connection pool maintained by the instance. Too large a size may cause memory overflow, while too small a size may cause the detection task to be blocked excessively.

[0090] Message queue QPS: Determines the number of times the message queue writes nodes to the compute component, preventing the compute component from accumulating messages. If the message queue buffer overflows, a discard strategy is adopted to ensure the normal operation of the entire task.

[0091] Aggregation timeout: This parameter is provided to the computing components to determine the timeout period for the information flow at the aggregation node. If it is too long, it may cause the computing components to accumulate, while if it is too short, a large amount of information may not be aggregated.

[0092] Calculation metrics: The metrics that the calculation component needs to calculate and the calculation methods, including operators, default values, etc.;

[0093] Task timeout: The maximum time the entire task can run;

[0094] External parameter configuration: Other parameters of the task are passed through various components of the system in a transparent manner, such as specifying probe instances, exception codes that can be ignored, etc.

[0095] Step 2: The selected probe instance obtains the task context, constructs, and starts the task. First, it requests connection resources from the connection pool. After obtaining the connection, it constructs a probe message based on the rID and flooding node information and sends the message. If the flooding node successfully returns node information, it sends the node information to the message queue, compares the rID with the rID in the progress database, and if the rID is greater than the rID in the process database, it acquires a read-write lock and writes the data. If the request to the flooding node times out and the retry also times out, the request is discarded, and the connection pool resources are released. The rID of the request sent by the probe instance is incremented based on the lower bound of the rID range in the task context. When a process or instance restarts due to certain reasons, the probe instance retrieves the running task number from the distributed system, retrieves the task context from the central control component based on the number, retrieves the task progress from the progress database, and quickly continues the task.

[0096] Step 3: The computation component obtains the task context, constructs, and starts the task. Node information enters the computation component from the message queue consumer, and the task context is broadcast to each node of the computation component. In the aggregation node, the IP address from the real-time task information is obtained. The node's historical information is searched in the database using the IP address as the keyword and entered into the aggregation node. The node's real-time information is copied, while the original node information flows out from the main stream and falls into the node information database. The copied real-time and historical node information are compared to determine whether aggregation is complete based on the amount of historical data in the historical summary information. Node information that has completed aggregation or timed out enters the computation node, performs metric calculations according to the configured computation metrics, and falls from the measurement stream into the node metric database.

[0097] In this invention, the information detected by the detection component in step two is fed into the calculation component in step three for index calculation. The calculation component outputs node information and the calculated node indicators, which are then fed into the data analysis component in step four to obtain the final result. Step three of this invention is streaming computation, and step four is batch data analysis, with the data derived from the calculations in step three.

[0098] Step 4: After receiving the status codes indicating task completion from the computing and detection components, the central control component notifies the data analysis component to begin its work. First, it identifies nodes with excessively low levels of both the significance of uniform rID distribution and the proportion of nodes with online time exceeding 24 hours. These nodes are listed as suspected anomalous nodes. Setting the threshold too high may lead to excessive overhead in analyzing suspected nodes, while setting it too low may result in missing attack nodes. Based on the number of existing I2P nodes and the cost of conducting a collusion attack, a minimum threshold of 1% to 5% is recommended. For suspected anomalous nodes, the historical online time and rID distribution are analyzed. Nodes with excessively close proximity are listed as anomalous nodes. From the efficiency curve of collusion attacks, excessive proximity refers to five or more consecutive I2P nodes with a spacing less than the average distance on the hash ring and an online time of less than one hour. Finally, nodes with excellent performance indicators such as average capacity, average bandwidth, average latency, and reachability ratio, and which are not anomalous, are identified as recommended nodes. Different election schemes can be selected based on communication needs; for example, nodes with higher bandwidth are selected for file transfer, nodes with lower latency are selected for video communication, and nodes with higher capacity are selected for mixed communication.

[0099] Step 5: The central control component probes the distributed file system instance and writes the task completion status code to the computing component path. It waits for each component to report the task release status code to clear the task path and writes the task information and logs to the database. The central control component obtains the task results from the data analysis component and generates the netDB file.

[0100] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0101] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An analysis method for a full-node fast data acquisition and analysis system in an I2P network, characterized in that, The analysis system includes: Central control component: manages all other components, obtains task information through interaction with the user, and triggers various tasks; Probe component: After receiving the task context, it starts the task in parallel through various probe nodes to quickly collect information from all nodes; Calculation components: Real-time node information generated by consumption and historical information in the database are used to calculate metrics; Data analysis component: After a detection mission is completed, the indicator information that has fallen into the database is analyzed to obtain recommended nodes and abnormal nodes; Communication between the central control component and the detection component, computing component, and data analysis component is accomplished using a distributed application coordination service; the detection component and the computing component are connected using a message queue capable of long-term connections. The analytical methods include: 1) The central control component obtains task information and creates task context through the front end and open interfaces, and obtains a list of online detection instances through the distributed file system, and selects a detection instance; 2) The detection component obtains the task context for the selected detection instance, constructs and starts the task, and generates a status code for task completion; 3) The computing component calculates the obtained task context, constructs and starts the task, and generates a status code for task completion; 4) After receiving the status codes from the computing and detection components indicating that they have completed their tasks, the central control component notifies the data analysis component to start working. The data analysis component then identifies recommended nodes and abnormal nodes. 5) The central control component writes the task completion status code to the distributed file system probe instance and the computing component path, waits for each component to report the task release status code to clear the task path, writes the task information and logs to the database, and obtains the task results from the data analysis component to obtain the generated netDB file; The specific content of step 1) is as follows: The central control component obtains task information through a front-end or open interface to create a task context. It then retrieves a list of online probe instances through a distributed file system, elects instances based on the number of probes, and writes the task number and task start status code to the corresponding instance path. The corresponding probe instance, upon receiving the task start status code, requests the task context from the central control component, constructs the task based on the task context, and writes the corresponding status code of the construction result back to the distributed file system. Simultaneously, the central control component writes the task number and task start status code to the distributed file system's computing component path. The main computing instance elects an instance to execute the task, requests the task context from the central control component, constructs the task based on the task context, and writes the corresponding status code of the construction result back to the distributed file system. The central control component also collects information on the task construction status of both components, decides whether to terminate or rerun the task, and begins continuous monitoring of the distributed file system to check for any abnormalities or successful completion of the task.

2. The analytical method according to claim 1, characterized in that, The central control component includes: Task triggering module: Triggers node collection tasks, returns recommended node tasks, and returns abnormal node tasks through user interaction mode or timed triggering mode; Task management module: Adopts a distributed file management architecture to uniformly manage all task contexts; Task Results Module: This module handles the use of task results, including result display and processing of netDB used directly in the generated I2P.

3. The analytical method according to claim 1, characterized in that, The detection component includes: Node detection module: Generates detection tasks based on the task context to collect node information of the I2P network; Progress saving module: Saves the task progress to prevent the task from having to start from the beginning if an error occurs and the task is restarted. Information transmission module: rewrites information according to requirements and writes it to the message queue producer.

4. The analytical method according to claim 1, characterized in that, The computing component includes: Information input module: Connects to the message queue consumer to receive streaming data and connects to the database to receive batch data; Information calculation module: Aggregates real-time node information and historical node information, and uses various operators to calculate indicators; Information output module: stores the calculation results in the database for use by the central control components; Information transmission module: rewrites information according to requirements and writes it to the message queue producer.

5. The analytical method according to claim 4, characterized in that, The data analysis component includes: Data storage module: Stores various information and metrics of nodes in the I2P network; Data Analysis Module: Analyzes node information and metrics in the database of the data storage module to obtain recommended nodes and abnormal nodes.

6. The analytical method according to claim 1, characterized in that, Step 2) contains the following details: First, request connection resources from the connection pool. After requesting, construct a probe message based on rID and flood node information, and send the message. If the flood node successfully returns node information, send the node information to the message queue, determine the size relationship between rID and rID in the progress database. If it is greater than rID in the process database, acquire a read-write lock and write the data. If the request to the flood node times out and the retry times out, discard the request and release the connection pool resources. When a process or instance restarts for some reason, the detection instance pulls the running task number from the distributed system, pulls the task context from the central control component based on the number, pulls the task progress from the progress database, and quickly continues the task.

7. The analytical method according to claim 6, characterized in that, Step 3) contains the following details: Node information enters the computing component from the message queue consumer, and the task context is broadcast to each node of the computing component. In the aggregation node, the IP address in the real-time task information is obtained, and the node's historical information in the database is searched using the IP address as the keyword. The node's real-time information is copied, and the original node information flows out from the main stream and falls into the node information database. The copied real-time node information and historical node information are aggregated based on the amount of historical data in the historical summary information. The node information that has completed aggregation or has timed out enters the computing node, and the index calculation is completed according to the configured computing index. The index falls from the flow measurement into the node index database.

8. The analytical method according to claim 7, characterized in that, Step 4) contains the following details: After receiving the status codes indicating that the computing and detection components have completed their tasks, the central control component notifies the data analysis component to begin working. First, it identifies nodes with excessively low levels of both uniform rID distribution and the proportion of nodes with online time exceeding 24 hours, classifying them as suspected anomalous nodes. For suspected anomalous nodes, it analyzes their historical online time and rID distribution, classifying those approaching the level of suspected anomalous nodes as anomalous nodes. Finally, it identifies nodes with excellent performance across multiple metrics, including average capacity, average bandwidth, average latency, and reachability, and classifies them as recommended nodes.

Images