Tor network hidden malicious organization mining method based on multi-dimensional fusion detection

Abstract

The invention discloses a Tor network hidden malicious organization mining method based on multi-dimensional fusion detection. Comprising the following steps: crawling a network state consensus published by Tor Metric for the last half year, arranging an online relay list and mining relay related information; performing multi-dimensional security assessment based on the relay list and the features, and screening out suspicious relays with abnormal scores higher than a threshold value; synthesizing a relay server descriptor file and network public asset surveying and mapping platform information, obtaining internal attribute behaviors and external intelligence characteristics of a relay node, and performing similarity modeling on a relay pair; a random forest algorithm is used to calculate the relay pair correlation degree; and on the basis of the relay pair correlation degree, performing density-based clustering by using an OPTICS clustering algorithm, and organizing and dividing suspicious relays. According to the method, internal attributes and external information features of relay nodes are combined, and the relay pair correlation degree is calculated and organization division is performed by using similarity modeling and a clustering algorithm, so that efficient identification of the hidden malicious organization of the Tor network is realized.

Description

Technical Field

[0001] The present invention belongs to the technical fields of computer security and anonymous network, and particularly relates to a method for mining hidden malicious organizations in the Tor network based on multi-dimensional fusion detection. Background Art

[0002] With the rapid development of Internet technology and anonymous communication networks, the problem of user anonymity protection has become increasingly prominent. As an important infrastructure for anonymous communication, The Onion Router (Tor) provides a highly privacy-protected network environment for users. However, this anonymity has also been misused by hidden malicious organizations to carry out various network attacks, and the Sybil attack may be used as a preparatory stage for multiple network attacks. In the Tor network, the Sybil attack is a means by which an attacker destroys network security by creating a large number of relays without correctly configured family information. These relays controlled by the same operator are called Sybil relays.

[0003] The official guide for automatic circuit construction in Tor stipulates that two nodes that violate specific criteria (such as belonging to the same family group or being within the same / 16 subnet) cannot be part of the same circuit. Obviously, such requirements cannot effectively control the Sybil attack. Hidden malicious organizations in the Tor network usually evade traditional Sybil attack detection methods by obfuscating relay configuration information, making it difficult for existing technologies to efficiently identify their covert behaviors.

[0004] Currently, the detection of hidden malicious organizations in the Tor network mainly relies on two types of methods. The first is detection based on feature matching: pattern matching is performed through predefined node attribute features (such as nicknames, external access ports). However, such methods have poor adaptability to more covert family relationships. The second is detection based on statistical models, which constructs a classification model using statistical features (such as the probability of node pairs appearing). However, these methods often ignore the temporal correlation of relay behaviors and are difficult to capture periodic or synchronous abnormal patterns.

[0005] In view of the above problems, the present invention proposes a method for mining hidden malicious organizations in the Tor network based on multi-dimensional fusion detection. By deeply evaluating the multi-dimensional security of Tor network relays and combining internal attributes and external intelligence features for similarity modeling and association clustering, the efficient identification of hidden malicious organizations is achieved. This method can effectively make up for the limitations of traditional detection technologies and provide a new solution for Tor network security protection. Summary of the Invention

[0006] In view of this, the present invention provides a method for mining hidden malicious organizations in the Tor network based on multi-dimensional fusion detection, aiming to mine the multi-dimensional abnormal attribute features and behavioral features of Tor relay nodes, conduct security evaluations, model the similarity of relay pairs, and finally construct a classification model to achieve efficient identification of hidden malicious organizations for combating Sybil attacks and other covert collaborative attacks in the Tor network.

[0007] A method for mining hidden malicious organizations in the Tor network based on multi-dimensional fusion detection, the method comprising:

[0008] S100. Crawl the network status consensus released by Tor Metric in the recent half year, sort out the list of relays online in the recent half year, and comprehensively mine the relevant information of each relay in the relay list by means of the relay server descriptor file, the historical malicious relay set, network scanning tools, etc.;

[0009] S200. Based on the relay list and relevant features in step S100, conduct multi-dimensional security evaluations on the relays in the relay list, and filter out suspicious relays with abnormal scores higher than the threshold;

[0010] S300. Based on the suspicious relays filtered out in step S200, comprehensively use the relay server descriptor file and the existing network public asset mapping platform to obtain the internal attribute behaviors and external intelligence features of each relay node, and model the similarity of relay pairs;

[0011] S400. Based on the relay pair similarity features calculated in step S300, use the random forest algorithm to calculate the correlation degree of relay pairs;

[0012] S500. Based on the correlation degree of relay pairs calculated in step S400, use the relay distance of the OPTICS clustering algorithm for density-based clustering, and divide the suspicious relays into organizations.

[0013] Further, in step S100, step S100 is specifically:

[0014] The Tor Metric is a public data platform provided by the Tor project, aiming to collect, analyze and display various statistical and performance data of the Tor network;

[0015] The network status consensus is a network status document released by the Tor directory authority every hour, and the consensus document consists of router status entries, which contain flags, heuristics for relay selection, etc.;

[0016] The relay server descriptor is a description file containing the basic information of the relay node provided by the relay node;

[0017] The historical malicious relay set is a malicious relay fingerprint data set officially released by Tor;

[0018] The network scanning tool is a network detection and security auditing tool, including Nmap or Masscan;

[0019] Mine all the relays that have been online in the past six months and related features according to multiple dimensions.

[0020] Further, in step S200, step S200 is specifically:

[0021] S210. According to the multi-dimensional security evaluation results of the relays, design a scoring formula and calculate the anomaly score of each relay:

[0022]

[0023] Where R i is a single relay, S(R i ) is the defined multi-dimensional anomaly score, d ∈ {1, 2, 3, 4} corresponds to four dimensions of its own attributes, external vulnerabilities, time series, and collaborative behavior, W d is the weight of each dimension, S d (R i ) is the normalized score of dimension d (range [0, 1], 0 means safe, 1 means abnormal);

[0024] Further, in step S210, step S210 is specifically:

[0025] S211. Evaluate the self-dimension of the relay according to the server descriptor file and the historical malicious node set. First, calculate the similarity of the configuration information between the currently online relay and the historical malicious relay:

[0026]

[0027] Where C mal is the set of configuration features of historical malicious relays, C i is the set of configuration features of the current relay (such as Tor version, operating system, port, contact information, etc.), and the Jaccard coefficient is used to measure the similarity;

[0028] Then calculate the similarity of the online time periods between the currently online relay and the historical malicious relay:

[0029]

[0030] Where T mal is the set of online time periods of historical malicious relays, T i is the set of online time periods of the current relay, and the Jaccard coefficient is also used to measure the similarity;

[0031] The final evaluation result of the inertial behavior dimension is:

[0032] S1(R i ) = α · S config (R i ) + β · S time (R i );

[0033] where α and β are weight coefficients;

[0034] S212. Conduct an external vulnerability dimension evaluation for relays based on the scanning results of network scanning tools. First, quantify the network risks in the C segment, and design scores according to different types of risks that may exist in the C segment network where the relay IP is located:

[0035]

[0036] where S is the risk score of the C segment network, r k is the risk coefficient of the kth type of risk, and n k is the quantity of the kth type of risk in the C segment network;

[0037] Then, quantify the risks of common port vulnerabilities of relays, and design scores according to the severity of different port vulnerabilities:

[0038]

[0039] where S risk is the risk score of the port vulnerability, cvss j is the CVSS (Common Vulnerability Scoring System) score of the jth port vulnerability, and P j is the importance weight of the jth port;

[0040] The final evaluation result of the external vulnerability dimension is:

[0041] S2(R i ) = α · S risk (R i ) + β · S risk (R i );

[0042] where α and β are weight coefficients;

[0043] S213. According to the network status consensus released by Tor, obtain the online time series of all relays in the recent half year, and conduct a time series dimension evaluation of relays in combination with the dates of global special events. First, calculate the coincidence degree of abnormal periods of relays based on the time weight function;

[0044] Define the set of counterintuitive periods T abnormal={t1, t2,..., t n}, and the weight w of each time point t i is determined by statistically analyzing the activity of historical malicious relays at different times during this period. i Then, the weighted coincidence degree can be calculated as follows:

[0045]

[0046] where T online is the online period of the relay on the current day, and w i satisfies ∑w i = 1, and the weight w i is higher during high-risk periods;

[0047] Next, calculate the activity of the relay on special dates based on Z-score. Statistically analyze the daily active durations {L1, L2,..., L m} of the relay during historical non-special periods, calculate the mean μ and variance σ, and then the abnormal activity can be calculated as follows:

[0048]

[0049] where L special is the online duration of the relay on the special date;

[0050] Then, the final evaluation result of the time series dimension is:

[0051] S3(R i ) = α·S overlap (R i ) + β·S special (R i );

[0052] where α and β are weight coefficients;

[0053] S214. According to the relay server descriptor file and network status consensus released by Tor, statistically analyze the online time and weight changes of all relays in the relay list in the recent half year to conduct a collaborative behavior dimension evaluation. First, statistically analyze the online time series of all relays R i , mark the online moment as 1 and the offline moment as 0, that is, T i (t) ∈ {0, 1}, (t = 1, 2,..., N), where N is the total number of time points. Then, the Pearson correlation coefficient between two relays is:

[0054]

[0055] where is the average online rate of relay R i ;

[0056] Use this Pearson coefficient to calculate the distance of the relay online time series:

[0057]

[0058] Then, use the distance of the relay online time series as the distance for single-link clustering:

[0059]

[0060] At this time, relays with similar online time patterns will be arranged together. For the clustered online time series matrix Let M be the number of relays, perform a discrete Fourier transform, extract the frequency domain amplitude A(f), and detect adjacent relays with common periodicity. If adjacent relays R i and R j satisfy at frequency f k : and where θ freq is the frequency amplitude threshold, then it is determined that the two have common periodicity;

[0061] Then, within the time window [t1, t2], if K adjacent relays satisfy: then these relays are determined to have abnormal online and offline cooperation behavior: S sync = 1;

[0062] Then, count the changes in the bandwidth weights of all relays in the recent half year. Let the bandwidth weight sequence of relay R i be W i (t), and define the moment when the bandwidth weight of the relay is 0 or the minimum value as the starting point: Define the moment when the bandwidth weight of the relay reaches the highest value as the ending point: Draw the bandwidth weight change trend curve of the relay from the starting point to the ending point. The weight change trend curve is normalized to:

[0063]

[0064] Calculate the average change rate of the weight from the starting point to the ending point:

[0065]

[0066] If k i > k thresh , it is determined to be a surge, and k thresh is the abnormal rate threshold;

[0067] The weight change trend similarity between relay R i and R j adopts the cosine similarity:

[0068]

[0069] If and the weight growth rate satisfies:

[0070] where then it is determined that the weight growth trends of the two are related;

[0071] Relays with a significant abnormal increase in weight and a relevant change trend are determined to have abnormal weight change behavior: S weight = 1;

[0072] Then the final evaluation result of the collaborative behavior dimension is:

[0073] S4(R i ) = α·S sync + β·S weight ;

[0074] where α and β are weight coefficients;

[0075] S220. Set a threshold for the abnormal score S(R i ) of the relay for security evaluation according to experience, and screen out the relays with an abnormal score higher than the threshold. These relays are determined to be suspicious relays.

[0076] Furthermore, in step S300, step S300 is specifically:

[0077] The network public asset mapping platform refers to tools and systems for scanning publicly exposed network assets on the Internet, including the Fofa platform or DayDayMap.

[0078] S310. Query the information of suspicious relays running on each suspicious IP according to the relay server descriptor, and obtain the IP addresses, Tor versions, nicknames, flags, operating systems, contacts, external access ports, long-term bandwidth capacity, burst bandwidth capacity, and routing policies of all suspicious relays;

[0079] S320. Use the public asset mapping platform to query the relays in the list of suspicious relays, and obtain the website titles, server products, common protocols, geographical locations, and autonomous systems used by the servers corresponding to each suspicious relay IP in the most recent half year;

[0080] S330. Calculate the similarity features of the relay pairs based on the relay features in steps S310 and S320;

[0081] Furthermore, in step S330, step S330 is specifically:

[0082] S331. Divide all the features of the relay into three categories. The self-attribute features are IP address, Tor version, nickname, logo, operating system, and contacts;

[0083] The network behavior features are external access ports, long-term bandwidth capacity, burst bandwidth capacity, and routing policies;

[0084] The external intelligence features are the website titles used, server products, common protocols, as well as the geographical location and autonomous system where the server is located;

[0085] Let the relay pair be (r i , r j ). Different types of features require different similarity calculation methods;

[0086] S332. For string-type features f text , such as IP address, Tor version, nickname, logo, operating system, contacts, geographical location, and autonomous system, calculate the similarity using the edit distance (Levenshtein distance):

[0087]

[0088] where Levenshtein(a, b) represents the edit distance between strings a and b, and len(a) represents the length of string a;

[0089] S333. For numerical-type features f num , such as external access ports, long-term bandwidth capacity, and burst bandwidth capacity, calculate the similarity using the normalized Euclidean distance:

[0090]

[0091] where num represents the index set of numerical-type features, and max(f k ) and min(f k ) are the maximum and minimum values of all relays for this feature, respectively;

[0092] S334. For set-type features f set , such as routing policies, website titles, server products, and common protocols, calculate the similarity using the Jaccard coefficient:

[0093]

[0094] S335. Calculate the similarity features of the relay pair:

[0095] S = (S attr , S behavior , S intelligence );

[0096] Among them S attr =w i S IP +w i S Tor_version +w i S IP +w i S nickname +w i S system +w i S contact , w i is the weight coefficient, i∈(1,6)

[0097] S behavior =k j S or_port +k j S bandwidth_avg +k j S bandwidth_burst +k j S policy , k j is the weight coefficient, j∈(1,4);

[0098] S intelligence =l m S title +l m S product +l m S protocol +l m S location +l m S as , l m is the weight coefficient, m∈(1,5).

[0099] Furthermore, in step S400, step S400 is specifically as follows:

[0100] S410. Pair all online relays in pairs to form a relay pair dataset, set the relay pairs belonging to the same family to have a label of family=1, and set the relay pairs that do not belong to the same family to have a label of family=0;

[0101] S420. Divide the relay pair data set D into a training set D train and the test set D test , where |D train |=0.7|D|,|D test |=0.3|D|, based on the similarity feature calculated in step S430 as input, the random forest algorithm is used to train and test the relay pair dataset to calculate the probability that the suspicious relay pairs belong to the same family.

[0102] Further, in step S420, step S420 specifically is:

[0103] S421. The random forest model consists of m decision trees. For the relay pair (r i , r j ), the probability that the k-th decision tree outputs belonging to the same family is Then the random forest model outputs the correlation degree P of the relay pair ij as:

[0104]

[0105] S422. During the training process, the cross-entropy loss function L is used to optimize the model parameters:

[0106]

[0107] The random forest model is trained by minimizing the loss function L, and finally the performance is evaluated on the test set D test .

[0108] Further, in step S500, step S500 specifically is:

[0109] The density-based clustering algorithm is a clustering algorithm that divides clusters by the density of data points, including DBSCAN or OPTICS;

[0110] S510. Based on the suspicious relays obtained in step S300 and the correlation degree of the relay pairs obtained in step S450, the OPTICS clustering algorithm is used to cluster the suspicious relays;

[0111] S520. The clusters with relays of different family information and the clusters composed of relays without family information are retained and defined as suspicious Sybil organizations. BRIEF DESCRIPTION OF THE DRAWINGS

[0112] To more clearly illustrate the technical solutions in the embodiments or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0113] Figure 1 is the flowchart of the Tor network hidden malicious organization detection method based on multi-dimensional security evaluation provided by the present invention;

[0114] Figure 2 is the multi-dimensional abnormal feature definition rule for evaluating the security of relays provided by the present invention; ]>

[0115] Figure 3 FIG. 0 is a sample diagram of an embodiment of a relay pair for similarity feature calculation provided by the present invention;

[0116] Figure 4 FIG. 5 is a flowchart for calculating the probability that a relay pair may belong to the same family provided by the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0117] The following further describes the specific embodiments of the present invention in detail with reference to the accompanying drawings and specific embodiments. The following embodiments or drawings are used to illustrate the present invention, but not to limit the scope of the present invention.

[0118] See Figure 1 , Figure 1 FIG. 17 is a flowchart of a method for detecting malicious organizations hidden in the Tor network based on multi-dimensional security evaluation provided by the present invention, including:

[0119] S100. Crawl the network status consensus released by Tor Metric in the recent six months, sort out the list of relays online in the recent six months, and comprehensively mine the relevant information of each relay in the relay list by combining the relay server descriptor file, the historical malicious relay set, network scanning tools, etc.;

[0120] Specifically, the Tor Metric is an open data platform provided by the Tor project, aiming to collect, analyze and display various statistical and performance data of the Tor network. The network status consensus is a network status document released by the Tor directory authority every hour. The consensus document consists of router status entries, which include flags, heuristics for relay selection, etc. The relay server descriptor is a description file containing the basic information of the relay node, and the historical malicious relay set is a malicious relay fingerprint data set released by Tor official;

[0121] For example: A complete relay node server descriptor specifically includes the descriptor type and version number, type, nickname, IP address, port number, main key of the Ed25519 algorithm, Tor software version and operating platform of each relay node. Tor protocol features, release time, fingerprint, running time, bandwidth information, extra information summary, whether it acts as a hidden service directory, operator contact information, bridge node distribution request, NTor Onion key, rejection rule, whether it supports tunnel directory servers and node summary;

[0122] S200. Based on the relay list and related features in step S100, perform multi-dimensional security evaluation on the relays in the relay list, and filter out suspicious relays with abnormal scores higher than the threshold;

[0123] In one embodiment, step S200 includes the following steps:

[0124] S210. Design a scoring formula based on the multi-dimensional security assessment results of the relays, and calculate the anomaly score for each relay:

[0125]

[0126] where R i is a single relay, S(R i ) is the defined multi-dimensional anomaly score, d ∈ {1, 2, 3, 4} corresponds to four dimensions of its own attributes, external vulnerability, time series, and collaborative behavior, W d is the weight of each dimension, S d (R i ) is the normalized score of dimension d (range [0, 1], 0 means secure, 1 means anomalous);

[0127] Furthermore, in step S210, step S210 is specifically:

[0128] S211. Conduct an assessment of the relay's own dimension based on the server descriptor file and the historical malicious node set. First, calculate the similarity of the configuration information between the currently online relay and the historical malicious relays:

[0129]

[0130] where C mal is the set of configuration features of historical malicious relays, C i is the set of configuration features of the current relay (such as Tor version, operating system, port, contact information, etc.), and the Jaccard coefficient is used to measure the similarity;

[0131] Then calculate the similarity of the online time periods between the currently online relay and the historical malicious relays:

[0132]

[0133] where T mal is the set of online time periods of historical malicious relays, T i is the set of online time periods of the current relay, and the Jaccard coefficient is also used to measure the similarity;

[0134] Then the final assessment result of the inertial behavior dimension is:

[0135] S1(R i ) = α · S config (R i ) + β · S time (R i );

[0136] where α and β are weight coefficients;

[0137] For example, a malicious relay in the historical malicious relay set contains the following features: configuration features: Tor version is 0.4.8.10, operating system is Linux Ubuntu 18.04, open ports include 8080, 3389, and the contact email contains the prefix "abc". The online period is from 22:00 to 6:00 every day (non-working hours, denoted as T mal ={22:00-6:00}, the currently to-be-evaluated relay R1 (IP: 192.168.1.100) contains the following feature configuration features: Tor version is 0.4.8.10, operating system is Linux Ubuntu 18.04, open ports include 9001 (Tor standard port), 8080 (new high-risk port), and the contact email is abc@relay.net. Online period: Online from 23:00 to 5:00 every day. The Jaccard coefficient with the historical malicious relay configuration feature set (Tor = 0.3.5.10, Ubuntu 18.04, 8080 / 3389 ports, email with the prefix "abc") is 0.75 (3 items match / 4 items in the union); the online period is from 23:00 to 5:00 every day, and the Jaccard coefficient with the historical malicious period (22:00-6:00) is 0.7 (420 minutes intersection / 600 minutes union). Through weighted combination (setting the configuration weight to 0.6 and the period weight to 0.4), the inertia behavior dimension score of relay R1 is obtained: S1(R1)=0.6·0.75 + 0.4·0.7 = 0.73

[0138] S212. Conduct an external vulnerability dimension assessment of the relay based on the network scanning tool scan results. First, quantify the network risk of the C segment. Design a score according to the different types of risks that may exist in the C segment network where the relay IP is located:

[0139]

[0140] where S is the risk score of the C segment network, r k is the risk coefficient of the kth type of risk, and n k is the quantity of the kth type of risk in the C segment network;

[0141] Then, quantify the risk of common port vulnerabilities of the relay. Design a score according to the severity of different port vulnerabilities:

[0142]

[0143] where S risk is the risk score of the port vulnerability, cvss j is the CVSS (Common Vulnerability Scoring System) score of the jth port vulnerability, and P j is the importance weight of the jth port;

[0144] Then the final external vulnerability dimension evaluation result is:

[0145] S2(R i ) = α·S risk (R i ) + β·S risk (R i );

[0146] Where α and β are weight coefficients;

[0147] For example: Suppose there is a relay R1 (IP: 172.16.100.50), and there are 3 high - risk IPs in the C - segment network (172.16.100.0 / 24) to which it belongs (172.16.100.10 was once a dark - web server, 172.16.100.20 was marked due to a DDoS attack, 172.16.100.3 has spread Trojans). The risk score of the C - segment network is S risk (R i ) = 0.8×1 + 0.6×1 + 0.5×1 = 0.8 + 0.6 + 0.5 = 1.9; and there are known CVE vulnerabilities in 3 high - risk ports opened by this relay (OpenSSH7.4p1 on port 22 of SSH has CVE - 2018 - 15473, Windows Server 2012R2 on port 3389 of RDP has CVE - 2019 - 0708, Samba 3.6.25 on port 445 of SMB has CVE - 2017 - 7494). The risk score of the port vulnerability is S risk (R i ) = 5.3×0.4 + 9.8×0.4 + 9.8×0.2 = 8.04; Through weighted combination (setting the C - segment risk weight to 0.3 and the port weight to 0.7), the external vulnerability dimension score of relay R1 is obtained as S2(R i ) = 0.3×1.9 + 0.7×8.04 = 6.2

[0148] S213. According to the network status consensus released by Tor official, obtain the online time series of all relays in the recent half - year, and combine the dates of global special events to conduct the time - series dimension evaluation of the relays. First, calculate the coincidence degree of abnormal periods of the relays based on the time - weight function;

[0149] Define the set of counter - intuitive periods T abnormal = {t1, t2,..., t n}; By counting the activity of historical malicious relays at different times in this period, determine the weight w i of each time point t i , then the weighted coincidence degree can be calculated:

[0150]

[0151] Among them, T online is the online period of the relay on the current day, and w i satisfies ∑w i = 1, and the weight of the high-risk period w i is higher;

[0152] Then calculate the special date activity of the relay based on Z-score, and count the daily active duration {L1, L2,..., L m} of the relay in the historical non-special period, calculate the mean μ and variance σ, then the abnormal activity can be calculated:

[0153]

[0154] Among them, L special is the online duration of the relay on the special date;

[0155] Then the final time series dimension evaluation result is:

[0156] S3(R i ) = α·S overlap (R i ) + β·S special (R i );

[0157] Among them, α and β are weight coefficients;

[0158] For example: Obtain the online time series of the relay R1 in the recent six months according to the Tor official network status consensus. Define the abnormal period set as 22:00 to 6:00 the next day every day (the active period of historical malicious relays). The weight of 22-24 points in this period is w1 = 0.4, and the weight of 0-6 points is w2 = 0.6. If it is online at 22-24 points and 0-4 points, then the abnormal time coincidence degree can be calculated as: Then calculate the special date activity. The average daily active duration of the relay R1 in the historical non-special period is μ = 5 hours, and the variance σ = 1. On a special date (such as a major network attack day), the relay R1 is online for L special = 8 hours, then the abnormal activity can be calculated as: The time series dimension score of the relay R1 is obtained as:

[0159] S3(R i ) = 0.5×0.8 + 0.5×3 = 1.9

[0160] S214. According to the relay server descriptor file and network status consensus released by Tor, count the online time and weight changes of all relays in the relay list in the recent six months, and conduct a collaborative behavior dimension evaluation. First, count all relays Ri For the online time series, mark the online time as 1 and the offline time as 0, i.e., T i (t) ∈ {0, 1}, (t = 1, 2,..., N), where N is the total number of time points. Then the Pearson correlation coefficient between the two relays is:

[0161]

[0162] where is the average online rate of relay R i ;

[0163] Use this Pearson coefficient to calculate the distance of the relay online time series:

[0164]

[0165] Then, based on the distance of the relay online time series as the distance for single-link clustering:

[0166]

[0167] At this time, the relays with similar online time patterns will be arranged together. For the clustered online time series matrix M is the number of relays. Perform discrete Fourier transform, extract the frequency domain amplitude A(f), and detect adjacent relays with common periodicity. If adjacent relays R i and R j at frequency f k satisfy: and where θ freq is the frequency amplitude threshold, then it is determined that the two have common periodicity;

[0168] Then, define that within the time window [t1, t2], if K adjacent relays satisfy: then these relays are determined to have abnormal online / offline coordination behavior: S sync = 1;

[0169] Then, count the changes in the bandwidth weights of all relays in the recent half year. Let the bandwidth weight sequence of relay R i be W i (t). Define the moment when the bandwidth weight of the relay is 0 or the minimum value as the starting point: Define the moment when the bandwidth weight of the relay reaches the maximum value as the ending point: Draw the bandwidth weight change trend curve of the relay from the starting point to the ending point. The weight change trend curve is normalized as:

[0170]

[0171] Calculate the average rate of change of the weight from the initial point to the termination point:

[0172]

[0173] If k i > k thresh , it is determined to be a surge, where k thresh is the abnormal rate threshold;

[0174] Relay R i and R j The similarity of the weight change trends is measured by cosine similarity:

[0175]

[0176] If and the weight growth rate satisfies:

[0177] where then it is determined that the weight growth trends of the two are related;

[0178] Relays with abnormal weight surges and related change trends are determined to have abnormal weight change behaviors: S weight = 1;

[0179] Then the final evaluation result of the collaborative behavior dimension is:

[0180] S4(R i ) = α·S sync + β·S weight ;

[0181] where α and β are weight coefficients;

[0182] For example: Suppose there are relays R1 (IP: 192.168.4.14) and R2 (IP: 192.168.4.15), and their 24-hour online time series are both [0,0,0,1,1,1,1,1,1,0,0,0,0,1,1,1,1,1,1,0,0,0,0,0]. The Pearson correlation coefficient between them is calculated to be 1, the online time distance is 0, and they are closely arranged during single-link clustering. After performing a discrete Fourier transform on their online time matrix, the frequency-domain amplitude spectra A A (f) and A (f) are obtained, where the dominant frequency is f0 = 1 / 24, corresponding to a 24-hour period. Calculate the ratio of the amplitude at this frequency to their respective maximum amplitudes: A (f), Set the periodicity determination threshold θ freq= 0.7. Since both ratios exceed the threshold, it is determined that relays R1 and R2 have a common periodicity. At the same time, within the time windows of [4, 9] hours and [14, 19] hours, ∑|T3((t)-T4(t)| = 0 is satisfied, that is, they are completely online at the same time. Therefore, both relay R1 and relay R2 have the up and down line coordination behavior: S sync = 1. Then calculate the change in bandwidth weight. Assume that the weight sequence of relay R3 (IP: 192.168.4.18) is [0, 200, 400, 700, 1100, 1600], and the average change rate is 300, which exceeds the abnormal rate threshold of 100, so it is determined to have a surge; the weight sequence of relay R4 (IP: 192.168.4.19) is [200, 300, 500, 800, 1200, 1700], and the cosine similarity of the normalized weight change sequences of relay R3 and R4 is 1. Therefore, relay R3 and R4 have abnormal weight change behavior: S weight = 1, which is related to the weight growth trend. Let α = 0.5 and β = 0.5, and the comprehensive score S collab == 0.5×1 + 0.5×1 = 1, indicating that these relays not only have a highly coordinated online time pattern, a common periodicity and simultaneous up and down line behavior, but also have closely related and surging bandwidth weight changes. They are very likely to be a relay cluster controlled by the same organization and there is significant abnormal coordination behavior.

[0183] S220. Set a threshold for the abnormal score S(R i ) of the relays for security assessment according to experience, and filter out the relays with abnormal scores higher than the threshold. These relays are determined to be suspicious relays.

[0184] S300. Based on the suspicious relays filtered in step S200, comprehensively use the relay server descriptor file and the existing network public asset mapping platform to obtain the internal attribute behavior and external intelligence characteristics of each relay node, and perform similarity modeling on the relay pairs;

[0185] Specifically, the network public asset mapping platform refers to tools and systems for scanning publicly exposed network assets on the Internet, including the Fofa platform or DayDayMap.

[0186] In one embodiment, step S300 includes the following steps:

[0187] S310. Query the information of the suspicious relays running on each suspicious IP according to the relay server descriptor, and obtain the IP addresses, Tor versions, nicknames, flags, operating systems, contacts, external access ports, long-term bandwidth capacity, burst bandwidth capacity, and routing policies of all suspicious relays;

[0188] S320. Query the relays in the suspicious relay list using a public asset mapping platform to obtain the website titles, server products, common protocols, geographical locations, and autonomous systems of the servers corresponding to each suspicious relay IP in the most recent six months.

[0189] S330. Calculate the similarity features of relay pairs based on the relay features in steps S310 and S320.

[0190] Further, in step S330, step S330 includes the following steps:

[0191] S331. Divide all the features of the relay into three categories. The self-attribute features are IP address, Tor version, nickname, flag, operating system, and contact person.

[0192] The network behavior features are external access ports, long-term bandwidth capacity, burst bandwidth capacity, and routing policies.

[0193] The external intelligence features are the website titles, server products, common protocols, geographical locations of the servers, and autonomous systems.

[0194] Let the relay pair be (r i , r j ). Different types of features require different similarity calculation methods.

[0195] S332. For string-type features f text , such as IP address, Tor version, nickname, flag, operating system, contact person, geographical location, and autonomous system, calculate the similarity using the edit distance (Levenshtein distance):

[0196]

[0197] Where Levenshtein(a, b) represents the edit distance between strings a and b, and len(a) represents the length of string a.

[0198] For example: Suppose the contact attribute of relay R1 is The contact attribute of relay R2 is Calculate the string length Calculate the edit distance To convert "abcde@ab.cde" to "abcdefg@ab.cde", it is necessary to insert "f" and "g" after "e", and the edit distance is 2. Substitute it into the formula to calculate the similarity:

[0199] S333. For numerical-type features f num, for external access ports, long-term bandwidth capacity, and burst bandwidth capacity, calculate similarity using the normalized Euclidean distance::

[0200]

[0201] where num represents the index set of numerical features, and max(f k ) and min(f k ) are the maximum and minimum values of all relays for this feature respectively;

[0202] For example: Suppose the bandwidth_avg attribute of relay R1 is 65536000, and the bandwidth_avg attribute of relay R2 is 52428800. The maximum bandwidth_avg attribute value among all relays is 1073741824, and the minimum is 0. Substitute into the formula to calculate similarity:

[0203] S334. For set features f set , such as routing policies, website titles, server products, and common protocols, calculate similarity using the Jaccard coefficient:

[0204]

[0205] For example: Suppose the policy attribute of relay R1 is {tls, dcerpc, rdp, http, ssh}, and the policy attribute of relay R2 is {rdp, http, ssh, tls}. Substitute into the formula to calculate similarity:

[0206] S335. Define the similarity feature of the relay pair as:

[0207] S ij =(S attr , S behavior , S intelligence );

[0208] where S attr =w i S IP +w i S Tor_version +w i S nickname +w i S sign +w i S system +w i S contact , w i is the weight coefficient, i ∈ (1, 6)

[0209] Sbehavior = k j S or_port + k j S bandwidth_avg + k j S bandwidth_burst + k j S policy , k j is the weight coefficient, j ∈ (1, 4);

[0210] S intelligence = l m S title + l m S product + l m S protocol + l m S location + l m S as , l m is the weight coefficient, m ∈ (1, 5);

[0211] For example: as Figure 3 shown, the Figure 3 is an example diagram for calculating the similarity features of relay pairs in this step. Relay pairs R1 and R2 include 15 attributes of their own attributes, network behaviors, and external intelligence, and the similarity features of this relay pair can be calculated: S attr = 0.175 + 0.143 + 0.12 + 0.1 + 0.087 + 0.14 = 0.765, S behavior = 0.012 + 0.219 + 0.219 + 0 = 0.45, S intelligence = 0.071 + 0.12 + 0.2 + 0.05 + 0.08 = 0.521. Therefore, the final similarity features of this relay pair are: S = (0.765, 0.45, 0.521)

[0212] S400. Based on the relay pair similarity features calculated in step S300, use the random forest algorithm to predict the association degree of all relay pairs;

[0213] In one embodiment, the Figure 4 is the step of calculating the association degree of relay pairs, and step S400 includes the following steps:

[0214] S410. Pair all online relays pairwise to form a relay pair dataset. Set the label family = 1 for relay pairs belonging to the same family, and set the label family = 0 for relay pairs not belonging to the same family;

[0215] S420. Divide the relay pair dataset D into a training set D train and a test set D test, where |D train | = 0.7|D|, |D test | = 0.3|D|. Using the similarity features calculated in step S335 as input, the relay pair dataset is trained and tested using the random forest algorithm to calculate the probability that a suspicious relay pair may belong to the same family.

[0216] Further, in step S420, step S420 includes the following steps:

[0217] S421. The random forest model consists of m decision trees. For the relay pair (r i , r j ), the probability of belonging to the same family output by the k-th decision tree is Then the correlation degree P of the relay pair output by the random forest model ij is:

[0218]

[0219] S422. During the training process, the cross-entropy loss function L is used to optimize the model parameters:

[0220]

[0221] The random forest model is trained by minimizing the loss function L, and finally the performance is evaluated on the test set D test .

[0222] S500. Based on the probability that the relay pair belongs to the same family calculated in step S400, the relay distance of the OPTICS clustering algorithm is used for density-based clustering to organize and divide the suspicious relays.

[0223] Specifically, the density-based clustering algorithm is a clustering algorithm that divides clusters by the density of data points, including DBSCAN or OPTICS;

[0224] In one embodiment, step S500 includes the following steps:

[0225] S510. Based on the suspicious relays obtained in step S300 and the relay pair correlation degree obtained in step S450, the OPTICS clustering algorithm is used to cluster the suspicious relays;

[0226] S520. The clusters with relays of different family information and the clusters composed entirely of relays without family information are retained and defined as suspicious Sybil organizations.

[0227] Finally, it should be noted that the above is only one embodiment of the present invention and is not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features.

[0228] Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for mining and identifying malicious organizations hidden in the Tor network based on multi-dimensional fusion detection, characterized in that Including the following steps: S100. Crawl the network status consensus released by Tor Metric in the recent six months, organize the Tor network relay list, and mine the relevant information of each relay in the relay list; S200. Based on the relay list and related features in step S100, conduct a multi-dimensional security assessment of the relays in the relay list, and filter out suspicious relays with abnormal scores higher than the threshold; S300. Based on the suspicious relays filtered out in step S200, obtain the internal attribute behaviors and external intelligence features of each relay node, and perform similarity modeling on the relay pairs; S400. Based on the relay pair similarity features calculated in step S300, use the random forest algorithm to calculate the correlation degree of the relay pairs; S500. Based on the relay pair correlation degree calculated in step S400, use the relay distance of the OPTICS clustering algorithm to perform density-based clustering, and organize and divide the suspicious relays.

2. The method for identifying hidden malicious mining in the Tor network based on multi-dimensional fusion detection according to claim 1, characterized in that, The information that can be mined for each relay in step S100 specifically includes the following: The basic configuration information, online period information, C-segment network information where it is located, open port information, and bandwidth weight change information of each relay node.

3. The method for identifying hidden malicious mining in the Tor network based on multi-dimensional fusion detection according to claim 1, wherein Step S200 includes the following steps: S210. Design a scoring formula based on the multi-dimensional security assessment results of the relays, and calculate the anomaly score S(R i ) for each relay; S220. Set a threshold for the relay anomaly score S(R i ) for the security assessment based on experience, and filter out the relay IPs with anomaly scores higher than the threshold. These IPs are determined to be suspicious relay IPs.

4. The method for identifying hidden malicious mining in the Tor network based on multi-dimensional fusion detection according to claim 3, wherein Step S210 includes the following steps: S211. First, calculate the similarity of the configuration information between the currently online relay and the historical malicious relay: Among them, C mal is the set of historical malicious relay configuration features, and C i is the set of current relay configuration features (such as Tor version, operating system, port, contact information, etc.), and the Jaccard coefficient is used to measure the similarity; Then calculate the similarity of the online periods between the currently online relay and the historical malicious relay: Among which T mal is the set of historical malicious relay online time periods, and T i is the set of current relay online time periods. The Jaccard coefficient is also used to measure the similarity; Finally, comprehensively evaluate the inertial behavior dimension based on the similarity of the configuration information and the similarity of the online periods; S212. First, quantify the C-segment network risk, and design scores according to the different types of risks that may exist in the C-segment network where the relay IP is located: Among them, S is the risk score of the C-section network, and r k is the risk coefficient of the k-th risk type, and n k is the quantity of the k-th risk type in the C-section network; Then quantify the vulnerability risk of the open ports of the relay, and design scores according to the severity of different port vulnerabilities: Among which S risk is the risk score of the port vulnerability, cvss j is the CVSS (Common Vulnerability Scoring System) score of the j-th port vulnerability, P j is the importance weight of the j-th port; Finally, comprehensively evaluate the external vulnerability dimension based on the C-segment network risk and port vulnerability information; S213. First, calculate the abnormal period coincidence degree of the relay based on the time weight function; Define the set \(T\) of counterintuitive time periods abnormal =\(\{t_1, t_2, \ldots, t\) n}\), and determine the weight \(w\) of each time point \(t\) by statistically analyzing the activity of historical malicious relays at different times during this time period i , then the weighted coincidence degree can be calculated as follows: i ​ Among which T online is the online period of the relay on the same day, w i satisfies ∑w i = 1, and the w during the high-risk period i is higher; Recalculate the activity of the relay on special dates based on Z-score, and count the daily active durations {L1, L2,..., L m} of the relay during historical non-special periods. Calculate the mean μ and variance σ, then it can be calculated as follows: Where L special is the online duration of the relay on special dates; Finally, comprehensively evaluate the external vulnerability dimension based on the counter-intuitive period coincidence degree and the activity on special dates; S214. First, count all relays R i 's online time series. Mark the online moment as 1 and the offline moment as 0, that is, T i (t) ∈ {0, 1}, (t = 1, 2,..., N), where N is the total number of time points. Then the Pearson correlation coefficient between two relays is as follows: Among them is the average online rate of relay R i ; Use this Pearson coefficient to calculate the distance of the relay online time series Then, based on the distance of the relay online time series as the distance of single-link clustering At this time, the relays with similar online time patterns will be arranged together. For the clustered online time series matrix Let M be the number of relays, perform discrete Fourier transform, extract the frequency domain amplitude A(f), and detect adjacent relays with common periodicity. If adjacent relays R i and R j at frequency f k satisfy: and where θ freq is the frequency amplitude threshold, then it is determined that the two have common periodicity; Within the redefined time window [t1, t2], if K adjacent relays satisfy: Then these relays are determined to have abnormal online and offline collaborative behavior: S sync = 1; Then, count the changes in the bandwidth weights of all relays in the most recent half year. Let the relay be R i and its bandwidth weight sequence be W i (t). Define the moment when the bandwidth weight of the relay is 0 or the minimum value as the initial point: Define the moment when the bandwidth weight of the relay reaches the highest value as the termination point: Plot the bandwidth weight transformation trend curve of the relay from the initial point to the termination point. The weight change trend curve is standardized as: Calculate the average change rate of the weight from the initial point to the termination point: If k i > k thresh , it is determined as a surge, and k thresh is the abnormal rate threshold; Relay R i and R j The similarity of the weight change trends is measured using cosine similarity: If and the weight growth rate satisfies: Among them it is determined that the weight growth trends of the two are related; Relays where there is a significant surge in weights and a correlation with the change trend are determined to have abnormal weight change behavior: S weight = 1; Finally, comprehensively evaluate the collaborative behavior dimension based on the abnormal online and offline behaviors and the abnormal weight growth behavior.

5. The method for identifying hidden malicious mining in the Tor network based on multi-dimensional fusion detection according to claim 1, wherein Step S300 includes the following steps: S310. Query the suspicious relay information running on each suspicious IP according to the relay server descriptor, and obtain the IP addresses, Tor versions, nicknames, flags, operating systems, contacts, external access ports, long-term bandwidth capacity, burst bandwidth capacity, and routing policies of all suspicious relays; S320. Use the public asset mapping platform to query the relays in the suspicious relay list, and obtain the website titles, server products, common protocols, geographical locations, and autonomous systems used by the servers corresponding to each suspicious relay IP in the recent six months; S330. Calculate the similarity feature S of the relay pair based on the relay features of steps S410 and S420 ij .

6. The method for identifying malicious mining hidden in the Tor network based on multi-dimensional fusion detection according to claim 5, characterized in that Step S330 includes the following steps: S331. Classify all the features of the relay into three categories. The self-attribute features are IP address, Tor version, nickname, logo, operating system, and contacts; the network behavior features are external access ports, long-term bandwidth capacity, burst bandwidth capacity, and routing policies; the external intelligence features are the titles of websites visited, server products, common protocols, as well as the geographical location and autonomous system where the server is located. S332. For the string-type feature f text , such as IP address, Tor version, nickname, logo, operating system, contact, geographical location, autonomous system, calculate the similarity using the edit distance (Levenshtein distance) S333. For the numerical feature f num , such as the external access port, long-term bandwidth capacity, and burst bandwidth capacity, calculate the similarity using the normalized Euclidean distance S334. For the set-type feature f set , such as routing policies, website titles, server products, and common protocols, calculate the similarity using the Jaccard coefficient: S335. Calculate the similarity features of relay pairs: S = (S attr , S behavior , S intelligence ).

7. The method for identifying hidden malicious mining in the Tor network based on multi-dimensional fusion detection according to claim 1, characterized in that, Step S400 includes the following steps: S410. Pair up all online relays to form a relay pair data set. Set the label family = 1 for relay pairs belonging to the same family, and set the label family = 0 for relay pairs not belonging to the same family. S420. Divide the relay pair dataset D into a training set D train and a test set D test , where |D train | = 0.7|D|, |D test | = 0.3|D|. Using the similarity features calculated in step S430 as input, train and test the relay pair dataset using the random forest algorithm, and calculate the correlation degree P of the suspicious relay pairs ij . During the training process, use the cross-entropy loss function L to optimize the model parameters:

8. The method for identifying hidden malicious mining in the Tor network based on multi-dimensional fusion detection according to claim 1, characterized in that, Step S500 includes the following steps: S510. Based on the suspicious relays obtained in step S300 and the relay pair correlation obtained in step S450, use the OPTICS clustering algorithm to cluster the suspicious relays. S520. Retain the clusters of relays with different family information and the clusters composed of relays without family information, and define them as suspicious Sybil organizations.

Images