Tor network hidden malicious organization mining method based on multi-dimensional fusion detection

Abstract

The application discloses a Tor network hidden malicious organization mining method based on multi-dimensional fusion detection. The method comprises the following steps: crawling the network state consensus published by Tor Metric in the latest half year, sorting the online relay list and mining the relay related information; performing multi-dimensional security evaluation based on the relay list and the characteristics, and screening out suspicious relays with abnormal scores higher than a threshold; comprehensively using the relay server descriptor file and the network public asset mapping platform information to obtain the internal attribute behavior and external intelligence features of the relay node, and modeling the similarity of relay pairs; using a random forest algorithm to calculate the correlation degree of relay pairs; based on the correlation degree of relay pairs, using an OPTICS clustering algorithm to perform density-based clustering, and dividing the suspicious relays into organizations. The application combines the internal attributes and external intelligence features of the relay node, respectively uses the similarity modeling and clustering algorithm to calculate the correlation degree of relay pairs and divide the organizations, and realizes efficient identification of the Tor network hidden malicious organizations.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer security and anonymous network, and particularly relates to a Tor network hidden malicious organization mining method based on multi-dimensional fusion detection. BACKGROUND

[0002] With the rapid development of Internet technology and anonymous communication network, the problem of user anonymity protection is increasingly prominent. The onion routing (Tor) as an important infrastructure of anonymous communication provides a highly private network environment for users. However, this anonymity is also abused by hidden malicious organizations to carry out various network attacks, and Sybil attack may serve as a preparation stage for multiple network attacks. In the Tor network, Sybil attack is a means for attackers to disrupt network security by creating a large number of relays without properly configuring family information, and these relays controlled by the same operator are called Sybil relays.

[0003] The official guide of Tor's automatic circuit construction stipulates that two nodes violating certain standards (such as belonging to the same family group or located in the same / 16 subnet) cannot be part of the same circuit, and obviously such requirements cannot effectively control Sybil attack. The hidden malicious organization in the Tor network usually evades the traditional Sybil attack detection method by confusing the relay configuration information, making it difficult for the prior art to efficiently identify its hidden behavior.

[0004] At present, the detection of hidden malicious organizations in the Tor network mainly relies on two methods, the first is feature matching-based detection: pattern matching is performed through pre-defined node attribute features (such as nickname, external access port). However, this method has poor adaptability to more hidden family relationships. The second is statistical model-based detection, which uses statistical features (such as node pair occurrence probability) to build a classification model. However, this method often ignores the time sequence correlation of relay behavior, and is difficult to capture periodic or synchronous abnormal patterns.

[0005] The present application proposes a Tor network hidden malicious organization mining method based on multi-dimensional fusion detection to solve the above problems. By deeply evaluating the multi-dimensional security of Tor network relays, combining internal attributes and external intelligence features for similarity modeling and association clustering, efficient identification of hidden malicious hidden malicious organizations is realized. This method can effectively overcome the limitations of traditional detection technology and provide a new solution for Tor network security protection. SUMMARY

[0006] In view of the above, the present application provides a Tor network hidden malicious organization mining method based on multi-dimensional fusion detection, aiming to mine the multi-dimensional abnormal attribute features and behavior features of Tor relay nodes, conduct security evaluation, model the similarity of relay pairs, and finally build a classification model to realize efficient identification of hidden malicious organizations, for resisting Sybil attacks and other hidden collaborative attacks in the Tor network.

[0007] A Tor network hidden malicious organization mining method based on multi-dimensional fusion detection, the method comprising:

[0008] S100. Crawling the network status consensus published by Tor Metric in the last half year, sorting the online relay list in the last half year, and comprehensively mining the relevant information of each relay in the relay list from the relay server descriptor file, the historical malicious relay set, and the network scanning tool;

[0009] S200. Based on the relay list and related features in step S100, performing multi-dimensional security evaluation on the relays in the relay list, and screening out suspicious relays with abnormal scores higher than the threshold;

[0010] S300. Based on the suspicious relays filtered out in step S200, comprehensively using the relay server descriptor file and the existing network public asset mapping platform to obtain the internal attribute behavior and external intelligence features of each relay node, and modeling the similarity of relay pairs;

[0011] S400. Based on the relay pair similarity features calculated in step S300, using the random forest algorithm to calculate the correlation degree of the relay pairs;

[0012] S500. Based on the correlation degree of the relay pairs calculated in step S400, using the relay distance of the OPTICS clustering algorithm to perform density-based clustering, and organizing and dividing the suspicious relays.

[0013] Further, in step S100, step S100 specifically comprises:

[0014] The Tor Metric is a public data platform provided by the Tor project, aiming to collect, analyze and display various statistical and performance data of the Tor network;

[0015] The network status consensus is a network status document published by the Tor directory authority every hour, and the consensus document is composed of router status entries, which contain flags, heuristic methods for relay selection, etc.;

[0016] The relay server descriptor is a description file provided by the relay node containing its basic information;

[0017] The historical malicious relay set is a malicious relay fingerprint data set officially released by Tor;

[0018] The network scanning tool is a network probing and security auditing tool, including Nmap or Masscan;

[0019] According to multiple dimensions, all online relays in the past six months and related features are mined.

[0020] Further, in step S200, step S200 specifically comprises:

[0021] S210. According to the multi-dimensional security evaluation result of the relay, a scoring formula is designed, and the abnormal score of each relay is calculated:

[0022]

[0023] Where R i is a single relay, S(R i ) is the defined multi-dimensional abnormal score, d∈{1,2,3,4} corresponds to four dimensions of self attribute, external vulnerability, time series, and cooperative behavior, W d is the weight of each dimension, S d (R i ) is the normalized score of dimension d (range [0,1], 0 represents safety, and 1 represents abnormality);

[0024] Further, in step S210, step S210 specifically comprises:

[0025] S211. According to the server descriptor file and the historical malicious node set, the self dimension evaluation of the relay is performed, and first, the configuration information similarity of the current online relay and the historical malicious relay is calculated:

[0026]

[0027] Where C mal is a set of historical malicious relay configuration features, C i is a set of current relay configuration features (such as Tor version, operating system, port, contact information, etc.), and Jaccard coefficient is used to measure similarity;

[0028] Then, the online period similarity of the current online relay and the historical malicious relay is calculated:

[0029]

[0030] Where T mal is a set of historical malicious relay online time periods, T i is a set of current relay online time periods, and Jaccard coefficient is also used to measure similarity;

[0031] Then the final inertia behavior dimension evaluation result is:

[0032] S1(R i )=α·S config (R i )+β·S time (R i );

[0033] Wherein α and β are weight coefficients;

[0034] S212. According to the network scanning tool scanning result, the external vulnerability dimension evaluation of the relay is carried out, first, the C segment network risk is quantified, and the score is designed according to the different types of risks that may exist in the C segment network where the relay IP is located:

[0035]

[0036] Wherein S is the risk score of the C segment network, r k is the risk coefficient of the kth risk type, and n k is the number of the kth risk type in the C segment network;

[0037] Then the relay common port vulnerability risk is quantified, and the score is designed according to the severity of different port vulnerabilities:

[0038]

[0039] Wherein S risk is the risk score of the port vulnerability, cvss j is the CVSS (Common Vulnerability Scoring System) score of the jth port vulnerability, and P j is the importance weight of the jth port;

[0040] Then the final external vulnerability dimension evaluation result is:

[0041] S2(R i )=α·S risk (R i )+β·S risk (R i );

[0042] Wherein α and β are weight coefficients;

[0043] S213. According to the network status consensus published by Tor official, the online time sequence of all relays in the last half year is obtained, and the time sequence dimension evaluation of the relay is carried out combined with the date of global special events, first, the abnormal period coincidence degree of the relay based on the time weight function is calculated;

[0044] Define the abnormal period set T abnormal= {t1, t2,..., t n}, the weight w i of each time point t i is determined by the statistical history of malicious relay activity at different times in the period, then the weighted degree of integration can be calculated:

[0045]

[0046] where T online is the online period of the relay on the day, w i satisfies ∑w i = 1, and the high-risk period w i is higher.

[0047] Then calculate the special date activity of the relay based on the Z-score, and count the daily active time length {L1, L2,..., L m} of the relay in the history of non-special periods, calculate the mean μ and variance σ, then the abnormal activity can be calculated:

[0048]

[0049] where L special is the online duration of the relay on the special date;

[0050] The final time series dimension evaluation result is:

[0051] S3(R i ) = α·S overlap (R i ) + β·S special (R i ) ;

[0052] Where α and β are weight coefficients;

[0053] S214. According to the Tor official release of the relay server descriptor file and the network state consensus, the online time of all relays in the relay list in the recent half year and the weight change are counted, and the collaborative behavior dimension evaluation is carried out. First, the online time sequence of all relays R i is counted, and the online time is marked as 1 and the offline time is marked as 0, that is, T i (t) ∈ {0, 1}, (t = 1, 2,..., N), N is the total number of time points, then the Pearson correlation coefficient of two relays is:

[0054]

[0055] Where is the average online rate of the relay R i ;

[0056] The distance of relay online time series is calculated using the Pearson coefficient:

[0057]

[0058] The distance of relay online time series is used as the distance of single-chain clustering:

[0059]

[0060] At this time, the relays with similar online time patterns are arranged together, and the online time series matrix after clustering is M is the number of relays, and the discrete Fourier transform is performed to extract the frequency domain amplitude A(f). Adjacent relays with common periodicity are detected. If adjacent relays R i and R j satisfy: at frequency f k , where θ freq is the frequency amplitude threshold, it is determined that the two have common periodicity;

[0061] If K adjacent relays satisfy: in the time window [t1, t2], these relays are determined to have abnormal online and offline collaborative behavior: S sync = 1;

[0062] The change of bandwidth weight of all relays in the last half year is counted. The bandwidth weight sequence of relay R i is W i (t). The time when the bandwidth weight of the relay is 0 or the minimum value is defined as the initial point: The time when the bandwidth weight of the relay reaches the maximum value is defined as the termination point: The bandwidth weight transformation trend curve of the relay from the initial point to the termination point is drawn, and the weight change trend curve is standardized as:

[0063]

[0064] The average change rate of weight from the initial point to the termination point is calculated:

[0065]

[0066] If k i > k thresh , it is determined to be a surge, and k thresh is the abnormal rate threshold;

[0067] The weight change trend similarity of relays R i and R j uses cosine similarity:

[0068]

[0069] If and the weight growth rate satisfies:

[0070] wherein the weight growth trends of the two are determined to be related;

[0071] The relay whose weight abnormally increases and the weight change trend are related is determined to have an abnormal weight change behavior: S weight = 1;

[0072] The final cooperative behavior dimension evaluation result is:

[0073] S4(R i ) = a S sync + b S weight ;

[0074] wherein a and b are weight coefficients;

[0075] S220. A threshold value is set for the abnormal score S(R i ) of the relay subjected to the security evaluation, and relays with abnormal scores higher than the threshold value are screened out, and these relays are determined to be suspicious relays.

[0076] Further, in step S300, step S300 specifically comprises:

[0077] The network public asset mapping platform refers to a tool and system for scanning network assets exposed on the Internet, including a Fofa platform or a DayDayMap.

[0078] S310. Query suspicious relay information run by each suspicious IP according to the relay server descriptor, and obtain IP addresses, Tor versions, nicknames, flags, operating systems, contacts, external access ports, long-term bandwidth capacity, burst bandwidth capacity, and routing strategies of all suspicious relays;

[0079] S320. Query the relays in the suspicious relay list by using the public asset mapping platform, and obtain website titles, server products, commonly used protocols, and geographic locations and autonomous systems where each suspicious relay IP corresponds to a server used in the past half year;

[0080] S330. Calculate similarity features of relay pairs based on relay features in steps S310 and S320;

[0081] Further, in step S330, step S330 specifically comprises:

[0082] S331. All features of a relay are divided into three categories, self-attribute features are IP address, Tor version, nickname, flag, operating system, contact;

[0083] Network behavior features are external access port, long-term bandwidth capacity, burst bandwidth capacity, routing strategy;

[0084] External intelligence features are used website title, server product, common protocol, and the geographical location and autonomous system of the server;

[0085] Let the relay pair be (r i ,r j ), different types of features need to use different similarity calculation methods;

[0086] S332. For string type features f text , such as IP address, Tor version, nickname, flag, operating system, contact, geographical location, and autonomous system, use the edit distance (Levenshtein distance) to calculate the similarity:

[0087]

[0088] Where Levenshtein(a,b) represents the edit distance between strings a and b, and len(a) represents the length of string a;

[0089] S333. For numerical features f num , such as external access port, long-term bandwidth capacity, and burst bandwidth capacity, use the normalized Euclidean distance to calculate the similarity:

[0090]

[0091] Where num represents the index set of numerical features, max(f k ) and min(f k ) are the maximum and minimum values of all relays for this feature, respectively;

[0092] S334. For set type features f set , such as routing strategy, website title, server product, and common protocol, use the Jaccard coefficient to calculate the similarity:

[0093]

[0094] S335. Calculate the similarity features of the relay pair:

[0095] S = (S attr ,S behavior ,S intelligence );

[0096] where S attr = w i S IP + w i S Tor_version + w i S IP + w i S nickname + w i S system + w i S contact , w i is a weight coefficient, i e (1, 6)

[0097] S behavior = k j S or_port + k j S bandwidth_avg + k j S bandwidth_burst + k j S policy , k j is a weight coefficient, j e (1, 4) ;

[0098] S intelligence = l m S title + l m S product + l m S protocol + l m S location + l m S as , l m is a weight coefficient, m e (1, 5).

[0099] Further, in step S400, step S400 is specifically:

[0100] S410. Pair all online relays two by two to form a relay pair dataset, set the label family = 1 for relay pairs belonging to the same family, and set the label family = 0 for relay pairs not belonging to the same family;

[0101] S420. Divide the relay pair dataset D into a training set D train and a test set D test , where |D train | = 0.7 |D|, |D test | = 0.3 |D|, based on the similarity features calculated in step S430 as input, use a random forest algorithm to train and test the relay pair dataset, and calculate the probability that a suspicious relay pair belongs to the same family.

[0102] Further, in step S420, step S420 is specifically:

[0103] S421. The random forest model is composed of m decision trees, and for a relay pair (r i , j ), the probability output by the kth decision tree that belongs to the same family is Then the random forest model outputs the relay pair correlation degree P ij :

[0104]

[0105] S422. In the training process, the model parameters are optimized using the cross-entropy loss function L:

[0106]

[0107] The random forest model is trained by minimizing the loss function L, and finally the performance is evaluated on the test set D test .

[0108] Further, in step S500, step S500 is specifically:

[0109] The density-based clustering algorithm is a clustering algorithm that divides clusters by the density of data points, including DBSCAN or OPTICS;

[0110] S510. Based on the suspicious relay obtained in step S300 and the relay pair correlation degree obtained in step S450, the OPTICS clustering algorithm is used to cluster the suspicious relay;

[0111] S520. The clusters with different family information relays and the clusters composed of family information relays are retained, defined as suspicious Sybil organizations. BRIEF DESCRIPTION OF DRAWINGS

[0112] To more clearly illustrate the technical solutions in the embodiments or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0113] Figure 1 A flowchart of a Tor network hidden malicious organization detection method based on multi-dimensional security evaluation is provided for the present application;

[0114] Figure 2 A multi-dimensional anomaly feature definition rule for evaluating relay security is provided for the present application;

[0115] Figure 3 FIG. 1 shows an example graph of a relay pair embodiment for similarity feature calculation of the present application;

[0116] Figure 4 FIG. 2 shows a flowchart for calculating the probability that a relay pair belongs to the same family of the present application; DETAILED DESCRIPTION

[0117] The specific embodiments of the present application will be further described in conjunction with the drawings and specific embodiments. The following examples or drawings are used to illustrate the present application, but not to limit the scope of the present application.

[0118] Reference Figure 1 , Figure 1 The flowchart of the Tor network malicious organization detection method based on multi-dimensional security assessment provided by the present application includes:

[0119] S100. Crawling the network status consensus published by Tor Metric in the past half year, sorting the relay list online in the past half year, and comprehensively mining the relevant information of each relay in the relay list from the relay server descriptor file, the historical malicious relay set, and the network scanning tool;

[0120] Specifically, the Tor Metric is a public data platform provided by the Tor project, aiming to collect, analyze and display various statistical and performance data of the Tor network. The network status consensus is a network status document published by the Tor directory authority every hour. The consensus document is composed of router status entries, which contain flags and heuristic methods for relay selection. The relay server descriptor is a description file provided by the relay node, which contains basic information about itself. The historical malicious relay set is a malicious relay fingerprint data set published by the Tor official;

[0121] For example: A complete relay node server descriptor specifically includes the descriptor type and version number, type, nickname, IP address, port number, Ed25519 algorithm master key, Tor software version and running platform of each relay node. Tor protocol characteristics, release time, fingerprint, running time, bandwidth information, additional information digest, whether to act as a hidden service directory role, operator contact information, bridge node distribution request, NTor Onion key, rejection rule, whether to support tunnel directory server and node digest;

[0122] S200. Based on the relay list and related features in step S100, the relays in the relay list are subjected to multi-dimensional security assessment, and suspicious relays with abnormal scores higher than the threshold value are screened out;

[0123] In one embodiment, the step S200 includes the following steps:

[0124] S210. Design a scoring formula according to the results of the multi-dimensional security evaluation of the relay, and calculate the anomaly score of each relay:

[0125]

[0126] where R i is a single relay, S(R i ) is the defined multi-dimensional anomaly score, d ∈ {1, 2, 3, 4} corresponds to the four dimensions of self attribute, external vulnerability, time series, and cooperative behavior, W d is the weight of each dimension, S d (R i ) is the normalized score of dimension d (range [0, 1], 0 represents security and 1 represents anomaly);

[0127] Further, in step S210, step S210 is specifically:

[0128] S211. According to the server descriptor file and the historical malicious node set, the self dimension evaluation of the relay is performed, and first, the configuration information similarity of the current online relay and the historical malicious relay is calculated:

[0129]

[0130] where C mal is a set of historical malicious relay configuration features, C i is a set of current relay configuration features (such as Tor version, operating system, port, contact information, etc.), and the Jaccard coefficient is used to measure similarity;

[0131] Then, the online period similarity of the current online relay and the historical malicious relay is calculated:

[0132]

[0133] where T mal is a set of historical malicious relay online time periods, T i is a set of current relay online time periods, and the Jaccard coefficient is also used to measure similarity;

[0134] Then the final inertia behavior dimension evaluation result is:

[0135] S1(R i ) = α·S config (R i ) + β·S time (R i );

[0136] Where α and β are weight coefficients;

[0137] For example: one of the historical malicious relay set contains the following features, configuration features: Tor version is 0.4.8.10, operating system is Linux Ubuntu 18.04, open port contains 8080, 3389, contact mailbox contains "abc" prefix. Online period is 22:00-6:00 every day (non-working period, recorded as T mal ={22:00-6:00}, the current relay R1 (IP: 192.168.1.100) to be evaluated contains the following feature configuration features: Tor version is 0.4.8.10, operating system is Linux Ubuntu 18.04, open port contains 9001 (Tor standard port), 8080 (new high-risk port), contact mailbox is abc@relay.net. Online period: 23:00-5:00 every day online, the Jaccard coefficient with the historical malicious relay configuration feature set (Tor = 0.3.5.10, Ubuntu 18.04, 8080 / 3389 ports, "abc" prefix mailbox) is 0.75 (3 matches / 4 union); the online period is 23:00-5:00 every day, and the Jaccard coefficient with the historical malicious period (22:00-6:00) is 0.7 (420 minutes intersection / 600 minutes union). By weighted combination (set configuration weight 0.6, period weight 0.4), the inertia behavior dimension score of relay R1 is: S1(R1) = 0.6·0.75 + 0.4·0.7 = 0.73

[0138] S212. According to the network scanning tool scanning result, the external vulnerability dimension of the relay is evaluated, first the C segment network risk is quantified, according to the different types of risks that may exist in the C segment network of the relay IP, the score is designed:

[0139]

[0140] Where S is the risk score of the C segment network, r k is the risk coefficient of the kth risk type, n k is the number of the kth risk type in the C segment network;

[0141] Then the common port vulnerability risk of the relay is quantified, and the score is designed according to the severity of different port vulnerabilities:

[0142]

[0143] Where S risk is the risk score of the port vulnerability, cvss j is the CVSS (Common Vulnerability Scoring System) score of the jth port vulnerability, P j is the importance weight of the jth port;

[0144] Then the final external vulnerability dimension assessment result is:

[0145] S2(R i )=α·S risk (R i )+β·S risk (R i );

[0146] Wherein α and β are weight coefficients;

[0147] For example: assuming there is a relay R1 (IP: 172.16.100.50) belonging to a C segment network (172.16.100.0 / 24) with 3 high-risk IPs (172.16.100.10 was a dark web server, 172.16.100.20 was marked due to DDoS attack, and 172.16.100.3 spreaded Trojan), the C segment network risk score is S risk (R i )=0.8×1+0.6×1+0.5×1=0.8+0.6+0.5=1.9; and the 3 high-risk ports opened by the relay have known CVE vulnerabilities (OpenSSH7.4p1 of SSH 22 port has CVE-2018-15473, Windows Server 2012R2 of RDP 3389 port has CVE-2019-0708, and Samba 3.6.25 of SMB 445 port has CVE-2017-7494), and the port vulnerability risk score is S risk (R i )=5.3×0.4+9.8×0.4+9.8×0.2=8.04; by weighted combination (setting the C segment risk weight as 0.3 and the port weight as 0.7), the external vulnerability dimension score of the relay R1 is S2(R i )=0.3×1.9+0.7×8.04=6.2

[0148] S213. According to the network status consensus released by Tor, the online time sequence of all relays in the last half year is obtained, and the time sequence dimension of the relay is evaluated in combination with the date of global special events. First, the abnormal time period coincidence degree of the relay based on the time weight function is calculated;

[0149] Define the abnormal time period set T abnormal ={t1,t2,...,t n}, determine the weight w i of each time point t i by counting the activity of historical malicious relays at different times in the time period, and then the weighted coincidence degree can be calculated:

[0150]

[0151] where T online is the online time of the relay on the day, w i satisfies∑w i =1, the high-risk period w i is higher.

[0152] Recalculate the special date activity of the relay based on the Z-score, and count the daily active time of the relay in the historical non-special period {L1, L2,..., L m}, calculate the mean μ and variance σ, then the abnormal activity can be calculated:

[0153]

[0154] where L special is the online time of the relay on the special day;

[0155] The final time series dimension evaluation result is:

[0156] S3(R i ) = α·S overlap (R i ) + β·S special (R i );

[0157] where α and β are weight coefficients;

[0158] For example: According to the official network status consensus of Tor, the online time series of relay R1 in the last half year is obtained. Define the abnormal period set as 22 o'clock to 6 o'clock every day (historical malicious relay active period), the weight of 22-24 o'clock in this period is w1=0.4, and the weight of 0-6 o'clock is w2=0.6. If the relay is online at 22-24 o'clock and 0-4 o'clock, the abnormal time overlap degree can be calculated as: Recalculate the special date activity, the mean value μ of the daily active time of the relay R1 in the historical non-special period is 5 hours, the variance σ is 1, and the relay R1 is online L special =8 hours on a special day (such as a major network attack day), then the abnormal activity can be calculated as: The time series dimension score of the relay R1 is:

[0159] S3(R i ) = 0.5×0.8 + 0.5×3 = 1.9

[0160] S214. According to the relay server descriptor file and network status consensus published by Tor, the online time of all relays in the relay list in the last half year and the weight change are counted, and the cooperative behavior dimension is evaluated. First, count the online time of all relays Ri The online time series is marked with 1 for online moments and 0 for offline moments, i.e., T i (t)∈{0,1}, (t=1,2,...,N), where N is the total number of time points, then the Pearson correlation coefficient between the two relays is:

[0161]

[0162] in For relay R i Average online rate;

[0163] Using the Pearson coefficient to calculate the distance of a relay online time series:

[0164]

[0165] Then, the distance between the relay online time series is used as the distance for single-chain clustering:

[0166]

[0167] At this point, relays with similar online time patterns will be grouped together, and the resulting clustered online time series matrix will be analyzed. M represents the number of relays. A Discrete Fourier Transform is performed to extract the frequency domain amplitude A(f). Adjacent relays with common periodicity are detected. If adjacent relays R... i and R j Re-frequency f k The following conditions must be met: and Where θ freq If the frequency amplitude threshold is used, then it is determined that the two have common periodicity;

[0168] Within a redefined time window [t1, t2], if K adjacent relays satisfy: These relays are then identified as exhibiting abnormal online / offline coordination behavior: S sync =1;

[0169] Next, we analyze the changes in bandwidth weights of all relays over the past six months, assuming the number of relays is R. i The bandwidth weight sequence is W i (t), the moment when the relay's bandwidth weight is 0 or at its minimum value is defined as the initial point: The point at which the relay's bandwidth weight reaches its maximum value is defined as the termination point: Plot the bandwidth weight change trend curve of the relay from the initial point to the terminal point. The standardization of the weight change trend curve is as follows:

[0170]

[0171] The average rate of change of the weight from the initial point to the terminal point is calculated:

[0172]

[0173] If k i > k thresh , it is determined that there is a surge, k thresh is an abnormal rate threshold value;

[0174] The weight change trend similarity of relay R i and R j uses the cosine similarity:

[0175]

[0176] If and the weight growth rate satisfies:

[0177] Wherein , it is determined that the weight growth trends of the two are related;

[0178] The relay whose weight abnormally surges and whose weight change trend has relevance is determined to have abnormal weight change behavior: S weight = 1;

[0179] The final cooperative behavior dimension evaluation result is:

[0180] S4(R i ) = α·S sync + β·S weight ;

[0181] Wherein α and β are weight coefficients;

[0182] For example: assuming that there are relays R1 (IP: 192.168.4.14) and R2 (IP: 192.168.4.15), and their 24-hour online time sequences are both [0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0], the Pearson correlation coefficient of the two is 1, the online time distance is 0, and they are closely arranged when single-chain clustering. After performing discrete Fourier transform on the online time matrix, the frequency domain amplitude spectrum A A (f) and A A (f) are obtained, wherein the dominant frequency is f0 = 1 / 24, corresponding to a 24-hour period, and the proportion of the amplitude at this frequency to the maximum amplitude of each is calculated: Set the periodicity determination threshold θ freq= 0.7, since both ratios exceed the threshold, it is determined that the relays R1 and R2 have common periodicity. Meanwhile, in the [4, 9] hour and [14, 19] hour windows, it satisfies that ∑|T3((t)-T4(t)| = 0, that is, completely simultaneous online, so the relays R1 and R2 both have up and down line cooperative behavior: S sync = 1. Again, calculate the bandwidth weight change, assuming that the weight sequence of the relay R3 (IP: 192.168.4.18) is [0, 200, 400, 700, 1100, 1600], the average change rate is 300, which exceeds the abnormal rate threshold 100, and it is determined to be a surge; the weight sequence of the relay R4 (IP: 192.168.4.19) is [200, 300, 500, 800, 1200, 1700], the cosine similarity of the normalized weight change sequence of the relays R3 and R4 is 1, so the relays R3 and R4 have abnormal weight change behavior: S weight = 1, the weight growth trend is related. Assuming that a = 0.5 and β = 0.5, the comprehensive score S collab = 0.5 x 1 + 0.5 x 1 = 1, indicating that these relays not only have highly cooperative online time patterns, common periodicity, and simultaneous up and down line behaviors, but also have closely related and surging bandwidth weight changes, and are extremely likely to be a relay cluster controlled by the same organization, and there is significant abnormal cooperative behavior.

[0183] S220. According to experience, set a threshold for the abnormal evaluation S(R i ) of the relays subjected to security evaluation, and filter out relays with an abnormal score higher than the threshold. These relays are determined to be suspicious relays.

[0184] S300. Based on the suspicious relays filtered out in step S200, obtain the internal attribute behavior and external intelligence characteristics of each relay node by comprehensively using the relay server descriptor file and the existing network public asset mapping platform, and model the similarity of relay pairs;

[0185] Specifically, the network public asset mapping platform refers to a tool and system for scanning publicly exposed network assets on the Internet, including the Fofa platform or DayDayMap.

[0186] In one embodiment, the step S300 includes the following steps:

[0187] S310. Query the suspicious relay information running on each suspicious IP according to the relay server descriptor, and obtain the IP address, Tor version, nickname, logo, operating system, contact, external access port, long-term bandwidth capacity, burst bandwidth capacity, and routing strategy of all suspicious relays;

[0188] S320. Querying the relays in the suspicious relay list using the public asset mapping platform to obtain the website title, server product, commonly used protocol, and the geographic location and autonomous system where the server corresponding to each suspicious relay IP has been used in the past half year;

[0189] S330. Calculating the similarity features of the relay pair based on the relay features in steps S310 and S320;

[0190] Further, in step S330, step S330 includes the following steps:

[0191] S331. Dividing all the features of the relay into three categories, the self attribute features being IP address, Tor version, nickname, logo, operating system, and contact;

[0192] The network behavior features being external access port, long-term bandwidth capacity, burst bandwidth capacity, and routing strategy;

[0193] The external intelligence features being the website title, server product, commonly used protocol, and the geographic location and autonomous system where the server is located;

[0194] Let the relay pair be (r i ,r j ), different types of features need to use different similarity calculation methods;

[0195] S332. For string type features f text , such as IP address, Tor version, nickname, logo, operating system, contact, geographic location, and autonomous system, use the edit distance (Levenshtein distance) to calculate the similarity:

[0196]

[0197] Where Levenshtein(a, b) represents the edit distance of strings a and b, and len(a) represents the length of string a;

[0198] For example, assume that the contact attribute of relay R1 is The contact attribute of relay R2 is The string length is calculated as The edit distance is calculated as Convert "abcde@ab.cde" to "abcdefg@ab.cde", which requires inserting "f" and "g" after "e", and the edit distance is 2. Substitute into the formula to calculate the similarity:

[0199] S333. For numerical value type features f numThe similarity is calculated using the normalized Euclidean distance: S = 1 - d / D

[0200]

[0201] where num represents the index set of numerical features, max(f k ) and min(f k ) are the maximum and minimum values of all relays relaying this feature, respectively;

[0202] For example, assuming that the bandwidth_avg attribute of relay R1 is 65536000, the bandwidth_avg attribute of relay R2 is 52428800, the maximum bandwidth_avg attribute value among all relays is 1073741824, and the minimum is 0, the similarity is calculated by substituting the formula:

[0203] S334. For set type features f set , such as routing policy, website title, server product, and commonly used protocol, the similarity is calculated using the Jaccard coefficient:

[0204]

[0205] For example, assuming that the policy attribute of relay R1 is {tls, dcerpc, rdp, http, ssh} and the policy attribute of relay R2 is {rdp, http, ssh, tls}, the similarity is calculated by substituting the formula:

[0206] S335. The similarity of a relay pair is defined as:

[0207] S ij = (S attr , S behavior , S intelligence );

[0208] where S attr = w i S IP + w i S Tor_version + w i S nickname + w i S sign + w i S system + w i S contact , w i is a weight coefficient, i∈(1, 6)

[0209] Sbehavior =k j S or_port +k j S bandwidth_avg +k j S bandwidth_burst +k j S policy k j Here are the weighting coefficients, j∈(1,4);

[0210] S intelligence =l m S title +l m S product +l m S protocol +l m S location +l m S as , l m , where m∈(1,5);

[0211] For example: Figure 3 As shown, the Figure 3 This is a sample diagram used in this step to calculate the similarity features of relay pairs. Relay pairs R1 and R2 contain 15 attributes, including their own attributes, network behavior, and external intelligence. The similarity features of this relay pair can be calculated as follows: S attr =0.175+0.143+0.12+0.1+0.087+0.14=0.765, S behavior =0.012+0.219+0.219+0=0.45,S intelligence =0.071+0.12+0.2+0.05+0.08=0.521, therefore the final similarity feature of this relay pair is: S=(0.765,0.45,0.521)

[0212] S400. Based on the relay pair similarity features calculated in step S300, use the random forest algorithm to predict the association degree of all relay pairs;

[0213] In one embodiment, the Figure 4 For the step of calculating the correlation degree of relay pairs, step S400 includes the following steps:

[0214] S410. Pair all online relays together to form a relay pair dataset. Label relay pairs belonging to the same family as family=1, and label relay pairs not belonging to the same family as family=0.

[0215] S420. Divide the relay pair dataset D into a training set D. train and test set D test, where |D train |=0.7|D|,|D test With |=0.3|D| as input, the similarity features calculated in step S335 are used to train and test the relay pair dataset using the random forest algorithm to calculate the probability that the suspicious relay pairs may belong to the same family.

[0216] Furthermore, in step S420, step S420 includes the following steps:

[0217] S421. A random forest model consists of m decision trees, for each relay pair (r i ,r j The probability that the k-th decision tree outputs belongs to the same family is... The relay pair correlation P output by the random forest model is... ij for:

[0218]

[0219] S422. During training, the cross-entropy loss function L is used to optimize the model parameters:

[0220]

[0221] The random forest model is trained by minimizing the loss function L, and finally tested on the test set D. test The performance was evaluated.

[0222] S500. Based on the probability that relay pairs belong to the same family calculated in step S400, density-based clustering is performed using the relay distance of the OPTICS clustering algorithm to organize and divide suspicious relays.

[0223] Specifically, the density-based clustering algorithm is a clustering algorithm that divides clusters by the density of data points, including DBSCAN or OPTICS;

[0224] In one embodiment, step S500 includes the following steps:

[0225] S510. Based on the suspicious relays obtained in step S300 and the relay pair correlation obtained in step S450, the suspicious relays are clustered using the OPTICS clustering algorithm;

[0226] S520. Clusters with different family information relays and clusters consisting entirely of no family information relays are retained and defined as suspicious Sybil organizations.

[0227] Finally, it should be noted that the above only describes one embodiment of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art will appreciate that modifications, equivalent replacements and improvements of the technical solutions described in the foregoing embodiments can still be made.

[0228] Any modifications, equivalent replacements and improvements made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A Tor network hidden malicious organization mining method based on multi-dimensional fusion detection, characterized in that, The method comprises the following steps: S100. Crawling the network status consensus published by Tor Metric in the last half year, sorting the relay list in the Tor network, and mining the related information of each relay in the relay list; S200. Based on the relay list and related features in step S100, performing multi-dimensional security evaluation on the relays in the relay list, and screening out suspicious relays with abnormal scores higher than a threshold value; S300. Based on the suspicious relays filtered out in step S200, obtaining the internal attribute behavior and external intelligence features of each relay node, and performing similarity modeling on the relay pairs; S400. Based on the similarity features of the relay pairs calculated in step S300, using a random forest algorithm to calculate the correlation degree of the relay pairs; S500. Based on the correlation degree of the relay pairs calculated in step S400, using the relay distance of the OPTICS clustering algorithm to perform density-based clustering, and organizing and dividing the suspicious relays. 2.The Tor network hidden malicious organization mining and identifying method based on multi-dimension fusion detection according to claim 1, characterized in that, The information that can be mined for each relay in step S100 specifically includes the following contents: Basic configuration information, online period information, network information in the C segment, open port information, bandwidth weight change information of each relay node. 3.The Tor network hidden malicious organization mining and identifying method based on multi-dimensional fusion detection according to claim 1, characterized in that, The step S200 comprises the following steps: S210. According to the multi-dimension security evaluation result of the relay, a scoring formula is designed to calculate the anomaly score of each relay ; S220. Score the relaying anomalies that are subjected to security evaluation according to experience A threshold is set to screen out the relaying IPs with scores higher than the threshold, and these IPs are determined as suspicious relaying IPs.

4. The Tor network hidden malicious organization mining and identifying method based on multi-dimensional fusion detection according to claim 1, characterized in that, The step S300 comprises the following steps: S310. Querying the suspicious relay information of each suspicious IP according to the relay server descriptor, obtaining the IP address, Tor version, nickname, flag, operating system, contact person, external access port, long-term bandwidth capacity, burst bandwidth capacity, and routing strategy of all suspicious relays; S320. Querying the relays in the suspicious relay list using a public asset mapping platform, obtaining the website title, server product, commonly used protocol, geographical location, and autonomous system of the server corresponding to each suspicious relay IP in the last half year; S330. Calculate the similarity feature of the relay pair based on the relay features of steps S310 and S320 .

5. The Tor network hidden malicious organization mining and identifying method based on multi-dimensional fusion detection according to claim 1, characterized in that, The step S500 comprises the following steps: S510. Based on the suspicious relays obtained in step S200 and the correlation degree of the relay pairs obtained in step S400, using the OPTICS clustering algorithm to cluster the suspicious relays; S520. Retaining the clusters with different family information relays and the clusters composed of no family information relays, and defining them as suspicious Sybil organizations.

Images