Method and device for clustering fraud-related entities based on network traffic and metadata

Through clustering methods and situational awareness technology based on network traffic and metadata, the problem of difficulty in identifying and tracking of short-term fraud-related network entities in the existing technology is solved, and the rapid identification and effective crackdown on fraud activities is achieved, and network security response is improved. ability.

CN119094190AActive Publication Date: 2024-12-06BEIJING FULE TECH CO LTD
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
CN202411190742.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-06
Estimated Expiration
2044-08-28

AI Technical Summary

Technical Problem

The prior art is difficult to effectively identify and track fraud-related network entities for short-term use, especially in the context of rapid changes in domain names and reduced reuse rates, traditional methods are inefficient and prone to missing key information.

Method used

Using fraud-related entities clustering methods based on network traffic and metadata, through preset clustering algorithms and situational awareness technology, network entities with similar characteristics or behavior patterns are automatically identified and tracked, combining multi-source data fusion and natural language processing. , monitor and track scam activities in real time.

Benefits of technology

It realizes accurate identification and tracking of fraud-related network entities, improves the response speed and strike capabilities of the network security team, reduces the cost of manual intervention, and has real-time monitoring and risk assessment capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119094190A_ABST
    Figure CN119094190A_ABST
Patent Text Reader

Abstract

The invention discloses a fraud-related entity clustering method and device based on network traffic and metadata, and relates to the field of network security. In the method, flow logs and metadata are obtained from a plurality of data sources, and the data sources comprise a CDN, a public cloud, a private cloud and a target area internal network; marking the traffic in the traffic log through a preset label propagation algorithm to distinguish CDN traffic, public cloud traffic, private cloud traffic, Whois privacy protection traffic, malicious traffic and service traffic; performing clustering analysis on the marked data through a preset clustering algorithm to identify potential target network entities related to fraud; and performing aggregation analysis on the metadata through a situation awareness technology to identify and track a target network entity from the potential target network entities, and performing risk assessment. By implementing the technical scheme provided by the invention, the fraud-related network entity can be accurately identified and tracked, and the network security protection capability is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of network security, and in particular to a method and device for clustering fraud-related entities based on network traffic and metadata. Background Art

[0002] With the rapid development of network technology, network security issues have become increasingly prominent, especially the frequent occurrence of online fraud activities, which has caused huge economic losses to individuals and enterprises.

[0003] Most of the existing methods for countering and tracking fraudulent entities rely on manual aggregation and analysis of domain names. However, due to the rapid changes and single-use characteristics of domain names, this method is not only inefficient, but also prone to missing key information. Some solutions may use basic data analysis and pattern matching to track fraudulent activities, but they usually lack the ability to handle large-scale data, as well as the in-depth understanding and processing capabilities of complex fraud patterns. More specifically, existing technologies mainly rely on collecting data after the fact to form a "cesspool" in an attempt to identify fraudulent information. However, in the context of the current situation where the reuse rate of the domain names involved in the case is gradually decreasing and new domain names are constantly emerging, especially many domain names are only used once or for a single day, this method has become difficult to effectively collect these short-term domain names.

[0004] Therefore, how to provide a technical solution that can accurately identify and track fraud-related network entities has become an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0005] The present application provides a method and device for clustering fraud-related entities based on network traffic and metadata. By performing cluster analysis on the marked data through a preset clustering algorithm, potential target network entities involved in fraud can be effectively identified, and network entities with similar characteristics or behavior patterns can be automatically discovered, thereby quickly locating potential fraud activities. Combined with situational awareness technology, metadata is aggregated and analyzed to further identify and track specific target network entities from potential targets. This real-time monitoring and tracking capability enables the network security team to respond quickly and effectively intervene in and combat fraud-related activities.

[0006] In a first aspect of the present application, a method for clustering fraud-related entities based on network traffic and metadata is provided, which is applied to a fraud-related network entity clustering and tracking platform. The method comprises: Obtain traffic logs and metadata from multiple data sources, including CDN, public cloud, private cloud, and internal network of the target region; The traffic in the traffic log is marked by a preset label propagation algorithm to distinguish CDN traffic, public cloud traffic, private cloud traffic, Whois privacy protection traffic, malicious traffic and business traffic; Perform cluster analysis on the marked data through a preset clustering algorithm to identify potential target network entities involved in fraud; The metadata is aggregated and analyzed through situation awareness technology to identify and track target network entities from the potential target network entities and perform risk assessment.

[0007] By adopting the above technical solution, traffic logs and metadata are obtained from multiple data sources such as CDN, public cloud, private cloud and internal network of the target area, ensuring the comprehensiveness and diversity of data. This multi-source data fusion method can more comprehensively reflect the real situation of network traffic and metadata, and provide a solid foundation for subsequent clustering analysis and situation awareness. Through the preset label propagation algorithm, the traffic in the traffic log is accurately marked, and CDN traffic, public cloud traffic, private cloud traffic, Whois privacy protection traffic, malicious traffic and business traffic are effectively distinguished. This refined traffic classification helps to more accurately identify fraudulent network entities in subsequent clustering analysis. The preset clustering algorithm is used to perform cluster analysis on the marked data to identify potential target network entities involved in fraud. The clustering algorithm can automatically discover the potential patterns and structures in the data, classify data points with similar characteristics into one category, and effectively identify fraudulent network entities. This method not only improves the analysis efficiency, but also reduces the cost of manual intervention. Through the situation awareness technology, the metadata is aggregated and analyzed, and the target network entities are further identified and tracked from the potential target network entities, and risk assessment is performed. Situational awareness technology can monitor changes in network status in real time, detect and respond to potential security threats in a timely manner. At the same time, by tracking target network entities, we can gain an in-depth understanding of their behavior patterns and characteristics, providing strong support for subsequent crackdowns and prevention.

[0008] Optionally, the marking the traffic in the traffic log by using a preset label propagation algorithm includes: Setting initial labels for nodes of known categories based on the configuration information, wherein the nodes of known categories include CDN nodes, public cloud nodes, and Whois privacy protection nodes; Determine a final label of each node according to a connection relationship between the nodes, wherein the connection relationship includes traffic or network connections recorded in a traffic log; Traffic is marked according to node labels, traffic characteristics and traffic behavior patterns of the nodes it passes through, wherein the traffic characteristics include protocol type, port number and request URL, and the traffic behavior pattern includes access frequency and request time distribution.

[0009] By adopting the above technical solution, initial labels are set for nodes of known categories (such as CDN nodes, public cloud nodes, and Whois privacy protection nodes), which can ensure the accuracy of these nodes during the label propagation process. These nodes serve as "anchor points" and provide a reliable benchmark for the label allocation of the entire network traffic. The final label of each node is determined based on the connection relationship between nodes (such as the traffic or network connection recorded in the traffic log), which fully considers the actual flow direction of network traffic and the interaction between nodes. Through the propagation algorithm, nodes with unknown labels can infer their own labels based on the information of the known label nodes connected to them, thereby improving the accuracy of label allocation. When marking traffic, not only the basic characteristics of traffic (such as protocol type, port number, and request URL) are considered, but also the behavior patterns of traffic (such as access frequency and request time distribution). This multi-dimensional consideration makes traffic marking more refined and can more accurately reflect the true attributes and potential risks of traffic. By introducing traffic behavior patterns (such as access frequency and request time distribution), the behavioral characteristics of traffic can be more deeply understood. These behavior patterns are often closely related to fraudulent activities, so by analyzing and marking them, potential fraudulent behaviors can be more effectively identified. Through the preset label propagation algorithm, labels can be automatically assigned to network traffic, reducing the cost and error rate of manual intervention. This automated label assignment method improves the efficiency and accuracy of network traffic analysis. Combining traffic characteristics and traffic behavior patterns for labeling enables the system to intelligently identify potential fraudulent behavior. This intelligent identification capability provides strong support for network security teams, helping them to promptly detect and respond to network fraud activities.

[0010] Optionally, the performing cluster analysis on the marked data by using a preset clustering algorithm to identify potential target network entities involved in fraud includes: Select target features according to the identification requirements of fraudulent behavior, and preprocess and normalize the target features, wherein the target features include domain name age, SSL certificate information, IP location similarity, domain name Hamming distance, IP service and port information; The neighborhood radius and the minimum number of points of the preset clustering algorithm are set according to the characteristics of the fraudulent behavior, and the marked data are clustered according to the preset clustering algorithm and the target features to generate a clustering result; Based on the clustering results, the characteristics and commonalities of different clusters are analyzed to identify potential target network entities involved in fraud.

[0011] By adopting the above technical solution, according to the identification requirements of fraudulent behavior, a series of features related to the target network entity are selected, such as domain name age, SSL certificate information, IP location similarity, domain name Hamming distance, IP service and port information, etc. The selection of these features is based on a deep understanding of network fraudulent behavior and can fully reflect the potential risks of network entities. By preprocessing and normalizing these target features, the dimensional differences between different features are eliminated, and the accuracy and reliability of cluster analysis are improved. According to the characteristics of fraudulent behavior, the neighborhood radius and minimum number of points of the preset clustering algorithm are customized. This customized processing makes the clustering algorithm more suitable for the identification scenario of fraudulent behavior and can more accurately discover network entities with similar characteristics and behavior patterns. The marked data is automatically clustered and analyzed by the preset clustering algorithm, and clustering results can be generated without manual intervention. This automated processing method greatly improves the efficiency of fraudulent behavior identification and reduces labor costs. Based on the clustering results, the system can automatically analyze the characteristics and commonalities of different clusters to identify potential target network entities involved in fraud. This intelligent identification method enables the system to detect and respond to potential fraud threats in a timely manner, improving the defense capabilities of network security. The cluster analysis process takes into account multiple dimensions of feature information, such as domain name, SSL certificate, IP location, etc. This information provides a comprehensive basis for risk assessment. Through the comprehensive analysis of these features, the potential risks of network entities can be more accurately assessed.

[0012] Optionally, performing cluster analysis on the marked data according to the preset clustering algorithm and the target feature to generate a clustering result includes: grouping the marked data according to the neighborhood radius, the minimum number of points, and the target feature; Traversing all data points in the marked data, when the number of data points within the domain radius of the target data point is greater than or equal to the minimum number of points, setting the target data point as a core point; Starting from the core point, adjacent data points are connected into a cluster through density reachability, and a clustering result is generated based on the cluster.

[0013] By adopting the above technical solution, a density-based clustering algorithm (such as DBSCAN, etc.) is used to control the compactness of clustering through two parameters: neighborhood radius and minimum number of points. This method can effectively identify clusters of any shape, thereby improving the accuracy of clustering analysis. In the process of traversing data points, the number of data points within the neighborhood radius is used to determine whether it is a core point. This judgment method ensures that only areas with sufficiently high density can be identified as the core part of the cluster, further improving the accuracy of clustering. Starting from the core point, adjacent data points are connected into a cluster through density accessibility. This connection method not only considers the spatial distance between data points, but also the density characteristics of data points, making the clustering results more stable and reliable. Even if there are noise or outliers in the data set, the influence of noise or outliers on the clustering results can be avoided by judging density accessibility. As key parameters of the clustering algorithm, the neighborhood radius and the minimum number of points can be flexibly adjusted according to the specific fraud behavior identification needs. This configurability enables the clustering algorithm to be applicable to data sets of different sizes and characteristics, improving the applicability and flexibility of the algorithm. By traversing the data points and connecting them based on density reachability, clustering algorithms can efficiently process large-scale data sets. At the same time, since the core point identification process is relatively simple and direct, the computational complexity of the entire clustering process is low, and clustering results can be generated in a shorter time.

[0014] Optionally, the performing aggregate analysis on the metadata by using situation awareness technology to identify and track the target network entity from the potential target network entities includes: Adopt data aggregation technology to aggregate pre-processed metadata to reveal potential fraud behavior patterns and trends; Use natural language processing and semantic analysis technology to understand the aggregated metadata to reveal the logical relationship between the data and determine the connection between different fraud behaviors and network assets through association reasoning algorithms; Time series analysis and deep learning models are used to predict fraud trends and network asset risks, and to draw fraud trend forecast charts.

[0015] By adopting the above technical solutions, the pre-processed metadata is effectively integrated through data aggregation technology, thereby revealing potential fraud behavior patterns and trends. Data aggregation not only helps to discover anomalies in a single data point, but also reveals the commonalities and laws of fraud behavior in terms of time, space, and behavior patterns through pattern matching and trend analysis. This helps the security team better understand the nature and evolution of fraud behavior. Natural language processing and semantic analysis technologies are used to deeply understand and analyze the aggregated metadata. These technologies can identify keywords, phrases, and sentences in the data and understand the logical relationship between them. Through the association reasoning algorithm, the intrinsic connection between different fraud behaviors and network assets can be further determined, providing strong support for subsequent tracking and response. Based on the results of semantic understanding and association reasoning, the security team can more accurately judge the nature and scope of the fraud behavior, thereby formulating more accurate and effective response strategies. This helps to improve the intelligence level of security decision-making and reduce the risk of human misjudgment and missed judgment. Through the application of time series analysis and deep learning models, the trend of fraud behavior and the risk of network assets can be accurately predicted. These models can capture subtle changes and potential laws in the data and predict future development trends based on them. By drawing a fraud trend prediction chart, the security team can intuitively understand the evolution trend and potential risk points of fraudulent behavior, providing strong guidance for future security prevention work.

[0016] Optionally, the data aggregation technology is used to aggregate the pre-processed metadata to reveal potential fraud behavior patterns and trends, including: The type of fraud behavior is used as the aggregation keyword, and the distribution of each type in the overall fraud behavior is counted to identify the type of fraud behavior with the highest incidence rate; The geographical location information of the fraudulent behavior is used as the aggregation keyword, and the number of fraudulent behaviors occurring in each geographical location is counted to draw a heat map or distribution map of the fraudulent behavior; The time information of fraudulent behaviors is used as the aggregation keyword, and the timeline diagram of fraudulent behaviors is drawn by counting the number of fraudulent behaviors occurring in each time period; A multi-dimensional cross-aggregation view is formed by cross-combining the analysis results of a single dimension, wherein the single dimension includes type, location, and time.

[0017] By adopting the above technical solution, the fraud behavior type is used as the aggregation keyword, and by counting the distribution of each type in the overall fraud behavior, the fraud behavior type with the highest incidence rate can be intuitively identified. This accurate identification helps the security team prioritize the limited resources to prevent and combat high-incidence fraud behaviors and improve the prevention efficiency. Using geographic location information as the aggregation keyword, by drawing a heat map or distribution map of fraud behavior, the distribution of fraud behavior in different geographical areas can be intuitively displayed. This visual presentation method helps the security team quickly locate high-risk areas and take corresponding preventive measures to reduce the occurrence of fraud. Using time information as the aggregation keyword, by counting the number of fraud behaviors occurring in each time period and drawing a timeline chart, the time distribution pattern of fraud behavior can be clearly displayed. This analysis helps the security team predict the high-incidence period of fraud in the future, and prepare in advance to effectively curb the spread of fraud. By cross-combining the analysis results of a single dimension to form a multi-dimensional cross-aggregation view, the complexity and diversity of fraud behavior can be more comprehensively understood. This multi-dimensional analysis not only helps to reveal the intrinsic connections and mutual influences between fraudulent behaviors, but also provides more accurate decision-making support for security teams, enabling them to formulate more comprehensive and effective prevention strategies.

[0018] Optionally, the use of natural language processing and semantic analysis technology to understand the aggregated metadata to reveal the logical relationship between the data, and to determine the connection between different fraudulent behaviors and network assets through an association reasoning algorithm includes: Extracting entities from the aggregated metadata using named entity recognition technology, including fraud method names, victim types, IP addresses, and domain names; Using relationship extraction technology, the relationships between the entities are identified and extracted. Combined with context information, the extracted entities and the relationships are semantically understood to reveal the logical relationship behind them. Different data points are associated according to preset rules, and the associated data points are represented in the form of a network graph, where nodes represent entities and edges represent the relationships between entities; Through the association reasoning algorithm, path search and influence analysis are performed in the network graph to reveal the connection between different fraud behaviors and network assets.

[0019] By adopting the above technical solutions, key entities such as fraud method names, victim types, IP addresses and domain names can be accurately extracted from massive and complex metadata. These entities are the basis for understanding the relationship between fraudulent behavior and network assets. Accurate extraction helps to improve the accuracy and efficiency of subsequent analysis. Using relationship extraction technology, the relationship between entities can be identified and extracted, and semantic understanding can be performed in combination with contextual information. This can not only reveal the direct and explicit relationship between entities, but also discover implicit and deep logical relationships through semantic reasoning. This in-depth understanding helps to more comprehensively grasp the complex relationship network between fraudulent behavior and network assets. The associated data points are represented in the form of a network graph, with nodes representing entities and edges representing the relationship between entities. This visual presentation makes the complex relationship network between fraudulent behavior and network assets intuitive and easy to understand, which helps the security team quickly locate key nodes and relationship paths, and provides strong support for subsequent analysis and decision-making. Path search and influence analysis in the network graph can efficiently reveal the connection between different fraudulent behaviors and network assets. Through the association reasoning algorithm, it is possible to identify how fraudulent behavior is spread and diffused through network assets. This information is of great significance for formulating targeted prevention strategies.

[0020] In a second aspect of the present application, a system for clustering fraud-related entities based on network traffic and metadata is provided, comprising a collection module, a marking module, a clustering module and an execution module, wherein: A collection module configured to obtain traffic logs and metadata from multiple data sources, including CDN, public cloud, private cloud, and internal network of the target area; A marking module configured to mark the traffic in the traffic log by a preset label propagation algorithm to distinguish CDN traffic, public cloud traffic, private cloud traffic, Whois privacy protection traffic, malicious traffic and business traffic; A clustering module configured to perform cluster analysis on the marked data by using a preset clustering algorithm to identify potential target network entities involved in fraud; The execution module is configured to perform aggregate analysis on the metadata through situation awareness technology to identify and track target network entities from the potential target network entities and perform risk assessment.

[0021] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0022] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any of the methods described above is executed.

[0023] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Obtain traffic logs and metadata from multiple data sources such as CDN, public cloud, private cloud, and internal network of the target area. This extensive and diverse data source ensures the comprehensiveness and accuracy of the data, providing a solid foundation for subsequent analysis; 2. The traffic in the traffic log is marked by a preset label propagation algorithm, effectively distinguishing CDN traffic, public cloud traffic, private cloud traffic, Whois privacy protection traffic, malicious traffic, and business traffic. This refined classification not only improves the accuracy of data analysis, but also provides clear boundary conditions for subsequent cluster analysis; 3. Use the preset clustering algorithm to perform cluster analysis on the marked data, which can automatically identify potential fraud-related target network entities. This data-driven method is more efficient and accurate than traditional manual analysis, and can quickly lock in fraud-related targets, providing strong support for subsequent processing; 4. Aggregate analysis of metadata through situational awareness technology not only reveals potential fraud behavior patterns and trends, but also further understands the logical relationship between data through natural language processing and semantic analysis technology, time series analysis and deep learning models, and predicts fraud behavior trends and network asset risks. This in-depth analysis helps security teams unravel complex data and accurately identify and track target network entities; 5. After identifying and tracking the target network entity, the method also conducts risk assessment. By comprehensively considering multiple factors (such as the type of fraudulent behavior, frequency of occurrence, scope of impact, etc.), it can accurately assess the risk level of the target network entity and provide a scientific basis for subsequent response measures; 6. It has the ability to process network traffic and metadata in real time, and can promptly detect and respond to potential fraudulent activities. At the same time, by dynamically adjusting the parameters of clustering algorithms and situational awareness technology, it can also adapt to changes in the network environment and maintain efficient and accurate performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flowchart of a method for clustering fraud-related entities based on network traffic and metadata disclosed in an embodiment of the present application; Figure 2 It is a schematic diagram of the overall architecture of the method for clustering fraud-related entities based on network traffic and metadata disclosed in the embodiment of the present application; Figure 3 It is a module schematic diagram of a system for clustering fraud-related entities based on network traffic and metadata disclosed in an embodiment of the present application; Figure 4 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0025] Explanation of the reference numerals: 301, acquisition module; 302, labeling module; 303, clustering module; 304, execution module; 401, processor; 402, communication bus; 403, user interface; 404, network interface; 405, memory. DETAILED DESCRIPTION

[0026] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0027] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0028] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0029] This embodiment discloses a method for clustering fraud-related entities based on network traffic and metadata, which is applied to a fraud-related network entity clustering and tracking platform. Figure 1 is a flow chart of a method for clustering fraud-related entities based on network traffic and metadata disclosed in an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps: S110, obtaining traffic logs and metadata from multiple data sources, where the data sources include CDN, public cloud, private cloud, and internal network of the target area; Figure 2This is a schematic diagram of the overall architecture of the method for clustering fraud-related entities based on network traffic and metadata disclosed in the embodiment of the present application, combined with Figure 2 The embodiments of the present application are described below. Figure 2 As shown, after data collection, metadata sets are constructed and labeled data is collected, and data is stored and processed.

[0030] CDN (Content Distribution Network) is an intelligent virtual network built on the basis of existing networks. Public cloud refers to cloud computing services owned, managed and operated by third-party cloud service providers. Private cloud refers to allowing enterprises or organizations to own and manage their own computing resources and services. Deploy network monitoring tools (such as Wireshark, tcpdump, Snort, etc.) in the internal network of the target area to capture real-time network traffic. Design and implement a distributed web crawler to crawl data from data sources that support web access (CDN, public cloud, private cloud, and internal network of the target area). Use libraries such as Scrapy and BeautifulSoup to parse web pages and extract required information. For data sources that provide API interfaces, write API client programs, call APIs regularly and crawl data. Store the crawled data in a distributed file system (such as HDFS, Ceph) for subsequent processing and analysis. At the same time, NoSQL databases (such as Cassandra, MongoDB) can be used to store structured or semi-structured data. Use message queues (such as Kafka, RabbitMQ) and stream processing frameworks (such as Apache Flink, Apache Spark Streaming) to process newly captured data in real time and update the information in the metadata engine. For metadata that is volatile over time, design a mechanism to regularly record its historical snapshots for retrospective analysis and auditing. Integrate data from different data sources into a unified metadata collection, including PassiveDNS, ICP, Domain, Host, Whois, SSL Cert, IP ASN, IP Location, and other information. Collect marked fraud case data from law enforcement agencies, security organizations, and other channels, including fraud host, domain, fraud type, and other information. For IPs, Domains, etc. involved in fraud, record their related metadata snapshots for in-depth analysis and tracking. Use encryption technology during data transmission and storage to ensure data security and privacy. Ensure that the data capture and processing process complies with the requirements of relevant laws and regulations, such as GDPR, CCPA, etc.

[0031] S120, marking the traffic in the traffic log by a preset label propagation algorithm to distinguish CDN traffic, public cloud traffic, private cloud traffic, Whois privacy protection traffic, malicious traffic and business traffic; like Figure 2 As shown in the figure, the data processing and security steps include hashing or encrypting sensitive information, traffic marking and classification through label propagation algorithms, and cluster analysis. After the data is collected, it is processed by a dedicated cleaning engine. This engine is responsible for identifying and removing invalid, duplicate or malformed data to ensure the data quality for subsequent analysis. Sensitive information (such as personal identity information, financial information, etc.) is hashed or encrypted during the cleaning process to prevent data leakage and abuse. This processing method not only retains the availability of data (for analysis and statistics) but also ensures the privacy of data. In order to better understand and utilize traffic data, a label propagation algorithm is introduced. The algorithm can automatically mark data into different categories based on the characteristics and attributes of traffic data, such as CDN traffic, public cloud traffic, private cloud traffic, Whois privacy protection traffic, malicious traffic, and business traffic. For CDN traffic, the algorithm will identify the IP addresses accelerated by CDN and label them accordingly. The distinction between public cloud traffic and private cloud traffic is based on information such as IP address, port number, protocol type, etc. This distinction helps enterprises understand their resource usage in different cloud environments and make corresponding resource allocation and cost optimization. Whois privacy protection data refers to domain name data that hides the registrant information through the Whois service. The algorithm will identify these domain names and label them with privacy protection to protect the privacy rights of the registrants. Identification of malicious traffic is an important part of network security. The algorithm will combine multiple features (such as request frequency, request content, source IP address, etc.) to determine whether the traffic is malicious and label it accordingly. This helps enterprises to promptly detect and prevent malicious attacks and ensure network security. Business traffic refers to the traffic generated by the normal business activities of the enterprise. The algorithm will regard the remaining traffic that is not clearly classified into any of the above categories as business traffic and label it accordingly. The above label propagation algorithm can effectively avoid the inappropriate clustering of unrelated network assets. This is because each traffic data is given a clear label, allowing data analysts to clearly understand the attributes and ownership of each data point. Clustering optimization not only improves the accuracy and efficiency of data analysis, but also provides strong support for enterprises to formulate targeted network strategies. For example, enterprises can optimize their network architecture and resource allocation based on the distribution of CDN traffic and public cloud traffic, and strengthen network security protection based on the identification results of malicious traffic.

[0032] Optionally, the marking the traffic in the traffic log by using a preset label propagation algorithm includes: Setting initial labels for nodes of known categories based on the configuration information, wherein the nodes of known categories include CDN nodes, public cloud nodes, and Whois privacy protection nodes; Determine a final label of each node according to a connection relationship between the nodes, wherein the connection relationship includes traffic or network connections recorded in a traffic log; Traffic is marked according to node labels, traffic characteristics and traffic behavior patterns of the nodes it passes through, wherein the traffic characteristics include protocol type, port number and request URL, and the traffic behavior pattern includes access frequency and request time distribution.

[0033] Initial labels are set for specific categories of nodes based on known configuration information or prior knowledge. These known categories of nodes include but are not limited to CDN nodes, public cloud nodes, and Whois privacy protection nodes. These initial labels are the starting point of the label propagation algorithm and provide a basis for the subsequent propagation process. The algorithm analyzes the connection relationships between nodes, which are mainly derived from the traffic or network connections recorded in the traffic log. By considering the interactions between nodes and the flow of traffic, the algorithm can infer the potential associations and similarities between nodes. Based on the connection relationships between nodes, the algorithm iteratively updates the label of each node. In each iteration, the node updates its own label based on the labels of its neighbor nodes and its own characteristics. This process continues until the labels of all nodes are stable (i.e. converged) or the preset maximum number of iterations is reached. After the label propagation process is completed, the algorithm further labels the traffic based on the final label of each node and the traffic characteristics and behavior patterns in the traffic log. Traffic characteristics include but are not limited to protocol type, port number, and request URL, while traffic behavior patterns may include access frequency, request time distribution, etc.

[0034] The specific markings are as follows: CDN traffic: Based on the CDN node labels and traffic characteristics (such as CDN IP address, common request headers, cache hits, etc.), mark the traffic that belongs to CDN acceleration.

[0035] Public cloud traffic: By identifying the labels and traffic characteristics of public cloud nodes (such as public cloud IP address segments, public cloud endpoints, API calls, etc.), the traffic originating from the public cloud is marked.

[0036] Private cloud traffic: Similarly, traffic from the enterprise private cloud environment is marked based on the labels and characteristics of the private cloud nodes (such as private cloud IP address segments, internal endpoints, enterprise-specific APIs, etc.).

[0037] Whois privacy protection traffic: marks the traffic related to domain name addresses using the Whois privacy protection service.

[0038] Malicious traffic: Combine external threat intelligence and traffic characteristics (such as abnormal access patterns, malware signatures, etc.) to mark potential malicious IPs and suspicious traffic.

[0039] Normal business traffic: For the parts that have been clearly identified as legitimate and normal business traffic, keep them in the whitelist status and conduct further analysis as needed.

[0040] Intranet traffic and VPN traffic: Mark legitimate traffic and VPN traffic in the intranet based on the intranet configuration and VPN protocol characteristics.

[0041] By presetting the label propagation algorithm, the initial label can be set based on nodes of known categories (such as CDN nodes, public cloud nodes, and Whois privacy protection nodes), and the connection relationship between nodes (such as traffic or network connections recorded in traffic logs) can be used to iteratively update the label until convergence. This method is more intelligent and accurate than the traditional classification method based on rules or simple statistics, and can better capture the inherent characteristics and associations of traffic data. Accurate traffic classification provides a solid foundation for subsequent data analysis. By marking different categories of traffic, it is easier to identify key traffic, abnormal traffic, and potential security threats in the network. This helps network administrators better understand the network status, formulate targeted network strategies, optimize network resource configuration, and improve network performance and security. In addition to basic traffic classification, this application also combines traffic characteristics and traffic behavior patterns (such as protocol type, port number, request URL, access frequency, request time distribution, etc.) to further mark traffic. This refined marking method enables network administrators to implement more accurate and effective traffic control and management measures, such as bandwidth allocation, access control, security policies, etc., according to the characteristics and needs of different traffic.

[0042] S130, performing cluster analysis on the marked data by using a preset clustering algorithm to identify potential target network entities involved in fraud; like Figure 2As shown, cluster analysis includes selecting parameters of clustering algorithms, processing noise, assigning feature weights, and feature engineering. After the data in the traffic log is effectively labeled and classified by the preset label propagation algorithm, the preset clustering algorithm is used to perform in-depth cluster analysis on the labeled data. The main purpose of this step is to identify potential target network entities that may be involved in fraud, attack, or other malicious behaviors. Cluster analysis is an unsupervised learning technique that can divide labeled traffic data into several groups or clusters so that objects in the same cluster are similar to each other, while objects in different clusters are quite different. In the embodiment of the present application, clustering can help identify network entities with abnormal behavior patterns that may be related to fraudulent activities. Choosing a suitable clustering algorithm is crucial to identifying potential target network entities. Commonly used clustering algorithms include K-means, DBSCAN, hierarchical clustering, etc. Each algorithm has its own characteristics and applicable scenarios. For example, the K-means algorithm is suitable for discovering spherical clusters, but the number of clusters needs to be specified in advance; the DBSCAN algorithm does not need to specify the number of clusters in advance, and can identify clusters of any shape, but it is sensitive to the choice of parameters. The preset clustering algorithm in the embodiments of the present application may include any of the above clustering algorithms.

[0043] Optionally, the performing cluster analysis on the marked data by using a preset clustering algorithm to identify potential target network entities involved in fraud includes: Select target features according to the identification requirements of fraudulent behavior, and preprocess and normalize the target features, wherein the target features include domain name age, SSL certificate information, IP location similarity, domain name Hamming distance, IP service and port information; The neighborhood radius and the minimum number of points of the preset clustering algorithm are set according to the characteristics of the fraudulent behavior, and the marked data are clustered according to the preset clustering algorithm and the target features to generate a clustering result; Based on the clustering results, the characteristics and commonalities of different clusters are analyzed to identify potential target network entities involved in fraud.

[0044] The embodiment of the present application is illustrated with the DBSCAN clustering algorithm. According to the identification requirements of fraudulent behavior, multiple key features are selected, including domain name age, SSL certificate information, IP location similarity, domain name Hamming distance, IP service and port information, etc. These features can reflect the typical characteristics and behavior patterns of fraudulent network assets. For example, fraudulent websites usually have a shorter domain name life; phishing websites and regular websites usually have significant differences in SSL certificates; fraudulent websites often use multiple IPs in the same geographical location; malicious domain names usually have higher information entropy than normal domain names; fraudulent websites usually open non-standard ports. The selected features are preprocessed as necessary, such as digitization of text information (hashing or word embedding), coordinate conversion of geographic location information, etc., to ensure data consistency and comparability. At the same time, the numerical features are normalized to eliminate the influence of different dimensions on the clustering results. According to the characteristics of fraudulent network assets, such as domain name age, registration information, etc., a suitable neighborhood radius (Eps) value is selected to define the neighborhood size of a point. The selection of this value needs to be determined based on experiments and data analysis to ensure that fraudulent websites can be effectively clustered. According to the complexity of the fraudulent network assets, a suitable minimum number of points (MinPts) value is selected to define the minimum number of points required for a dense area. The selection of this value also needs to be based on experiments and data analysis to ensure that there are enough points in the dense area to represent a real fraudulent network entity. By testing and setting reasonable Eps and MinPts parameters, any point with an average distance of more than 0.7 from all points in the dense area will be marked as noise, identifying noise points that are not related to fraudulent network assets. Further, the noise is filtered by using the density-based local outlier factor (LOF) and the threshold filtering based on the domain name reuse rate to process the points identified as noise. The DBSCAN clustering algorithm is used to perform cluster analysis on the marked data according to the preset neighborhood radius and minimum number of points. The DBSCAN algorithm can identify clusters of any shape based on density, which is very suitable for identifying complex fraud patterns. After the clustering analysis is completed, the clustering results are generated, that is, similar network assets are clustered together to form different clusters. Based on the clustering results, the characteristics and commonalities of different clusters are analyzed. By comparing the similarities and differences between different clusters, potential target network entities with fraudulent behavior characteristics can be identified. Based on the analysis results, determine which clusters represent potential target network entities involved in fraud. These entities may have similar characteristics such as domain names, registration information, page content or behavior patterns. Use dimensionality reduction techniques and visualization tools (such as PCA and t-SNE) to visualize the clustering results so that the distribution and structure of fraudulent network assets can be more intuitively understood. At the same time, interpretative analysis of the clustering results is performed through tools such as SHAP (Shapley Additive Explanations) to reveal the common characteristics and differences of fraudulent network assets.Based on cluster analysis and combined with actual needs, fraudulent websites can be tracked quickly and accurately, and case chaining can be automatically and intelligently performed based on clustering results, thereby improving the efficiency and accuracy of combating fraud.

[0045] Examples of scenarios for identifying suspicious traffic: (1) Centralized release of new domain name addresses, which may be characterized by multiple different domain names pointing to the same server at a certain moment, for example, large-scale release of new domain names pointing to the same cname; (2) Geographical location aggregation, which may be characterized by fraudulent activities originating from a specific geographic area or IP address range, for example, IP addresses from a certain country or region actively requesting communications; (3) Similar network assets, which may be characterized by similar domain names, registration information, and page content; (4) Frequent access to specific resources, which may be characterized by high frequency of access to specific resource files or pages; (5) Periodic requests, which may be characterized by repeated requests or behaviors within a fixed time period.

[0046] The data features of centralized release of new domain name addresses include: domain name, target IP address or CNAME, and record addition time. The domain name, target IP / CNAME, and timestamp can be used as features for DBSCAN clustering to identify cases where centralized release of new domain names points to the same server. The data features of geographic location clustering include: source IP address and geographic location information. DBSCAN clustering can be performed through the geographic location information of the IP address to identify fraud activities concentrated in a specific geographic area. The data features of similar network assets include: domain name, registration information, and page content text information digitization (hash or word embedding). These hash values ​​can be used as features for DBSCAN clustering to identify network assets with similar registration information or page content. The data features of frequent access to specific resources include: access frequency and URL path (hash). DBSCAN clustering can be performed through access frequency and URL path hash value to identify behavioral patterns of frequent access to specific resources. The data features of periodic requests include: timestamp and access type. DBSCAN clustering can be performed through timestamp and access type features to identify patterns of periodic requests.

[0047] Different distance metrics are applied in different scenarios. For example, when clustering geographical location related clusters, the distance metric related to IP geographical location is preferred. Cosine similarity is used to quantify the similarity between SSL certificates. Different weights are assigned to different features according to the importance of fraudulent network assets. According to the actual test results, relatively high feature weights are assigned to domain age, IP location, and SLL certificates, and relatively low weights are assigned to other features.

[0048] By selecting features that are highly correlated with the target fraudulent behavior (such as domain age, SSL certificate information, IP location similarity, domain Hamming distance, IP service and port information, etc.), it is possible to ensure that cluster analysis can focus on those features that are most likely to reveal fraudulent behavior patterns, thereby improving the accuracy of identification. Cluster analysis is an automated data processing technology that can quickly process large amounts of data and discover hidden patterns and structures in the data. Compared with manual review, this method can significantly improve processing speed and efficiency, especially in the field of network security that requires rapid response. By adjusting the parameters of the preset clustering algorithm (such as neighborhood radius and minimum number of points), it can flexibly adapt to different types of fraudulent behavior characteristics, making this method widely applicable. At the same time, as new types of fraudulent methods emerge, the system's effectiveness and timeliness can be maintained by updating the feature set and algorithm parameters. Cluster analysis can not only help identify known fraudulent behaviors, but also reveal potential fraud threats by discovering abnormal clusters or unclassified entities in the data. This is of great significance for preventing future fraudulent activities.

[0049] Optionally, performing cluster analysis on the marked data according to the preset clustering algorithm and the target feature to generate a clustering result includes: grouping the marked data according to the neighborhood radius, the minimum number of points, and the target feature; Traversing all data points in the marked data, when the number of data points within the domain radius of the target data point is greater than or equal to the minimum number of points, setting the target data point as a core point; Starting from the core point, adjacent data points are connected into a cluster through density reachability, and a clustering result is generated based on the cluster.

[0050] All data points are initially marked as "unvisited". This is to ensure that each data point is processed only once during the clustering process to avoid duplicate calculations. A random "unvisited" point is selected from the marked data as the starting point. This step is to randomize the beginning of the clustering process and helps with datasets with multiple dense areas. For the selected point, all points in its neighborhood are calculated according to the preset neighborhood radius. This usually involves calculating the distance between the selected point and all other points and finding the set of points whose distance is less than or equal to the neighborhood radius. If the number of data points in the neighborhood of the selected point is greater than or equal to the preset minimum number of points (MinPts), the point is marked as a "core point" and is considered sufficient to form the starting point of a new cluster. If the number of data points in the neighborhood is less than MinPts, the point is marked as a "noise point". These points may be located in sparse areas of the dataset or have too low density compared to other clusters to form an independent cluster. However, it is worth noting that in some implementations, these noise points may be re-evaluated in subsequent iterations and may be added to other clusters, although they usually remain as noise in the original DBSCAN algorithm. Starting from each core point, adjacent data points are connected into a cluster through density reachability (i.e., starting from a core point, reaching another core point through a series of density-connected core points). This process involves recursively checking the points in the neighborhood of each core point and adding them to the corresponding cluster until there are no more points to add. Continue to traverse the data points until all points have been visited and classified into the corresponding cluster (as core points, noise points, or part of a cluster). According to the above process, one or more clusters and a possible set of noise points will eventually be generated. These clusters and noise points constitute the final result of the cluster analysis.

[0051] By using the neighborhood radius and the minimum number of points as clustering criteria, the process can identify clusters of any shape and size in the data. In the clustering process, those points whose number of data points in the neighborhood is less than the minimum number of points are marked as noise or outliers. This processing method helps to reduce the impact of noise on the clustering results, making the clustering results more accurate and reliable. At the same time, since the noise points are clearly distinguished, users can further analyze or process them as needed. By connecting adjacent data points into a cluster through density accessibility, the embodiment of the present application can ensure that the data points in the cluster are connected in density. This continuity not only helps to form a compact and meaningful cluster, but also helps to understand the inherent relationship between data. Since the embodiment of the present application performs clustering based on local density, it has certain robustness for local changes or noise in the data. At the same time, as the size of the data set increases, the embodiment of the present application can still maintain good clustering performance and has certain scalability. In addition to generating clusters, the embodiment of the present application can also provide information about noise points. This information is of great significance for tasks such as further data analysis, anomaly detection or data cleaning.

[0052] S140: Performing aggregate analysis on the metadata through situation awareness technology to identify and track target network entities from the potential target network entities, and perform risk assessment.

[0053] like Figure 2As shown in the figure, situational awareness and tracking include metadata aggregation and intelligent analysis at the observation level, semantic analysis and association reasoning at the understanding level, trend prediction and risk assessment at the prediction level, and multi-dimensional interactive situational awareness dashboard. The main purpose of situational awareness is to observe elements in the environment within a certain time and space, understand the meaning of these elements, and predict the status of these elements in the near future. Situational awareness consists of three levels: observation, understanding, and prediction, and its output will be directly fed into the cycle of decision-making and action. Observation level: This level involves the operator's sensory detection of significant information in the system he is operating and the environment in which the system operates. For example, the operator needs to be able to see the relevant display or hear the alarm signal. Basic situational awareness includes the perception of the status of elements such as various fraud types, fraud landing websites, fraud system nodes, fraud network asset clusters, related historical records, and related historical IP addresses. Through web crawlers and API interfaces, collect websites, domain names, IP addresses, traffic, and related data involved in fraud. Understanding level: Situational awareness does not stop at observing the data displayed on the computer screen, but needs to be combined with the operator's goals to understand the meaning or significance of this information. In this process, the information understood should be gradually integrated to gradually develop a comprehensive picture of the system, so as to form a more comprehensive overall understanding of the events that are happening. In terms of situational understanding, it mainly revolves around the landing scenarios of fraudulent behaviors, the detection characteristics of fraudulent behaviors, which fraudulent behaviors may be related to each other, and the correct prioritization of competing events. Machine learning algorithms, including decision trees and support vector machines (SVMs), are used to classify and label the collected data. Prediction layer: This layer contains forward time inferences of information to determine how it will affect the future state of the operating environment. This is combined with an understanding of the current situation and a mental model formed for the system, so that it can be used to predict what may happen next. Specifically based on time series analysis and recurrent neural networks (RNNs), new fraud network landing methods in the network are predicted, the change from normal assets to fraudulent network assets is predicted, and the clustering changes of the same assets over time are recorded.

[0054] Use the data analysis tools and methods in situational awareness technology to aggregate and analyze the pre-processed metadata. This includes but is not limited to association analysis (identifying the association between different data sources), cluster analysis (grouping similar data points), time series analysis (analyzing the trend of data changes over time), etc. Through these analysis methods, valuable information and patterns can be extracted from massive data. Based on the results of aggregate analysis, extract key features related to the target network entity. These features may include network behavior patterns, abnormal traffic features, user behavior features, etc. Match the extracted features with the known target network entity feature library to identify potential target network entities. If the match is successful, further track the activity trajectory and behavior pattern of the entity. Continuously monitor the identified target network entity and collect its latest activity data and metadata so as to adjust the analysis strategy and risk assessment results in a timely manner. According to the behavior pattern and characteristics of the target network entity, evaluate the type and severity of the threat it may pose. This includes evaluating its attack intention, attack means, attack scope, etc. Analyze the system or network vulnerabilities that the target network entity may exploit and evaluate the possibility of its successful attack. Generate a risk assessment report based on the results of threat assessment and vulnerability assessment. The report should include a detailed description of the target network entity, threat type, vulnerability points, risk assessment level, and recommended countermeasures.

[0055] Optionally, the performing aggregate analysis on the metadata by using situation awareness technology to identify and track the target network entity from the potential target network entities includes: Adopt data aggregation technology to aggregate pre-processed metadata to reveal potential fraud behavior patterns and trends; Use natural language processing and semantic analysis technology to understand the aggregated metadata to reveal the logical relationship between the data and determine the connection between different fraud behaviors and network assets through association reasoning algorithms; Time series analysis and deep learning models are used to predict fraud trends and network asset risks, and to draw fraud trend forecast charts.

[0056] Through data aggregation technology, the pre-processed metadata (such as log files, network traffic data, user behavior records, etc.) are summarized and integrated to extract valuable information from large amounts of scattered data. Specifically, the original data is cleaned, deduplicated, formatted, etc. to ensure data quality and consistency. Data aggregation algorithms (such as hash tables, database aggregation queries, etc.) are used to merge similar or related data items to form a data set that is easier to analyze. Aggregated data can reveal potential fraud behavior patterns (such as frequent login failures, abnormal capital flows, etc.) and trends, providing a basis for subsequent analysis. Through natural language processing and semantic analysis technology, the content of aggregated metadata is deeply understood, the logical relationship between data is revealed, and potential clues to fraud behavior are discovered. Specifically, text data (such as user comments, chat records, etc.) is converted into a computer-understandable format, keywords, phrases, etc. are extracted, the meaning and contextual relationship of text data are analyzed, potential semantic patterns or anomalies are identified, and association reasoning algorithms (such as graph theory algorithms, clustering algorithms, etc.) are used to perform association analysis on data from different sources (such as user behavior, network traffic, system logs, etc.) to determine the intrinsic connection between different fraud behaviors and network assets. Through time series analysis and deep learning models, we predict the trend of fraudulent behavior and network asset risks, providing a basis for formulating effective defense strategies. Specifically, we analyze the time change patterns in historical data, identify the periodicity and seasonality of fraudulent behavior, and use deep learning algorithms (such as recurrent neural networks (RNNs) and long short-term memory networks (LSTMs)) to train large amounts of data and learn the complex patterns of fraudulent behavior. Based on the trained models, we predict the future trend of fraudulent behavior and draw a fraud trend prediction chart to intuitively display the risk changes.

[0057] Through data aggregation technology, large amounts of scattered metadata can be integrated into more meaningful and highly correlated data sets. This integration helps reveal fraudulent behavior patterns and trends hidden behind complex data, thereby significantly improving the accuracy of fraudulent behavior identification. Combined with natural language processing and semantic analysis technology, the system can deeply understand the content of aggregated metadata and explore the logical relationship between data. This not only helps to identify direct fraud clues, but also discovers indirect but important connections between different fraudulent behaviors and network assets through association reasoning algorithms, and builds a more comprehensive fraud network map. Using time series analysis and deep learning models, future trends of fraudulent behavior can be predicted. These models can learn patterns in historical data and infer future development trends based on them. At the same time, drawing fraud trend prediction maps can provide decision makers with intuitive and visual references, helping them to formulate response strategies in a timely manner and reduce potential risks. By continuously monitoring and analyzing metadata in the network environment, the system can promptly discover and track target network entities, including potential fraudsters. This real-time monitoring and tracking capability can significantly improve the security and defense capabilities of the network and protect network assets from harm.

[0058] Optionally, the data aggregation technology is used to aggregate the pre-processed metadata to reveal potential fraud behavior patterns and trends, including: The type of fraud behavior is used as the aggregation keyword, and the distribution of each type in the overall fraud behavior is counted to identify the type of fraud behavior with the highest incidence rate; The geographical location information of the fraudulent behavior is used as the aggregation keyword, and the number of fraudulent behaviors occurring in each geographical location is counted to draw a heat map or distribution map of the fraudulent behavior; The time information of fraudulent behaviors is used as the aggregation keyword, and the timeline diagram of fraudulent behaviors is drawn by counting the number of fraudulent behaviors occurring in each time period; A multi-dimensional cross-aggregation view is formed by cross-combining the analysis results of a single dimension, wherein the single dimension includes type, location, and time.

[0059] Use the type of fraud information as the aggregation keyword. This includes various fraud methods, such as phishing, telephone fraud, identity theft, etc. By counting the distribution of each type in the overall fraud, the fraud type with the highest incidence can be identified. This analysis helps to identify the hot spots and key prevention targets of current fraud activities. Display the statistical results in the form of charts, such as fraud type distribution maps, to intuitively reflect the proportion and distribution of various types of fraud. Use the geographical location information of the fraud as the aggregation keyword. This includes the IP address of the fraud initiator, the geographical location of the domain name resolution, etc. By counting the number of frauds occurring in each geographical location, a heat map or distribution map of fraud can be drawn. This map display method can intuitively reveal the regional characteristics and concentrated areas of fraud activities. Based on the aggregation results of geographical locations, relevant departments can be guided to strengthen prevention and crackdown efforts in specific areas in a targeted manner. Use the time information of the fraud as the aggregation keyword. This includes the timestamp and duration of the fraud. By counting the number of frauds occurring in each time period, a timeline diagram of fraud can be drawn. This timeline display method can reveal the cyclical, seasonal or sudden characteristics of fraud activities. Based on the aggregation results of time series, the prediction model can be used to predict the trend of future fraud activities, so as to take preventive measures in advance. The above single-dimensional analysis results (type, location, time) are cross-combined to form a multi-dimensional cross-aggregation view. This cross-aggregation can reveal the inherent connection and mutual influence between different dimensions. By comprehensively analyzing the results of multi-dimensional cross-aggregation, the complexity and diversity of fraudulent behavior can be more comprehensively understood. For example, the distribution of specific types of fraud in different geographical locations and time periods can be analyzed, or the evolution trend of different types of fraud in the same geographical location can be analyzed. The results of multi-dimensional cross-aggregation provide strong data support for the formulation of more accurate and effective anti-fraud strategies.

[0060] By using the type information of fraudulent behavior as the aggregation keyword, the system can accurately count the distribution of various types of fraudulent behavior in the population, thereby quickly identifying the fraud type with the highest incidence rate. This precise identification helps the security team prioritize and deal with high-risk fraud activities. Using geographic location information as the aggregation keyword, combined with the display method of heat map or distribution map, the geographical distribution characteristics of fraudulent behavior can be intuitively reflected. This map visualization technology enables security personnel to quickly locate the hot spots of fraudulent activities, providing strong support for subsequent field investigations and crackdowns. Using time information as the aggregation keyword, the system can count the number of fraudulent behaviors in each time period and draw a timeline chart. This time series analysis helps to reveal the cyclical, seasonal or sudden characteristics of fraudulent activities, and provides a time reference for the formulation of targeted preventive measures. By cross-combining the analysis results of a single dimension (type, location, time) to form a multi-dimensional cross-aggregation view, the system can reveal the inherent connection and mutual influence between different dimensions. This comprehensive analysis helps the security team to more fully understand the complexity and diversity of fraudulent behavior, so as to formulate a more comprehensive and effective response strategy.

[0061] Optionally, the use of natural language processing and semantic analysis technology to understand the aggregated metadata to reveal the logical relationship between the data, and to determine the connection between different fraudulent behaviors and network assets through an association reasoning algorithm includes: Extracting entities from the aggregated metadata using named entity recognition technology, including fraud method names, victim types, IP addresses, and domain names; Using relationship extraction technology, the relationships between the entities are identified and extracted. Combined with context information, the extracted entities and the relationships are semantically understood to reveal the logical relationship behind them. Different data points are associated according to preset rules, and the associated data points are represented in the form of a network graph, where nodes represent entities and edges represent the relationships between entities; Through the association reasoning algorithm, path search and influence analysis are performed in the network graph to reveal the connection between different fraud behaviors and network assets.

[0062] Key entities are extracted from the aggregated metadata using named entity recognition (NER) technology. These entities are the basis for understanding the relationship between fraudulent behavior and network assets, including but not limited to the name of fraudulent means, victim type, IP address and domain name. Named entity recognition helps the system quickly locate and identify key information in the text, providing basic data for subsequent relationship extraction and semantic understanding. Relation extraction (RelationExtraction) technology is used to identify and extract the relationship between the above entities. This process not only involves simple entity-to-relationship mapping, but also requires in-depth semantic understanding of the extracted entities and relationships in combination with contextual information. Relation extraction and semantic understanding can reveal complex logical relationships between entities, such as the association between an IP address and multiple fraudulent domain names, or the correspondence between a certain fraudulent means and a specific victim type. According to preset rules or algorithms, different data points are associated and these associated data points are represented in the form of a network graph. In the network graph, nodes represent entities (such as IP addresses, domain names, fraudulent means, etc.), and edges represent relationships between entities (such as association, similarity, influence, etc.). The network diagram representation can intuitively display the complex relationship between data, allowing operators to see at a glance the inherent connection between different fraudulent behaviors and network assets. Path search and influence analysis are performed in the network diagram through association reasoning algorithms. These algorithms can identify key nodes and paths in the network, revealing the potential connection and mutual influence between different fraudulent behaviors and network assets. The association reasoning algorithm can not only help the system discover new fraud patterns and trends, but also provide a scientific basis for the formulation of targeted preventive measures. By updating the network diagram and association reasoning results in real time, the system can maintain a keen insight into fraudulent activities and the ability to respond quickly.

[0063] The following are the implementation details of the specific levels of situational awareness: (1) Observation-level metadata aggregation and intelligent analysis: This level uses advanced data aggregation technology to deeply aggregate the collected metadata to reveal potential fraud patterns and trends. Using machine learning and big data analysis, it analyzes the dynamic changes of various fraud network assets and their correlation with environmental factors in real time, providing richer and more accurate data support for subsequent understanding and prediction levels.

[0064] Fraud type distribution map: Displays the distribution of different types of fraud, such as the fraudster's geographical location, the domain name used, etc. Specifically, collect information such as the geographical location, domain name, and type of fraud involved in the fraud incident, aggregate the data by type and geographical location, and use geographic maps (such as heat maps) to display the distribution of different types of fraud.

[0065] Fraudulent network asset clustering diagram: Use a map or network diagram to display the clustering of fraudulent network assets, such as the relationship between IP addresses and domain names. Specifically, collect IP addresses, domain names, fraudulent pages, fraudulent applications, fraudulent types and their associations, and use a network diagram (such as NetwrokX) to display the association between IP addresses, domain names, fraudulent pages, applications and fraudulent types.

[0066] Historical record timeline: Displays the historical changes of fraudulent network assets, such as domain name registration and usage. Specifically, collects information such as domain name registration, changes, usage and timestamps, arranges the data in chronological order, and uses a timeline chart to display the historical records.

[0067] Alarm and notification panel: Real-time display of alarms and notifications of fraudulent network assets, such as newly discovered fraudulent domain names, abnormal traffic, etc. Specifically, collect real-time alarm and notification data, such as abnormal traffic, suspicious domain names, IP addresses, fraud types, etc., filter, classify and prioritize alarm data (such as by the severity of the fraud type), and use alarm panels (such as Dash) to display real-time alarms.

[0068] (2) Understanding-level semantic analysis and association reasoning: This level uses natural language processing and semantic analysis technology to deeply understand the aggregated metadata and reveal the potential meaning and logical relationship behind the data. Through association reasoning algorithms, it explores the inherent connection between different fraud behaviors and network assets, providing a scientific basis for identifying and preventing new types of fraud.

[0069] Fraud behavior analysis chart: Use charts and graphs to show the characteristics and patterns of fraud behavior, such as common fraud methods, target victims, etc. Specifically, collect data on fraud behavior, such as methods, target victims, number of times, fraud amount, etc., aggregate data based on fraud methods and target victim types, and use bar charts, pie charts and other charts to show the distribution of common fraud methods and target victims.

[0070] Correlation analysis diagram: Shows the correlation and similarity between different fraudulent behaviors, such as fraudulent domain names using the same IP address or registration information. Specifically, collect data containing related information, such as IP addresses, registration information, domain names, etc., associate them based on the same IP address or registration information, and use a network diagram to show the correlation.

[0071] Event Priority Chart: Display the priority of fraud events through lists or charts to help operators determine the order and strategy of response. Specifically, collect data such as fraud amount and fraud type containing priority information, sort them according to priority, and use lists or bar charts to display the priority ranking.

[0072] (3) Forecasting level trend forecasting and risk assessment: This level uses time series analysis and deep learning models to accurately predict future fraud trends and network asset risks. Combined with a multi-dimensional risk assessment model, it assesses the risk level of various types of network assets in real time, providing forward-looking decision-making support for fraud prevention.

[0073] Fraud trend forecast chart: Use time series analysis to show the future trend of fraudulent network assets, such as predicting fraudulent domain names that may appear in the future. Specifically, collect time series data such as the registration time and access frequency of fraudulent domain names, process the time series data, including missing value filling and smoothing, use time series models such as ARIMA and Prophet to predict trends, draw time series trend charts, and display historical data and forecast data. The periodicity, seasonality, and interval distribution of the data can be used for secondary in-depth analysis Risk Assessment Chart: By analyzing the characteristics and historical records of fraudulent network assets, the risk levels of different assets are displayed, such as risk scores based on historical fraudulent behaviors. Specifically, the characteristics of each network asset (such as registration information, associated IP, access frequency, etc.) and historical fraud records are collected, and the risk score of each asset is calculated according to the predefined scoring rules or models. The risk score chart is drawn to display the risk levels of different network assets.

[0074] Change association graph: record and visualize the clustering changes of the same asset over time. If they are consistent, further associate the two. Specifically, collect the clustering results of the same asset at different time points, record the clusters to which the assets belong at different time points, and use the change graph to show the clustering changes of the same asset. If the changes are consistent, further display the detailed association information of the two.

[0075] (4) Multi-dimensional interactive situational awareness dashboard: This dashboard integrates the analysis results of the three levels of observation, understanding, and prediction, and provides operators with all-round, multi-dimensional situational awareness through an interactive visual interface. Users can customize views and layouts according to their needs, adjust analysis parameters in real time, and deeply explore various types of information on fraudulent network assets to achieve more flexible and accurate situational awareness.

[0076] Integrate visualization at the three levels of observation, understanding, and prediction: Through a comprehensive dashboard, the visualization content at the three levels of observation, understanding, and prediction is integrated to provide comprehensive situational awareness. Operators are allowed to customize views as needed, such as selecting displayed charts, adjusting layouts, etc. Interactive functions are provided, such as clicking on a fraudulent domain name to view detailed information, adjusting the time range to view historical records, etc.

[0077] In order to achieve a comprehensive dashboard that integrates observation, understanding, and prediction, and provides customized views and interactive functions, modern visualization frameworks such as Plotly Dash, Streamlit, or Bokeh can be used. Users can customize the view as needed by selecting different chart types from the drop-down menu. Users can click on a category in the fraud behavior analysis chart to view detailed information, and the specific data clicked will be displayed in the window.

[0078] In addition to using the DBSCAN clustering algorithm in the embodiments of the present application, other clustering algorithms can also be used to achieve automatic tracking and clustering of fraudulent network assets. For example, the K-Means clustering algorithm: by selecting a suitable K value, the fraudulent network assets can be divided into K clusters, and then the best cluster center can be found through iterative optimization; hierarchical clustering: by gradually merging or splitting clusters, a hierarchical clustering tree can be formed, so as to better understand the relationship between fraudulent network assets.

[0079] The embodiments of the present application can use deep learning methods, such as autoencoders or deep clustering networks, to learn complex patterns of fraudulent network assets and achieve more accurate clustering. Autoencoders: Learn low-dimensional representations of fraudulent network assets by training autoencoders, and then use these representations for clustering. Deep clustering networks: Directly learn the clustering assignments of fraudulent network assets through end-to-end training.

[0080] The embodiments of the present application can use graph analysis methods to represent and analyze the relationships between fraudulent network assets. For example, social network analysis: represent fraudulent network assets as nodes in a graph, represent the relationships between them as edges, and then use social network analysis methods to identify important nodes and communities; graph embedding: use graph embedding methods to learn low-dimensional representations of fraudulent network assets, and then use these representations for clustering.

[0081] The embodiments of the present application can achieve more powerful and flexible fraud network asset tracking and clustering by combining multiple methods. For example, ensemble learning: by combining the results of multiple clustering algorithms, more robust and accurate clustering can be achieved; multi-view clustering: by integrating information from different views or sources, more comprehensive and accurate clustering can be achieved.

[0082] This embodiment also discloses a system for clustering fraud-related entities based on network traffic and metadata. Figure 3 is a module diagram of a system for clustering fraud-related entities based on network traffic and metadata disclosed in an embodiment of the present application, such as Figure 3 As shown, the system includes a collection module 301, a marking module 302, a clustering module 303 and an execution module 304, wherein: A collection module 301 is configured to obtain traffic logs and metadata from multiple data sources, including CDN, public cloud, private cloud, and internal network of the target area; A marking module 302 is configured to mark the traffic in the traffic log by a preset label propagation algorithm to distinguish CDN traffic, public cloud traffic, private cloud traffic, Whois privacy protection traffic, malicious traffic and business traffic; A clustering module 303 is configured to perform cluster analysis on the marked data by using a preset clustering algorithm to identify potential target network entities involved in fraud; The execution module 304 is configured to perform aggregate analysis on the metadata through situation awareness technology to identify and track target network entities from the potential target network entities and perform risk assessment.

[0083] Optionally, the marking module 302 is configured to: Setting initial labels for nodes of known categories based on the configuration information, wherein the nodes of known categories include CDN nodes, public cloud nodes, and Whois privacy protection nodes; Determine a final label of each node according to a connection relationship between the nodes, wherein the connection relationship includes traffic or network connections recorded in a traffic log; Traffic is marked according to node labels, traffic characteristics and traffic behavior patterns of the nodes it passes through, wherein the traffic characteristics include protocol type, port number and request URL, and the traffic behavior pattern includes access frequency and request time distribution.

[0084] Optionally, the clustering module 303 is configured to: Select target features according to the identification requirements of fraudulent behavior, and preprocess and normalize the target features, wherein the target features include domain name age, SSL certificate information, IP location similarity, domain name Hamming distance, IP service and port information; The neighborhood radius and the minimum number of points of the preset clustering algorithm are set according to the characteristics of the fraudulent behavior, and the marked data are clustered according to the preset clustering algorithm and the target features to generate a clustering result; Based on the clustering results, the characteristics and commonalities of different clusters are analyzed to identify potential target network entities involved in fraud.

[0085] Optionally, the clustering module 303 is configured to: grouping the marked data according to the neighborhood radius, the minimum number of points, and the target feature; Traversing all data points in the marked data, when the number of data points within the domain radius of the target data point is greater than or equal to the minimum number of points, setting the target data point as a core point; Starting from the core point, adjacent data points are connected into a cluster through density reachability, and a clustering result is generated based on the cluster.

[0086] Optionally, the execution module 304 is configured to: Adopt data aggregation technology to aggregate pre-processed metadata to reveal potential fraud behavior patterns and trends; Use natural language processing and semantic analysis technology to understand the aggregated metadata to reveal the logical relationship between the data and determine the connection between different fraud behaviors and network assets through association reasoning algorithms; Time series analysis and deep learning models are used to predict fraud trends and network asset risks, and to draw fraud trend forecast charts.

[0087] Optionally, the execution module 304 is configured to: The type of fraud behavior is used as the aggregation keyword, and the distribution of each type in the overall fraud behavior is counted to identify the type of fraud behavior with the highest incidence rate; The geographical location information of the fraudulent behavior is used as the aggregation keyword, and the number of fraudulent behaviors occurring in each geographical location is counted to draw a heat map or distribution map of the fraudulent behavior; The time information of fraudulent behaviors is used as the aggregation keyword, and the timeline diagram of fraudulent behaviors is drawn by counting the number of fraudulent behaviors occurring in each time period; A multi-dimensional cross-aggregation view is formed by cross-combining the analysis results of a single dimension, wherein the single dimension includes type, location, and time.

[0088] Optionally, the execution module 304 is configured to: Extracting entities from the aggregated metadata using named entity recognition technology, including fraud method names, victim types, IP addresses, and domain names; Using relationship extraction technology, the relationships between the entities are identified and extracted. Combined with context information, the extracted entities and the relationships are semantically understood to reveal the logical relationship behind them. Different data points are associated according to preset rules, and the associated data points are represented in the form of a network graph, where nodes represent entities and edges represent the relationships between entities; Through the association reasoning algorithm, path search and influence analysis are performed in the network graph to reveal the connection between different fraud behaviors and network assets.

[0089] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0090] This embodiment also discloses an electronic device, referring to Figure 4 The electronic device may include: at least one processor 401 , at least one communication bus 402 , a user interface 403 , a network interface 404 , and at least one memory 405 .

[0091] The communication bus 402 is used to realize the connection and communication between these components.

[0092] The user interface 403 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 403 may also include a standard wired interface and a wireless interface.

[0093] The network interface 404 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0094] Among them, the processor 401 may include one or more processing cores. The processor 401 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 405, and calling data stored in the memory 405. Optionally, the processor 401 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 401 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 401, and it can be implemented separately through a chip.

[0095] Among them, the memory 405 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 405 includes a non-transitory computer-readable storage medium. The memory 405 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 405 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 405 may also be optionally at least one storage device located away from the aforementioned processor 401. As Figure 4 As shown, the memory 405 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program of a method for clustering fraud-related entities based on network traffic and metadata.

[0096] exist Figure 4 In the electronic device shown, the user interface 403 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 401 can be used to call the application program of the method of clustering fraud-related entities based on network traffic and metadata stored in the memory 405. When executed by one or more processors 401, the electronic device executes one or more methods in the above-mentioned embodiments.

[0097] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0098] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0100] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0101] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0102] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory 405, including a number of instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory 405 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk.

[0103] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for clustering fraud-related entities based on network traffic and metadata, characterized in that: Applied to a fraud-related network entity clustering and tracking platform, the method includes: Obtaining traffic logs and metadata from multiple data sources, including CDN, public cloud, private cloud, and internal network of the target region; The traffic in the traffic log is marked by a preset label propagation algorithm to distinguish CDN traffic, public cloud traffic, private cloud traffic, Whois privacy protection traffic, malicious traffic and business traffic; Perform cluster analysis on the marked data through a preset clustering algorithm to identify potential target network entities involved in fraud; The metadata is aggregated and analyzed through situation awareness technology to identify and track target network entities from the potential target network entities and perform risk assessment.

2. The method for clustering fraud-related entities based on network traffic and metadata according to claim 1, characterized in that: The marking of the traffic in the traffic log by using a preset label propagation algorithm includes: Setting initial labels for nodes of known categories based on the configuration information, wherein the nodes of known categories include CDN nodes, public cloud nodes, and Whois privacy protection nodes; Determine a final label of each node according to a connection relationship between the nodes, wherein the connection relationship includes traffic or network connections recorded in a traffic log; Traffic is marked according to node labels, traffic characteristics and traffic behavior patterns of the nodes it passes through, wherein the traffic characteristics include protocol type, port number and request URL, and the traffic behavior pattern includes access frequency and request time distribution.

3. The method for clustering fraud-related entities based on network traffic and metadata according to claim 1, characterized in that: The clustering analysis of the marked data by using a preset clustering algorithm to identify potential target network entities involved in fraud includes: Select target features according to the identification requirements of fraudulent behavior, and preprocess and normalize the target features, wherein the target features include domain name age, SSL certificate information, IP location similarity, domain name Hamming distance, IP service and port information; The neighborhood radius and the minimum number of points of the preset clustering algorithm are set according to the characteristics of the fraudulent behavior, and the marked data are clustered according to the preset clustering algorithm and the target features to generate a clustering result; Based on the clustering results, the characteristics and commonalities of different clusters are analyzed to identify potential target network entities involved in fraud.

4. The method for clustering fraud-related entities based on network traffic and metadata according to claim 3, characterized in that: The performing cluster analysis on the marked data according to the preset clustering algorithm and the target feature to generate a clustering result includes: grouping the marked data according to the neighborhood radius, the minimum number of points, and the target feature; Traversing all data points in the marked data, when the number of data points within the domain radius of the target data point is greater than or equal to the minimum number of points, setting the target data point as a core point; Starting from the core point, adjacent data points are connected into a cluster through density reachability, and a clustering result is generated based on the cluster.

5. The method for clustering fraud-related entities based on network traffic and metadata according to claim 1, characterized in that: The step of performing aggregate analysis on the metadata by using situation awareness technology to identify and track the target network entity from the potential target network entities includes: Adopt data aggregation technology to aggregate pre-processed metadata to reveal potential fraud behavior patterns and trends; Use natural language processing and semantic analysis technology to understand the aggregated metadata to reveal the logical relationship between the data and determine the connection between different fraud behaviors and network assets through association reasoning algorithms; Time series analysis and deep learning models are used to predict fraud trends and network asset risks, and to draw fraud trend forecast charts.

6. The method for clustering fraud-related entities based on network traffic and metadata according to claim 5, characterized in that: The data aggregation technology is used to aggregate the pre-processed metadata to reveal potential fraud behavior patterns and trends, including: The type of fraud behavior is used as the aggregation keyword, and the distribution of each type in the overall fraud behavior is counted to identify the type of fraud behavior with the highest incidence rate; The geographical location information of the fraudulent behavior is used as the aggregation keyword, and the number of fraudulent behaviors occurring in each geographical location is counted to draw a heat map or distribution map of the fraudulent behavior; The time information of fraudulent behaviors is used as the aggregation keyword, and the timeline diagram of fraudulent behaviors is drawn by counting the number of fraudulent behaviors occurring in each time period; A multi-dimensional cross-aggregation view is formed by cross-combining the analysis results of a single dimension, wherein the single dimension includes type, location, and time.

7. The method for clustering fraud-related entities based on network traffic and metadata according to claim 5, characterized in that: The natural language processing and semantic analysis technology is used to understand the aggregated metadata to reveal the logical relationship between the data. Through the association reasoning algorithm, the connection between different fraudulent behaviors and network assets is determined, including: Extracting entities from the aggregated metadata using named entity recognition technology, including fraud method names, victim types, IP addresses, and domain names; Using relationship extraction technology, the relationships between the entities are identified and extracted. Combined with context information, the extracted entities and the relationships are semantically understood to reveal the logical relationship behind them. Different data points are associated according to preset rules, and the associated data points are represented in the form of a network graph, where nodes represent entities and edges represent the relationships between entities; Through the association reasoning algorithm, path search and influence analysis are performed in the network graph to reveal the connection between different fraud behaviors and network assets.

8. A system for clustering fraud-related entities based on network traffic and metadata, characterized in that: It includes a collection module, a marking module, a clustering module and an execution module, among which: A collection module configured to obtain traffic logs and metadata from multiple data sources, including CDN, public cloud, private cloud, and internal network of the target area; A marking module configured to mark the traffic in the traffic log by a preset label propagation algorithm to distinguish CDN traffic, public cloud traffic, private cloud traffic, Whois privacy protection traffic, malicious traffic and business traffic; A clustering module configured to perform cluster analysis on the marked data by using a preset clustering algorithm to identify potential target network entities involved in fraud; The execution module is configured to perform aggregate analysis on the metadata through situation awareness technology to identify and track target network entities from the potential target network entities and perform risk assessment.

9. An electronic device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Citation Information

Patent Citations

  • Fraud user identification method and device, computer equipment and storage medium

    CN109784636A

  • Method and system for mining and checking fraud gang relationship in Internet

    CN110413707A

  • Fraudulent behavior tracking method based on security intelligence and device thereof and related equipment

    CN113988886A

  • Medical insurance cash-out fraud behavior identification method and system

    CN115330546A

  • Malicious domain name generation method and device, equipment and medium

    CN118138382A