A clustering analysis method based on network behavior data

Abstract

The application discloses a clustering analysis method based on network behavior data and relates to the field of data analysis, which comprises the following steps: S1, obtaining a data set to be processed, wherein the data set comprises at least one behavior clue; S2, matching each behavior clue with an existing clue set, adding the behavior clues matched successfully into the clue set, adding the behavior clues not matched successfully into a cache, and entering S3; S3, analyzing the behavior clues not matched successfully, and clustering to form a plurality of central regions; S4, calculating the similarity of each two central regions, merging the two central regions with the similarity exceeding a threshold to form a new central region; S5, judging whether each central region meets the condition of becoming an organization, if yes, removing the central region from the cache to establish a new clue set, and performing persistent storage on the central region; otherwise, continuously keeping the central region in the cache and performing persistent storage on the cache; and the network behavior is subjected to clustering analysis, so that the behavior of the Internet can be traced back quickly and relatively in real time.

Description

Technical Field

[0001] This invention relates to the field of data analysis, and in particular to a clustering analysis method based on network behavior data. Background Technology

[0002] Currently, internet security threats are increasing. While targeted cyberattacks can be detected by reconstructing internet traffic data, current detection methods only address the act of attack; they lack effective means to detect the source of the attack. At present, tracing the origin of cyberattacks mostly relies on manual methods, which are often inefficient and time-consuming. Summary of the Invention

[0003] The purpose of this invention is to design a clustering analysis method based on network behavior data in order to solve the above problems.

[0004] The present invention achieves the above objectives through the following technical solutions:

[0005] A clustering analysis method based on network behavior data includes:

[0006] S1. Obtain the dataset to be processed, which includes at least one behavioral clue;

[0007] S2. Each behavioral clue is matched with the existing clue set of the organization. The behavioral clues that match successfully are added to the clue set, and the behavioral clues that do not match successfully are added to the cache and then proceed to S3.

[0008] S3. Analyze the unmatched behavioral cues and cluster them into multiple central regions, each of which contains at least two behavioral cues.

[0009] S4. Calculate the similarity between any two central regions, and merge two central regions whose similarity exceeds the threshold to form a new central region.

[0010] S5. Determine whether each central region meets the conditions to become an organization. If it does, remove the central region from the cache, establish a new thread set, and persist it. Otherwise, keep it in the cache and persist it.

[0011] The beneficial effects of this invention are: by using the method of this invention to aggregate and analyze network behavior, it is possible to quickly and relatively in real time trace the source of Internet behavior. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating a clustering analysis method based on network behavior data according to the present invention.

[0013] Figure 2This is a schematic diagram of the clustering process in a clustering analysis method based on network behavior data according to the present invention. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0015] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0016] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0017] In the description of this invention, the term "clue" refers to a data carrier of a cyberattack that has been identified, such as a phishing email. The term "element" refers to key features extracted from a clue, such as the sending IP address. The term "organization" refers to an entity or natural person that launched a cyberattack, which in this invention is represented as a collection of clues. "Central region" refers to an organization temporarily generated during the processing of the method described in this invention that has not yet met the threshold condition. "Matching degree" refers to an indicator of the credibility of a clue belonging to an organization. "Similarity" refers to an indicator of the degree of homology between two organizations; the higher the similarity, the more likely the two organizations are to be homologous. "Hard match" refers to a match where the key features extracted from a clue are the same as the key features extracted from an organization. "Soft match" refers to a match where the key features extracted from a clue are not the same as the key features in an organization, but have a certain degree of homology, and the homology reaches a certain threshold. "Persistence" refers to the process of storing the analysis results or temporary data of this invention on disk so that this data can still be accessed in the event of a power outage or system restart.

[0018] In the description of this invention, it should be understood that the terms "upper," "lower," "inner," "outer," "left," "right," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used to facilitate the description of this invention and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0019] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, terms such as "configuration," "data," "cachance," and "persistent storage" should be interpreted broadly. For example, "persistent storage" can be a relational database or other non-relational data storage components; it can be row-based storage or column-based storage; it can be record-based storage or file-based storage. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0020] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0021] like Figure 1 , Figure 2 As shown, a clustering analysis method based on network behavior data includes:

[0022] S1. Obtain the dataset to be processed. The dataset includes at least one behavioral clue.

[0023] S2. Each behavioral clue is matched against the existing clue set of the organization. Successfully matched behavioral clues are added to the clue set, while unsuccessfully matched behavioral clues are added to the cache and then proceed to S3; specifically including:

[0024] S21. Extract the key features of each behavioral cue;

[0025] S22. Perform a hard match between the key features of each behavioral clue and the key features extracted from the existing clue set. Add the successfully matched behavioral clues to the clue set, and add the unmatched behavioral clues to the cache, and proceed to S3.

[0026] S3. Analyze unmatched behavioral cues and cluster them into multiple central regions, each containing at least two behavioral cues; specifically including:

[0027] S31. Unmatched behavioral cues are clustered according to pre-set rules and thresholds to obtain multiple central regions; specifically:

[0028] S311. Determine the mapping between each key feature and the corresponding list of clues;

[0029] S312. Traverse the mapping obtained in S311 and take the intersection of each pair of clues in the mapping value list. The intersection represents the set of behavioral clues with at least two key features.

[0030] S313. Determine whether the set can be used as a central region based on whether the number of behavioral clues in the set reaches a preset threshold. If it can, store the set as a central region in the cache; otherwise, continue to add it to the next aggregation analysis.

[0031] S32. Determine whether all successfully matched behavioral clues have been added to the central area. If yes, proceed directly to S4; otherwise, proceed to S33.

[0032] S33. Behavioral cues not added to the central region perform soft matching of their key features with the key features of each central region in turn, and determine whether the matching degree between the behavioral cues and the central region exceeds a preset threshold. If it exceeds the threshold, the behavioral cues are added to the central region and proceed to S34. Otherwise, they are saved in the cache and added to the next aggregation analysis.

[0033] S34. The behavioral cues of all central regions are recalculated to match the degree of each central region, and added to the central region with the highest matching degree. Then proceed to S4. The matching degree is a weighted score calculated based on the weights of the key features that match the behavioral cues with the central regions.

[0034] S4. Calculate the similarity between any two central regions. Merge two central regions with similarity exceeding a threshold to form a new central region. The similarity is the proportion of the number of identical key features in the two central regions to the total number of key features in the central region.

[0035] S5. Determine whether each central region meets the conditions to become an organization. If the number of behavioral clue data and the average matching degree of the central region both reach the preset persistence threshold, then the central region meets the conditions to become an organization. If it does, remove the central region from the cache, establish it as a new clue set, and persist it. Otherwise, keep it in the cache and persist it.

[0036] The above method improves the efficiency and real-time performance of tracing network behavior data, which includes network attacks and other non-aggressive network activities.

[0037] The technical solutions of the present invention are not limited to the specific embodiments described above. Any technical modifications made in accordance with the technical solutions of the present invention fall within the protection scope of the present invention.

Claims

1. A clustering analysis method based on network behavior data, characterized by, Comprise: S1, obtaining a data set to be processed, the data set comprising at least one behavior clue; S2, each behavior clue is matched with an existing organization clue set, the behavior clue matched successfully is added to the clue set, the behavior clue unmatched successfully is added to the cache, and S3 is entered; S3, analyzing the behavior clue unmatched successfully, clustering to form a plurality of central regions, each central region comprising at least two behavior clues; specifically comprising: S31, the behavior clue unmatched successfully is clustered and analyzed according to a pre-set rule and threshold to obtain a plurality of central regions; specifically: S311, determining the mapping of each key feature and the corresponding clue list; S312, traversing the mapping obtained in S311, and taking the intersection between the clue lists in the mapping value two by two; S313, judging whether the intersection can be used as a central region, if yes, the intersection is stored in the cache as a central region, otherwise, it is continuously added to the next aggregation analysis; S32, judging whether all the behavior clues matched successfully are added to the central region, if yes, S4 is directly entered, otherwise, S33 is entered; S33, the behavior clue not added to the central region is sequentially soft-matched with the key features of each central region, and it is judged whether the matching degree between the behavior clue and the central region exceeds a pre-set threshold, if yes, the behavior clue is added to the central region, and S34 is entered, otherwise, it is saved in the cache and added to the next aggregation analysis; S34, the behavior clues of all central regions are re-calculated with the matching degree of each central region, and are added to the central region with the highest matching degree, and S4 is entered; S4, calculating the similarity of each two central regions, and merging two central regions with similarity exceeding a threshold to form a new central region; S5, judging whether each central region meets the condition of becoming an organization, if yes, the central region is removed from the cache to establish a new clue set, and is stored persistently; otherwise, it is continuously reserved in the cache and stored persistently.

2. The clustering analysis method based on network behavior data according to claim 1, characterized in that, In S2, comprising: S21, extracting the key features of each behavior clue; S22, hard-matching the key features of each behavior clue with the key features extracted from the existing organization clue set, adding the behavior clue matched successfully to the clue set, adding the behavior clue unmatched successfully to the cache, and entering S3.

3. The method of claim 1, wherein the method further comprises: In S313, it is judged whether the intersection can be used as a central region according to whether the number of behavior clues in the intersection reaches a pre-set threshold.

4. The method of claim 1, wherein the method further comprises: In S5, the conditions are that the number of behavior clue data of the central region and the average matching degree both reach a pre-set persistence threshold.

Images