Method and apparatus for processing relationship, electronic device and storage medium

Abstract

The application relates to the technical field of network security, and discloses a processing method and device for accompanying relations, electronic equipment and a storage medium, which comprise obtaining behavior data of a subject set and a preset first accompanying threshold value, wherein the behavior data records operation sequences of each subject in the subject set on an object within a statistical time window; an accompanying strength is determined according to a gap between the preset first accompanying threshold value and a total number of subjects in the subject set; and a spatial accompanying object combination is extracted from the behavior data according to an extraction strategy corresponding to the accompanying strength, the spatial accompanying object combination being an object combination that appears in operation sequences of no less than the first accompanying threshold value of subjects. When the accompanying strength is different, the application can control the overall calculation amount within an acceptable range by adaptively switching optimal strategies, thereby meeting real-time requirements.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a method, apparatus, electronic device, and storage medium for processing companion relationships. Background Technology

[0002] With the large-scale deployment of IoT devices and the increasing richness of network applications, massive amounts of behavioral data are generated in the network environment, such as device communication logs, DNS query records, and application access records. These data contain rich correlation information between subjects (such as devices and users) and objects (such as domain names, IP addresses, and behavioral actions).

[0003] In scenarios such as cybersecurity analysis, user behavior understanding, and asset management, it is often necessary to discover the key information of "which objects are always used together" from massive amounts of data. For example, certain domain names may always be accessed by the same group of devices at the same time, and certain behaviors may always occur in pairs. This kind of accompaniment relationship between objects is of great value for identifying malicious families, reconstructing attack chains, and building user profiles.

[0004] However, traditional methods for mining adjoint relationships typically employ direct intersection calculations. When the number of subjects and objects reaches millions, the computational complexity increases dramatically, making it difficult to meet real-time requirements. Summary of the Invention

[0005] The purpose of this invention is to provide at least one method, apparatus, electronic device, and storage medium for processing adjoint relations, which can at least solve the technical problem that existing methods, when the number of subjects and objects both reach millions, are difficult to meet real-time requirements by directly finding the intersection, and can at least improve the efficiency of adjoint relation extraction.

[0006] To address the aforementioned technical problems, at least one embodiment of this application provides a method for processing accompaniment relationships, including: Obtain behavioral data of a set of subjects and a preset first accompanying threshold, wherein the behavioral data records the sequence of operations performed by each subject on an object within a statistical time window; The accompanying strength is determined based on the difference between a preset first accompanying threshold and the total number of subjects in the subject set; According to the extraction strategy corresponding to the accompanying intensity, spatial accompanying object combinations are extracted from the behavioral data. The spatial accompanying object combination is an object combination that appears in the operation sequence of no less than the first accompanying threshold number of subjects.

[0007] At least one embodiment of this application also provides an apparatus for processing accompaniment relationships, comprising: The acquisition module is used to acquire behavioral data of the subject set and a preset first accompanying threshold, wherein the behavioral data records the operation sequence of each subject in the subject set on the object within a statistical time window; The determination module is used to determine the accompanying intensity based on the difference between a preset first accompanying threshold and the total number of subjects in the subject set; An extraction module is used to extract spatial accompanying object combinations from the behavioral data according to an extraction strategy corresponding to the accompanying intensity. The spatial accompanying object combination is an object combination that appears in the operation sequence of no less than the first accompanying threshold number of subjects.

[0008] At least one embodiment of this application also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described method for processing the accompaniment relationship.

[0009] At least one embodiment of this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for processing the accompaniment relationship.

[0010] The accompanying relationship processing method, apparatus, electronic device, and storage medium provided in the embodiments of this application do not use the same brute-force intersection method for all scenarios. Instead, they first divide the accompanying relationships into different intensity levels using a calculable index, "the difference between the first accompanying threshold and the total number of subjects," and then match a corresponding extraction strategy for each intensity level. This means that the system uses different computational paths and algorithms under different accompanying intensities, rather than uniformly using brute-force intersection. When the accompanying intensities are different, the matched extraction strategy is necessarily superior in computational complexity to a uniform brute-force intersection. Thus, even in large-scale scenarios where both subjects and objects reach millions, the overall computational load can still be controlled within an acceptable range by adaptively switching the optimal strategy, meeting real-time requirements.

[0011] In some optional embodiments, the accompanying intensity, from low to high, includes partial accompanying, basic accompanying, and complete accompanying, and the extraction strategy corresponding to the accompanying intensity includes: When it is a partial accompaniment, the extraction strategy is a vertical bitmap bitwise AND algorithm or a slice voting algorithm; when it is a basic accompaniment, the extraction strategy is a difference algorithm for missing sets, a frequent pattern tree algorithm, or Boolean matrix bitwise operations; when it is a complete accompaniment, the extraction strategy is to directly find the intersection of the object sets in the operation sequences corresponding to each subject.

[0012] In this embodiment, by binding the intensity of the accompaniment to a dedicated extraction strategy, precise routing of computational paths is achieved: for low-intensity partial accompaniments, approximation algorithms such as vertical bitmap bitwise AND / OR slicing voting are used to compress millions of serial table lookups into single hardware-level parallel operations or a small number of sampling votes, significantly reducing computational overhead; for medium-intensity basic accompaniments, algorithms such as difference sets or frequent pattern trees are used to transform forward verification into reverse missing data computation, avoiding traversing massive amounts of entities one by one; for high-intensity complete accompaniments, intersection can be directly calculated, and since candidate combinations are naturally very few, the computational load is controllable. The synergy of these three approaches ensures that the system uses the algorithm with the optimal computational complexity for each intensity scenario.

[0013] In some optional embodiments, it also includes: The statistical time window is divided into multiple sub-time windows; For any subject, if a candidate object combination appears simultaneously within more than a preset second accompanying threshold number of the sub-time windows, then the candidate object combination is extracted as the time-accompanying object combination on the corresponding subject.

[0014] In this embodiment, by subdividing the statistical time window into multiple sub-time windows and setting a second accompanying threshold, it is possible to accurately identify combinations of objects that frequently appear simultaneously within a short time interval within the same subject, thereby effectively filtering out accidental or long-term co-occurrences and significantly improving the accuracy and reliability of time-related relationships.

[0015] In some optional embodiments, it also includes: If a candidate object combination appears in the operation sequence of each of at least the first accompanying threshold objects, and constitutes a temporal accompanying relationship on each object, then the candidate object combination is determined to be a spatiotemporally accompanying object combination.

[0016] In this embodiment, by simultaneously requiring that object combinations be prevalent in multiple subjects (spatial condition) and frequently co-occur for short periods within each subject (temporal condition), it is possible to filter out strongly correlated combinations that are consistent across subjects and have a tight temporal sequence. This provides a more reliable analytical foundation for subsequent tasks such as tag propagation, attack chain reconstruction, and user profiling.

[0017] In some optional embodiments, it also includes: A similarity analysis is performed on the objects in the extracted combination of the same accompanying objects. If the similarity reaches a preset similarity threshold, the accompanying relationship between the objects in the combination of accompanying objects is determined to be a similar accompanying relationship. Based on similarity association, the labels of known objects in the accompanying object combination are used to classify other unknown objects in the same accompanying object combination.

[0018] In this embodiment, by performing similarity discrimination on the combination of accompanying objects, the association with significantly similar features is classified as similar companions, and other unknown objects in the same combination are automatically labeled using a label propagation algorithm with a small number of objects with known labels, thereby achieving low-cost and high-efficiency batch labeling and category expansion.

[0019] In some optional embodiments, it also includes: If the similarity does not reach the preset similarity threshold, the accompanying relationship between objects in the accompanying object combination is determined to be functional accompaniment; Based on the functional accompaniment relationship, the attack behavior process and / or the attacker's tactical characteristics are reconstructed according to the behavioral sequence represented by the combination of accompaniment objects.

[0020] In this embodiment, by identifying non-similar but frequently co-occurring object combinations as functional companions, and using their behavioral sequences to reconstruct the attack process or construct the attacker's tactical characteristics, it is possible to effectively reveal cross-stage and multi-step collaborative attack patterns. This makes up for the insufficiency of relying solely on similarity analysis to discover functionally complementary attack chains, and provides key behavioral analysis basis for network security situation awareness, threat tracing, and attack organization profiling.

[0021] In some optional embodiments, before obtaining the behavioral data of the subject set, the method further includes: Obtain a first type of raw data and a second type of raw data. The first type of raw data includes a subject identifier, an exit address, and an exit port. The second type of raw data includes a source address, a source port, and an object identifier. By utilizing the matching relationship between the exit address and exit port and the source address and source port, the first type of raw data and the second type of raw data are fused and aligned to generate the behavioral data.

[0022] In this embodiment, by fusing and aligning the first type of data, which includes the subject identifier, exit address, and port, with the second type of data, which includes the source address, port, and object identifier, the silos between multi-source heterogeneous data can be broken down, and the behavioral mapping relationship between the subject and the object can be accurately established, providing a complete and reliable behavioral data foundation for subsequent accompaniment relationship extraction. Attached Figure Description

[0023] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.

[0024] Figure 1 This is a flowchart of a method for processing accompaniment relationships provided in one embodiment of this application; Figure 2 This is a schematic diagram of a system for processing accompaniment relationships provided in one embodiment of this application; Figure 3 This is a schematic diagram of a processing apparatus for accompaniment relationships provided in another embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details are presented in the various embodiments of this application to facilitate a better understanding of the application. However, the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments. The division of the various embodiments below is for ease of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0026] The following is a detailed description of the implementation details of the method for handling the accompaniment relationship in this embodiment. The following content is only for the convenience of understanding and is not necessary for implementing this solution.

[0027] Example 1: The method for processing companion relationships in this embodiment can be applied to electronic devices with communication, computing, and data storage capabilities. Its specific process can be as follows: Figure 1 As shown, it includes: Step 110: Obtain behavioral data of the subject set and a preset first accompanying threshold, wherein the behavioral data records the operation sequence of each subject in the subject set on the object within a statistical time window; Specifically, the subject set refers to a group of behavior initiators to be analyzed, such as user accounts, IoT devices, and mobile terminals on the network. Behavioral data records the sequence of actions each subject takes towards objects within a pre-defined statistical time window (e.g., one day, one hour). The object is the target of the subject's actions, such as a domain name visited, a product purchased, or a geographical location traversed. Each subject's action sequence is arranged chronologically and may contain repeated objects, reflecting the subject's complete behavioral trajectory within the time window.

[0028] For example, define a set of subjects A = { ,……, A set of objects B = { ,……, Within the statistical time window T, each The behavior includes several , that is =[ ,……, ]( Sort by time (duplicates may exist). Dataset Data={ , ...} represents a mixture of all data of entity A. Extract the combination of entities B that have a companion relationship from Data.

[0029] The preset first co-occurrence threshold is a value set according to business needs, used to define the strictness of "spatial co-occurrence". Specifically, if the goal is to discover object combinations that commonly co-occur among a large number of subjects, the threshold can be set higher; if the goal is to discover specific combinations that appear only among a few subjects, it can be set lower. This threshold is a key parameter for subsequent intensity judgment and strategy selection.

[0030] Step 120: Determine the accompanying intensity based on the difference between the preset first accompanying threshold and the total number of subjects in the subject set; Specifically, the magnitude of the difference between the first accompanying threshold and the total number of subjects directly determines the distribution characteristics of the object combinations to be mined in the data, thus affecting the selection of subsequent extraction strategies: When the first associated threshold is close to the total number of subjects (i.e., the difference is very small), a combination of objects must appear in the vast majority or even all of the subjects. Such combinations are usually very few in number, but once they exist, their association is very strong; When the first accompanying threshold is much smaller than the total number of subjects (i.e., the difference is large), it is only required that the combination of objects appears in some subjects. The number of such combinations can be very large, and the data scanning range involved is correspondingly expanded; When the gap is in the middle, the number and distribution of combinations are somewhere in between.

[0031] Therefore, the accompanying strength can also be regarded as an indicator reflecting the "generality of the target combination": the smaller the gap, the higher the strength (the stricter the requirement); the larger the gap, the lower the strength (the more lenient the requirement). Different accompanying strengths correspond to different data scales, sparsity, and combination number characteristics, requiring different algorithmic approaches to handle them.

[0032] Step 130: Extract spatial accompanying object combinations from the behavioral data according to the extraction strategy corresponding to the accompanying intensity. The spatial accompanying object combination is an object combination that appears in the operation sequence of no less than the first accompanying threshold number of subjects.

[0033] Specifically, a spatially associated object combination is defined as a group of objects that appear in the operational sequences of at least a first associated threshold number of subjects. In other words, a group of objects is considered to have a spatially associated relationship if it appears simultaneously in the behavioral records of a sufficient number of different subjects. The extraction strategy is selected based on the associated strength determined in step 120. The statistical characteristics of the behavioral data differ significantly under different strengths: When the accompaniment strength is high (the threshold is close to the total number), there are very few combinations of objects that meet the conditions. We can quickly obtain the common objects by directly finding the intersection of the subject sets, and then combine them into candidate pairs. When the accompaniment strength is moderate, we need more refined algorithms, such as using an inverted index to count the subject list in which the object appears, and then judging whether the threshold is met by the size of the intersection of the lists. When the accompaniment strength is low (the threshold is much smaller than the total number), there are many candidate combinations. We need to use an approximation method based on bitmaps or sampling to significantly improve the calculation speed with an acceptable loss of accuracy.

[0034] This embodiment does not limit the specific algorithm used, but emphasizes the adaptive idea of ​​"strength determining strategy". The system has multiple pre-set extraction logics, and automatically selects the most suitable one based on the strength parameter at runtime, thereby avoiding the high complexity problem caused by using the same calculation framework regardless of the threshold (such as directly enumerating all object combinations) in traditional methods.

[0035] Ultimately, the extracted spatial accompaniment object combinations can be output as results for subsequent storage, analysis, classification, or visualization.

[0036] In summary, the method for handling accompaniment relationships provided in this embodiment does not use the same brute-force intersection method for all scenarios. Instead, it first divides accompaniment relationships into different intensity levels using a calculable indicator: the difference between the first accompaniment threshold and the total number of subjects. Then, it matches a corresponding extraction strategy for each intensity level. Under different accompaniment intensities, the system uses different computational paths and algorithms, rather than uniformly performing brute-force intersection. When the accompaniment intensities are different, the matched extraction strategy is necessarily more computationally efficient than a uniform brute-force intersection. Thus, even in large-scale scenarios with millions of subjects and objects, the overall computational load can still be kept within an acceptable range by adaptively switching the optimal strategy, meeting real-time requirements.

[0037] In some optional embodiments, the accompanying strength, from low to high, includes partial accompanying, basic accompanying, and complete accompanying. The extraction strategy corresponding to the accompanying strength includes: when it is the partial accompanying, the extraction strategy is a vertical bitmap bitwise AND algorithm or a slice voting algorithm; when it is the basic accompanying, the extraction strategy is a difference algorithm for missing sets, a frequent pattern tree algorithm, or Boolean matrix bitwise operations; when it is the complete accompanying, the extraction strategy is to directly find the intersection of the object sets in the operation sequence corresponding to each subject.

[0038] Specifically, based on the first accompanying threshold and total number of subjects n The relative size of the components also affects the strength of the association represented by the accompanying relationship: (1) equaln That is, each All include ( ,……, This kind of accompaniment is defined as complete accompaniment; (2) Slightly less than n (e.g.) (0.8n), that is, the vast majority All include ( ,……, Only a small amount This condition is not met. Such an accompaniment is defined as a fundamental accompaniment; even though( ,……, It is indeed because of functional collaboration that the DGA is used when the same app makes requests or when the same malicious code communicates. n When the number of occurrences is large, it is difficult to reach the standard of complete association; in other words, complete association is often a theoretical condition. In real-world scenarios, due to factors such as network jitter and missing log parsing data, only the standard of basic association may be met.

[0039] (3) Less than n ,and It is a relatively large value (e.g.) (where 0.5n), this association is defined as a partial association.

[0040] In the process of extracting spatially associated object combinations, in order to improve the time and space efficiency of computation, according to and n The methods for extracting adjoint relationships differ depending on their relative and absolute sizes. These methods include, but are not limited to, the following: (1) equal n (Complete accompaniment) You can directly consider calculating each one. The intersection of.

[0041] (2) Slightly smaller n When (basic adjoint) is involved, the algorithms that can be used include, but are not limited to: Inverted Index and Difference (dEclat Algorithm): Abandoning the conventional "intersection" approach, this algorithm leverages the extremely small number of missing elements (B) under high thresholds. Instead of "finding the intersection of present sets," it transforms massive data computation into computation with limited data, significantly improving efficiency by "recording the missing sets (Diffset) and finding the union of the missing sets." Core logic: If the combination { , The number of common missing subjects ≤ (n - If ), then the combination is a target accompaniment relationship.

[0042] Frequent Pattern Tree (FP-Growth Algorithm): By scanning the behavioral data of subject A twice, the data is compressed into an FP-Tree (frequent pattern tree). The nodes in the tree record the frequency of occurrence and association of element B. In high threshold scenarios, the FP-Tree has a "slender single chain + short branch" structure, and the path traversal and pattern extraction speed is extremely fast.

[0043] Boolean matrices and bitwise operations: When the total number of elements in B m Not too big ( m When <10000), construct n × m Boolean matrix (rows = , column = element = 1 indicates Include (0 indicates not included), utilizes BitMap / BitSet to compress memory, and performs CPU hardware-level bitwise AND operations to quickly verify the frequency of occurrence of B element combinations, achieving a rapid optimization of brute-force cracking.

[0044] (3) Less than n (Partially associated), the algorithms that can be used include, but are not limited to: Vertical Data Representation and Bitwise Intersection (BitMap / Eclat Algorithm): The core of this method lies in perspective transformation and bitwise operation acceleration: It transforms the traditional horizontal recording method of "which behaviors a subject contains" into a vertical bitmap format of "which subjects each behavior appears in" (e.g., using a string of binary bits to record the trigger distribution of a specific security alarm or micro-action across all affected devices); it quickly counts and eliminates data with a total occurrence count lower than [a certain threshold]. After identifying low-frequency behaviors, the highly efficient bitwise AND operation of modern CPUs is used to perform a series of intersection calculations on the bitmaps of the remaining high-frequency behaviors. As long as the number of 1s in the intersection result still meets the threshold, frequently co-occurring behaviors can be quickly identified. ,……, This behavior combination transforms the complex comparison of massive amounts of data into lightweight hardware-level computing.

[0045] Slice Voting Algorithm: When dealing with extremely large datasets and limited resources, this proposal suggests an approximate spatiotemporal adjoint extraction method based on slice voting. That is, r% of elements in A are randomly selected as... Set an extremely short t as the time window for analysis. Each of them Its division forms slices (each slice consists of several...) Composition) as a set of candidate slices ={ ( ,……, ),……}. Then for the remaining elements in A (A- Divide it into segments using t as the time window to form ,use right Candidate slices are voted on, and slices that reach the spatiotemporal association threshold are considered as a group of objects with spatiotemporal association.

[0046] Due to the randomness of the slices, if the slices are too small, the spatiotemporal adjoint set obtained by this method may not be the maximum spatiotemporal adjoint set; if the slices are too large, some spatiotemporal adjoint sets may be missed.

[0047] However, this method is relatively simple to implement and is suitable for data streaming input scenarios such as Kafka (where the first arriving data can be selected as the input). (Later data is used for voting).

[0048] In this embodiment, by binding the intensity of the accompaniment to a dedicated extraction strategy, precise routing of computational paths is achieved: for low-intensity partial accompaniments, approximation algorithms such as vertical bitmap bitwise AND / OR slicing voting are used to compress millions of serial table lookups into single hardware-level parallel operations or a small number of sampling votes, significantly reducing computational overhead; for medium-intensity basic accompaniments, algorithms such as difference sets or frequent pattern trees are used to transform forward verification into reverse missing data computation, avoiding traversing massive amounts of entities one by one; for high-intensity complete accompaniments, intersection can be directly calculated, and since candidate combinations are naturally very few, the computational load is controllable. The synergy of these three approaches ensures that the system uses the algorithm with the optimal computational complexity for each intensity scenario.

[0049] In some optional embodiments, the method further includes: dividing the statistical time window into multiple sub-time windows; for any subject, if a candidate object combination appears simultaneously within more than a preset second accompanying threshold number of the sub-time windows, then the candidate object combination is extracted as the time-accompanying object combination on the corresponding subject.

[0050] Specifically, the statistical time window T (e.g., hour) is divided into extremely short sub-time windows t (e.g., 1 minute). That is, T = [ ,……, Based on this, a =[ ,……, Divided into =[ ,……, ]=[[ ,…]……,[…, If a group ( ,……, When the second accompanying threshold is exceeded indivual If it appears in the text, then define the group ( ,……, )exist They are related in terms of time.

[0051] In this embodiment, by subdividing the statistical time window into multiple sub-time windows and setting a second accompanying threshold, it is possible to accurately identify combinations of objects that frequently appear simultaneously within a short time interval within the same subject, thereby effectively filtering out accidental or long-term co-occurrences and significantly improving the accuracy and reliability of time-related relationships.

[0052] In some optional embodiments, the method further includes: if a candidate object combination appears in the operation sequence of each of at least the first accompanying threshold objects and constitutes a temporal accompanying relationship on each object, then the candidate object combination is determined to be a spatiotemporally accompanying object combination.

[0053] Specifically, if a group ( ,……, In more than indivual ={ ,……, It appears in} and for Each of them , ( ,……, )exist If it is also a companion relationship in the time dimension, then define this group ( ,……, The relationship between them is a spatiotemporal association on the set of entities A.

[0054] Of the three types of accompaniment relationships mentioned above, spatiotemporal accompaniment reflects the closest accompaniment relationship. Temporal accompaniment can only reflect (…). ,……, In a single The simultaneous occurrence of these phenomena across time may possess unique characteristics. Spatial accompaniment can only reflect (…). ,……, Within a long time window T, entities appearing simultaneously in entity set A may not have a clear correlation. Spatiotemporal accompaniment, however, refers to simultaneous accompaniment in both time and space dimensions, representing (…). ,……, The relationship between them is stronger.

[0055] In this embodiment, by simultaneously requiring that object combinations be prevalent in multiple subjects (spatial condition) and frequently co-occur for short periods within each subject (temporal condition), it is possible to filter out strongly correlated combinations that are consistent across subjects and have a tight temporal sequence. This provides a more reliable analytical foundation for subsequent tasks such as tag propagation, attack chain reconstruction, and user profiling.

[0056] In some optional embodiments, the method further includes: performing similarity analysis on objects in the extracted same combination of accompanying objects; if the similarity reaches a preset similarity threshold, determining the accompanying relationship between objects in the combination of accompanying objects as similar accompanying relationships; based on the similar accompanying relationship, classifying other unknown objects in the same combination of accompanying objects using the labels of known objects in the combination of accompanying objects; if the similarity does not reach the preset similarity threshold, determining the accompanying relationship between objects in the combination of accompanying objects as functional accompanying relationships; and based on the functional accompanying relationship, reconstructing the attack behavior flow and / or constructing the attacker's tactical characteristics according to the behavioral sequence represented by the combination of accompanying objects.

[0057] Specifically, if a group of domains with an accompanying relationship exhibits significant similarities, such as similar textual features, similar registration times, identical DNS resolution results (e.g., both being NXDomain), or similar webpage structures, then the accompanying relationship between these domains is classified as similarity-based. If a group of domains with an accompanying relationship is known to have high similarity, this accompanying relationship can be used to classify unknown data based on a small amount of labeled data and label propagation algorithms. For example, if a domain is known to belong to the malware family A, then it can be assumed that other accompanying domains also belong to the malware family A.

[0058] If the extracted set of domain names with accompanying relationships do not have any obvious similarities or associations, such as differences in text features, second-level suffixes, or resolution results, this set of accompanying domain names may be a collaborative relationship arising from functional synergy. Based on the functional accompanying relationship, the attack behavior flow and / or the attacker's tactical characteristics are reconstructed according to the behavioral sequence represented by the accompanying object combination.

[0059] Existing techniques for detection based on association relationships typically assume that entities with association relationships are of the same class. However, in reality, entities with association relationships may have cooperative relationships rather than being of the same class. Therefore, this crude association-based classification may lead to a high false positive rate. This solution further classifies the extracted association relationships to determine their nature, thereby improving the accuracy of subsequent association relationship applications.

[0060] In some optional embodiments, before obtaining the behavioral data of the subject set, the method further includes: obtaining a first type of raw data and a second type of raw data, wherein the first type of raw data includes a subject identifier, an exit address, and an exit port, and the second type of raw data includes a source address, a source port, and an object identifier; and using the matching relationship between the exit address and the exit port and the source address and the source port, the first type of raw data and the second type of raw data are fused and aligned to generate the behavioral data.

[0061] Specifically, behavioral data can be collected from raw data such as traffic / logs. If the data volume is too large and includes some known irrelevant data, the data preprocessing module filters the irrelevant data based on whitelists or other rules to reduce subsequent computation and storage consumption. If the entities and behaviors involved in the accompanying relationships span different data types, the data needs to be merged and aligned.

[0062] For example, the external connection log data of IoT devices is currently known. The format of a single IoT log entry is (device_id, external_ip, external_port, timestamp). `device_id` is a unique identifier for the IoT device. The IoT device connects to the base station via an IoT SIM card, dynamically obtaining an exit IP address and binding it to a port to achieve communication with the Internet. `external_ip` and `external_port` are the IP address and bound port number obtained by the device based on the IoT SIM card, respectively. `timestamp` is the time when the connection occurred.

[0063] Meanwhile, the DNS log data is also known. The format of a single DNS log entry is (src_ip, src_port, dst_ip, domain). src_ip and src_port are the source address and source port of the DNS resolution request, respectively. dst_ip is the destination address of the DNS resolution request, and domain is the domain name of the DNS resolution request.

[0064] To analyze the associated relationships between domain names accessed by IoT devices, IoT device log data and DNS log data need to be merged and aligned based on external_ip and external_port (i.e., src_ip, src_port).

[0065] In this embodiment, by fusing and aligning the first type of data, which includes the subject identifier, exit address, and port, with the second type of data, which includes the source address, port, and object identifier, the silos between multi-source heterogeneous data can be broken down, and the behavioral mapping relationship between the subject and the object can be accurately established, providing a complete and reliable behavioral data foundation for subsequent accompaniment relationship extraction.

[0066] Example 2: Based on the above embodiments, this embodiment provides an application example of a method for processing adjoint relationships. For example... Figure 2 As shown, the accompanying relationship processing system proposed in this embodiment mainly consists of five parts: a data preprocessing module, an accompanying relationship extraction module, an accompanying relationship storage module, an accompanying relationship classification module, and an accompanying relationship application module. The specific functions of each module are as follows: 1. Data Preprocessing Module The data preprocessing module is responsible for all preprocessing procedures preceding extraction, including but not limited to the collection, parsing, initial filtering, and fusion alignment of raw data such as traffic / logs. If the data volume is too large and includes some known irrelevant data, the data preprocessing module filters the irrelevant data based on whitelists or other rules to reduce subsequent computation and storage costs. If the entities and behaviors involved in the accompanying relationships span different data types, data fusion alignment is required.

[0067] For example, the external connection log data of IoT devices is currently known. The format of a single IoT log entry is (device_id, external_ip, external_port, timestamp). `device_id` is a unique identifier for the IoT device. The IoT device connects to the base station via an IoT SIM card, dynamically obtaining an exit IP address and binding it to a port to achieve communication with the Internet. `external_ip` and `external_port` are the IP address and bound port number obtained by the device based on the IoT SIM card, respectively. `timestamp` is the time when the connection occurred.

[0068] Meanwhile, the DNS log data is also known. The format of a single DNS log entry is (src_ip, src_port, dst_ip, domain). src_ip and src_port are the source address and source port of the DNS resolution request, respectively. dst_ip is the destination address of the DNS resolution request, and domain is the domain name of the DNS resolution request.

[0069] To analyze the associated relationships between domain names accessed by IoT devices, IoT device log data and DNS log data need to be merged and aligned based on external_ip and external_port (i.e., src_ip, src_port).

[0070] 2. Accompanying Extraction Module The adjoint extraction module is the core module of this framework, extracting adjoint relationships from preprocessed data. Specifically, it can be divided into two steps. The first step is defining the adjoint relationships, that is, defining the subjects used for clustering and the objects that have adjoint relationships. The second step is extracting the adjoint relationships between objects. The specific details of each step are as follows: 2.1 Accompanying Definition 2.1.1 Definition of Companionship Define a set of entities A = { ,……, }, a set of entities B = { ,……, Within a relatively long time window T, each The behavior includes several , that is =[ ,……, ]( Sort by time (may contain duplicates). Dataset Data={ ,……} represents a mixture of all data of entity A. The accompanying extraction module extracts combinations of entities B that have accompanying relationships from the Data. Here, A = { ,……, } is called the accompanying relation subject, and B = { ,……, This is called the object of the accompanying relationship.

[0071] From the perspective of accompanying dimensions, they can be divided into three types: temporal accompanying, spatial accompanying, and temporal and spatial accompanying (spatiotemporal accompanying). (1) Time accompanies Divide the time window T into extremely short time windows t, i.e., T = [ ,……, Based on this, a =[ ,……, Divided into =[ ,……, ]=[[ ,…]……,[…, If a group ( ,……, When the second accompanying threshold is exceeded indivual If it appears in the text, then define the group ( ,……, )exist They are related in terms of time.

[0072] (2) Spatial accompaniment If a group ( ,……, When the first accompanying threshold is exceeded indivual If it appears in the text, then define the group ( ,……, In the spatial dimension of entity set A, there is a companion relationship.

[0073] (3) Spatiotemporal accompaniment If a group ( ,……, In more than indivual ={ ,……, It appears in} and for Each of them , ( ,……, )exist If it is also a companion relationship in the time dimension, then define this group ( ,……, The relationship between them is a spatiotemporal association on the set of entities A.

[0074] Among the three types of accompaniment relationships mentioned above, spatiotemporal accompaniment reflects the closest accompaniment relationship.

[0075] Time can only reflect ( ,……, In a single The simultaneous occurrence of these phenomena across the time dimension may have unique characteristics.

[0076] Space can only reflect ( ,……, Within a long time window T, entities appearing simultaneously on entity set A may not have an obvious association.

[0077] Spatiotemporal accompaniment, on the other hand, refers to simultaneous accompaniment in both time and space dimensions, representing ( ,……, The relationship between them is stronger.

[0078] 2.1.2 The significance of companion relationships The association represented by the accompanying relationship is not always a similarity relationship; it may also be a synergistic relationship. Furthermore, the association represented by the accompanying relationship does not only exist in (…). ,……, It may also reflect this. ={ ,……, }middle The relationships between them.

[0079] For example, a carrier user's internet access log contains fields such as (user_id, src_ip, src_port, dst_ip, dst_port, app, host, timestamp). The design of these relationships can be considered from several perspectives, including but not limited to the following: (1) Reference relationships between different hosts. If, in a scenario involving external resource references, site A (hosta) references images from site B (hostb) and resources from site C (hostc), then when a user visits hosta, they will simultaneously access hostb and hostc due to the automatic loading of resources. This reference relationship between hosta and hostb / hostc is reflected in the spatiotemporal accompaniment relationship between hosta and the three hosts hostb / hostc in the dimension of user behavior.

[0080] (2) Different behaviors implemented by a single App. For example, when a mobile payment App (apppay) runs, it first calls the relevant domains for risk control and security to defend against attacks and risks (hostd, hoste). Subsequently, in order to complete user authentication, payment transactions, and other business operations, it will access the relevant core basic business domains (hostf). Therefore, the different behaviors implemented by a single App, that is, the coordination between hostd, hoste, and hostf, are reflected in the spatiotemporal accompaniment relationship between the three hosts (hostd, hoste, and hostf) in the dimension of user behavior. And this spatiotemporal accompaniment relationship is associated with apppay, that is, it is reflected in the spatiotemporal accompaniment relationship between the three hosts (hostd, hoste, and hostf) in the App dimension.

[0081] (3) Correlation of user behavior. For example, if a user, User0, has cryptocurrency investment behavior, they may browse several cryptocurrency websites (hostg, hosth, hosti) continuously during their free time. Then, the correlation of User0's browsing of cryptocurrency websites is reflected in the accompanying relationship of hostg, hosth, and hosti in User0's time dimension. Further, if (hostg, hosth, hosti) has a time accompanying relationship in the time dimension of a group of users such as (User1, User2, ...), then this accompanying relationship can be used as a user profile of (User1, User2, ...) - a cryptocurrency investment user.

[0082] Accompanying relationships can be defined based on actual data conditions and application scenarios.

[0083] 2.1.3 Strength of the adjoining relationship Taking spatial accompaniment as an example, according to and n The relative size of the components and the strength of the association represented by the accompanying relationship also differ.

[0084] (1) equal n That is, each All include ( ,……, This kind of accompaniment is defined as complete accompaniment.

[0085] (2) Slightly less than n: that is, the vast majority All include ( ,……, Only a small amount This condition is not met. This type of accompaniment is defined as the fundamental accompaniment.

[0086] even though( ,……, It is indeed because of functional collaboration that the DGA is used when the same app makes requests or when the same malicious code communicates. n When the number of occurrences is large, it is difficult to reach the standard of complete association; in other words, complete association is often a theoretical condition. In real-world scenarios, due to factors such as network jitter and missing log parsing data, only the standard of basic association may be met.

[0087] (3) Less than n ,and It is a relatively large value. This kind of association is defined as a partial association. A partial association usually represents A = { ,……, A portion of them have They share similar characteristics. For example, in the scenario described above where user behavior is correlated, some users are cryptocurrency investors.

[0088] 2.2 Adjoint Relation Extraction In the extraction process, the basic theoretical logic of temporal and spatial accompaniment extraction is the same, namely, finding the intersection of object B. Spatiotemporal accompaniment is the intersection of temporal and spatial accompaniment.

[0089] In actual operation, in order to improve the time and space efficiency of computation, according to and n The methods for extracting adjoint relationships differ depending on their relative and absolute sizes. These methods include, but are not limited to, the following: 2.2.1 equal n (Complete accompaniment) You can directly consider calculating each one. The intersection of.

[0090] 2.2.2 Slightly smaller n When (basic adjoint) is involved, the algorithms that can be used include, but are not limited to: Inverted index and difference set (dEclat algorithm). This is a relatively mature method. It abandons the conventional "intersection" approach and leverages the characteristic that the number of missing elements B is extremely small under high thresholds. Instead of "finding the intersection of present sets," it transforms massive data computation into small data computation, significantly improving efficiency by "recording the missing sets (Diffset) and finding the union of the missing sets." Core logic: If the combination { , The number of common missing subjects ≤ ( n - If ), then the combination is a target accompaniment relationship.

[0091] Frequent Pattern Tree (FP-Growth Algorithm). This is a relatively mature method. By scanning the behavioral data of subject A twice, the data is compressed into an FP-Tree (Frequent Pattern Tree). The nodes in the tree record the frequency of occurrence of element B and its association relationships. In high-threshold scenarios, the FP-Tree has a "slender single chain + short branch" structure, and the path traversal and pattern extraction speed is extremely fast. Mature algorithm libraries can be directly reused, reducing development costs.

[0092] Boolean matrices and bitwise operations. This is a relatively mature method. When the total number of elements m in B is not too large (m < 10000), construct an n×m Boolean matrix (rows = ...). , column = element = 1 indicates Include (0 indicates not included), utilizes BitMap / BitSet to compress memory, and performs CPU hardware-level bitwise AND operations to quickly verify the frequency of occurrence of B element combinations, achieving a rapid optimization of brute-force cracking.

[0093] 2.2.3 Less than n (Partially accompanied) Vertical data representation and bitwise intersection (BitMap / Eclat algorithm). This is a relatively mature method. The core of the method lies in perspective transformation and bitwise operation acceleration: it flips the traditional horizontal recording method of "which behaviors a subject contains" into a vertical bitmap format of "which subjects each behavior appears in" (e.g., using a string of binary bits to record the trigger distribution of a specific security alarm or micro-action across all affected devices). After quickly counting and eliminating low-frequency behaviors with a total occurrence count below the threshold space, it uses the extremely efficient bitwise AND operation of modern CPUs to perform a series of intersections on the bitmaps of the remaining high-frequency behaviors. As long as the number of 1s in the intersection result still meets the threshold, frequently co-occurring behaviors can be quickly identified. ,……, This behavior combination transforms the complex comparison of massive amounts of data into lightweight hardware-level computing.

[0094] Slice voting Faced with extremely large datasets and limited resources, this proposal suggests an approximate spatiotemporal adjoint extraction method based on slice voting. That is, r% of elements in A are randomly selected as... Set an extremely short t as the time window for analysis. Each of them Its division forms slices (each slice consists of several...) Composition) as a set of candidate slices ={ ( ,……, ),……}. Then for the remaining elements in A (A- Divide it into segments using t as the time window to form ,use right Candidate slices are voted on, and slices that reach the spatiotemporal association threshold are considered as a group of objects with spatiotemporal association.

[0095] Due to the randomness of the slices, if the slices are too small, the spatiotemporal adjoint set obtained by this method may not be the maximum spatiotemporal adjoint set; if the slices are too large, some spatiotemporal adjoint sets may be missed.

[0096] However, this method is relatively simple to implement and is suitable for data streaming input scenarios such as Kafka (where the first arriving data can be selected as the input). (Later data is used for voting).

[0097] 3. Accompanying Relationship Storage Module The companion relationship storage module is responsible for storing the extracted companion relationships. For ease of subsequent application, this module needs to store the companion objects, i.e. ( ,……, The accompanying relationships between them also need to be stored, as well as the inclusion relationships between the accompanying subject and the accompanying object, that is... right( ,……, The inclusion relationship can be defined. In addition, the tags for each subject and each object, as well as the tags for accompanying relationships, can be stored as needed.

[0098] 4. Accompanying Classification Module This module is responsible for classifying the associated subjects, objects, and relationships stored in the database.

[0099] 4.1 Classification of Companion Relationships For different subjects and objects, the accompanying relationships can be classified based on similarity, relevance, and other factors. Taking domain names as an example... (1) Similarity of accompanying objects If a group of domains with a companion relationship has obvious similar characteristics, such as similar text features, similar registration time, the same resolution result (e.g., both are NXDomain), and similar web page composition, then the companion relationship between this group of domains is a similarity-based companion relationship.

[0100] (2) Irrelevantness of the accompanying object If the extracted set of domain names with accompanying relationships do not have any obvious similarities or associations, such as different text features, second-level suffixes, or resolution results, this set of accompanying domain names may be due to a collaborative relationship generated by functional synergy.

[0101] 4.2 Classification of Accompanying Objects If a group of domains with a known association relationship have a high degree of similarity, this association relationship can be used to classify unknown data based on a small amount of labeled data and label propagation algorithms. For example, if a domain is known to belong to the malware family A, then it can be assumed that other associated domains also belong to the malware family A.

[0102] 4.3 Accompanying Subject Classification If it is known that a group of domain names with a companion relationship belong to a group of domain names for a specific type of application, such as the domain names of cryptocurrency trading websites, then user profiles can be created based on the users who frequently use this group of domain names—cryptocurrency traders.

[0103] The categorized data tags will be stored in the companion relationship storage module and used in subsequent companion relationship application scenarios. In addition, the tag data can also be used for data filtering in the data preprocessing module.

[0104] 5. Application of Companionship Relationship The application of companion relationships includes, but is not limited to, the following scenarios.

[0105] 5.1 Classification Classifying network entities based on association relationships is the most common application scenario, including but not limited to the domain name scenario described in 4.1 and 4.2.

[0106] 5.2 Association Taking domain names as an example, it is known that domain name A and domain name B are used by the same attack group, and domain name C and domain name D are used by the same attack group. Assuming that domain name B and C have a companion relationship, it can be inferred that A, B, C, and D may come from the same group.

[0107] 5.3 Portraits In addition to the user profiles described in 4.3, in the context of network attack detection, the accompanying relationship of a certain network behavior may represent different steps of the same attack behavior. By capturing the accompanying relationship, on the one hand, we can depict the sequential steps of an attack behavior, that is, reconstruct the process of the attack behavior, and on the other hand, we can use it to depict the profile of the relevant organization, such as to construct the tactical characteristics of the attacker.

[0108] Example 3: Another embodiment of this application relates to a processing apparatus for accompaniment relationships. The implementation details of this accompaniment relationship processing apparatus are described below. The following implementation details are provided for ease of understanding and are not essential for implementing this solution. A schematic diagram of the accompaniment relationship processing apparatus of this embodiment can be seen as follows: Figure 3 As shown, it includes an acquisition module 310, a determination module 320, and an extraction module 330.

[0109] The acquisition module 310 is used to acquire behavioral data of the subject set and a preset first accompanying threshold, wherein the behavioral data records the operation sequence of each subject in the subject set on the object within a statistical time window; The determination module 320 is used to determine the accompanying intensity based on the difference between a preset first accompanying threshold and the total number of subjects in the subject set; Extraction module 330 is used to extract spatial accompanying object combinations from the behavioral data according to an extraction strategy corresponding to the accompanying intensity. The spatial accompanying object combination is an object combination that appears in the operation sequence of no less than the first accompanying threshold number of subjects.

[0110] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units are absent in this embodiment.

[0111] In some optional embodiments, the accompanying relationship processing apparatus can implement the accompanying relationship processing method described in any of the above embodiments.

[0112] Example 4: Another embodiment of this application relates to an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the companion relationship processing method in the above embodiments.

[0113] In this embodiment, the memory and processor are connected via a bus, which can include any number of interconnected buses and bridges, connecting various circuits of one or more processors and the memory together. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be further described in this embodiment. A bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0114] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0115] Example 5: Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0116] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. A method for processing accompaniment relationships, characterized in that, include: Obtain behavioral data of a set of subjects and a preset first accompanying threshold, wherein the behavioral data records the sequence of operations performed by each subject on an object within a statistical time window; The accompanying strength is determined based on the difference between a preset first accompanying threshold and the total number of subjects in the subject set; According to the extraction strategy corresponding to the accompanying intensity, spatial accompanying object combinations are extracted from the behavioral data. The spatial accompanying object combination is an object combination that appears in the operation sequence of no less than the first accompanying threshold number of subjects.

2. The method for processing accompaniment relationships according to claim 1, characterized in that, The accompanying intensities, from low to high, include partial accompanying, basic accompanying, and complete accompanying, and the extraction strategies corresponding to the accompanying intensities include: When it is a partial accompaniment, the extraction strategy is a vertical bitmap bitwise AND algorithm or a slice voting algorithm; when it is a basic accompaniment, the extraction strategy is a difference algorithm for missing sets, a frequent pattern tree algorithm, or Boolean matrix bitwise operations; when it is a complete accompaniment, the extraction strategy is to directly find the intersection of the object sets in the operation sequences corresponding to each subject.

3. The method for processing accompaniment relationships according to claim 1, characterized in that, Also includes: The statistical time window is divided into multiple sub-time windows; For any subject, if a candidate object combination appears simultaneously within more than a preset second accompanying threshold number of the sub-time windows, then the candidate object combination is extracted as the time-accompanying object combination on the corresponding subject.

4. The method for processing accompaniment relationships according to claim 3, characterized in that, Also includes: If a candidate object combination appears in the operation sequence of each of at least the first accompanying threshold objects, and constitutes a temporal accompanying relationship on each object, then the candidate object combination is determined to be a spatiotemporally accompanying object combination.

5. The method for processing the accompanying relationship according to any one of claims 1-4, characterized in that, Also includes: A similarity analysis is performed on the objects in the extracted combination of the same accompanying objects. If the similarity reaches a preset similarity threshold, the accompanying relationship between the objects in the combination of accompanying objects is determined to be a similar accompanying relationship. Based on similarity association, the labels of known objects in the accompanying object combination are used to classify other unknown objects in the same accompanying object combination.

6. The method for processing accompaniment relationships according to claim 5, characterized in that, Also includes: If the similarity does not reach the preset similarity threshold, the accompanying relationship between objects in the accompanying object combination is determined to be functional accompaniment; Based on the functional accompaniment relationship, the attack behavior process and / or the attacker's tactical characteristics are reconstructed according to the behavioral sequence represented by the combination of accompaniment objects.

7. The method for processing accompaniment relationships according to claim 1, characterized in that, Before obtaining the behavioral data of the subject set, the following is also included: Obtain a first type of raw data and a second type of raw data. The first type of raw data includes a subject identifier, an exit address, and an exit port. The second type of raw data includes a source address, a source port, and an object identifier. By utilizing the matching relationship between the exit address and exit port and the source address and source port, the first type of raw data and the second type of raw data are fused and aligned to generate the behavioral data.

8. A device for processing accompaniment relationships, characterized in that, include: The acquisition module is used to acquire behavioral data of the subject set and a preset first accompanying threshold, wherein the behavioral data records the operation sequence of each subject in the subject set on the object within a statistical time window; The determination module is used to determine the accompanying intensity based on the difference between a preset first accompanying threshold and the total number of subjects in the subject set; An extraction module is used to extract spatial accompanying object combinations from the behavioral data according to an extraction strategy corresponding to the accompanying intensity. The spatial accompanying object combination is an object combination that appears in the operation sequence of no less than the first accompanying threshold number of subjects.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method for processing the accompaniment relationship as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for processing the accompaniment relationship as described in any one of claims 1 to 7.

Images