A multi-host connection event anomaly detection method and system

Abstract

The application discloses a multi-host connection event anomaly detection method and system, and belongs to the technical field of network security and terminal behavior analysis. The technical problem to be solved is that the existing connection anomaly detection has the problems of amplifying the general degree of the object group, the current event interference, the cloud environment IP drift, the unified rule being unable to adapt to different program behaviors, and the cold start sample being insufficient, resulting in the problem of false alarm. The technical solution points are that four types of cooperative baselines of entities, entity-objects, similar entities and similar objects are constructed; the connection event is preprocessed to locate the program entity and the similar set, the historical weight is attenuated before writing the current event, the entity side feature and the similar coverage general degree are calculated, and the entity historical contribution correction increment is generated; after the anomaly discrimination is completed, the four types of baselines are updated, and the detection result is output in combination with the multi-dimensional features.

Description

Technical Field

[0001] This invention belongs to the field of network security and terminal behavior analysis technology, specifically relating to a method and system for detecting anomalies in multi-host connection events. Background Technology

[0002] Enterprise terminals, servers, and cloud hosts continuously generate proactive connection events initiated by programs to remote IPs. Anomaly detection for these connection events is a core technical aspect of endpoint security. Existing connection anomaly detection solutions typically determine whether a current connection is abnormal based on whether the remote object is appearing for the first time, the number of connections within a preset time window, historical connection frequency, or the object's prevalence among similar programs. In real-world deployment environments with multiple hosts, diverse program types, and continuously changing connection objects, existing technical solutions suffer from the following problems: (1) Connection frequency statistics cannot accurately represent the coverage of objects by similar entities; existing solutions usually directly use the cumulative number of connection events to count the prevalence of objects in similar programs. However, a single program on the same host may repeatedly connect to the same remote object in a short period of time. The high frequency of repeated connections on a single host will unduly amplify the commonness of the object in the group. "High number of connections" is not the same as "high number of similar hosts connecting to the object". Existing statistical methods do not distinguish between the connection strength of the object and the coverage of similar entities, which leads to the distortion of the object prevalence calculation and makes it easy to misjudge private objects with high frequency of connections on a single host as public objects of similar programs. (2) Single IP dimension judgment cannot adapt to the scenario of dynamic changes in service address in cloud environment; In cloud service, content delivery network, load balancing and service migration scenarios, the same service organization often corresponds to multiple or dynamically changing IP addresses. The existing solution only judges the anomaly based on whether the IP is appearing for the first time, which is easy to misjudge normal IP address drift under familiar service organization as a new connection target; when the IP, the organization to which it belongs and the country to which it belongs have no historical connection support, it is more likely to indicate that the remote target has undergone substantial changes. The existing single dimension judgment method has a high false alarm rate in cloud environment. (3) The unified detection rules cannot adapt to the differences in behavior patterns of different types of programs; browser and proxy-type open access programs usually connect to a large number of different remote objects, while heartbeat reporting, version update or fixed business programs often only connect to a few fixed targets for a long time. The existing solution uses the same new object judgment rules or unified weighted thresholds for all types of programs, which cannot distinguish the normal multi-object connection behavior of open access programs from the connection pattern breakthrough behavior of fixed connection mode programs, and it is difficult to accurately identify the abnormal external connection behavior of fixed business programs; (4) Insufficient baseline samples during the cold start phase will reduce the reliability of the detection results. New installation programs, new online hosts, or similar program sets with small sample sizes have not yet formed a stable historical behavior baseline. When the baseline samples are insufficient, the existing scheme still directly judges the object as a high-risk anomaly based on the lack of historical records, without considering the impact of baseline maturity on the confidence of the detection results, which is prone to generating a large number of low-quality false alarms.

[0003] In view of this, the present invention is hereby proposed. Summary of the Invention

[0004] To address the aforementioned technical problems in the existing technology, this invention provides a method and system for detecting anomalies in multi-host connection events. This solves the problems of existing connection anomaly detection methods, such as amplifying the prevalence of high-frequency connections on a single host, interference of the current event with the detection basis, false alarms due to IP drift in cloud environments, inability of unified rules to adapt to different program behaviors, and excessive false alarms due to insufficient cold start samples.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: Firstly, a method for detecting anomalies in multi-host connection events includes: S1. Receive program-initiated connection event, preprocess the connection event, determine the program entity according to the host identifier and program family, determine the set of similar programs according to the host group and program family, and decompose the connection event into connection objects corresponding to multiple object dimensions; S2. Read the entity baseline state and entity-object historical baseline state corresponding to the program entity, and decay the historical weight in the entity baseline state and entity-object historical baseline state to the time of the current connection event. S3. Calculate the entity-side detection features based on the entity baseline state that has not yet been written to the current connection event and the entity-object historical baseline state. S4. Based on the historical connection time in the entity-object historical baseline state and the historical activity time in the entity baseline state, calculate the historical remaining contribution of the current program entity that has been included in the same type of baseline state, and generate the object coverage correction increment and the same type of effective entity correction increment. S5. Read the baseline status of similar entities and the baseline status of similar entities-objects corresponding to the similar program set, and decay the aggregation weight in the baseline status of similar entities and the baseline status of similar entities-objects to the time of the current connection event. S6. Based on the baseline state of similar entities that have not yet been written to the current connection event and the baseline state of similar entity-object coverage, calculate the similar coverage universality of the current connection object; S7. Combine the entity-side detection features with the coverage prevalence of similar objects to perform anomaly detection and generate detection results; S8. After the detection results are generated, update the entity baseline status and the entity-object historical baseline status, and update the same type of entity baseline status and the same type of entity-object coverage baseline status using the object coverage correction increment and the same type of effective entity correction increment. S9. Output the detection results and retain the updated four baseline states for subsequent connection event processing.

[0006] Furthermore, the step of decomposing the connection event into connection objects corresponding to multiple object dimensions includes: The connection event is decomposed into connection objects with three independent object dimensions: IP object, organization object, and country object. For each object dimension, maintain the corresponding entity-object historical baseline state and the same type of entity-object coverage baseline state.

[0007] Furthermore, the time for decaying historical weights or aggregated weights to the current connection event includes: The remaining proportion of historical weights is calculated using a half-life exponential decay function. This half-life exponential decay function satisfies the property of time divisibility. Performing an overall decay on aggregated weights of the same type is equivalent to decaying the individual contribution of each program entity in the same set separately.

[0008] Furthermore, the entity-side detection features include persistent familiarity and intensity familiarity; The sustained familiarity is calculated based on the decay bucket weight of the current connected object in the entity history and the decay effective active bucket weight of the entity in the dimension to which the current object belongs, with a smoothing parameter added during the calculation process. The strength familiarity is calculated based on the weight of decay events of the current connected object in the entity's history, and the total weight of all decaying valid connected events of the entity in the dimension to which the current object belongs, with a smoothing parameter added during the calculation process.

[0009] Furthermore, the calculation of the generality of similar coverage of the current connected object includes: Using program entities as the statistical unit, the sum of the coverage weights of similar entities that have been connected to the current connected object is used as the numerator, and the sum of the weights of similar entities that have generated valid connected objects under the dimension to which the current object belongs is used as the denominator to calculate the generality of similar coverage. The maximum weight of a single program entity over a single object is 1.

[0010] Furthermore, the generated object coverage correction increment and the correction increment of similar valid entities include: Remove the remaining historical contribution of the current program entity to the current connected object and the remaining historical activity contribution of the current program entity to the same type of set from the corresponding same type of aggregate weights, respectively. Write the unit contribution of the current program entity to the current connected object and the unit activity contribution of the current program entity to the same type of set into the corresponding same type of aggregate weight respectively; The object coverage correction increment is the difference between the unit contribution and the object's historical remaining contribution, and the same type of effective entity correction increment is the difference between the unit activity contribution and the historical activity remaining contribution.

[0011] Furthermore, the entity-side detection features also include entity behavior features, which include connection activity strength, activity persistence rate, active bucket connection density, and object effective diversity. In the anomaly detection step, the detection rules corresponding to different object relationships are adjusted according to the entity behavior characteristics.

[0012] Furthermore, the anomaly detection step is preceded by: The entity baseline maturity is determined based on the entity observation duration, the weight of historical effective events, and the weight of active buckets; the baseline maturity of the same type is determined based on the weight of the same type of effective entities and the proportion of effective objects. The corresponding detection path is selected based on the combination of the entity's baseline maturity and the maturity of similar baselines. If the baseline does not meet the maturity conditions, no high-confidence abnormal results will be output.

[0013] Furthermore, it also includes baseline maintenance steps: Set a remaining contribution threshold and a quiet period for the object state of each object dimension. When the remaining contribution of an object is lower than the corresponding threshold and exceeds the quiet period, mark the object state as expired and delete it. When the number of object states exceeds the preset limit, expired objects and low-contribution objects are deleted first. When deleting objects whose remaining contribution is higher than the threshold, the corresponding aggregation weight is adjusted simultaneously, and the baseline truncation state is marked. The confidence of the detection results is adjusted according to the baseline truncation state.

[0014] Secondly, a multi-host connection event anomaly detection system, applied to any of the multi-host connection event anomaly detection methods described above, includes: The event preprocessing module is used to receive program-initiated connection events, preprocess the connection events, determine program entities based on host identifiers and program families, determine a set of similar programs based on host groups and program families, and decompose the connection events into connection objects corresponding to multiple object dimensions. The entity baseline processing module is used to read the entity baseline status and entity-object historical baseline status corresponding to the program entity, and decay the historical weight in the entity baseline status and entity-object historical baseline status to the time of the current connection event. The entity feature calculation module is used to calculate entity-side detection features based on the entity baseline state that has not yet been written to the current connection event and the entity-object historical baseline state. The correction increment generation module is used to calculate the historical remaining contribution of the current program entity that has been included in the same type of baseline state based on the historical connection time in the entity-object historical baseline state and the historical activity time in the entity baseline state, and generate the object coverage correction increment and the correction increment of the same type of effective entity. The baseline processing module is used to read the baseline status of the same entity and the baseline status of the same entity-object coverage of the same program set, and to decay the aggregation weight in the baseline status of the same entity and the baseline status of the same entity-object coverage to the time of the current connection event. The class prevalence calculation module is used to calculate the class coverage prevalence of the current connected object based on the baseline state of class entities that have not yet been written to the current connection event and the coverage baseline state of class entities-objects. The anomaly detection module is used to combine the entity-side detection features with the coverage prevalence of similar features to perform anomaly detection and generate detection results; The baseline update module is used to update the entity baseline status and the entity-object historical baseline status after the detection results are generated, and to update the baseline status of the same type of entity and the coverage baseline status of the same type of entity using the object coverage correction increment and the correction increment of the same type of effective entity. The result output module is used to output the detection results and retain the updated four baseline states for subsequent connection event processing.

[0015] The present invention has at least the following beneficial effects: (1) Similar coverage is updated by entity correction, which can prevent the high-frequency repeated connection of a single entity from amplifying the universality of the group; (2) The entity's history and similar groups are used together to make the judgment, which can distinguish between similar public objects and host-specific stable objects; (3) Joint judgment by IP, organization and country at the level can reduce false alarms caused by changes in cloud service address and identify substantial changes in remote targets; (4) Entity features are used to interpret the connection objects rather than simply participating in linear weighting, and can be adapted to fixed connection type, centralized task type and open access type programs; (5) Maturity and object validity control can reduce low-quality alarms caused by cold start and completion failure; (6) A single event only accesses the relevant statistical status of the current entity, the current object, and the current collection of the same type, which is suitable for online processing of high-throughput connection events on multiple hosts. Attached Figure Description

[0016] Figure 1A flowchart of a multi-host connection event anomaly detection method provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are not all embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0018] It should be noted that, unless otherwise specifically stated, the relative arrangement and numerical expressions of the components and steps described in these embodiments should not be construed as limiting the scope of the invention.

[0019] The following description of exemplary embodiments is merely illustrative and is not intended to limit the invention or its application or use in any way. Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail herein, but where applicable, such techniques, methods, and apparatus should be considered part of this specification.

[0020] Example 1 See Figure 1 , Figure 1 This is a flowchart of a multi-host connection event anomaly detection method proposed in this invention, applied to a multi-host cluster terminal security protection scenario. It performs real-time anomaly detection for program-initiated outward connection events. First, it defines the core object and baseline state: Core object definition: Program Entity Defined as:

[0021] in, For host identification, The normalized program family; Collection of similar programs Defined as:

[0022] in, For customers, business groups, device groups, or other comparable sets of hosts; The dimension of the connection object is:

[0023] IP, Organization, and Country represent different semantic levels for the same remote connection target:

[0024] Objects that cannot be reliably identified can be marked as Unknown, but they are not included in the calculation of familiarity, object diversity, and commonality coverage as normal objects.

[0025] Four types of baseline state definitions: Entity baseline status: Overall connectivity statistics for stored program entities, including event weights, active bucket weights, sum of squared object weights, most recent active time, and most recent active bucket; Entity-Object Historical Baseline Status: Stores statistical data on the relationship between entities and specific objects, including object event weights, object occurrence bucket weights, most recent occurrence time, and most recent occurrence bucket; Baseline status of similar entities: Stores the effective entity aggregation weights and update times for similar sets; Same-type entity-object coverage baseline status: Stores the entity coverage aggregation weight and update time of a specific object in the same type of collection.

[0026] The detection method in this embodiment includes the following steps: S1. Event reception and preprocessing: Receive program-initiated connection events, preprocess the connection events, determine program entities based on host identifiers and program families, determine similar program sets based on host groups and program families, and decompose the connection events into connection objects corresponding to multiple object dimensions. The event timestamps are standardized in a uniform format and divided into time buckets according to a preset granularity; program paths, process names, and versions are normalized to obtain a unified program family identifier; IP address formats are standardized, and the organization and country attributes corresponding to the IP are completed; invalid objects are filtered out, and objects whose attributes cannot be resolved are marked as "Unknown" and do not enter the normal baseline process; finally, the current program entity is located based on the host and program family. Locate the set of similar programs based on host group and program family. The current event is broken down into three dimensions: IP, organization, and country, which are the objects to be detected.

[0027] S2, Entity-side baseline reading and time decay: Read the entity baseline state and entity-object historical baseline state corresponding to the program entity, and decay the historical weight in the entity baseline state and entity-object historical baseline state to the time of the current connection event; This embodiment uses a half-life exponential decay function, and the basic decay formula is:

[0028] in, Half-life, This represents the elapsed time or number of time buckets between the current event time and the last status update time. This represents the remaining proportion of the historical weights after this time interval.

[0029] For any historical cumulative weight Its decay to the current event time The weights of those not yet written to the current event are:

[0030] in, This is the time when this status was last updated. This indicates the historical weight before the update. After the current event is identified, the corresponding new value or overwrite correction increment for the current event is written:

[0031] in, To write the new weight after the current event, This refers to the incremental impact of this event. In this article, Indicates the weight associated with the current object. Represents the overall reference weight of an entity or a collection of similar entities.

[0032] For the sum of squared weights of object events:

[0033] in, This refers to all historical objects belonging to the same dimension as the current object. Because the weights of each object are based on... Decay, the sum of squares decays by its square:

[0034] Different statistics can be configured with separate half-lives for sustained familiarity, intensity familiarity, and similar coverage. This invention is not limited to exponential decay; other time-weighting functions capable of calculating historical residual weights based on time intervals can also be used.

[0035] S3. Calculation of entity-side detection features before update: Based on the entity baseline state that has not yet been written to the current connection event and the entity-object historical baseline state, the entity-side detection features are calculated; specifically including: Continuous Familiarity: Calculated based on the decay bucket weight of the current connected object in the entity's history, and the decay effective active bucket weight of the entity in the dimension to which the current object belongs, with a smoothing parameter added during the calculation; the specific formula is:

[0036] in, For entities For objects Continued familiarity; For object In entity The decay weight appears in the historical active bucket; the weight is increased by 1 when an object first appears in a bucket, and repeated connections in the bucket are not repeated. For entities Decaying effective active bucket weight under the dimension to which the current object belongs; incremented by 1 when an entity first generates a valid object of this type within a time bucket; , The smoothing parameters represent the prior weights for the appearance and absence of objects, respectively, to avoid the denominator being zero and to reduce the problem of excessively extreme score values ​​when the sample size is small.

[0037] High sustained familiarity indicates that the object appears consistently over a long period of time; low sustained familiarity indicates that the object appears for the first time, has not appeared for a long time, or only appears in a small number of time buckets.

[0038] Strength Familiarity: Calculated based on the weight of decaying events of the current connected object in the entity's history, and the total weight of all decaying valid connected events of the entity in the dimension to which the current object belongs, with a smoothing parameter added during the calculation; the specific formula is: in, For entities For objects Familiarity with the intensity; For entities Connection object The decaying event weight, each pointing to an object The number of valid events increases by 1; For entities The total weight of all valid connection events in the dimension to which the current object belongs is reduced by 1 for each valid event pointing to this dimension of the entity. , The meaning is the same as the smoothing parameter in continuous familiarity.

[0039] When both persistent familiarity and strong familiarity are used together: when both are low, the object is close to being a true stranger; when persistent familiarity is low and strong familiarity is high, it indicates that the object has appeared in a concentrated manner recently, which is a short-term burst connection rather than a long-term stable object.

[0040] Entity-side detection features also include entity behavior features, which include connection activity strength, activity persistence rate, active bucket connection density, and object effective diversity. Connection activity strength: represents the entity The frequency of recent connection events is expressed as follows:

[0041] in, This represents the recent connection activity strength of the entity; The decay weight is applied to all connection events of an entity, increasing by 1 for each connection event, without duplicate increases for the three object levels: IP, organization, and country. The conversion factor from weight to connection frequency can be taken as follows when using exponential decay:

[0042] in, This is the half-life of the connection event weights. If... In terms of days, then This can be roughly interpreted as the number of recent connections per day.

[0043] Active Duration Rate: Represents the entity's activity duration. Whether it continues to appear in the elapsed time bucket, the specific expression is:

[0044] in, For entity activity persistence rate; The decaying active bucket weight is assigned to all connection activities of an entity, and the weight within the bucket increases by only 1 at the same time. This represents the theoretical maximum weight an entity can achieve from its first appearance in the bucket until it remains active in the current bucket.

[0045] Let the current time bucket be The first physical appearance of the bucket is The interval between the two is ,but:

[0046] in, The half-life of the active bucket weight, This indicates the number of buckets from the current bucket to the previous bucket.

[0047] Active bucket connection density: represents entities The average number of connections generated within each active bucket is expressed as follows:

[0048] in: For active bucket connection density; The decay weight is applied to all connection events of an entity, increasing by 1 for each connection event, without duplicate increases for the three object levels: IP, organization, and country. The decaying active bucket weight is assigned to all connection activities of an entity, and the weight within the bucket increases by only 1 at the same time. To prevent small constants with a denominator of zero.

[0049] Object effective diversity: Indicates whether entity connection activities are concentrated or dispersed among multiple objects in the dimension to which the current object belongs. The specific expression is:

[0050] in, Effective diversity of objects in the dimension to which the current object belongs; For entities The total weight of all valid connected events in this dimension is reduced by 1 for each valid event pointing to this dimension for the entity. This is the sum of squares of the decay event weights for each object within this dimension; The sum of squares of the decay event weights for each object in this dimension is defined as follows:

[0051] in,- For entities Connection object The decaying event weight, each pointing to an object The number of valid events increases by 1.

[0052] Low diversity indicates that connections are concentrated in a few objects, while high diversity indicates that connections are more evenly distributed across multiple objects. Let the current object... The old weights, decayed to the current time before the event was written, are:

[0053] After the current event is written, the object weight is changed from Become The sum of squares can be updated incrementally using the following formula:

[0054] Therefore, diversity can be updated simply by accessing the current entity and the current object; it is not necessary to traverse all historical objects connected to that entity. When the weight of valid events of this type is low, diversity is only used as a low-confidence feature and is not used alone to determine the entity's behavioral pattern.

[0055] S4. Calculation of Remaining Historical Contribution and Generation of Correction Increment: Based on the historical connection time in the entity-object historical baseline state and the historical activity time in the entity baseline state, calculate the remaining historical contribution of the current program entity that has been included in the same type of baseline state, and generate the object coverage correction increment and the correction increment of the same type of effective entity; specifically including: Remove the remaining historical contribution of the current program entity to the current connected object and the remaining historical activity contribution of the current program entity to the same type of set from the corresponding same type of aggregate weights, respectively. Write the unit contribution of the current program entity to the current connected object and the unit activity contribution of the current program entity to the same type of set into the corresponding same type of aggregate weight respectively; The object coverage correction increment is the difference between the unit contribution and the object's historical remaining contribution, while the correction increment for similar effective entities is the difference between the unit activity contribution and the historical activity remaining contribution.

[0056] S5. Same-side baseline reading and time decay: Read the same-side entity baseline status and the same-side entity-object coverage baseline status corresponding to the same-side program set, and decay the aggregation weight in the same-side entity baseline status and the same-side entity-object coverage baseline status to the time of the current connection event; S6. Calculation of the universality of similar coverage before update: Calculate the universality of the current connected object based on the baseline state of similar entities that have not yet been written to the current connection event and the baseline state of similar entity-object coverage. Using program entities as the statistical unit, the sum of the coverage weights of similar entities that have been connected to the current connected object is used as the numerator, and the sum of the weights of similar entities that have generated valid connected objects under the dimension to which the current object belongs is used as the denominator. The generality of similar coverage is calculated using the following formula:

[0057] in, A collection of similar program entities that share the same machine group and program family; For object In the same set The coverage of the content; Recently connected objects before writing to the current event The sum of the coverage weights of entities of the same type; Before the current event is written, it is the sum of the weights of similar entities that have recently produced valid objects under the type of the current object.

[0058] Object coverage weights are derived from the objects of each entity in the same collection. The sum of historical effective contributions:

[0059] in, For entities At any moment For objects Historical coverage contribution. The entity's most recently connected object. After that, its coverage contribution was set to 1 and gradually decayed over time.

[0060] Accordingly, the weights of valid entities of the same type are derived from the sum of the historical valid activity contributions of each entity in the dimension to which the current object belongs:

[0061] in, For entities At any moment Historical contribution to the total number of valid entities of the same type. After an entity most recently generates valid connected objects, its valid activity contribution is set to 1 and gradually decays over time.

[0062] entity Most recent connection object The time is The time when a valid object for the current dimension was last generated was ; Before the current event is written, the historical contribution of this object to the current entity is:

[0063] The entity's historical contributions are:

[0064] Before calculating the historical contributions mentioned above, the aggregated state is first decayed to the current event time. This is because exponential decay satisfies:

[0065] Therefore, performing an overall decay on the aggregate weight is equivalent to decaying the contribution of each entity to the current time, without having to traverse all entities of the same type.

[0066] The entity's old contribution is already included in the decayed aggregate state. After this event, simply replace the entity's old contribution with 1, that is, only supplement the difference between the old contribution and 1:

[0067]

[0068] The updated status is as follows:

[0069]

[0070] The above update is equivalent to removing the old contribution of the current entity from the aggregate state and then writing the new unit contribution, but without having to re-traverse and sum the similar sets. When the same entity repeatedly connects to the same object within a short period of time, the historical contribution is close to 1, and the current increment is close to 0; when it is the first connection or has not been connected for a long time, the increment is larger. This avoids amplifying the similarity of objects by frequently connecting a single entity repeatedly.

[0071] Both the numerator and denominator use program entities as the statistical unit, not the number of connection events. No matter how many times a single entity repeatedly connects to an object within a short period, its effective contribution to the object coverage weight will not exceed 1.

[0072] S7. Multi-dimensional Joint Anomaly Detection: Anomaly detection is performed by combining the entity-side detection features with the general coverage of similar entities, generating detection results. In the anomaly detection step, the discrimination rules corresponding to different object relationships are adjusted based on the entity's behavioral characteristics. Specifically, this includes: S71. Determine the entity baseline maturity based on entity observation duration, historical valid event weights, and active bucket weights; determine the similar baseline maturity based on the weights of similar valid entities and the proportion of valid objects; select the corresponding detection path based on the combination of entity baseline maturity and similar baseline maturity; use all features for double maturity, prioritize mature-side features for single maturity, and output the observation results for double immaturity; do not output high-confidence abnormal results when the baseline has not reached maturity conditions; see Table 1 for details: Table 1

[0073] S72. Object State Recognition: Based on continuous familiarity and strong familiarity, the current object is identified as a stable object, a truly unfamiliar object, or a short-term concentrated object.

[0074] S73. Classification of Dual-Referential Relationships: Combining familiar relationships between entities and general relationships of the same type, connections are divided into four categories:

[0075] S74. Joint identification of three-layer objects: Familiarity and prevalence are calculated separately for the three dimensions of IP, organization, and country, and the object change type is determined by combination:

[0076] S75. Behavioral Pattern Risk Correction: Adjusting risk levels based on entity behavioral patterns. For fixed-connection programs with high persistence and low diversity, the risk level is increased and marked as a fixed-connection pattern breakthrough when new organizations / countries with unfamiliar entities and rare similarities appear. A high-persistence, low-density heartbeat / reporting program that directly marks changes in connected objects as a fixed-pattern breach. Low-duration, high-density timed task programs are marked as short-term clustered connections when the short-term strength familiarity of new objects increases. Highly diverse open access programs reduce risk levels only when the IP address is unfamiliar, and rely more on organization, country, and similar coverage results.

[0077] S76. Data quality verification: Check the validity of objects, attribute completion status, late events, and status deletion markers; mark data quality as abnormal when the proportion of Unknown objects is too high or key information is missing.

[0078] S77. Anomaly Types and Risk Levels Output: The final output includes 15 types of structured anomaly results, including common stable connections, first access of similar public objects, host-specific stable connections, dual unfamiliar connections, IP changes under familiar organizations, connections to new organizations in known regions, cross-level unfamiliar connections, short-term concentrated connections, breakthroughs in fixed connection patterns, sudden changes in connection behavior, insufficient baselines to be observed, object completion / data quality anomalies, baseline truncation to be reviewed, high-cardinality object connections, and limited object state capacity. Among them, the sustained familiarity threshold, the intensity familiarity threshold, and the similar coverage prevalence threshold are respectively denoted as: , and The minimum support level for similar valid entities is denoted as Each threshold can be configured separately based on the program family, machine group, and object type.

[0079] Common stable connection: The entity's continuous familiarity is not less than And the coverage of similar types is no less than The weight of similar effective entities is not less than ; When similar public objects first enter the current entity: both the entity's sustained familiarity and intensity familiarity are below the corresponding thresholds, and the general coverage of similar objects is not less than [a certain threshold]. Furthermore, the current entity does not have any history records or baseline truncation markers for this object; Host-specific stable connection: Entity's continuous familiarity is no less than However, the coverage of similar products is less widespread than that of other products. ; Double unfamiliarity connection: Entity's sustained familiarity is lower than Strength familiarity is lower than Furthermore, the coverage prevalence in mature, similar baselines is lower than that of other baselines. ; Familiarity with IP changes within the organization: The IP object’s sustained familiarity and intensity familiarity are both below the corresponding threshold, but at least one of its organization’s sustained familiarity, intensity familiarity, or similar coverage reaches the corresponding threshold; New organizational connections within a known geographic region: The entity familiarity of both the IP and the organization is below the corresponding threshold, but the entity familiarity of the country or the general coverage of similar entities reaches the corresponding threshold. Cross-level unfamiliar connections: The familiarity of entities at the IP, organization, and country levels is below the corresponding threshold, and the general coverage of similar entities at all three levels is below [a certain threshold]. ; Short-term clustered connections: object familiarity is lower than Strength familiarity not less than Furthermore, the weight of recent effective events has reached the minimum support level; Breakthrough in fixed connection mode: Mature entity baseline, object diversity not exceeding the low diversity threshold. Furthermore, the newly emerging organizations or countries simultaneously meet the conditions of being both entities that are unfamiliar and those that are rare in the same category. Abrupt connection behavior: The entity's historical connection density is not higher than the low density threshold. However, the connection strength or connection density within the current observation period has reached a high threshold. or ; Baseline Insufficient to Observe: The entity observation duration, historical effective event weight, or active bucket weight have not reached the entity maturity threshold, and the weight of similar effective entities has not reached the similar maturity threshold. Object completion or data quality anomaly: The proportion of invalid or unknown objects within a preset time range is not less than the data anomaly threshold. Or, the key object information of the current event cannot be reliably obtained; Baseline truncation pending review: The current entity or similar set has a baseline truncation marker, or non-low-contribution objects have been deleted due to quantity restrictions within a preset time range; High cardinality object connections: The number of different objects connected to an entity within a preset statistical period is not less than a high cardinality threshold. Or the rate at which different objects are added is not lower than the threshold. ; Object state capacity is limited: The number of states stored by the current entity or collection of the same type under the corresponding object type has reached the maximum limit. or .

[0080] When an entity or similar baseline does not meet the maturity conditions, a high-confidence anomaly should be output based on low familiarity or low coverage prevalence. When there is baseline truncation, status deletion, or capacity limitation, the corresponding status flag should be output and the discrimination confidence should be reduced.

[0081] Risk levels are categorized as follows: Low risk / Normal (commonly stable, host-specific stable, similar public objects supported by the organizational level); Medium risk (familiar with the country but unfamiliar with the entity / similar entity, short-term concentrated new objects, change of ownership); High risk (rare new organizations / countries appear in mature fixed-connection programs, double unfamiliarity at all three levels); Uncertain (insufficient baseline, invalid object, state truncation affects judgment and outputs low confidence results, does not trigger high confidence alarms).

[0082] Baseline truncation, object deletion, and capacity limitation do not increase the risk level themselves, but are only used to reduce the confidence of the results; the thresholds can be configured independently according to the program family, host group, object dimension, and business risk, or they can be implemented by the classification model.

[0083] By combining the entity-side detection features with the coverage prevalence of similar features, anomaly detection is performed to generate detection results: S8. Baseline update and maintenance after discrimination: After the detection result is generated, update the entity baseline status and the entity-object historical baseline status, and update the same type of entity baseline status and the same type of entity-object coverage baseline status using the object coverage correction increment and the same type of effective entity correction increment. After the detection results are generated, a baseline update is performed. All update operations are idempotent, and the correction and deletion of the same event are performed only once to avoid duplicate deduction of contributions. S81, Regular Update: Write the current event contribution to the two types of baselines on the entity side, and update the weight and timestamp; write the two types of correction increments to the two types of baselines on the same side; the same physical connection event only increases the overall event weight of the entity once, and the three object dimensions update their respective states; Unknown and invalid objects are not written to the normal baseline; the number of objects is maintained by independent counting state increments, without relying on traversing all objects; S82. Lazy Expiration and Capacity Control: To limit the size of the online baseline state, an upper limit is set for the number of object states for each program entity and each object dimension, and a lazy expiration mechanism based on the remaining contribution is adopted.

[0084] When an object's state is accessed or quantitative cleanup is performed, its persistence weight, intensity weight, and similar coverage contribution are first decayed to the current time. Let the object... The current remaining contributions are as follows:

[0085]

[0086]

[0087] When the remaining contributions of the three categories are all below their corresponding ignore thresholds, and the object has exceeded the preset silent time, the object's state is marked as logically expired and deleted:

[0088] Maximum retention limits are set for the three object dimensions: IP, organization, and country. When the number of object states exceeds the limit, objects that have logically expired are deleted first; if the limit is still exceeded, objects with low continuous contribution, low intensity contribution, and low coverage contribution of the same type are deleted first.

[0089] When deleting an object, if its remaining contribution is below the ignore threshold, its impact on relevant metrics can be directly ignored; if the object deleted due to quantity limitations still has a non-negligible contribution, the relevant aggregation state is adjusted synchronously.

[0090]

[0091]

[0092] in, The total weight of valid events in the current object dimension. This represents the sum of squared weights of object events within this dimension. For object The weight of the same type of entity coverage.

[0093] Deleting the state of a single object does not reduce the overall event weight of the entity. Overall Active Bucket Weight of Entities and similar effective entity weights Only when the entire entity's valid activity state in the current dimension is deleted will it be removed from the current dimension. The activity contribution of the entity that has not yet decayed is deducted.

[0094] When objects that still have a non-negligible contribution are deleted due to quantity limitations, a baseline truncation marker is recorded, and the baseline confidence of the corresponding detection results is reduced.

[0095] S9. Result Output and State Retention: Output the detection results and retain the updated four baseline states for subsequent connection event processing; The system outputs structured detection results, including connection relationship type, anomaly type, risk level, triggering basis, baseline confidence, and status flags. The updated four baseline types are persistently stored for subsequent event processing. A single event only needs to access the statistical status of the current entity, current object, and current set of similar entities, without needing to traverse all historical events, historical objects, or all entities of the same type. This supports high-throughput, multi-host connection event online real-time processing.

[0096] Example 2 The present invention proposes a multi-host connection event anomaly detection system, which specifically includes: The event preprocessing module is used to receive program-initiated connection events, preprocess the connection events, determine program entities based on host identifiers and program families, determine a set of similar programs based on host groups and program families, and decompose the connection events into connection objects corresponding to multiple object dimensions. The entity baseline processing module is used to read the entity baseline status and entity-object historical baseline status corresponding to the program entity, and decay the historical weight in the entity baseline status and entity-object historical baseline status to the time of the current connection event. The entity feature calculation module is used to calculate entity-side detection features based on the entity baseline state that has not yet been written to the current connection event and the entity-object historical baseline state. The correction increment generation module is used to calculate the historical remaining contribution of the current program entity that has been included in the same type of baseline state based on the historical connection time in the entity-object historical baseline state and the historical activity time in the entity baseline state, and generate the object coverage correction increment and the correction increment of the same type of effective entity. The baseline processing module is used to read the baseline status of the same entity and the baseline status of the same entity-object coverage of the same program set, and to decay the aggregation weight in the baseline status of the same entity and the baseline status of the same entity-object coverage to the time of the current connection event. The class prevalence calculation module is used to calculate the class coverage prevalence of the current connected object based on the baseline state of class entities that have not yet been written to the current connection event and the coverage baseline state of class entities-objects. The anomaly detection module is used to combine the entity-side detection features with the coverage prevalence of similar features to perform anomaly detection and generate detection results; The baseline update module is used to update the entity baseline status and the entity-object historical baseline status after the detection results are generated, and to update the baseline status of the same type of entity and the coverage baseline status of the same type of entity using the object coverage correction increment and the correction increment of the same type of effective entity. The result output module is used to output the detection results and retain the updated four baseline states for subsequent connection event processing.

[0097] Example 3 This embodiment further illustrates the specific implementation process and effects of this solution by combining three actual deployment scenarios: Scenario 1: Detection of high-frequency repeated connections on a single host A company's business machine group deploys 100 hosts with identical configurations, running K-type business programs, forming 100 similar program entities. During the monitoring period, only the K-type program entity on host A initiates connections to remote IP-X, and generates 10,000 connection events pointing to IP-X within a short period. This solution handles this according to the following process: When the first connection event from host A to IP-X is received, the entity’s historical coverage contribution to IP-X is 0, the calculated object coverage correction increment is about 1, and the coverage weight of similar objects increases by 1 accordingly. When a repeated connection event to IP-X is subsequently received from the entity, the time interval between the entity's most recent connection to IP-X and the current time interval is extremely short. The historical coverage contribution is close to 1 after decay, and the calculated object coverage correction increment is close to 0. The same type of coverage weight is not repeatedly accumulated. Ultimately, the coverage prevalence of IP-X remains low, indicating that it is a host-specific connection and will not be mistakenly identified as a common object of similar programs due to the high frequency of tens of thousands of connections per host.

[0098] Scenario 2: Cloud Service IP Drift Detection An office application running on an office terminal has a long history of stable connections to the cloud service address corresponding to organization O1. Both the entity baseline and similar baselines store records of high familiarity with organization O1. Currently, the application is detected initiating a connection to a completely new IP address. After attribute completion, it is confirmed that the IP address still belongs to organization O1, and has a stable historical support at the country level. This solution will handle this according to the following process: Familiarity was calculated for three dimensions: IP, organization, and country. If both the sustained familiarity and intensity familiarity of the IP were below the threshold, it was determined to be unfamiliar with the IP. If the sustained familiarity of the organization was above the threshold, it was determined to be familiar with the organization. Familiarity and commonality of the country both met the requirements. Based on the high diversity of objects and the fact that this office application is an open access application, the risk level has been adjusted accordingly. The final output is "IP address changes under familiar organization", which is not judged as a new abnormal target across levels, thus avoiding false alarms caused by IP drift of cloud service nodes.

[0099] Scenario 3: Fixed connection mode anomaly bypassing detection A status reporting program running on a server exhibits long-term baseline characteristics of high activity persistence, low connection density, and low object diversity. Historically, it has only connected to addresses of two fixed business organizations, classifying it as a typical fixed-connection program. Currently, the program is detected initiating a connection to a completely new address. Attribute completion confirms that the IP address, its associated organization, and its country have no historical support in either the entity baseline or similar baselines. This solution will handle this according to the following process: Calculate entity familiarity and similarity prevalence: If the entity familiarity in all three dimensions is below the threshold and the similarity coverage prevalence is also below the threshold, it is determined to be a double unfamiliar relationship; Based on the program's behavior characteristics of a low-diversity fixed connection mode, the risk level is correspondingly increased. The final output identifies high-risk results for "cross-level unfamiliar connections" and "breakthrough of fixed connection patterns," accurately identifying abnormal external connection behaviors of fixed business procedures.

[0100] The above specific embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to examples, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting anomalies in multi-host connection events, characterized in that, include: S1. Receive program-initiated connection event, preprocess the connection event, determine the program entity according to the host identifier and program family, determine the set of similar programs according to the host group and program family, and decompose the connection event into connection objects corresponding to multiple object dimensions; S2. Read the entity baseline state and entity-object historical baseline state corresponding to the program entity, and decay the historical weight in the entity baseline state and entity-object historical baseline state to the time of the current connection event. S3. Calculate the entity-side detection features based on the entity baseline state that has not yet been written to the current connection event and the entity-object historical baseline state. S4. Based on the historical connection time in the entity-object historical baseline state and the historical activity time in the entity baseline state, calculate the historical remaining contribution of the current program entity that has been included in the same type of baseline state, and generate the object coverage correction increment and the same type of effective entity correction increment. S5. Read the baseline status of similar entities and the baseline status of similar entities-objects corresponding to the similar program set, and decay the aggregation weight in the baseline status of similar entities and the baseline status of similar entities-objects to the time of the current connection event. S6. Based on the baseline state of similar entities that have not yet been written to the current connection event and the baseline state of similar entity-object coverage, calculate the similar coverage universality of the current connection object; S7. Combine the entity-side detection features with the coverage prevalence of similar objects to perform anomaly detection and generate detection results; S8. After the detection results are generated, update the entity baseline status and the entity-object historical baseline status, and update the same type of entity baseline status and the same type of entity-object coverage baseline status using the object coverage correction increment and the same type of effective entity correction increment. S9. Output the detection results and retain the updated four baseline states for subsequent connection event processing.

2. The multi-host connection event anomaly detection method according to claim 1, characterized in that, The step of decomposing connection events into connection objects corresponding to multiple object dimensions includes: The connection event is decomposed into connection objects with three independent object dimensions: IP object, organization object, and country object. For each object dimension, maintain the corresponding entity-object historical baseline state and the same type of entity-object coverage baseline state.

3. The multi-host connection event anomaly detection method according to claim 1, characterized in that, The time for decaying historical weights or aggregated weights to the current connection event includes: The remaining proportion of historical weights is calculated using a half-life exponential decay function. This half-life exponential decay function satisfies the property of time divisibility. Performing an overall decay on aggregated weights of the same type is equivalent to decaying the individual contribution of each program entity in the same set separately.

4. The multi-host connection event anomaly detection method according to claim 1, characterized in that, The entity-side detection features include persistent familiarity and intense familiarity; The sustained familiarity is calculated based on the decay bucket weight of the current connected object in the entity history and the decay effective active bucket weight of the entity in the dimension to which the current object belongs, with a smoothing parameter added during the calculation process. The strength familiarity is calculated based on the weight of decay events of the current connected object in the entity's history, and the total weight of all decaying valid connected events of the entity in the dimension to which the current object belongs, with a smoothing parameter added during the calculation process.

5. The multi-host connection event anomaly detection method according to claim 1, characterized in that, The calculation of the generality of similar coverage of the current connected object includes: Using program entities as the statistical unit, the sum of the coverage weights of similar entities that have been connected to the current connected object is used as the numerator, and the sum of the weights of similar entities that have generated valid connected objects under the dimension to which the current object belongs is used as the denominator to calculate the generality of similar coverage. The maximum weight of a single program entity over a single object is 1.

6. The multi-host connection event anomaly detection method according to claim 5, characterized in that, The generated object coverage correction increment and the correction increment of similar effective entities include: Remove the remaining historical contribution of the current program entity to the current connected object and the remaining historical activity contribution of the current program entity to the same type of set from the corresponding same type of aggregate weights, respectively. Write the unit contribution of the current program entity to the current connected object and the unit activity contribution of the current program entity to the same type of set into the corresponding same type of aggregate weight respectively; The object coverage correction increment is the difference between the unit contribution and the object's historical remaining contribution, and the same type of effective entity correction increment is the difference between the unit activity contribution and the historical activity remaining contribution.

7. The multi-host connection event anomaly detection method according to claim 1, characterized in that, The entity-side detection features also include entity behavior features, which include connection activity strength, activity persistence rate, active bucket connection density, and object effective diversity. In the anomaly detection step, the detection rules corresponding to different object relationships are adjusted according to the entity behavior characteristics.

8. The multi-host connection event anomaly detection method according to claim 1, characterized in that, The anomaly detection step is preceded by: The entity baseline maturity is determined based on the entity observation duration, the weight of historical effective events, and the weight of active buckets; the baseline maturity of the same type is determined based on the weight of the same type of effective entities and the proportion of effective objects. The corresponding detection path is selected based on the combination of the entity's baseline maturity and the maturity of similar baselines. If the baseline does not meet the maturity conditions, no high-confidence abnormal results will be output.

9. The method for detecting anomalies in multi-host connection events according to claim 1, characterized in that, It also includes baseline maintenance steps: Set a remaining contribution threshold and a quiet period for the object state of each object dimension. When the remaining contribution of an object is lower than the corresponding threshold and exceeds the quiet period, mark the object state as expired and delete it. When the number of object states exceeds the preset limit, expired objects and low-contribution objects will be deleted first. When deleting objects whose remaining contribution is higher than the threshold, the corresponding aggregation weights are simultaneously corrected, and the baseline truncation state is marked. The confidence of the detection results is adjusted according to the baseline truncation state.

10. A multi-host connection event anomaly detection system, characterized in that, The method for detecting anomalies in multi-host connection events according to any one of claims 1-9 includes: The event preprocessing module is used to receive program-initiated connection events, preprocess the connection events, determine program entities based on host identifiers and program families, determine a set of similar programs based on host groups and program families, and decompose the connection events into connection objects corresponding to multiple object dimensions. The entity baseline processing module is used to read the entity baseline status and entity-object historical baseline status corresponding to the program entity, and decay the historical weight in the entity baseline status and entity-object historical baseline status to the time of the current connection event. The entity feature calculation module is used to calculate entity-side detection features based on the entity baseline state that has not yet been written to the current connection event and the entity-object historical baseline state. The correction increment generation module is used to calculate the historical remaining contribution of the current program entity that has been included in the same type of baseline state based on the historical connection time in the entity-object historical baseline state and the historical activity time in the entity baseline state, and generate the object coverage correction increment and the correction increment of the same type of effective entity. The baseline processing module is used to read the baseline status of the same entity and the baseline status of the same entity-object coverage of the same program set, and to decay the aggregation weight in the baseline status of the same entity and the baseline status of the same entity-object coverage to the time of the current connection event. The class prevalence calculation module is used to calculate the class coverage prevalence of the current connected object based on the baseline state of class entities that have not yet been written to the current connection event and the coverage baseline state of class entities-objects. The anomaly detection module is used to combine the entity-side detection features with the coverage prevalence of similar features to perform anomaly detection and generate detection results; The baseline update module is used to update the entity baseline status and the entity-object historical baseline status after the detection results are generated, and to update the baseline status of the same type of entity and the coverage baseline status of the same type of entity using the object coverage correction increment and the correction increment of the same type of effective entity. The result output module is used to output the detection results and retain the updated four baseline states for subsequent connection event processing.

Images