Network security intelligent detection method based on big data

By constructing a network entity behavior gene map and a dynamic baseline model, and combining negative and positive feedback mechanisms, the problems of single analysis dimensions and environmental adaptability of existing network security detection methods are solved. This enables efficient identification of unknown and internal threats, reduces false alarm rates, and improves detection accuracy and adaptability.

CN121486056APending Publication Date: 2026-02-06BEIJING JINBO SHUNCHANG NETWORK TECHNOLOGY CO LTD

Patent Information

Application Number
CN202511762533.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-11-27
Publication Date
2026-02-06

AI Technical Summary

Technical Problem

Existing network security detection methods have a single analytical dimension and lack multi-dimensional dynamic quantification. They cannot adapt to changes in the network environment and cannot learn from false positives and confirmed positives to achieve self-evolution, resulting in a high false positive rate and an inability to effectively identify unknown threats.

Method used

Collect multi-source heterogeneous security data, construct a behavioral gene map with network entities as the core, align behavioral events through dynamic time warping algorithm, establish a dynamic behavioral baseline model, compare deviation in real time and optimize detection through unsupervised learning, and adjust weights and thresholds using negative and positive feedback mechanisms to achieve adaptive detection.

Benefits of technology

It achieves accurate identification of unknown and internal threats, reduces false alarm rate, improves detection accuracy, and builds an adaptive security protection system with self-optimization capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121486056A_ABST
    Figure CN121486056A_ABST
Patent Text Reader

Abstract

The invention relates to the field of data security, in particular to a network security intelligent detection method based on big data, and the method comprises the steps: collecting network flow data, a terminal system call sequence and a user operation behavior log, and carrying out the data fusion processing to generate a unified behavior event flow; selecting a key behavior event based on an information entropy threshold value, performing time alignment through a dynamic time warping algorithm, and constructing a behavior gene map containing a communication association gene, an operation sequence gene and a behavior time sequence gene; a dynamic behavior baseline model is established by using unsupervised learning, and gene mutation detection and alarm are realized by calculating the deviation degree of each gene dimension; the detection performance is evaluated based on the false alarm rate, model parameters are optimized through a negative feedback mechanism, and acknowledged attack features are stored in a sharable threat gene feature library through a positive feedback mechanism. According to the method, the detection accuracy is continuously improved through a closed-loop learning mechanism, and a self-adaptive safety protection system with self-optimization capability is constructed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of data security, and in particular to a network security intelligent detection method based on big data. Background Technology

[0002] Traditional cybersecurity detection methods primarily rely on signature-based matching and rule-based detection. While effective against known attacks, these methods struggle to address unknown threats such as zero-day vulnerabilities, advanced persistent threats (APTs), and malicious insider operations. Furthermore, traditional systems are static and inflexible, resulting in high false alarm rates, leading to alert fatigue among security analysts, and they cannot learn and optimize from operational practices.

[0003] While some existing technologies incorporate User Behavior Analysis (UEBA), most of them rely on a single analysis dimension and lack multi-dimensional and dynamic quantification of the essence of behavior. Furthermore, these systems are typically open-loop systems, where the detection model remains fixed once deployed, making it unable to adapt to normal changes in the network environment or learn from false positives and confirmed positives to achieve self-evolution.

[0004] Patent document CN111291015A discloses a method for detecting abnormal user behavior. This method involves first retrieving the user's current-time user log from a server. This log represents user behavior data and may include transaction logs, login logs, etc. Next, a set of behavioral features related to the user's behavior is extracted from the user log. This set of behavioral features is then input into a pre-trained user behavior vector representation model to obtain a behavior vector representation, which includes the user's behavior classification result from the previous time step. Finally, the behavior vector representation is input into a behavior classification model to obtain the classification result corresponding to the user log. However, this method cannot learn from the massive amounts of feedback information generated in daily operations, cannot automatically reduce the false positive rate, and cannot solidify newly discovered hacker attack techniques into new detection knowledge.

[0005] Therefore, there is an urgent need for an intelligent security detection method that can accurately characterize entity behavior, adapt to environmental changes, and have continuous learning capabilities. Summary of the Invention

[0006] To address these issues, this invention provides a big data-based intelligent network security detection method to overcome the problems of existing technologies, such as limited analytical dimensions, lack of multi-dimensional and dynamic quantification of behavioral essence, fixed deployment, inability to adapt to normal changes in the network environment, and inability to learn from false alarms and confirmed alarms to achieve self-evolution.

[0007] To achieve the above objectives, this invention provides a big data-based intelligent network security detection method, which includes: Collect multi-source heterogeneous security data from the protected network environment, including at least network traffic data, terminal system call sequences, and user operation behavior logs. Then, clean, standardize, and correlate and fuse the multi-source heterogeneous security data to generate a unified behavior event stream with network entities as the core dimension. Key behavioral events are selected from the unified behavioral event stream based on the information entropy threshold, and behavioral events from different sources are aligned in time based on the dynamic time warping algorithm to construct a behavioral gene map consisting of multiple behavioral gene dimensions for each network entity. Using the behavioral gene map of the network entity during its historical normal operation cycle, a dynamic behavioral baseline model is established for each network entity; the current behavioral gene map is compared with the corresponding behavioral baseline model in real time, and its deviation in each behavioral gene dimension is calculated. When any deviation exceeds the set adaptive threshold, a gene mutation event is determined to have occurred and an initial screening security alarm is generated. The system collects feedback results generated after responding to the initial screening safety alarms, including confirmed alarms and false alarms; and evaluates whether the current detection capability is qualified based on the ratio of initial screening safety alarms judged as false alarms per unit time. When the evaluation is deemed unqualified, a negative feedback optimization mechanism is activated, and the following actions are performed: the weight of the behavioral gene dimension associated with recently generated false alarm reports is reduced in the baseline model, and its adaptive threshold is increased.

[0008] Furthermore, the correlation and fusion processing of multi-source heterogeneous security data includes: Assign a timestamp, source entity identifier, and behavior type identifier in a uniform format to each piece of behavioral data; Based on the source entity identifier, behavioral data from different data sources are associated, and based on the timestamp and behavior type identifier, a sequence of behavioral events centered on the network entity is constructed.

[0009] Furthermore, the selection of key behavioral events from the unified behavioral event stream based on an information entropy threshold includes: Calculate the information entropy value of a specific behavior type or operation target within a preset time window; The information entropy value is compared with a preset entropy threshold. When the information entropy value exceeds the entropy threshold range, the event within the time window is determined to be a key behavioral event and is extracted.

[0010] Furthermore, the behavioral gene dimension includes at least communication-related genes, operational sequence genes, and behavioral temporal genes; The communication association gene is quantified by at least one of the target address set, communication port distribution, and communication traffic timing pattern of the network entity. The operation sequence gene is quantified by at least one of the following: the sequence pattern of system calls executed by the network entity, the order of file access, and the habitual flow of application usage; the behavior temporal gene is quantified by at least one of the following: the distribution of active periods of the network entity, the statistical characteristics of the time interval of operation events, and the periodicity of behavioral sessions.

[0011] Furthermore, the time alignment of behavioral events from different sources based on the dynamic time warping algorithm includes: Extract event sequences representing the same potential activity from network traffic data and terminal operation behavior logs, respectively; The dynamic time warping algorithm is applied to calculate the optimal curved path between two event sequences to align related events that have offsets or velocity differences on the time axis.

[0012] Furthermore, in the process of establishing the dynamic behavioral baseline model, the unsupervised learning algorithm used is a clustering-based method, a reconstruction-based method, or an isolation-based method; The behavioral baseline model is updated periodically by incorporating new, confirmed normal behavioral gene map data through incremental learning.

[0013] Furthermore, the deviation of the calculated deviation in each behavioral gene dimension is as follows: For each behavioral gene dimension, calculate the statistical distance between the current gene map data and the behavioral baseline model in that dimension; The statistical distance of each dimension is compared with the corresponding adaptive threshold to determine whether a deviation has occurred.

[0014] Furthermore, the initial value of the adaptive threshold is set based on the statistical distribution quantile of historical normal behavior gene map data; The adaptive threshold is also periodically adjusted according to the overall state of the network environment; The overall status includes the overall network load or the switching between weekdays and holidays.

[0015] Furthermore, in the negative feedback optimization mechanism, reducing the weight and increasing the adaptive threshold are performed according to a predefined mathematical formula: The weight update formula is: Wnew = Wold × α, where α is a decay factor between 0 and 1; The threshold update formula is: Tnew = Told × β, where β is an amplification factor greater than 1.

[0016] Furthermore, it also includes: when the evaluation is qualified, a positive feedback optimization mechanism is activated to perform the following operations: extract the behavioral gene mutation pattern corresponding to the security alarm that is determined to be confirmed as an attack gene feature and store it in a shareable threat gene feature library; The attack gene features are stored in the threat gene feature library as a structured feature template; The feature template includes at least: the behavioral gene dimension identifier that triggers the alarm, the deviation vector of each dimension, the associated network entity context information, and the attack mode label; The system performs similarity matching between the detected gene mutation patterns and the feature templates in the feature library. If the similarity exceeds the matching threshold, it directly associates the pattern with known attack patterns to trigger an alert.

[0017] Compared with existing technologies, the advantages of this invention lie in its ability to construct a behavioral gene map centered on network entities by collecting and fusing multi-source data such as network traffic, terminal system calls, and user operation logs. This quantifies entity behavioral characteristics from three dimensions: communication association, operation sequence, and behavioral temporal sequence. A dynamic behavioral baseline model is established based on unsupervised learning, and anomaly detection is achieved by comparing the deviation between the gene map and the baseline in real time. When the deviation exceeds an adaptive threshold, a gene mutation alarm is triggered. The system automatically evaluates detection performance by analyzing the false alarm rate. When performance is unsatisfactory, a negative feedback mechanism is activated to reduce the weight of the false alarm gene dimension and increase the threshold to optimize detection accuracy. When performance is satisfactory, a positive feedback mechanism is activated to extract the gene mutation patterns of confirmed attacks as shareable attack gene features and store them in a threat feature database. This solution realizes a paradigm shift in detection from feature matching to behavioral analysis, effectively identifying both unknown and internal threats, and continuously improving detection accuracy through a closed-loop learning mechanism, ultimately constructing an adaptive security protection system with self-optimization capabilities. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the intelligent network security detection method based on big data, as described in an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating the process of correlation and fusion processing of multi-source heterogeneous security data according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the process of selecting key behavioral events from the unified behavioral event stream based on an information entropy threshold according to an embodiment of the present invention. Figure 4 This is a schematic diagram illustrating the process of aligning the time of behavioral events from different sources based on a dynamic time warping algorithm according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0020] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0021] It should be noted that in the description of this invention, the terms such as up, down, left, right, inside, and outside indicate the direction or positional relationship based on the direction or positional relationship shown in the drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0022] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms installation, connection, and linking should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0023] Please see Figure 1 ,like Figure 1 The diagram shown is a flowchart illustrating a big data-based intelligent network security detection method according to an embodiment of the present invention. The embodiment of the present invention provides a big data-based intelligent network security detection method, comprising: Collect multi-source heterogeneous security data from the protected network environment, including at least network traffic data, terminal system call sequences, and user operation behavior logs. Then, clean, standardize, and correlate and fuse the multi-source heterogeneous security data to generate a unified behavior event stream with network entities as the core dimension. Key behavioral events are selected from the unified behavioral event stream based on the information entropy threshold, and behavioral events from different sources are aligned in time based on the dynamic time warping algorithm. A behavioral gene map consisting of multiple behavioral gene dimensions is then constructed for each network entity. The behavioral gene dimensions include at least communication association genes, operation sequence genes, and behavioral temporal genes. Based on unsupervised learning algorithms, the behavioral gene map of the network entities during the historical normal operation cycle is used to establish a dynamic behavioral baseline model for each network entity; the current behavioral gene map is compared with the corresponding behavioral baseline model in real time, and its deviation in each behavioral gene dimension is calculated. When any deviation exceeds the set adaptive threshold, a gene mutation event is determined to have occurred and an initial screening safety alarm is generated. The system collects feedback results generated after responding to the initial screening safety alarms, including confirmed alarms and false alarms; and evaluates whether the current detection capability is qualified based on the ratio of initial screening safety alarms judged as false alarms per unit time. When the evaluation is unsatisfactory, a negative feedback optimization mechanism is activated to perform the following operations: reduce the weight of the behavioral gene dimension associated with recently generated false alarm reports in the baseline model and increase its adaptive threshold. When the evaluation is qualified, a positive feedback optimization mechanism is activated to perform the following operations: the behavioral gene mutation pattern corresponding to the security alarm that is determined to be confirmed is extracted as an attack gene feature and stored in a shareable threat gene feature library.

[0024] Specifically, this invention collects raw data across the network, including: deploying traffic probes on core network switches to capture network-wide connection logs, recording who accessed the server, when, and how much data was transmitted; installing lightweight proxy programs on all protected servers and terminal computers to continuously record all important system calls, such as process creation, file access, and network connections; aggregating user login and operation records from VPNs, office systems, databases, and other applications using a log collector; and converting all data into a unified JSON format to ensure that key fields such as timestamps and IP addresses adhere to the same standard. This invention requires a roster (such as a CMDB configuration management database) to associate IP addresses, MAC addresses, usernames, and department information. Thus, when the network side detects IP 192.168.1.100 abnormally transmitting data, it can immediately locate the computer belonging to Zhang San in the R&D department and discover that he was accessing the proxy server, thereby piecing together fragmented events into a behavioral chain centered on Zhang San.

[0025] This invention does not simply list behaviors, but extracts essential features to construct a behavioral gene map for each entity. This map mainly contains genes in three dimensions: Communication association genes can describe an entity's network social circle. This embodiment can analyze which IPs / domains it typically communicates with, which ports it uses, and whether its traffic patterns are stable or bursty. If a financial server suddenly starts communicating extensively with an unfamiliar external IP, this gene will send an abnormal signal.

[0026] Operation sequence genes can characterize an entity's operational habits. This embodiment can learn its typical command sequences and file access patterns. For example, a normal administrator operation is to log in, view logs, and perform maintenance, while a malicious operation might become logging in, packaging core data, and initiating external transmission.

[0027] Behavioral timing genes can reflect an entity's biological clock. This embodiment can record its active periods and operational rhythms. If an employee account that always works during the day suddenly logs in and accesses a sensitive system at 2 AM, the timing gene will light up a red light. Due to the massive amount of data, information entropy calculation is used to focus on key abnormal behaviors. For example, if an employee usually only accesses a few fixed servers (low entropy value), and if they suddenly start scanning a large number of IPs on the internal network (entropy value rises sharply), the system will capture all behaviors during this time period for in-depth analysis. The system uses unsupervised learning algorithms (such as Isolation Forest) to train a dynamic behavioral baseline model for the entity based on normal behavioral gene data from the past few weeks. This baseline is not fixed; it will be incrementally learned, periodically incorporating new, confirmed normal data to adapt to the normal development and changes in business.

[0028] During the operational phase, this embodiment can calculate the latest behavioral gene data for each entity in real time and compare it with its respective dynamic baseline model to calculate an anomaly score. Each behavioral gene dimension has an adaptive threshold. The initial threshold is set based on the statistical quantile (e.g., 95th percentile) of historical normal data. More importantly, this threshold is intelligently adjusted according to the context: for example, the threshold of the behavioral time-series gene is automatically widened on weekday evenings to allow employees to work overtime; or the threshold of the communication gene is raised during peak business periods to tolerate normal traffic fluctuations. When the anomaly score of any dimension exceeds its current threshold, the system determines that a gene mutation has occurred and generates a preliminary security alert.

[0029] The core evaluation metric in this embodiment is the false positive rate. If the false positive rate remains excessively high (e.g., exceeding 20%) within a unit of time, the current detection capability is deemed unqualified. When in an unqualified state, negative feedback is activated. This automatically reduces the weight of behavioral gene dimensions that recently generated false positives in the model and increases their judgment threshold. This essentially tells the system: for this type of behavior, it is not necessary to be so sensitive in the future, thereby effectively reducing future false positives. When an alarm is confirmed as a real attack, positive feedback is activated. It extracts the behavioral gene mutation pattern of this attack (including which gene dimensions are abnormal, specific deviation vectors, attack context, etc.), forming a structured attack gene feature, and stores it in a shareable threat gene feature library. Subsequently, when any other entity exhibits the same or similar behavioral patterns on the network, it can be quickly identified and alerted through real-time matching.

[0030] Specifically, this invention collects and integrates multi-source data such as network traffic, terminal system calls, and user operation logs to construct a behavioral gene map centered on network entities. It quantifies entity behavioral characteristics from three dimensions: communication association, operation sequence, and behavioral temporal sequence. A dynamic behavioral baseline model is established based on unsupervised learning. Anomaly detection is achieved by comparing the deviation between the gene map and the baseline in real time. When the deviation exceeds an adaptive threshold, a gene mutation alarm is triggered. Detection performance is automatically evaluated by analyzing the false alarm rate. When performance is unsatisfactory, a negative feedback mechanism is activated to reduce the weight of the false alarm gene dimension and increase the threshold to optimize detection accuracy. When performance is satisfactory, a positive feedback mechanism is activated to extract the gene mutation patterns of confirmed attacks as shareable attack gene features and store them in a threat feature database. This invention realizes a paradigm shift in detection from feature matching to behavioral analysis, effectively identifying unknown and internal threats while continuously improving detection accuracy through a closed-loop learning mechanism, ultimately constructing an adaptive security protection system with self-optimization capabilities.

[0031] Specifically, such as Figure 2 As shown, the correlation and fusion processing of multi-source heterogeneous security data includes: Assign a timestamp, source entity identifier, and behavior type identifier in a uniform format to each piece of behavioral data; Based on the source entity identifier, behavioral data from different data sources are associated, and based on the timestamp and behavior type identifier, a sequence of behavioral events centered on the network entity is constructed.

[0032] Specifically, in the data association and fusion phase, all collected raw logs (such as network traffic records, terminal system calls, and application operation logs) are first labeled with unified identity tags and type markers: a unified format timestamp ensures that all events can be sorted with millisecond-level precision; source entity identifiers (such as User:Alice@Host:PC-01) anchor scattered data to specific users, devices, or applications; and behavior type identifiers (such as UserLogin, FileDownload, and ExternalConnection) categorize the behaviors themselves. Using the source entity identifier as a unique key, data from different probes are associated and concatenated, and sorted according to timestamps, thereby constructing a complete sequence of behavioral events arranged chronologically for each network entity. For example, a record of PC-01 from the firewall uploading data to an external IP address is precisely associated with a record of user Alice from the terminal probe compressing core code files on PC-01, as consecutive events in the same sequence, completely reproducing a potential data leakage process.

[0033] Specifically, by assigning unified identifiers to data, the system breaks down barriers between different security data sources, constructs a unified perspective across networks, terminals, and applications, and solves the problem of data silos. By associating data with source entity identifiers as the core, the system can reconstruct fragmented attack traces into high-fidelity behavioral event chains centered on people or devices, greatly improving the contextual integrity and traceability of threat analysis. Finally, by constructing a clear timeline of behavioral event sequences, the system not only provides high-quality, structured input data for subsequent behavioral gene mapping but also enables security analysts to intuitively and quickly understand the full picture of an attack, thereby reducing the time for root cause analysis from hours to minutes, and improving the overall efficiency and accuracy of security operations.

[0034] Specifically, such as Figure 3 As shown, the selection of key behavioral events from the unified behavioral event stream based on an information entropy threshold includes: Calculate the information entropy value of a specific behavior type or operation target within a preset time window; The information entropy value is compared with a preset entropy threshold. When the information entropy value exceeds the entropy threshold range, the event within the time window is determined to be a key behavioral event and is extracted.

[0035] Specifically, when filtering key events from massive behavioral event streams, information entropy calculation is used as the core quantitative filtering tool. In practice, a sliding time window (e.g., 10 minutes) is first set, and within this window, the information entropy value is calculated for specific behavior types (e.g., accessed file types) or operation targets (e.g., connected server IP addresses). Information entropy is used here to measure the uncertainty or randomness of behavior: the lower the entropy value, the more concentrated and regular the behavior; the higher the entropy value, the more dispersed and disordered the behavior. A normal fluctuation range of entropy values ​​is pre-calculated based on historical normal data as a threshold. In real-time analysis, when the entropy value of an entity's behavior suddenly exceeds the preset threshold range—for example, a developer who usually only accesses a few internal servers, whose entropy value for the target IP address suddenly spikes in a short period—indicating that they may be randomly scanning the internal network, all relevant events within that time window are identified as key behavioral events and extracted completely for subsequent behavioral gene mapping and deep analysis.

[0036] Specifically, by quantifying the disorder of behavioral patterns, intelligent dimensionality reduction and precise filtering of massive amounts of raw security data are achieved, enabling the keen detection of potentially high-risk behavioral fragments that deviate from the norm. This effectively solves the core pain points of traditional security detection, such as low processing efficiency and the submersion of key signals due to data overload. It ensures that the subsequent analysis engine can focus on the most valuable abnormal behaviors, thereby improving the overall system detection efficiency while significantly reducing the unnecessary consumption of computing resources, making it possible to discover hidden and low-frequency attacks in real time and accurately.

[0037] Specifically, the communication association gene is quantified by at least one of the target address set communicated by the network entity, the distribution of communication ports, and the timing pattern of communication traffic. The operational sequence gene is quantified by at least one of the following: the sequence pattern of system calls executed by the network entity, the order of file access, and the habitual flow of application usage. The behavioral temporal gene is quantified by at least one of the following: the distribution of active periods of the network entity, the statistical characteristics of the time interval of operation events, and the periodicity of behavioral sessions.

[0038] Specifically, in constructing the behavioral gene map, the behavior of network entities is refined and quantified from three core dimensions. The characterization of communication-related genes is achieved by analyzing the interaction characteristics between entities and external nodes: the system records the set of target addresses for communication to identify abnormal connections, analyzes the distribution patterns of communication ports to discover unconventional service access, and monitors the temporal patterns of communication traffic to capture sudden traffic behaviors such as data leaks. The establishment of operation sequence genes relies on the analysis of the entity's internal operational logic: learning the sequence patterns of its executed system calls to identify malicious instruction combinations, tracing the sequential patterns of file access to discover abnormal data reading behavior, and analyzing the habitual processes of applications to detect abnormal operations that violate normal workflows. The quantification of behavioral temporal genes focuses on the temporal characteristics of entity activities: the system establishes a normal working time model by analyzing the distribution of its active periods, statistically analyzes the time interval characteristics of operational events to identify abnormal operation rhythms, and mines the periodic patterns of behavioral sessions to discover abnormal activity patterns that violate normal norms.

[0039] Specifically, by analyzing communication-related genes, abnormal network connections can be accurately identified, and risks of unauthorized external connections and data leaks can be detected in a timely manner. By modeling operational sequence genes, malware operation and abnormal data access behavior can be effectively detected, and internal threats can be accurately identified. By monitoring behavioral temporal genes, the system can keenly detect security events such as abnormal account usage and disruption of time patterns. The synergistic effect of these three dimensions constructs a three-dimensional behavioral analysis system, which not only significantly improves the accuracy of threat detection and reduces false positives, but more importantly, achieves a technological leap from single-point anomaly detection to comprehensive behavioral analysis, enabling the system to have global perception and identification capabilities for complex attack chains.

[0040] Specifically, such as Figure 4 As shown, the time alignment of behavioral events from different sources based on the dynamic time warping algorithm includes: Extract event sequences representing the same potential activity from network traffic data and terminal operation behavior logs, respectively; The dynamic time warping algorithm is applied to calculate the optimal curved path between two event sequences to align related events that have offsets or velocity differences on the time axis.

[0041] Specifically, when processing behavioral events from different data sources, the system employs a dynamic time warping algorithm to address time synchronization issues. The implementation process is as follows: First, it is necessary to extract behavioral sequences describing the same potential security event from both the network and terminal sides. For example, from network traffic data, extract the event chain of initiating a connection to an external IP, transmitting encrypted data, and terminating the connection; simultaneously, extract the operation sequence of accessing sensitive files, starting a compression program, and generating temporary files from terminal operation logs. Due to factors such as network transmission latency and system clock differences, these two event sequences describing the same activity often exhibit asynchronous behavior on the timeline. At this point, the dynamic time warping algorithm is used to calculate the optimal warping path between the two sequences. This algorithm can intelligently stretch or compress the timeline, finding the best correspondence between the two sequences. Even if the two sequences differ in their occurrence speed and specific time points, it can accurately align related events in the time dimension, thereby establishing a precise causal relationship.

[0042] Specifically, by introducing a dynamic time warping algorithm for cross-data source time alignment, the problem of time asynchrony in multi-source data caused by different collection points is effectively solved. This enables accurate matching of related events from network devices and terminal systems, thereby constructing a complete and coherent attack chain. This not only significantly improves the accuracy of behavioral gene map construction and provides a reliable data foundation for subsequent anomaly detection, but more importantly, it enables the system to reconstruct complex attack scenarios. It can correlate seemingly isolated security events into a complete attack process with causal relationships, greatly enhancing the detection and analysis capabilities for advanced persistent threats, while reducing misjudgments caused by time asynchrony.

[0043] Specifically, the dynamic behavioral baseline model established based on unsupervised learning algorithms can be a clustering-based method, a reconstruction-based method, or an isolation-based method. The behavioral baseline model is updated periodically by incorporating new, confirmed normal behavioral gene map data through incremental learning.

[0044] Specifically, when establishing the behavioral baseline model, the system employs unsupervised learning algorithms to automatically discover normal behavioral patterns of network entities. In implementation, the system provides three optional algorithm paths: clustering-based methods (such as K-means or DBSCAN) aggregate historical normal behavioral gene maps into multiple typical behavioral clusters, with the baseline serving as the boundary of these clusters; reconstruction-based methods (such as autoencoders) learn compressed representations of normal behavioral data, defining the normal range through reconstruction errors; and isolation-based methods (such as isolated forests) establish a tree structure for detecting anomalies by randomly partitioning the feature space. During model operation, the system continuously optimizes the baseline model through an incremental learning mechanism, periodically incorporating newly generated behavioral gene map data confirmed as normal by security analysts into the training set. This allows the baseline to adaptively evolve with environmental changes such as business changes and organizational restructuring, avoiding the problem of increased false alarms caused by baseline rigidity.

[0045] Specifically, the application of unsupervised learning frees the system from dependence on labeled data, enabling it to automatically learn normal patterns from massive behavioral data, significantly reducing deployment and maintenance costs. The introduction of incremental learning mechanisms ensures that the baseline model has the ability to continuously evolve, adapting to normal changes in the network environment while maintaining detection accuracy, and continuously optimizing model accuracy by incorporating new normal samples. Ultimately, this achieves an intelligent detection system that can maintain detection stability and adaptively optimize with the environment, effectively solving the industry problem of detection performance degradation caused by environmental changes in traditional static baseline systems.

[0046] Specifically, the calculation of its deviation in each behavioral gene dimension is as follows: For each behavioral gene dimension, calculate the statistical distance between the current gene map data and the behavioral baseline model in that dimension; The statistical distance of each dimension is compared with the corresponding adaptive threshold to determine whether a deviation has occurred.

[0047] Specifically, in the anomaly detection phase, the system accurately assesses the degree of behavioral deviation by quantifying statistical distances. The implementation process is as follows: The system independently calculates the deviation for each behavioral gene dimension. For example, in the communication association gene dimension, it calculates the Jaccard distance or cosine similarity between the current set of target addresses and the normal address distribution in the baseline model; in the operation sequence gene dimension, it uses edit distance or dynamic time warping distance to measure the difference between the current system call sequence and the baseline sequence; in the behavioral temporal gene dimension, it uses relative entropy or Euclidean distance to compare the difference between the current activity time distribution and the baseline time pattern. The statistical distance calculated for each dimension is immediately compared with an independently set adaptive threshold for that dimension—when the statistical distance of a dimension exceeds its specific threshold, the system determines that the dimension has deviated significantly. This mechanism of independent calculation and judgment across dimensions enables the system to accurately locate the specific behavioral characteristics of the anomaly.

[0048] Specifically, by independently calculating statistical distance across dimensions, the system can accurately pinpoint the specific feature dimensions of abnormal behavior, providing clear diagnostic directions for security analysis. Using statistical distance as a quantitative indicator provides a clear mathematical basis for anomaly determination, significantly improving the objectivity and interpretability of the detection results. By comparing adaptive thresholds for each dimension, the system ensures that different feature dimensions can be set with appropriate sensitivity according to their own characteristics, while also ensuring that the system can adapt to the differences in normal fluctuations across dimensions. Ultimately, this improves detection accuracy while effectively reducing the false alarm rate, forming a scientific and reliable anomaly determination mechanism.

[0049] Specifically, the initial value of the adaptive threshold is set based on the statistical distribution quantile of historical normal behavior gene map data; The adaptive threshold is also periodically adjusted according to the overall state of the network environment; The overall status includes the overall network load or the switching between weekdays and holidays.

[0050] Specifically, in terms of adaptive threshold management, the system adopts a hierarchical intelligent adjustment strategy. During the initial threshold setting phase, the system calculates the 95th percentile of statistical distances across various dimensions based on behavioral gene data from the entity's historical normal period (e.g., the last 30 days) as the initial threshold, ensuring that alarms are only generated for the most abnormal 5% of behaviors. During the operation and maintenance phase, the system establishes a periodic threshold adjustment mechanism, automatically reassessing threshold settings every 24 hours. The adjustment process comprehensively considers dynamic changes in the network environment: when the overall network load exceeds 80% of the normal level, the thresholds for communication-related genes and operational sequence genes are automatically increased to avoid false alarms caused by normal traffic surges during peak business periods; on weekends and public holidays, the system automatically relaxes the threshold range for behavioral time-series genes, allowing employees reasonable activities outside of working hours; when the environment returns to normal, the thresholds are correspondingly lowered back to the baseline level, forming a flexible threshold management system that is linked to environmental conditions.

[0051] Specifically, the initial setting based on historical data quantiles provides the system with a scientifically reliable baseline sensitivity, ensuring a robust start for the detection strategy. Periodic dynamic adjustments enable the system to intelligently distinguish between real threats and environmental noise, automatically reducing sensitivity to traffic fluctuations during peak business periods and appropriately relaxing monitoring of work schedule patterns during holidays. This effectively avoids contextual false alarms, thereby significantly improving alarm accuracy while maintaining a high detection rate. The system possesses the contextual awareness to differentiate between malicious behavior and normal business changes, greatly reducing the alarm processing burden on security analysts.

[0052] Specifically, in the negative feedback optimization mechanism, reducing the weight and increasing the adaptive threshold are performed according to a predefined mathematical formula: the weight update formula is: Wnew=Wold×α, where α is a decay factor between 0 and 1; The threshold update formula is: Tnew = Told × β, where β is an amplification factor greater than 1.

[0053] Specifically, in the negative feedback optimization mechanism, the system automatically adjusts parameters using precise mathematical formulas. When a specific behavioral gene dimension (such as behavioral time-series genes) generates a series of false positives recently, the system initiates a weight decay process: multiplying the current weight value Wold by a preset decay factor α (typically between 0.8 and 0.95) to obtain a new weight Wnew = Wold × α. This means that the importance of this dimension in the overall evaluation is systematically reduced. Simultaneously, the system performs an amplification operation on the adaptive threshold for this dimension: multiplying the current threshold Told by an amplification factor β (typically between 1.05 and 1.2) to obtain a new threshold Tnew = Told × β, which directly increases the anomaly detection threshold for this dimension. These adjustment operations are automatically executed based on preset trigger conditions. For example, when a gene dimension generates more than three false positives within 24 hours, the optimization process is immediately triggered, ensuring that the system can quickly respond to bias issues in the detection strategy.

[0054] Specifically, through the exponential decay of weights and the multiplicative amplification of thresholds, the system can quickly and stably converge to the optimal detection state, effectively suppressing persistent false alarms in specific dimensions. The formulaic adjustment method ensures the reliability and predictability of the optimization process, avoiding the subjective arbitrariness of manual adjustments. At the same time, this adaptive learning capability enables the system to automatically adapt to the normal changes in the network environment and user behavior, continuously maintaining the best balance between detection accuracy and false alarm rate. Ultimately, this achieves a technological leap from static configuration to dynamic self-optimization of the security detection system, significantly reducing operation and maintenance costs.

[0055] Specifically, the attack gene features are stored in a threat gene feature library as a structured feature template; The feature template includes at least: the behavioral gene dimension identifier that triggers the alarm, the deviation vector of each dimension, the associated network entity context information, and the attack mode label; It also includes: matching the gene mutation patterns detected in real time with the feature templates in the feature library based on similarity; if the similarity exceeds the matching threshold, it will directly associate the known attack pattern label to issue an alarm.

[0056] Specifically, in the threat intelligence accumulation and application stage, the system achieves standardized management of attack patterns through structured feature templates. Each confirmed attack event is extracted for key features, forming a feature template containing the following elements: recording the specific genetic dimension that triggered the alarm (e.g., communication-related genes + operational sequence genes), storing quantified deviation data for each dimension to form a deviation vector, saving the associated entity context (e.g., finance department users + database servers), and labeling with attack pattern tags (e.g., data theft - database breach). These feature templates are stored in a standardized format in a threat genetic feature library, forming a searchable knowledge system. In real-time detection, the system performs similarity matching between newly generated genetic mutation patterns and templates in the feature library, using a cosine similarity algorithm to calculate the degree of agreement between the current behavior pattern and historical attack patterns. When the similarity exceeds a preset 85% threshold, a high-confidence alarm is immediately generated by associating known attack tags, achieving rapid identification of attack patterns.

[0057] Specifically, the standardized storage of feature templates enables the systematic accumulation and efficient reuse of attack knowledge, transforming single detection experiences into sustainable collective defense capabilities. The real-time detection mechanism based on similarity matching allows the system to quickly identify variants of known attacks and recurring security events, significantly improving the detection efficiency of serialized and organized attacks. At the same time, by associating known attack pattern tags, it provides security analysts with rich contextual information and handling suggestions, significantly accelerating the speed of event response. Ultimately, it constructs a complete closed loop from attack discovery to knowledge accumulation to intelligent response, achieving a qualitative leap in security defense capabilities from passive response to proactive immunity.

[0058] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0059] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A network security intelligent detection method based on big data, characterized in that, include: Collect multi-source heterogeneous security data from the protected network environment, including at least network traffic data, terminal system call sequences, and user operation behavior logs. Then, clean, standardize, and correlate and fuse the multi-source heterogeneous security data to generate a unified behavior event stream with network entities as the core dimension. Key behavioral events are selected from the unified behavioral event stream based on the information entropy threshold, and behavioral events from different sources are aligned in time based on the dynamic time warping algorithm to construct a behavioral gene map consisting of multiple behavioral gene dimensions for each network entity. Using the behavioral gene map of the network entity during its historical normal operation cycle, a dynamic behavioral baseline model is established for each network entity; the current behavioral gene map is compared with the corresponding behavioral baseline model in real time, and its deviation in each behavioral gene dimension is calculated. When any deviation exceeds the set adaptive threshold, a gene mutation event is determined to have occurred and an initial screening security alarm is generated. The feedback results generated after the safety response to the initial screening safety alarm are collected, and the current detection capability is evaluated based on the ratio of the initial screening safety alarms that are judged as false alarms within a unit of time. When the evaluation is deemed unqualified, a negative feedback optimization mechanism is activated, and the following actions are performed: the weight of the behavioral gene dimension associated with recently generated false alarm reports is reduced in the behavioral baseline model, and its adaptive threshold is increased.

2. The intelligent network security detection method based on big data according to claim 1, characterized in that, The correlation and fusion processing of multi-source heterogeneous security data includes: Assign a timestamp, source entity identifier, and behavior type identifier in a uniform format to each piece of behavioral data; Based on the source entity identifier, behavioral data from different data sources are associated, and based on the timestamp and behavior type identifier, a sequence of behavioral events centered on the network entity is constructed.

3. The intelligent network security detection method based on big data according to claim 2, characterized in that, The selection of key behavioral events from the unified behavioral event stream based on an information entropy threshold includes: Calculate the information entropy value of a specific behavior type or operation target within a preset time window; The information entropy value is compared with a preset entropy threshold. When the information entropy value exceeds the entropy threshold range, the event within the time window is determined to be a key behavioral event and is extracted.

4. The intelligent network security detection method based on big data according to claim 3, characterized in that, The behavioral gene dimension includes at least communication-related genes, operational sequence genes, and behavioral temporal genes; The communication association gene is quantified by at least one of the target address set, communication port distribution, and communication traffic timing pattern of the network entity. The operational sequence gene is quantified by at least one of the following: the sequence pattern of system calls executed by the network entity, the order of file access, and the habitual flow of application usage. The behavioral temporal gene is quantified by at least one of the following: the distribution of active periods of the network entity, the statistical characteristics of the time interval of operation events, and the periodicity of behavioral sessions.

5. The intelligent network security detection method based on big data according to claim 4, characterized in that, The time alignment of behavioral events from different sources based on the dynamic time warping algorithm includes: Extract event sequences representing the same potential activity from network traffic data and terminal operation behavior logs, respectively; The dynamic time warping algorithm is applied to calculate the optimal curved path between two event sequences to align related events that have offsets or velocity differences on the time axis.

6. The intelligent network security detection method based on big data according to claim 5, characterized in that, In the process of establishing the dynamic behavioral baseline model, the unsupervised learning algorithm used is a clustering-based method, a reconstruction-based method, or an isolation-based method. The behavioral baseline model is updated periodically by incorporating new, confirmed normal behavioral gene map data through incremental learning.

7. The intelligent network security detection method based on big data according to claim 6, characterized in that, The process of calculating its deviation in each behavioral gene dimension is as follows: For each behavioral gene dimension, calculate the statistical distance between the current gene map data and the behavioral baseline model in that dimension; The statistical distance of each dimension is compared with the corresponding adaptive threshold to determine whether a deviation has occurred.

8. The intelligent network security detection method based on big data according to claim 7, characterized in that, The initial value of the adaptive threshold is set based on the statistical distribution quantile of historical normal behavior gene map data; The adaptive threshold is periodically adjusted according to the overall state of the network environment; The overall status includes the overall network load or the switching between weekdays and holidays.

9. The intelligent network security detection method based on big data according to claim 8, characterized in that, In the negative feedback optimization mechanism, reducing the weight and increasing the adaptive threshold are performed according to a predefined mathematical formula: The weight update formula is: Wnew = Wold × α, where α is a decay factor between 0 and 1; The threshold update formula is: Tnew = Told × β, where β is an amplification factor greater than 1.

10. The intelligent network security detection method based on big data according to claim 9, characterized in that, Also includes: When the evaluation is qualified, the positive feedback optimization mechanism is activated and the following operations are performed: the behavioral gene mutation pattern corresponding to the security alarm that is determined to be confirmed is extracted as an attack gene feature and stored in a shareable threat gene feature library. The attack gene features are stored in the threat gene feature library as structured feature templates. The feature templates include at least: the behavioral gene dimension identifier that triggers the alarm, the deviation vector of each dimension, the associated network entity context information, and the attack mode label. The system performs similarity matching between the detected gene mutation patterns and the feature templates in the feature library. If the similarity exceeds the matching threshold, it directly associates the pattern with known attack patterns to trigger an alert.

Citation Information

Patent Citations

  • User behavior anomaly detection method and device

    CN111291015A

  • Data processing method and system for cloud computing and storage medium

    CN120256196A

  • Information security analysis method and system based on big data

    CN120474818A

  • Risk management and control method and system based on real-time behavior analysis

    CN120896706A

  • Computer network intelligent security protection system based on big data

    CN120896761A

Cited By

  • Transaction process-oriented risk sample adaptive selection and labeling method

    CN121765387A