Intelligent network threat detection system based on adaptive learning

Abstract

The invention relates to the technical field of network security detection, in particular to an intelligent network threat detection system based on adaptive learning. The system comprises a data acquisition layer which is deployed on a network node and is used for acquiring network traffic, log data and user behavior data in real time and integrating the network traffic, the log data and the user behavior data into original network data; the data processing layer is used for preprocessing the original network data integrated by the data acquisition layer to obtain preprocessed network behavior data; and the adaptive learning module is used for training a preset threat detection model according to the preprocessed network behavior data and historical threat data to obtain a trained threat detection model. According to the invention, the network threat existing in the network data can be improved, and the network security is improved.

Description

Technical Field

[0001] This application relates to the field of network security detection technology, and in particular to an intelligent network threat detection system based on adaptive learning. Background Technology

[0002] Existing network data is at high risk of leakage, and this risk increases with the volume and complexity of enterprise data. Traditional encryption technologies and privacy protection measures are insufficient to fully address this challenge. In the event of a cyberattack or threat, traditional emergency response mechanisms often rely on manual judgment and operation, which is inefficient and prone to errors. Therefore, there is an urgent need for a system capable of real-time dynamic identification of cyber threats to accurately identify and respond to cyber threats present in existing network data, thereby addressing the problem of low network security. Summary of the Invention

[0003] To address the aforementioned issues, this application provides an intelligent network threat detection system based on adaptive learning, which can improve the detection of network threats present in network data and enhance network security.

[0004] According to one aspect of the embodiments of this application, an intelligent network threat detection system based on adaptive learning is proposed, the system comprising: A data acquisition layer is deployed on network nodes and is used to acquire network traffic, log data and user behavior data in real time, and integrate the network traffic, log data and user behavior data into raw network data. A data processing layer is used to preprocess the raw network data integrated by the data acquisition layer to obtain preprocessed network behavior data. An adaptive learning module is used to train a preset threat detection model based on the preprocessed network behavior data and historical threat data to obtain a trained threat detection model. The historical threat data refers to data on the existence of network threats.

[0005] In the above scheme, the steps for preprocessing the original network data include data cleaning, noise reduction, and format unification.

[0006] In the above scheme, the step of the adaptive learning module training the preset threat detection model based on the preprocessed network behavior data and historical threat data includes: Receive the preprocessed network behavior data sent by the data processing layer; The weights of the preset threat detection model are iteratively optimized using an incremental learning algorithm or an online learning algorithm to obtain the trained threat detection model.

[0007] In the above solution, the system further includes a threat detection engine, which includes: The feature extraction module is used to extract key features from the preprocessed network behavior data, including IP address, port number, request frequency, and data packet size. The threat assessment module is used to input the key features into the trained threat detection model for threat assessment.

[0008] In the above scheme, after the threat assessment is performed, the trained threat detection model outputs a threat score corresponding to the preprocessed network behavior data, so as to perform corresponding early warning operations based on the threat score.

[0009] In the above scheme, the step of performing the corresponding early warning operation based on the threat score includes: If the threat score is greater than or equal to a preset score threshold, an alarm mechanism is triggered. Under the alarm mechanism, network communication is blocked or isolated, and the triggering of the alarm mechanism is logged. If the threat score is less than the preset score threshold, the alarm mechanism will not be triggered, and normal network communication will be maintained.

[0010] In the above scheme, the system further includes a response execution layer, which is used to execute the instructions issued by the threat detection engine to perform corresponding early warning operations according to the instructions issued by the threat detection engine.

[0011] In the above scheme, the system also includes a user interface, which is used to display the system's operating status, the threat score, and network security policy configuration information.

[0012] In the above scheme, the system also includes a model performance evaluation system, which is used to evaluate the performance of the trained threat detection model.

[0013] In the above scheme, the performance evaluation indicators consist of detection accuracy, false alarm rate, false negative rate, and response time.

[0014] The beneficial effects of this application are as follows: This application collects network traffic, log data, and user behavior data through a data acquisition layer and integrates them into raw network data. The data processing layer preprocesses the raw network data to obtain preprocessed network behavior data. An adaptive learning module is used to train a preset threat detection model based on the preprocessed network behavior data and historical threat data. By learning the correlation between historical threat data and current network behavior data, the trained threat detection model can achieve better performance and higher prediction accuracy. Attached Figure Description

[0015] Figure 1 This is an architecture diagram of the intelligent network threat detection system based on adaptive learning provided in the embodiments of this application; Figure 2 This is a schematic diagram of an intelligent network threat detection system module based on adaptive learning, provided in an embodiment of this application. Detailed Implementation

[0016] To enable those skilled in the art to better understand the solutions of this application, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] It should be noted that while some processes described in the specification, claims, and accompanying drawings include multiple steps appearing in a specific order, it should be clearly understood that these steps may not be performed in the order they appear herein, or may be performed in parallel. The step numbers are merely used to distinguish different steps and do not themselves represent any execution order. Furthermore, descriptions such as "first," "second," or "objective" in this document are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. "Multiple" in this document refers to at least two.

[0018] It is worth noting that, in the specific embodiments of this application, data such as network traffic, log data, and user behavior data are involved. When the above embodiments of this application are applied to specific products or technologies, permission or consent from the target object is required, and the collection, use, and processing of the relevant data must comply with relevant laws, regulations, and standards. For example, when an embodiment of this application needs to obtain data such as network traffic, log data, and user behavior data, separate permission or consent from the target object can be obtained through pop-ups or redirection to a confirmation page. After obtaining the separate permission or consent from the target object, the necessary network traffic, log data, and user behavior data required for the normal operation of the embodiments of this application can then be obtained.

[0019] The following provides a detailed description of the specific implementation methods of the embodiments of this application: Please see Figure 1 , Figure 1 This is an architecture diagram of an intelligent network threat detection system based on adaptive learning provided in this application embodiment. The intelligent network threat detection system based on adaptive learning provided in this application embodiment includes: A data acquisition layer is deployed on network nodes and is used to acquire network traffic, log data and user behavior data in real time, and integrate the network traffic, log data and user behavior data into raw network data. A data processing layer is used to preprocess the raw network data integrated by the data acquisition layer to obtain preprocessed network behavior data. An adaptive learning module is used to train a preset threat detection model based on the preprocessed network behavior data and historical threat data to obtain a trained threat detection model. The historical threat data refers to data on the existence of network threats.

[0020] Specifically, this application collects network traffic, log data, and user behavior data through a data acquisition layer and integrates them into raw network data. The data processing layer preprocesses the raw network data to obtain preprocessed network behavior data. An adaptive learning module trains a preset threat detection model based on the preprocessed network behavior data and historical threat data. By learning the correlation between historical threat data and current network behavior data, the trained threat detection model can achieve better performance and higher prediction accuracy.

[0021] like Figure 2 As shown, the adaptive learning module designed in this application can continuously collect and analyze multi-source heterogeneous data such as security events, user behavior, and system logs in the network.

[0022] Machine learning algorithms (such as neural networks and decision trees) are used to train models, creating intelligent models tailored to different security scenarios. These models can identify abnormal behavior patterns and predict potential security threats.

[0023] A feedback mechanism is introduced to compare actual security incidents with model predictions, continuously optimizing and updating the intelligent model to improve its ability to identify new and variant malicious attacks. Through continuous learning and optimization, the adaptive learning module can cope with the ever-changing threat environment, improve the system's dynamic threat identification capabilities, reduce the cost of manual intervention and rule maintenance, and enhance the system's automation and intelligence levels.

[0024] An intelligent threat detection and defense system is built based on the intelligent model generated by the adaptive learning module.

[0025] This intelligent threat detection and defense system can monitor key indicators such as network traffic, user behavior, and system status in real time, and identify potential security threats through methods such as pattern matching and anomaly detection. Once a security threat is detected, the system automatically triggers defense mechanisms (i.e., alarm mechanisms), such as isolating infected devices, blocking malicious traffic, and initiating emergency response procedures, thereby achieving early detection, early warning, and early handling of security threats and reducing the impact of security incidents on network users.

[0026] In some embodiments, the preprocessing steps for the raw network data include data cleaning, noise reduction, and format standardization.

[0027] In some embodiments, the step of the adaptive learning module training a preset threat detection model based on the preprocessed network behavior data and historical threat data includes: Receive the preprocessed network behavior data sent by the data processing layer; The weights of the preset threat detection model are iteratively optimized using an incremental learning algorithm or an online learning algorithm to obtain the trained threat detection model.

[0028] The number of pre-defined threat detection models proposed in this application is not limited to one; there can be multiple models. Therefore, various machine learning network threat models (such as random forests, neural networks, etc.) can be trained using historical threat data and normal network behavior data (i.e., pre-processed network behavior data) to form a basic threat detection model library. In this way, different network threat models can be used for prediction and defense in different threat scenarios.

[0029] In some embodiments, the system further includes a threat detection engine, the threat detection engine comprising: The feature extraction module is used to extract key features from the preprocessed network behavior data, including IP address, port number, request frequency, and data packet size. The threat assessment module is used to input the key features into the trained threat detection model for threat assessment.

[0030] Specifically, key features such as IP address, port number, request frequency, and packet size are extracted from the processed network behavior data.

[0031] Threat assessment: The extracted features are input into a model trained by the adaptive learning module to perform a threat assessment. The model outputs a threat score, indicating the likelihood that the data sample represents malicious behavior.

[0032] Alarms and Decisions: Based on the threat score, the system will trigger an alarm mechanism and perform corresponding operations according to the preset alarm policies (such as blocking, isolation, logging, etc.).

[0033] In some embodiments, the system further includes a response execution layer, which is used to execute instructions issued by the threat detection engine to perform corresponding early warning operations according to the instructions issued by the threat detection engine.

[0034] Response Execution Layer: Responsible for executing instructions issued by the threat detection engine, such as blocking malicious traffic, isolating infected devices, and updating security policies.

[0035] User Interface: Provides a visual interface displaying system operating status, threat detection results, security policy configurations, and other information. Administrators can use the interface to monitor network security posture, adjust security policies, and view audit logs.

[0036] In summary, this application integrates multiple functional modules such as adaptive learning, intelligent collaborative defense, balance between privacy protection and data utilization, and customization for business scenarios, forming a complete network threat detection and defense solution.

[0037] The adaptive learning module of this application can continuously learn and optimize intelligent models to cope with ever-changing cyber threats.

[0038] Level of intelligence: This application can also achieve intelligent data analysis, anomaly detection and threat response by introducing AI technology.

[0039] In summary, this application significantly improves the ability to identify and defend against potential security threats through adaptive learning and intelligent threat detection and defense systems, effectively resisting various new and variant malicious attacks.

[0040] Furthermore, the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or apparatus.

[0041] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0042] It should be understood that in the description of the embodiments of this application, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0043] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0044] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0045] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0046] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0047] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.

[0048] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0049] The above is a detailed description of the embodiments of this application. However, this application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. An intelligent network threat detection system based on adaptive learning, characterized in that, The system includes: A data acquisition layer is deployed on network nodes and is used to acquire network traffic, log data and user behavior data in real time, and integrate the network traffic, log data and user behavior data into raw network data. A data processing layer is used to preprocess the raw network data integrated by the data acquisition layer to obtain preprocessed network behavior data. An adaptive learning module is used to train a preset threat detection model based on the preprocessed network behavior data and historical threat data to obtain a trained threat detection model. The historical threat data refers to data on the existence of network threats.

2. The intelligent network threat detection system based on adaptive learning according to claim 1, characterized in that, The preprocessing steps for the raw network data include data cleaning, noise reduction, and format standardization.

3. The intelligent network threat detection system based on adaptive learning according to claim 1, characterized in that, The steps of the adaptive learning module training the preset threat detection model based on the preprocessed network behavior data and historical threat data include: Receive the preprocessed network behavior data sent by the data processing layer; The weights of the preset threat detection model are iteratively optimized using an incremental learning algorithm or an online learning algorithm to obtain the trained threat detection model.

4. The intelligent network threat detection system based on adaptive learning according to claim 1, characterized in that, The system also includes a threat detection engine, which comprises: The feature extraction module is used to extract key features from the preprocessed network behavior data, including IP address, port number, request frequency, and data packet size. The threat assessment module is used to input the key features into the trained threat detection model for threat assessment.

5. The intelligent network threat detection system based on adaptive learning as described in claim 4, characterized in that, After the threat assessment is performed, the trained threat detection model outputs a threat score corresponding to the preprocessed network behavior data, so as to perform corresponding early warning operations based on the threat score.

6. The intelligent network threat detection system based on adaptive learning according to claim 5, characterized in that, The step of performing corresponding early warning operations based on the threat score includes: If the threat score is greater than or equal to a preset score threshold, an alarm mechanism is triggered. Under the alarm mechanism, network communication is blocked or isolated, and the triggering of the alarm mechanism is logged. If the threat score is less than the preset score threshold, the alarm mechanism will not be triggered, and normal network communication will be maintained.

7. The intelligent network threat detection system based on adaptive learning according to claim 6, characterized in that, The system also includes a response execution layer, which is used to execute the instructions issued by the threat detection engine to perform corresponding early warning operations according to the instructions issued by the threat detection engine.

8. The intelligent network threat detection system based on adaptive learning according to claim 8, characterized in that, The system also includes a user interface, which is used to display the system's operating status, threat score, and network security policy configuration information.

9. The intelligent network threat detection system based on adaptive learning according to claim 1, characterized in that, The system also includes a model performance evaluation system, which is used to evaluate the performance of the trained threat detection model.

10. The intelligent network threat detection system based on adaptive learning according to claim 9, characterized in that, The performance evaluation metrics consist of detection accuracy, false alarm rate, false negative rate, and response time.

Images