Model training method, attack detection method and related device
By adjusting the pre-trained model and using questionable data and labeled data to train the target prediction model, the lag and dependency problems of traditional network attack detection are solved, and more efficient network threat detection is achieved.
Patent Information
- Application Number
- CN202211215041.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-30
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2042-09-30
AI Technical Summary
Traditional network attack detection technology has lags and dependencies, making it difficult to extract effective attack features and rules, resulting in high rates of missed reports and false positives, poor flexibility, and an inability to effectively respond to complex network threats.
By training and adjusting the pre-trained model, using questionable data, control data and labeled data, the model parameters are adjusted to obtain the target prediction model, thereby improving the applicability and detection capabilities of the model in specific task scenarios.
It improves the flexibility and accuracy of network attack detection, reduces missed and false alarm rates, and enhances network security defense capabilities. It is suitable for products such as firewalls and security situation awareness.
Smart Images
Figure CN116015703B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of network security technology, and in particular to a model training method, an attack detection method, and related devices. Background Art
[0002] With the continuous development of the information age, network security issues are becoming increasingly serious, and the social impact and economic losses caused are also increasing, which has put forward higher demands and challenges for the detection and defense of network threats.
[0003] However, traditional network attack detection technology is mainly based on rule filtering. It collects currently known network attack data, summarizes their attack characteristics, and constructs a rule base. The rule base is then used to compare and detect the data to be tested to determine whether an attack exists.
[0004] Therefore, traditional attack detection methods generally have lags and dependencies. Only when the attack is exposed can the corresponding rules be extracted. At the same time, as the obfuscation and bypass capabilities of attack techniques increase, it becomes increasingly difficult to extract effective attack features and rules, making traditional attack detection methods less flexible and prone to missed reports and false positives of attacks. Summary of the Invention
[0005] The embodiments of the present application provide a model training method, an attack detection method, and related devices for improving the detection capability of network attacks.
[0006] A first aspect of an embodiment of the present application provides a model training method, comprising:
[0007] Obtaining control data for the questionable data, wherein the questionable data is determined from sample data by a pre-trained, pre-used model, and the questionable data and the control data are data that may have attack characteristics and have a similarity with the labeled data exceeding a similarity threshold, and the labeled data is data that is labeled with a corresponding attack type;
[0008] The model parameters of the pre-use model are trained and adjusted using the labeled data, the questionable data and the control data, and the training is stopped when the pre-use model reaches a convergence condition to obtain a target prediction model.
[0009] A second aspect of the embodiments of the present application provides an attack detection method, including:
[0010] Get the data to be tested;
[0011] The data to be tested is input into the target prediction model as described in the first aspect above to output a prediction result corresponding to the data to be tested, wherein the prediction result includes whether the data to be tested is intrusion data indicating an attack.
[0012] A third aspect of the present application provides a model training system, including:
[0013] a first acquisition unit, configured to obtain control data for questionable data, wherein the questionable data is determined from sample data by a pre-trained, pre-used model, the questionable data and the control data being data that may have attack characteristics and whose similarity to the labeled data exceeds a similarity threshold, and the labeled data being data that is labeled with a corresponding attack type;
[0014] The first processing unit is used to train and adjust the model parameters of the pre-use model using the labeled data, the questionable data and the control data, and stop training when the pre-use model reaches a convergence condition to obtain a target prediction model.
[0015] A third aspect of the present application provides an attack detection system, including:
[0016] A second acquisition unit, configured to acquire the data to be tested;
[0017] The second processing unit is used to input the data to be tested into the target prediction model as described in the first aspect above to output a prediction result corresponding to the data to be tested, and the prediction result includes whether the data to be tested is intrusion data with an attack.
[0018] A fourth aspect of the embodiments of the present application provides an electronic device, including:
[0019] CPU, memory and input / output interfaces;
[0020] The memory is a transient storage memory or a persistent storage memory;
[0021] The central processing unit is configured to communicate with the memory and execute instruction operations in the memory to perform the method described in the first aspect or the second aspect of the embodiment of the present application.
[0022] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium comprising instructions, which, when executed on a computer, enables the computer to execute a method as described in any specific implementation of the first aspect or the second aspect of the embodiment of the present application.
[0023] A fifth aspect of the embodiments of the present application provides a computer program product comprising instructions, which, when run on a computer, enables the computer to execute the method described in any specific implementation of the first aspect or the second aspect of the embodiments of the present application.
[0024] It can be seen from the above technical solutions that the embodiments of the present application have at least the following advantages:
[0025] Using questionable data, control data and labeled data with corresponding attack type labels, the pre-trained pre-use model is retrained to obtain the target prediction model, which helps to discover more data that may have attack characteristics and improve the applicability of the model in specific task scenarios, such as predicting whether the data to be tested actually contains attack behavior, reducing the traditional attack detection dependence and lag caused by the data rules themselves, thereby further meeting the defense needs of network security. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0027] Figure 1 A schematic diagram of an application scenario of an embodiment of the present application;
[0028] Figure 2 A flow chart of the model training method according to an embodiment of the present application;
[0029] Figure 3 This is another flowchart of the model training method according to an embodiment of the present application;
[0030] Figure 4 A flowchart of the attack detection method according to an embodiment of the present application;
[0031] Figure 5 A schematic diagram of the structure of the model training system according to an embodiment of the present application;
[0032] Figure 6 This is a schematic diagram of the structure of the attack detection system according to an embodiment of the present application;
[0033] Figure 7 This is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0034] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0035] The terms "first," "second," "third," "fourth," and the like (if any) in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequential sequence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0036] In the following description, references to "one embodiment" or "an example" and similar expressions describe a subset of all possible embodiments, but it is understood that "one embodiment" or "an example" may refer to the same subset or different subsets of all possible embodiments and may be combined with each other without conflict. In the following description, the term "a plurality" refers to at least two. The reference to a certain value reaching a threshold value (if any) in this application includes the case where the former is greater than the threshold value and the latter is greater than the threshold value.
[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.
[0038] To facilitate understanding and explanation, the nouns and terms involved in the embodiments of the present application are explained below, and these nouns and terms are subject to the following explanations.
[0039] Cross-site scripting (XSS) is one of the most common web application security vulnerabilities. This vulnerability allows attackers to embed malicious script code into web pages that are typically accessed by legitimate users. When legitimate users access the page, the embedded malicious script code executes, thereby achieving malicious attacks on users.
[0040] SQL injection: refers to a situation where a web application fails to judge the legitimacy of user input data or does not filter it strictly. An attacker can add additional SQL statements to the end of pre-defined query statements in the web application, performing illegal operations without the administrator's knowledge. This is to deceive the database server into executing unauthorized arbitrary queries and further obtain the corresponding data information.
[0041] Bypass attack: An attack method that evades detection by defense tools by obfuscating the original attack payload. Common bypass techniques include encoding bypass, keyword destruction, and deformation obfuscation.
[0042] Machine learning: A discipline that studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.
[0043] Language model: Assuming that all possible sentences in a language conform to a probability distribution, the task of the language model is to model this underlying probability distribution in order to calculate the probability of each sentence appearing in the language. A good language model should assign a higher probability to sentences that conform to language expression habits and a probability close to zero to incorrect sentences.
[0044] Word vectors: A vector mechanism that maps words into a high-dimensional space. Word vectors contain both grammatical and semantic information about the words. With the development of natural language processing, word vectors are often used as input for various machine learning and deep learning models.
[0045] To facilitate understanding and explanation, before further describing the embodiments of the present application in detail, a brief description of conventional technologies will be given using a World Wide Web (WEB) attack as an example.
[0046] The current mainstream WEB attack detection technology mainly makes a deterministic description of known WEB attack data to form corresponding rules and summarize them into a rule base or feature base. In the process of one-to-one comparison, if the collected network data to be detected matches the rules in the rule base, then it indicates that this is an intrusion behavior data, which can be simply called attack data.
[0047] In addition, records show that many companies use machine learning-related technologies to detect web attacks. However, these traditional machine learning solutions are essentially combinations of features such as hazard functions, and still require manual identification and extraction of features or rules. At the same time, due to the limitations of the data itself, it is difficult to learn a more reliable voice model.
[0048] As can be seen, traditional network attack detection methods mainly rely on extracting attack rules from collected attack data or writing rules based on bypass techniques. This has poor flexibility and makes it easy to overlook suspicious data, resulting in high attack omission and false alarm rates, thereby affecting network information security and user experience. In view of this, the present application provides a model training method, an attack detection method, and related devices to effectively improve the detection capabilities of network attacks.
[0049] The method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 101 through the network, and the data storage system can store the data that the server 101 needs to process. The data storage system 100 can be integrated on the server 101, or it can be placed on the cloud or other network servers. The terminal 102 can obtain the test data input by the user and send the test data to the server 101. The server 101 can perform attack detection based on the obtained test data, and can also adjust the corresponding strategy for the test data based on the detection results (prediction results). In addition, the server 101 can also send the prediction results to the terminal 102. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server 101 can be implemented as an independent server or a server cluster composed of multiple servers. It should be noted that the method provided in the embodiment of the present application can be implemented by the terminal device and the server as described above, or it can be implemented entirely on the server side, or it can be implemented entirely on the terminal device side. The specific method can be determined according to the actual application scenario and is not limited here.
[0050] The attack detection method of the present application will be described in further detail below.
[0051] See also Figures 2 to 3 In a first aspect, the present application provides an embodiment of a model training method, which includes steps 201 to 202:
[0052] 201. Obtain control data for questionable data.
[0053] Among them, the questionable data is determined from the sample data by a pre-trained pre-use model. The questionable data and control data are data that may have attack characteristics and whose similarity with the labeled data exceeds the similarity threshold. The labeled data is data marked with the corresponding attack type.
[0054] The attack types mentioned in this application include but are not limited to SQL injection attacks and XSS attacks. The presence or absence of labels mentioned mainly refers to whether the data has been marked with attack type labels. Of course, data that does not involve attack behavior can be regarded as unlabeled data.
[0055] As a possible implementation, the process of obtaining the pre-use model (pre-training process) includes: using sample data containing unlabeled data to pre-train the initial prediction model to obtain the pre-use model.
[0056] Specifically, the sample data (training samples) of the initial prediction model may include one or more combinations of attack data and vulnerability data that have been made public on the Internet, and attack data and vulnerability data that have been collected or suffered by the device to be tested (such as the user host); of course, these sample data can be analyzed and refined separately.
[0057] In real-world scenarios, most collected network data is unlabeled with attack type labels, and labeled data is often a minority. Modeling directly using labeled data is difficult to achieve effective predictions, so it's necessary to mine unlabeled samples. Furthermore, given that text sample data is composed of characters, which form words, sentences, and ultimately semantics, it naturally possesses universal characteristics and can be mined through pre-training.
[0058] In some specific examples, the initial prediction model can be selected from one or more models such as the continuous bag of words model CBOW (Continuous BagOfWords), the Skip-Gram word vector model, and the language model such as the autoregressive language model (Auto Regressive Language Model) and the auto encoding language model (Auto Encoding Language Model). Of course, other specific algorithms or algorithm combinations that can learn the required word vector representation can also be selected.
[0059] Taking the CBOW model as an example, the process of pre-training the initial prediction model is to express word A through the vocabulary B, C, D, etc. in the context of word A. For example, taking XSS attacks as an example, its common payload examples are as follows:
[0060] <ahref="javascript:alert('"xss"');"> test
[0061] <ahref="javascript:prompt('"xss"');"> test
[0062] Because the characters before and after the words "alert" and "prompt" are similar, when "alert" and "prompt" are input into the initial prediction model to complete word vector learning, the resulting word vectors are very similar. In subsequent applications, the word vector of "alert" (which can be regarded as word A) can be used to find words in the vector library that correspond to vectors similar to the word vector of "alert" (such as the aforementioned "prompt"). These words are very likely to be suspicious words related to attack behavior (called black words or suspected words).
[0063] In some specific examples, the frequency of occurrence of each attack type in a large amount of sample data can be calculated, as well as the conditional probability estimates of each attack type for each attack type, given the attack data. These calculation results are then used to train an initial prediction model to obtain a pre-use model. Furthermore, in this application, the aforementioned model training can be specifically performed using a naive Bayesian algorithm or a machine learning classification algorithm.
[0064] 202. Use the labeled data, questionable data, and control data to train the pre-used model to obtain the target prediction model.
[0065] Specifically, the model parameters of the pre-used model are trained and adjusted using labeled data, questionable data, and control data. Training is terminated when the pre-used model reaches a convergence condition, thereby obtaining a target prediction model. The convergence condition may be that the loss function value of the pre-used model fluctuates within a preset loss range over a period of time.
[0066] As can be seen from the above description, the purpose of retraining the pre-used model is to obtain the pre-used model by training the initial prediction model with sample data containing a large amount of unlabeled data. Therefore, the model parameters of the pre-used model are also universal parameters. Therefore, to more accurately use the model in a specific task scenario, further adjustment (which can be understood as fine-tuning) is required. This fine-tuning process can be similar to the operation process of obtaining the pre-used model described above (with different sample data). Using the trained pre-used model parameters as the initial values, the model is retrained using labeled data of specific tasks (such as attack types such as XSS and SQL), questionable data, and reference data. The pre-used model parameters are adjusted until the model converges and adapts to the specific task application. During this process, unlike the unlabeled data used in the pre-training process, at least part of the (sample) data used in the fine-tuning process will be labeled with the corresponding attack type (specifically, manually labeled), allowing for supervised learning. In addition, the sample data at this time also includes questionable data and reference data that may have attack characteristics (or intrusion characteristics), which enables the model to detect suspected intrusion data more quickly and accurately.
[0067] Based on the above example, the specific implementation process of step 201 may include:
[0068] Query whether there is known data that meets the preset feature rules compared with the questionable data. The preset feature rules are used to indicate that there are at least partially identical features between the two data being compared; if so, generate comparison data that meets the preset feature rules with the known data, and use the known data and the comparison data as control data; if not, generate control data for the questionable data according to the preset feature rules.
[0069] The data mentioned in this application may include word segmentation vocabulary in text type data, and of course, also include identification information such as protocol type and / or IP address. The data can be flexibly set according to actual conditions, and this application does not impose any restrictions. Taking text type data as an example, words such as alert, `;alert, and prompt usually appear in similar contexts, so the three sample data (or called test data) containing these three words respectively have certain similarities, that is, they meet certain preset feature rules. Therefore, if one of them is questionable data, the other two sample data can be queried through the preset feature rules. In some practical applications, comparison data similar to the questionable data can also be generated based on this rule, thereby increasing the accuracy and foresight of discovering suspicious intrusion data and reducing the risk of data attacks.
[0070] On the other hand, the process of determining the questionable data in step 201 may include:
[0071] For each sample data feature vector output by the pre-used model, the vector library is searched to see if there is a known feature vector whose similarity with the feature vector meets the vector similarity threshold. A known feature vector is a feature vector corresponding to data of known attack type. If so, the sample data is determined to be suspicious data. The detailed implementation of this process can be found in the description of one of the measures in step 302 below, and will not be detailed here.
[0072] The first aspect mentioned above mainly explains the training process of the model. The second aspect below mainly explains the actual application process of the model in detail. The specific implementation process of the first aspect mentioned above can be implemented with reference to the operation content of the second aspect.
[0073] See also Figure 3 and Figure 4 The second aspect of the present application provides an embodiment of an attack detection method, the method comprising:
[0074] 301. Obtain the data to be tested.
[0075] The data mentioned in this application may include segmented vocabulary in text type data, and of course, also include identification information such as protocol type and / or IP address. The data can be flexibly set according to actual conditions and this application does not impose any restrictions.
[0076] 302. Input the data to be tested into the target prediction model to output the prediction results corresponding to the data to be tested.
[0077] The prediction result includes whether the data to be tested is intrusion data that may be involved in an attack.
[0078] When the user needs to confirm whether the data to be tested has an attack, a specific implementation of step 302 includes:
[0079] The target feature vector corresponding to the test data output by the target prediction model, such as the word vector;
[0080] Search the vector library for known feature vectors whose similarity to the target feature vector reaches a vector similarity threshold. The known feature vectors are feature vectors corresponding to data of known attack types.
[0081] If it exists, it is determined that the data to be tested is intrusion data; otherwise, if it does not exist, it is determined that the data to be tested is not intrusion data.
[0082] Furthermore, if the user needs to determine the specific attack type of the data being tested, the attack type corresponding to the known feature vector can be used as the target attack type for the data being tested. If there are multiple known feature vectors, the attack type corresponding to the known feature vector with the highest similarity to the target feature vector can be selected as the target attack type.
[0083] In some specific examples, another specific implementation of step 302 includes:
[0084] The target prediction model is used to predict the probability estimate of the test data belonging to different attack types;
[0085] determining whether there is a probability estimate that satisfies a predetermined probability range;
[0086] If it exists, it is determined that the data to be tested is intrusion data.
[0087] Of course, if the user needs to clarify what specific attack type the data to be tested belongs to when it is intrusion data, the attack type corresponding to the probability estimate that meets the preset probability range and has the largest value can also be determined as the target attack type of the data to be tested.
[0088] Based on the above example, if the user needs to clarify what other suspected intrusion features may be associated when the data to be tested is intrusion data, that is, when the data therein is an intrusion feature, then the prediction result at this time may also include the suspected intrusion features found when the data to be tested is intrusion data. Accordingly, step 302 may also include any of the following similarity search measures:
[0089] One of the measures is to search for existing feature vectors in the vector library whose vector similarity with the known feature vector reaches a preset threshold, and determine the data pointed to by the known feature vector and the existing feature vector respectively as suspected intrusion features.
[0090] The second measure is to classify the data to be tested and all known data into feature categories, and identify known data that belongs to the same category as the data to be tested as suspected intrusion features. For example, K-means clustering can be used for feature classification.
[0091] The first of the above measures, after finding a known feature vector that has a certain similarity with the target feature vector, searches for an existing feature vector that is similar to the known feature vector, and the second measure, helps to further discover more suspicious bypass technique features or attack black words, that is, to discover more suspicious expressions, reduce the missed reporting rate of the attack detection link, and thus effectively improve the security defense capabilities of network data.
[0092] Specifically, the prediction model obtained after training can output the word vector of each word, such as the corresponding relationship between words and word vectors in the following table:
[0093] Serial number word Word vectors 1 alert A 2 `;alert B 3 prompt C
[0094] Table 1 Corresponding examples of vocabulary and word vectors
[0095] Since alert, `alert, and prompt usually appear in similar contexts, their corresponding word vectors A, B, and C are very similar. Therefore, in specific applications, the word vector A of alert can be used to search for the most similar word vectors B, C, or even vectors D and E in the word vector library, thereby discovering suspicious bypass techniques or black words, thereby effectively avoiding network attacks.
[0096] The specific implementation process of the second aspect of this application can be implemented with reference to the operational content of the first aspect mentioned above. In summary, this application can effectively utilize massive amounts of unlabeled data, and through pre-training schemes, efficiently mine the value of massive amounts of data, thereby improving the ability to detect network attacks; in addition, the target prediction model obtained after further fine-tuning is more flexible and generalizable, and can be efficiently applied to attack detection scenarios such as the Web, and applied to products such as firewalls and security situation awareness. At the same time, the model of this application can effectively mine similar black words, discover more possible bypass techniques, and further enhance the security of the network operating environment.
[0097] See also Figure 5 In a third aspect, the present application provides an embodiment of a model training system, the system comprising:
[0098] A first acquisition unit 401 is configured to obtain reference data for questionable data, wherein the questionable data is determined from sample data using a pre-trained, pre-used model. The questionable data and reference data are data that may have attack characteristics and whose similarity to the labeled data exceeds a similarity threshold. The labeled data is data that is labeled with a corresponding attack type.
[0099] The first processing unit 402 is used to train and adjust the model parameters of the pre-used model using the labeled data, the questionable data and the control data, and stop the training when the pre-used model reaches the convergence condition to obtain the target prediction model.
[0100] Optionally, the first acquiring unit 401 is specifically configured to:
[0101] Check whether there is known data that meets the preset feature rules compared with the questionable data. The preset feature rules are used to indicate that the two data being compared have at least some of the same features.
[0102] If so, generate comparative data that meets the preset feature rules with the known data, and use the known data and the comparative data as control data;
[0103] If it does not exist, generate reference data for the questionable data based on the preset feature rules.
[0104] Optionally, the first acquiring unit 401 is specifically configured to:
[0105] For each feature vector of sample data output by the pre-used model, search the vector library for a known feature vector whose similarity with the feature vector reaches a vector similarity threshold. The known feature vector is the feature vector corresponding to data with a known attack type.
[0106] If so, the sample data is determined to be questionable data.
[0107] In the embodiment of the present application, the operations performed by each unit of the model training system are similar to the operations described in any specific method embodiment of the first aspect above, and the details will not be repeated here.
[0108] See also Figure 6 According to a fourth aspect of the present application, an embodiment of an attack detection system is provided, the system comprising:
[0109] The second acquisition unit 501 is used to acquire the data to be tested;
[0110] The second processing unit 502 is used to input the test data into the target prediction model as described in the first aspect above to output a prediction result corresponding to the test data, where the prediction result includes whether the test data is intrusion data indicating an attack.
[0111] Optionally, the second processing unit 502 is specifically configured to:
[0112] For the target feature vector corresponding to the test data output by the target prediction model, search the vector library for a known feature vector whose similarity with the target feature vector reaches a vector similarity threshold. The known feature vector is a feature vector corresponding to data with a known attack type.
[0113] If so, the data to be tested is determined to be intrusion data;
[0114] If it does not exist, it is determined that the data to be tested is not intrusion data.
[0115] Optionally, when the prediction result further includes an attack type corresponding to when the data to be tested is intrusion data, the second processing unit 502 is specifically configured to:
[0116] Determine the attack type to which the known feature vector corresponds, which is the target attack type corresponding to the intrusion data.
[0117] Optionally, when the prediction result further includes suspected intrusion features found when the data to be tested is intrusion data, the second processing unit 502 is specifically configured to:
[0118] Searching for a stored feature vector in the vector library whose vector similarity with the known feature vector reaches a preset threshold;
[0119] Determine the data pointed to by the known feature vector and the stored feature vector as suspected intrusion features;
[0120] or,
[0121] The data to be tested and all known data are classified by features, and the known data that belongs to the same category as the data to be tested is determined as a suspected intrusion feature.
[0122] Optionally, the second processing unit 502 is specifically configured to:
[0123] The target prediction model is used to predict the probability estimate of the test data belonging to different attack types;
[0124] determining whether there is a probability estimate that satisfies a predetermined probability range;
[0125] If so, the data to be tested is determined to be intrusion data, and the attack type corresponding to the probability estimate that meets the preset probability range and has the largest value is determined as the target attack type of the data to be tested.
[0126] In the embodiment of the present application, the operations performed by each unit of the attack detection system are similar to the operations described in any specific method embodiment of the second aspect above, and will not be described in detail here.
[0127] See also Figure 7 The electronic device 600 of the embodiment of the present application may include one or more central processing units (CPUs) 601 and a memory 605 , in which one or more applications or data are stored.
[0128] Memory 605 may be volatile or persistent storage. The program stored in memory 605 may include one or more modules, each of which may include a series of instruction operations on the electronic device. Furthermore, the central processing unit 601 may be configured to communicate with memory 605 and execute the series of instruction operations in memory 605 on the electronic device 600.
[0129] The electronic device 600 may also include one or more power supplies 602, one or more wired or wireless network interfaces 603, one or more input and output interfaces 604, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.
[0130] The central processing unit 601 can execute the operations performed by any specific method embodiment of the first aspect or the second aspect mentioned above, and the details will not be repeated here.
[0131] It can be understood that in the various embodiments of the present application, the serial number of each step does not mean the order of execution. The order of execution of each step should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0132] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0133] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system or device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection of some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0134] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0135] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.
[0136] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product (computer program product) is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a business server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
Claims
1. A model training method, characterized in that: include: Obtaining control data for the questionable data, wherein the questionable data is determined from sample data by a pre-trained, pre-used model, the questionable data and the control data are respectively data with possible attack characteristics whose similarity to the labeled data exceeds a similarity threshold, the labeled data being data correspondingly labeled with a corresponding attack type, and the questionable data and the control data satisfying a preset feature rule; The model parameters of the pre-use model are trained and adjusted using the labeled data, the questionable data and the control data, and the training is stopped when the pre-use model reaches a convergence condition to obtain a target prediction model.
2. The model training method according to claim 1, characterized in that The control data for obtaining questionable data includes: Checking whether there is known data that meets a preset feature rule compared with the questionable data, wherein the preset feature rule is used to indicate that at least some of the features are the same between the two compared data; If so, generating comparison data that meets the preset feature rule with the known data, and using the known data and the comparison data as the control data; If it does not exist, the comparison data of the questionable data is generated according to the preset feature rules.
3. The model training method according to claim 1, characterized in that The process of determining the questionable data includes: For each feature vector of sample data output by the pre-used model, searching the vector library for a known feature vector whose similarity with the feature vector reaches a vector similarity threshold, wherein the known feature vector is a feature vector corresponding to data of a known attack type; If so, the sample data is determined to be the questionable data.
4. An attack detection method, characterized in that: include: Get the data to be tested; The data to be tested is input into the target prediction model according to any one of claims 1 to 3 to output a prediction result corresponding to the data to be tested, wherein the prediction result includes whether the data to be tested is intrusion data indicating an attack.
5. The attack detection method according to claim 4, characterized in that: Outputting the prediction result corresponding to the data to be tested includes: For the target feature vector corresponding to the data to be tested output by the target prediction model, searching a vector library for a known feature vector whose similarity with the target feature vector reaches a vector similarity threshold, wherein the known feature vector is a feature vector corresponding to data of known attack type; If so, determining that the data to be tested is the intrusion data; If not, it is determined that the data to be tested is not the intrusion data.
6. The attack detection method according to claim 5, characterized in that: When the prediction result further includes an attack type corresponding to the data to be tested being intrusion data, outputting the prediction result corresponding to the data to be tested includes: The attack type to which the known feature vector corresponds is determined, which is the target attack type corresponding to when the data to be tested is intrusion data.
7. The attack detection method according to claim 5 or 6, characterized in that: When the prediction result further includes suspected intrusion features found when the data to be tested is intrusion data, outputting the prediction result corresponding to the data to be tested includes: Searching for a stored feature vector in a vector library whose vector similarity with the known feature vector reaches a preset threshold; Determining the data pointed to by the known feature vector and the stored feature vector as the suspected intrusion feature; or, The data to be tested and all known data are subjected to feature classification, and the known data belonging to the same category as the data to be tested are determined as the suspected intrusion features.
8. The attack detection method according to claim 4, characterized in that: Outputting the prediction result corresponding to the data to be tested includes: Predicting the probability estimation value of the test data belonging to different attack types by the target prediction model; Determining whether there is a probability estimate that satisfies a predetermined probability range; If so, the data to be tested is determined to be the intrusion data, and the attack type corresponding to the probability estimate that meets the preset probability range and has the largest value is determined as the target attack type of the data to be tested.
9. A model training system, characterized in that: include: a first acquisition unit, configured to obtain control data for questionable data, wherein the questionable data is determined from sample data by a pre-trained, pre-used model, the questionable data and the control data are respectively data with possible attack characteristics whose similarity to the labeled data exceeds a similarity threshold, the labeled data being data correspondingly labeled with a corresponding attack type, and the questionable data and the control data satisfying a preset feature rule; The first processing unit is used to train and adjust the model parameters of the pre-use model using the labeled data, the questionable data and the control data, and stop training when the pre-use model reaches a convergence condition to obtain a target prediction model.
10. An attack detection system, characterized in that: include: A second acquisition unit, configured to acquire the data to be tested; The second processing unit is used to input the data to be tested into the target prediction model according to any one of claims 1 to 3 to output a prediction result corresponding to the data to be tested, wherein the prediction result includes whether the data to be tested is intrusion data indicating an attack.
11. An electronic device, characterized in that: include: CPU, memory and input / output interfaces; The memory is a transient storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the method according to any one of claims 1 to 8.
12. A computer-readable storage medium, characterized in that The method comprises instructions, which, when executed on a computer, cause the computer to perform the method according to any one of claims 1 to 8.
13. A computer program product comprising instructions, characterized in that When the computer program product is run on a computer, the computer is caused to perform the method according to any one of claims 1 to 8.