Security data identification method and system based on natural language technology
By generating enhanced security data and training a security data identification model, the problems of scarce, poor adaptability and insufficient processing capabilities in the prior art are solved, and higher generalization capabilities and adaptability are achieved, and more accurate abnormal security data identification and more targeted network security protection decision support are provided.
Patent Information
- Application Number
- CN202411611345.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-12
- Publication Date
- 2025-03-04
AI Technical Summary
In the absence of data scarcity, poor adaptability and insufficient processing capabilities, it is difficult for the existing technology to effectively identify new network security incidents and unusual attack patterns, resulting in insufficient generalization capabilities of the model and the inability to respond to changes in the network security environment in a timely manner.
By collecting security data from historical small samples, augmented security data is generated, and the security data is trained to identify the model using the enhanced security data, and the model is updated through a pre-trained feedback mechanism to improve the generalization ability and adaptability of the model.
It greatly expands the scale of training data, improves the generalization ability of the model, can more accurately identify abnormal security data, and adapt to the ever-changing network security environment, providing more targeted decision-making support for network security protection.
Smart Images
Figure CN119557902A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of network security technology, and specifically to a security data identification method and system based on natural language technology. Background Art
[0002] With the rapid development of information technology, network attack methods are constantly being updated, and the complexity and diversity of security threats continue to increase. The inventors found the following problems in their research on existing security data identification:
[0003] 1. Data scarcity. In many cases, especially for new network security incidents or rare attack patterns, the amount of data that can be collected for training is very limited, which makes it difficult for traditional security data identification methods to fully learn the characteristics of various security incidents, resulting in insufficient generalization ability of the model and inability to accurately identify new or uncommon security anomalies;
[0004] 2. Poor performance in adapting to changes in the network security environment. The network security environment is in a dynamic state of change, and new attack strategies and security threats are constantly emerging. Therefore, the existing solidified security data identification often cannot be adjusted and updated in a timely manner, making it difficult to effectively respond to these changes;
[0005] 3. The processing of security data is often not comprehensive and in-depth enough, and the capabilities of data type identification, analysis, and semantic migration are limited. It is impossible to accurately identify abnormal security data and it is difficult to provide targeted decision-making support for network security protection, which brings huge risks to network security.
[0006] The rapid development of natural language technology provides the necessary technical means to fully exploit security data and thus solve the above problems. Therefore, in summary, the existing technology urgently needs a new technical solution for security data identification based on natural language technology. Summary of the invention
[0007] The purpose of this application is to provide a security data identification method and system based on natural language technology to solve the technical problems raised in the above background technology.
[0008] To achieve the above objectives, this application discloses the following technical solutions:
[0009] In a first aspect, the present application discloses a security data identification method based on natural language technology, the method comprising:
[0010] S1: Collect historical small sample security data; wherein the security data is used to describe network security incidents;
[0011] S2: Generate enhanced security data using the security data; wherein the scale of the enhanced security data is much larger than the scale of the security data;
[0012] S3: Using the enhanced security data to train a security data recognition model and output it; wherein the security data recognition model is used to perform data type recognition, data type analysis and semantic migration based on the security data, and output corresponding abnormal security data, wherein the abnormal security data is security data in the security data that characterizes network security anomalies;
[0013] S4: When there is generated real-time safety data, the safety data recognition model is run to output corresponding real-time abnormal safety data;
[0014] S5: Update the security data model using the pre-trained feedback mechanism, and re-execute steps S1 to S3.
[0015] Preferably, the process of collecting the historical small sample safety data specifically includes:
[0016] Determine the source of the security data of the historical small sample, which source includes at least network security logs and security incident reports; wherein, for the network security logs, use a preset data collection program to regularly scan the storage location, based on the log format and identification, filter out the records corresponding to the network security incidents, analyze the log structure, extract the corresponding information fields, and use natural language technology to transform the records and the information fields to obtain the corresponding security data; wherein, for the security incident reports, use a preset interface with the security management system to read and use natural language technology to transform the corresponding security incident reports to obtain the corresponding security data;
[0017] The sample size of the small sample is determined, and the sample size is positively correlated with the size of the network security log and the size of the security incident.
[0018] Preferably, the process of generating the enhanced security data specifically includes:
[0019] The security data is enhanced to obtain the enhanced security data; wherein the data enhancement at least includes randomly replacing text, adding or deleting text, and back-translating text.
[0020] Preferably, the training process of the security data recognition model specifically includes:
[0021] A security data recognition pre-model is constructed by using deep learning, and the security data recognition pre-model is trained by using the enhanced security data to perform the following steps A1 to A4:
[0022] A1: Obtain the security data;
[0023] A2: Using a preset attention mechanism to perform the data type identification, the data type analysis, and the semantic migration on the security data to obtain corresponding security features; wherein the attention mechanism is used to assign corresponding continuous weights to the input security data and perform the data type analysis and the semantic migration;
[0024] A3: using the security feature to match in a preset abnormal security feature library, when a corresponding abnormal security feature is matched, marking the corresponding security feature; wherein the abnormal security feature library stores the abnormal security feature corresponding to the abnormal security data;
[0025] A4: Outputting the security data corresponding to the security feature marked in step A3 and defining it as the abnormal security data;
[0026] The security data recognition pre-model that meets a preset accuracy threshold is defined as a security data recognition model and output.
[0027] Preferably, the data type analysis and semantic migration in step S2 specifically include:
[0028] A20: using a preset data type label to identify the data type of the security data, and when a corresponding data type label is identified, obtaining the corresponding security feature and executing step A3, otherwise executing steps A21 to A22;
[0029] A21: Utilize the attention mechanism to perform the data type analysis; wherein the data type analysis is specifically:
[0030] Using the attention mechanism to assign corresponding continuous weights to the input security data, calculate the feature vector corresponding to the security data, and output it;
[0031] A22: Using the attention mechanism to perform the semantic transfer; wherein the semantic transfer is specifically:
[0032] Call a preset semantic database, use the feature vector corresponding to the security data to match in the semantic database, output the matching result, migrate the semantics corresponding to the security data to the semantics corresponding to the matching result, obtain the corresponding security feature and execute step A3; wherein the semantic data stores the semantics corresponding to the feature vector with a semantic migration relationship, and the semantic migration relationship is used to characterize the relationship between the security data for semantic migration.
[0033] Preferably, the process of generating the real-time safety data specifically includes:
[0034] Continuously monitor the data packet traffic in the network and extract the corresponding real-time security data;
[0035] Continuously monitor the process activities in the network and extract the corresponding real-time security data.
[0036] Preferably, the output process of the real-time abnormal safety data specifically includes:
[0037] S40: Continue to determine whether the real-time security data exists, if yes, execute step S41, otherwise, repeat step S40;
[0038] S41: using a preset attention mechanism to perform data type identification, real-time data type analysis, and real-time semantic migration on the real-time security data to obtain corresponding real-time security features;
[0039] S42: using the real-time security feature to perform matching in the abnormal security feature library, and when a corresponding abnormal security feature is matched, marking the corresponding real-time security feature;
[0040] S43: Output the real-time safety data corresponding to the safety feature marked in step S42 and define it as the real-time abnormal safety data.
[0041] Preferably, the real-time data type analysis and the real-time semantic migration in step S41 specifically include:
[0042] S410: using the data type label to identify the data type of the real-time security data, when the corresponding data type label is identified, the corresponding real-time security feature is obtained and step S43 is executed, otherwise, steps S411 to S412 are executed;
[0043] S411: Utilizing the attention mechanism to perform the real-time data type analysis; wherein the real-time data type analysis specifically includes:
[0044] Using the attention mechanism to assign corresponding continuous weights to the input real-time safety data, calculate the real-time feature vector corresponding to the real-time safety data, and output it;
[0045] S412: Using the attention mechanism to perform the real-time semantic migration; wherein the real-time semantic migration is specifically:
[0046] Call the semantic database, use the real-time feature vector corresponding to the real-time security data to match in the semantic database, output the matching result, migrate the semantics corresponding to the real-time security data to the semantics corresponding to the matching result, obtain the corresponding real-time security feature and execute step S43.
[0047] Preferably, the updating process of the security data model specifically includes:
[0048] Running the feedback mechanism to receive user feedback on the abnormal safety data identification result and automatically comparing the difference between historical safety data and real-time safety data;
[0049] When there is a new data type, the model update is triggered and steps S1 to S3 are executed;
[0050] Obtain a preset verification set and verify the updated security data recognition model.
[0051] In a second aspect, the present application discloses a security data identification system based on natural language technology, the system is applicable to the security data identification method based on natural language technology as described above, and the system comprises:
[0052] A data collection module, wherein the data collection module is configured to: collect historical small sample security data; wherein the security data is used to describe network security events;
[0053] A data enhancement module, wherein the data enhancement module is configured to: generate enhanced security data using the security data; wherein the scale of the enhanced security data is much larger than the scale of the security data;
[0054] A model output module, wherein the model output module is configured to: train a security data recognition model using the enhanced security data and output the model; wherein the security data recognition model is used to perform data type recognition, data type analysis and semantic migration based on the security data, and output corresponding abnormal security data, wherein the abnormal security data is security data in the security data that characterizes network security anomalies;
[0055] A model operation module, wherein the model operation module is configured to: when there is generated real-time safety data, run the safety data recognition model to output corresponding real-time abnormal safety data;
[0056] A model updating module, wherein the model updating module is configured to: update the security data model using a pre-trained feedback mechanism, and re-run the data collection module, the data enhancement module, and the model output module.
[0057] Beneficial effects: The security data identification method and system based on natural language technology of the present application, when data is scarce, uses security data to generate enhanced security data, greatly expanding the scale of training data. The security data identification model trained based on enhanced security data learns more abundant security events, improves the generalization ability of the model, and further enhances the ability of the model to adapt to the ever-changing network security environment with the help of feedback mechanism; the data type identification, data type analysis and semantic migration of the security data identification model realize the comprehensive and in-depth processing of security data identification based on natural language technology, realize more accurate identification of abnormal security data, and provide more targeted decision support for network security protection. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.
[0059] Figure 1 A flowchart of a method for identifying secure data based on natural language technology provided in an embodiment of the present application;
[0060] Figure 2 A structural block diagram of a security data identification system based on natural language technology provided in an embodiment of the present application. DETAILED DESCRIPTION
[0061] The technical solutions in the embodiments of the present application are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.
[0062] In this article, the term "comprising" is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "comprising..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0063] In a first aspect, this embodiment discloses Figure 1 A security data identification method based on natural language technology is shown, the method comprising:
[0064] S1: Collect historical small sample security data; the security data is used to describe network security incidents;
[0065] S2: Generate enhanced security data using security data; wherein the scale of the enhanced security data is much larger than the scale of the security data;
[0066] S3: Use the enhanced security data to train a security data recognition model and output it; wherein the security data recognition model is used to perform data type recognition, data type analysis and semantic migration based on the security data, and output corresponding abnormal security data, where the abnormal security data is security data in the security data that represents network security anomalies;
[0067] S4: When there is generated real-time safety data, the safety data recognition model is run to output corresponding real-time abnormal safety data;
[0068] S5: Update the security data model using the pre-trained feedback mechanism and re-execute steps S1 to S3.
[0069] Based on the above, this embodiment uses security data to generate enhanced security data when data is scarce, which greatly expands the scale of training data. The security data recognition model trained based on enhanced security data learns more abundant security events, improves the generalization ability of the model, and further enhances the ability of the model to adapt to the ever-changing network security environment with the help of feedback mechanism. The data type recognition, data type analysis and semantic migration of the security data recognition model realize the comprehensive and in-depth processing of security data recognition based on natural language technology, realizes more accurate identification of abnormal security data, and provides more targeted decision support for network security protection.
[0070] Specifically, the process of collecting historical small sample safety data includes:
[0071] Determine the source of the historical small sample security data, which source includes at least network security logs and security incident reports; wherein, for network security logs, use a preset data collection program to regularly scan the storage location, based on the log format and identifier, filter out the records corresponding to the network security incident, analyze the log structure, extract the corresponding information field, and use natural language technology to transform the record and the information field to obtain the corresponding security data; wherein, for security incident reports, use a preset interface with the security management system to read and use natural language technology to transform the corresponding security incident report to obtain the corresponding security data;
[0072] Determine the sample size of the small sample, which is positively correlated with the size of the network security log and the size of the security incident.
[0073] In a specific application, for extracting text data from network security logs, a special data collection program is set up to regularly scan the network security log storage location to identify record entries containing specific keywords or conforming to a specific format. For example, keywords that may indicate security incidents, such as "abnormal access" and "unauthorized", are screened to extract relevant timestamps, source IP addresses, target addresses, operation descriptions and other information to form security data. For security incident reports, an interface with an existing security management system is used. When a new security incident report is generated, the report content in the form of a structured or semi-structured document is automatically read using natural language technology. It is understandable that in actual applications, this embodiment can also use existing web crawler technology to crawl discussion posts related to specific security topics in well-known network security technology forums in technical forum discussions, and use natural language technology to extract text content such as security incident experiences, analysis insights and solution descriptions shared by users.
[0074] Through the above, this embodiment collects security data from multiple channels, broadens the data sources, and can obtain more comprehensive and diversified security data, providing a richer information basis for subsequent security data identification, and realizing efficient collection and processing of small sample security data.
[0075] Specifically, the generation process of enhanced security data includes:
[0076] Performing data enhancement on the security data to obtain enhanced security data; wherein the data enhancement at least includes randomly replacing text, adding or deleting text, and back-translating text.
[0077] In a specific application, in a random text replacement operation, a text vocabulary is established based on the experience known to those skilled in the art, and the common words related to network security are stored in the text vocabulary. In the security data, a certain proportion of text is randomly selected, and a similar text is randomly selected from the text vocabulary for replacement. For example, if there is "hacker attack" in the original text, it can be randomly replaced with "network invasion" and other similar expressions; in the new text operation, text is also randomly selected from the text vocabulary and inserted at a random position in the security data, wherein the inserted word can be an adjective, adverb, noun or other different parts of speech, to increase the complexity of the text. For example, in "system In the text deletion operation, a certain proportion of the text in the security data is randomly selected for deletion, wherein the importance of the words can be considered when deleting to avoid deleting key information. For example, some less critical auxiliary words and prepositions can be deleted. In the back-translation operation, an online translation tool or a translation software package is used to translate the security data into another language, such as English, French, etc., and then translated back to the original language. The purpose of this design in this embodiment is that due to the different expressions and grammatical structures of different languages, the back-translated text may have some changes, thereby increasing the diversity of the data.
[0078] Based on the above, this embodiment effectively expands the scale of small sample security data through data enhancement, and provides more training samples for the security data recognition model.
[0079] Specifically, the training process of the security data recognition model includes:
[0080] Use deep learning to build a security data recognition pre-model, and use enhanced security data to train the security data recognition pre-model to perform the following steps A1 to A4:
[0081] A1: Obtain security data;
[0082] A2: Use the preset attention mechanism to identify the data type, analyze the data type, and transfer the semantics of the security data to obtain the corresponding security features. The attention mechanism is used to assign corresponding continuous weights to the input security data and perform data type analysis and semantic transfer.
[0083] A3: using the security feature to match in a preset abnormal security feature library, when matching the corresponding abnormal security feature, marking the corresponding security feature; wherein the abnormal security feature library stores the abnormal security features corresponding to the abnormal security data;
[0084] A4: Output the security data corresponding to the security feature marked in step A3 and define it as abnormal security data;
[0085] A security data recognition pre-model that meets a preset accuracy threshold is defined as a security data recognition model and output.
[0086] In a specific application, this embodiment uses a deep learning-based convolutional neural network architecture and natural language technology to construct and train a security data recognition pre-model, represents the input security data with word vectors in the security data recognition pre-model, converts the security data into a high-dimensional vector, and uses multiple convolutional layers with convolution kernels of different sizes to capture text patterns of different lengths to achieve feature extraction of the text. The pooling layer is used to reduce the dimensionality of the extracted features, and the most significant security features are screened out for matching in the abnormal security feature library.
[0087] It should be noted that the abnormal safety feature library of this embodiment is obtained based on the extraction of abnormal safety features known to those skilled in the art, and the preset accuracy threshold of this embodiment is an empirical value of the accuracy threshold selected when building a known model.
[0088] Based on the above, this embodiment realizes automatic learning of complex features and patterns in security data through the convolutional neural network and natural language technology of the security data recognition model, thereby improving the accuracy and efficiency of security data recognition.
[0089] Specifically, the data type analysis and semantic migration in step S2 specifically include:
[0090] A20: using a preset data type label to identify the data type of the security data, and when a corresponding data type label is identified, obtaining a corresponding security feature and executing step A3, otherwise executing steps A21 to A22;
[0091] A21: Use the attention mechanism to analyze data types; the data type analysis is specifically as follows:
[0092] Use the attention mechanism to assign corresponding continuous weights to the input security data, calculate the feature vector corresponding to the security data, and output it;
[0093] A22: Use the attention mechanism for semantic transfer; the semantic transfer is specifically:
[0094] Call the preset semantic database, use the feature vector corresponding to the security data to match in the semantic database, output the matching result, migrate the semantics corresponding to the security data to the semantics corresponding to the matching result to obtain the corresponding security feature and execute step A3; wherein the semantic data stores the semantics corresponding to the feature vector with a semantic migration relationship, and the semantic migration relationship is used to characterize the relationship between security data for semantic migration.
[0095] As a preferred implementation of this embodiment, on the basis of convolutional neural networks and natural language technology, an attention mechanism is added to calculate the importance weight of each feature, highlight the key features, and make the security data recognition model pay more attention to the important parts in the text to improve the accuracy of data type recognition and analysis. In this way, a more accurate feature vector is provided for semantic migration. It should be noted that the semantic database of this embodiment is built based on the semantics corresponding to the security data known to those skilled in the art. In the operation of semantic migration, the attention mechanism, convolutional neural network and natural language technology are used to parse the meaning and context of the security data, and classify or cluster them, so as to output the corresponding security features. For example, for the description of network security events, it can be judged whether it belongs to an abnormal security event, such as a network attack, vulnerability exploitation, etc., based on the semantic features.
[0096] Based on the above, this embodiment uses data type analysis and semantic migration to fully utilize the semantic database, improve the accuracy and efficiency of security feature extraction, and reduce dependence on small sample security data based on semantic data, thereby improving the generalization ability of the model.
[0097] Specifically, the process of generating real-time security data includes:
[0098] Continuously monitor the data packet traffic in the network and extract the corresponding real-time security data;
[0099] Continuously monitor process activities in the network and extract corresponding real-time security data.
[0100] In a simple example, this embodiment uses a network traffic monitoring device to monitor the data packet traffic in the network in real time, including information such as the source, destination, protocol type, and traffic size of the data packet. By monitoring the network traffic, abnormal traffic patterns can be discovered in a timely manner, such as a sudden increase in traffic, traffic from unknown sources, etc.; this embodiment uses security sensors installed in different locations of the system, such as servers and terminal devices, to monitor the status and behavior of the system in real time. For example, it monitors the process activities of the system, including file access, user login, and other behaviors.
[0101] Based on the above, this embodiment provides a real-time data basis for the operation of the security data identification model through the generation of real-time security data, and provides a basis for discovering abnormal security data in the network and taking corresponding security measures.
[0102] Specifically, the output process of real-time abnormal safety data includes:
[0103] S40: Continue to determine whether there is real-time security data, if yes, execute step S41, otherwise, repeat step S40;
[0104] S41: Use the preset attention mechanism to perform data type identification, real-time data type analysis, and real-time semantic migration on real-time security data to obtain corresponding real-time security features;
[0105] S42: using the real-time security feature to match in the abnormal security feature library, when matching the corresponding abnormal security feature, marking the corresponding real-time security feature;
[0106] S43: Output the real-time safety data corresponding to the safety feature marked in step S42 and define it as real-time abnormal safety data.
[0107] Specifically, the real-time data type analysis and real-time semantic migration in step S41 specifically include:
[0108] S410: using the data type label to identify the data type of the real-time security data, when the corresponding data type label is identified, the corresponding real-time security feature is obtained and step S43 is executed, otherwise, steps S411 to S412 are executed;
[0109] S411: Perform real-time data type analysis using the attention mechanism; wherein the real-time data type analysis specifically includes:
[0110] Use the attention mechanism to assign corresponding continuous weights to the input real-time security data, calculate the real-time feature vector corresponding to the real-time security data, and output it;
[0111] S412: Using the attention mechanism to perform real-time semantic migration; wherein the real-time semantic migration is specifically as follows:
[0112] Call the semantic database, use the real-time feature vector corresponding to the real-time security data to match in the semantic database, output the matching result, migrate the semantics corresponding to the real-time security data to the semantics corresponding to the matching result, obtain the corresponding real-time security feature and execute step S43.
[0113] It should be noted that the output of real-time abnormal safety data, real-time data type analysis and real-time semantic migration of the present embodiment correspond to the output, data type analysis and semantic migration of the aforementioned abnormal safety data, and only the processing objects therein are converted into real-time safety data. Therefore, the output of real-time abnormal safety data, real-time data type analysis and real-time semantic migration of the present embodiment that are not specifically described (including but not limited to technical effects, specific technical means, etc.) can be referred to the specific records in the aforementioned abnormal safety data output, data type analysis and semantic migration, and this text will not go into details here.
[0114] Specifically, the update process of the security data model includes:
[0115] The operation feedback mechanism receives user feedback on abnormal safety data identification results and automatically compares the differences between historical safety data and real-time safety data;
[0116] When there is a new data type, the model update is triggered and steps S1 to S3 are executed;
[0117] Obtain the preset verification set to verify the updated security data recognition model.
[0118] In a specific application, the feedback mechanism of this embodiment receives user feedback on abnormal security data identification results, designs a corresponding user feedback interface, and security personnel view the abnormal security data results output by the security data identification model on the interface, and confirm or correct them, and collect the security personnel's evaluation and suggestions on the identification results; when automatically comparing the differences between historical security data and real-time security data, a data comparison cycle is constructed, and based on the cycle, historical security data of a certain time period is regularly extracted from the historical security database, and compared with the real-time collected security data in multiple dimensions, for example, such as data type distribution, security event frequency, abnormal data characteristics, etc., so as to determine whether there are new data types, etc.
[0119] It should be noted that the validation set of this embodiment is an independent data set obtained based on historical security data, which is used to identify the performance and generalization ability of the updated security data model. The method of obtaining the validation set is any of the methods of obtaining the validation set in the existing model construction.
[0120] Based on the above, this embodiment provides a more comprehensive basis for updating the security data identification model through the integration of user feedback and the combination of automatic comparison technology, and continuously improves the security data identification model to make it better adapt to the ever-changing network security environment.
[0121] In a second aspect, this embodiment discloses Figure 2 A security data identification system based on natural language technology is shown, and the system is applicable to the security data identification method based on natural language technology as described above, and the system includes:
[0122] The data collection module is configured to: collect historical small sample security data; wherein the security data is used to describe network security events;
[0123] A data enhancement module, the data enhancement module is configured to: generate enhanced security data using the security data; wherein the scale of the enhanced security data is much larger than the scale of the security data;
[0124] The model output module is configured to: train the security data recognition model using the enhanced security data and output it; wherein the security data recognition model is used to perform data type recognition, data type analysis and semantic migration based on the security data, and output the corresponding abnormal security data, where the abnormal security data is security data in the security data that characterizes network security anomalies;
[0125] The model operation module is configured to: when there is generated real-time safety data, run the safety data recognition model to output corresponding real-time abnormal safety data;
[0126] Model update module,The model update module is configured to: update the security data model using the pre-trained feedback mechanism, and re-run the data collection module, the data enhancement module, and the model output module.
[0127] It should be noted that the security data identification system based on natural language technology of this embodiment corresponds to the aforementioned security data identification method based on natural language technology. Therefore, the parts that are not specifically described in the security data identification system based on natural language technology (including but not limited to technical effects, specific technical means, etc.) can refer to the specific records in the aforementioned security data identification method based on natural language technology, and this text will not go into details here.
[0128] In summary, the security data identification method and system based on natural language technology of this embodiment, when data is scarce, uses security data to generate enhanced security data, greatly expanding the scale of training data. The security data identification model trained based on enhanced security data learns more abundant security events, improves the generalization ability of the model, and further enhances the ability of the model to adapt to the ever-changing network security environment with the help of feedback mechanism. The data type identification, data type analysis and semantic migration of the security data identification model realize the comprehensive and in-depth processing of security data identification based on natural language technology, realize more accurate identification of abnormal security data, and provide more targeted decision support for network security protection.
[0129] In the embodiments provided in the present application, it should be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code or any appropriate combination thereof. For hardware implementation, the processor can be implemented in one or more of the following units: application specific integrated circuit (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), field programmable gate array (FPGA), processor, controller, microcontroller, microprocessor, other electronic units designed to implement the functions described herein or their combination. For software implementation, part or all of the flow of the embodiment can be completed by instructing the relevant hardware through a computer program. When implemented, the above program can be stored in a computer-readable storage medium or transmitted as one or more instructions or codes on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein the communication medium includes any medium that is convenient for transmitting a computer program from one place to another. The storage medium can be any available medium that a computer can access. The computer-readable storage medium can include but is not limited to RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer.
[0130] Finally, it should be noted that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. A security data identification method based on natural language technology, characterized in that: The method includes: S1: Collect historical small sample security data; wherein the security data is used to describe network security incidents; S2: Generate enhanced security data using the security data; wherein the scale of the enhanced security data is much larger than the scale of the security data; S3: Using the enhanced security data to train a security data recognition model and outputting the model; wherein the security data recognition model is used to perform data type recognition, data type analysis and semantic migration based on the security data, and output corresponding abnormal security data, wherein the abnormal security data is security data in the security data that characterizes network security anomalies; S4: When there is generated real-time safety data, the safety data recognition model is run to output corresponding real-time abnormal safety data; S5: Update the security data model using the pre-trained feedback mechanism, and re-execute steps S1 to S3.
2. The method for identifying security data based on natural language technology according to claim 1, characterized in that: The process of collecting the historical small sample safety data specifically includes: Determine the source of the security data of the historical small sample, which source includes at least network security logs and security incident reports; wherein, for the network security logs, use a preset data collection program to regularly scan the storage location, based on the log format and identification, filter out the records corresponding to the network security incidents, analyze the log structure, extract the corresponding information fields, and use natural language technology to transform the records and the information fields to obtain the corresponding security data; wherein, for the security incident reports, use a preset interface with the security management system to read and use natural language technology to transform the corresponding security incident reports to obtain the corresponding security data; The sample size of the small sample is determined, and the sample size is positively correlated with the size of the network security log and the size of the security incident.
3. The method for identifying security data based on natural language technology according to claim 1, characterized in that: The generation process of the enhanced security data specifically includes: The security data is enhanced to obtain the enhanced security data; wherein the data enhancement at least includes randomly replacing text, adding or deleting text, and back-translating text.
4. The method for identifying security data based on natural language technology according to claim 1, characterized in that: The training process of the security data recognition model specifically includes: A security data recognition pre-model is constructed by using deep learning, and the security data recognition pre-model is trained by using the enhanced security data to perform the following steps A1 to A4: A1: Obtain the security data; A2: Using a preset attention mechanism to perform the data type identification, the data type analysis, and the semantic migration on the security data to obtain corresponding security features; wherein the attention mechanism is used to assign corresponding continuous weights to the input security data and perform the data type analysis and the semantic migration; A3: using the security feature to match in a preset abnormal security feature library, when a corresponding abnormal security feature is matched, marking the corresponding security feature; wherein the abnormal security feature library stores the abnormal security feature corresponding to the abnormal security data; A4: Outputting the security data corresponding to the security feature marked in step A3 and defining it as the abnormal security data; The security data recognition pre-model that meets a preset accuracy threshold is defined as a security data recognition model and output.
5. The method for identifying security data based on natural language technology according to claim 4, characterized in that: The data type analysis and semantic migration in step S2 specifically include: A20: using a preset data type label to identify the data type of the security data, and when a corresponding data type label is identified, obtaining the corresponding security feature and executing step A3, otherwise executing steps A21 to A22; A21: Utilize the attention mechanism to perform the data type analysis; wherein the data type analysis is specifically: Using the attention mechanism to assign corresponding continuous weights to the input security data, calculate the feature vector corresponding to the security data, and output it; A22: Using the attention mechanism to perform the semantic transfer; wherein the semantic transfer is specifically: Call a preset semantic database, use the feature vector corresponding to the security data to match in the semantic database, output the matching result, migrate the semantics corresponding to the security data to the semantics corresponding to the matching result, obtain the corresponding security feature and execute step A3; wherein the semantic data stores the semantics corresponding to the feature vector with a semantic migration relationship, and the semantic migration relationship is used to characterize the relationship between the security data for semantic migration.
6. The method for identifying security data based on natural language technology according to claim 1, characterized in that: The generation process of the real-time safety data specifically includes: Continuously monitor the data packet traffic in the network and extract the corresponding real-time security data; Continuously monitor the process activities in the network and extract the corresponding real-time security data.
7. The method for identifying security data based on natural language technology according to claim 5, characterized in that: The output process of the real-time abnormal safety data specifically includes: S40: Continue to determine whether the real-time security data exists, if yes, execute step S41, otherwise, repeat step S40; S41: using a preset attention mechanism to perform data type identification, real-time data type analysis, and real-time semantic migration on the real-time security data to obtain corresponding real-time security features; S42: using the real-time security feature to perform matching in the abnormal security feature library, and when a corresponding abnormal security feature is matched, marking the corresponding real-time security feature; S43: Output the real-time safety data corresponding to the safety feature marked in step S42 and define it as the real-time abnormal safety data.
8. The method for identifying security data based on natural language technology according to claim 7, characterized in that: The real-time data type analysis and the real-time semantic migration in step S41 specifically include: S410: using the data type label to identify the data type of the real-time security data, when the corresponding data type label is identified, the corresponding real-time security feature is obtained and step S43 is executed, otherwise, steps S411 to S412 are executed; S411: Utilizing the attention mechanism to perform the real-time data type analysis; wherein the real-time data type analysis specifically includes: Using the attention mechanism to assign corresponding continuous weights to the input real-time safety data, calculate the real-time feature vector corresponding to the real-time safety data, and output it; S412: Using the attention mechanism to perform the real-time semantic migration; wherein the real-time semantic migration is specifically: Call the semantic database, use the real-time feature vector corresponding to the real-time security data to match in the semantic database, output the matching result, migrate the semantics corresponding to the real-time security data to the semantics corresponding to the matching result, obtain the corresponding real-time security feature and execute step S43.
9. The method for identifying security data based on natural language technology according to claim 1, characterized in that: The updating process of the security data model specifically includes: Running the feedback mechanism to receive user feedback on the abnormal safety data identification result and automatically comparing the difference between historical safety data and real-time safety data; When there is a new data type, the model update is triggered and steps S1 to S3 are executed; Obtain a preset verification set and verify the updated security data recognition model.
10. A security data identification system based on natural language technology, the system is applicable to the security data identification method based on natural language technology as described in any one of claims 1 to 9, characterized in that: The system includes: A data collection module, wherein the data collection module is configured to: collect historical small sample security data; wherein the security data is used to describe network security events; A data enhancement module, wherein the data enhancement module is configured to: generate enhanced security data using the security data; wherein the scale of the enhanced security data is much larger than the scale of the security data; A model output module, wherein the model output module is configured to: train a security data recognition model using the enhanced security data and output the model; wherein the security data recognition model is used to perform data type recognition, data type analysis and semantic migration based on the security data, and output corresponding abnormal security data, wherein the abnormal security data is security data in the security data that characterizes network security anomalies; A model operation module, wherein the model operation module is configured to: when there is generated real-time safety data, run the safety data recognition model to output corresponding real-time abnormal safety data; A model updating module, wherein the model updating module is configured to: update the security data model using a pre-trained feedback mechanism, and re-run the data collection module, the data enhancement module, and the model output module.