A data analysis method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a data analysis method and device, electronic equipment and storage medium, the method comprising: obtaining a topic set of a security event document; the topic set comprising at least one topic feature sub-set, the topic feature sub-set corresponding to a topic of the security event document, and the topic feature sub-set comprising a first word feature belonging to the corresponding topic; obtaining a plurality of category sets in a sample library; the category set corresponding to a category, and the category set comprising a second word feature belonging to the corresponding category; determining a target category to which the security event document belongs according to a first conditional probability of each category set under the topic set; and updating the second word features included in the target category set corresponding to the target category by using the first word features of at least part of the topic feature sub-sets in the topic set. As the category set comprises more and more second word features with high accuracy, the accuracy of classifying the security event document by using the sample library will also be improved.

Description

Technical Field

[0001] This application relates to the field of data analysis technology, and more specifically, to a data analysis method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of the internet and the continuous maturation of digital technologies, the application and services of big data are constantly deepening and expanding in scale. Faced with technical pressures and resource constraints in both data management and analysis, the use of machine learning and training for massive data mining has significantly reduced labor costs. However, improving the accuracy of data analysis during model-based data mining and analysis remains a pressing technical challenge in this field. Summary of the Invention

[0003] The purpose of this application is to provide a data analysis method, apparatus, electronic device, and storage medium to achieve the technical effect of improving the accuracy of data analysis.

[0004] A first aspect of this application provides a data analysis method, the method comprising:

[0005] Obtain a set of topics for security event documents; the set of topics includes at least one subset of topic features, the subset of topic features corresponds to the topic of the security event document, and the subset of topic features includes a first word feature belonging to the corresponding topic.

[0006] Obtain multiple category sets from the sample library; the category sets correspond to categories, and the category sets include second word features belonging to the corresponding categories;

[0007] The target category to which the security event document belongs is determined based on the first conditional probability of each category set under the topic set;

[0008] The second word features included in the target category set corresponding to the target category are updated using the first word features of at least a subset of the topic features in the topic set.

[0009] In the above implementation process, after obtaining the topic set of security event documents and multiple category sets in the sample library, the security event documents are classified according to multiple first conditional probabilities to determine their respective categories. Subsequently, the first word features of the topic feature subset are updated to the target category set, making the second word features included in the category sets in the sample library increasingly richer. Furthermore, since the category set to be updated is determined based on the category to which the security event document belongs, the accuracy of the word features added to the target category set is guaranteed. As the category set includes more and more high-accuracy second word features, the accuracy of classifying security event documents using the sample library also improves. Thus, improving the accuracy of the sample library improves the accuracy of data analysis.

[0010] Furthermore, the set of topics for obtaining security event documents includes:

[0011] Obtain the security event document;

[0012] The topics of the security event documents are extracted using a topic generation model, resulting in a topic feature subset corresponding to each topic and the topic set.

[0013] In the above implementation process, the topic of the security event document is extracted by the topic generation model, thereby obtaining a topic feature subset with its first word feature, so that the security event document can be classified according to the first conditional probability of each category set under the topic set, and the category to which the security event document belongs can be determined.

[0014] Furthermore, obtaining the security event document includes:

[0015] Capture security incident data from network security devices;

[0016] The security event data is subjected to target processing to obtain security event documents; the target processing includes one or more of the following: standardization processing, cleaning processing, merging processing, and storage processing.

[0017] In the above implementation process, security event data is captured from network security devices, and the sample library is updated using real network traffic data, making the second-word features in the sample library more closely resemble real-world situations. Furthermore, preprocessing the security event data improves the accuracy of subsequent topic and other feature extraction.

[0018] Furthermore, the process of obtaining the first word feature includes:

[0019] Retrieve all keywords belonging to the topic from the security incident document;

[0020] Calculate the similarity between each keyword and the topic;

[0021] Keywords with a similarity greater than a threshold are identified as the first word features belonging to the topic.

[0022] In the above implementation process, keywords are filtered based on their similarity to the topic, ensuring that the topic feature subset includes word features of keywords closely related to the topic. This improves the accuracy of subsequent classification and ensures that the word features added to the target category set have a high correlation with the category when updating the target category set using the first word features.

[0023] Furthermore, the sample library includes an interface for receiving updated data; the updated data is used to update the sample library.

[0024] In the above implementation process, by embedding an interface into the sample library to receive data for updating the sample library, the quality of the sample library is improved, thereby improving the accuracy of classification using the sample library.

[0025] Further, determining the target category to which the security event document belongs based on the first conditional probability of each category set under the topic set includes:

[0026] In response to the sample library receiving the updated data through the interface, the updated category set is determined;

[0027] Calculate the second conditional probability of the updated category set under the topic set;

[0028] The target category to which the security document belongs is determined based on the first conditional probability and the second conditional probability.

[0029] In the above implementation process, for the dynamically changing sample library, this embodiment proposes to perform secondary classification of security event documents by calculating the second conditional probability of the updated category set under the topic set during the classification process, so as to obtain more accurate data.

[0030] Furthermore, the security event document includes multiple documents; the method further includes:

[0031] Acquire knowledge and intelligence data;

[0032] Based on the knowledge intelligence data, establish the association between multiple security event documents to obtain attack chain data.

[0033] In the above implementation process, knowledge intelligence data is introduced to establish relationships between multiple security incident documents, thereby forming an attack chain from the attacker to the victim. This reveals the implicit relationships between multiple data points, yielding valuable patterns and rules, and enhancing the data's value.

[0034] A second aspect of this application provides a data analysis apparatus, the apparatus comprising:

[0035] The first acquisition module is used to acquire a set of topics for security event documents; the set of topics includes at least one subset of topic features, the subset of topic features corresponds to the topic of the security event document, and the subset of topic features includes a first word feature belonging to the corresponding topic;

[0036] The second acquisition module is used to acquire multiple category sets in the sample library; the category set corresponds to a category, and the category set includes second word features belonging to the corresponding category;

[0037] The classification module is used to determine the target category to which the security event document belongs based on the first conditional probability of the topic set under multiple category sets;

[0038] An update module is used to update the second word features included in the target category set corresponding to the target category using the first word features of at least a subset of the topic features in the topic set.

[0039] A third aspect of this application provides an electronic device, the electronic device comprising:

[0040] processor;

[0041] Memory used to store processor-executable instructions;

[0042] Wherein, when the processor invokes the executable instructions, it implements the operation of any of the methods described in the first aspect.

[0043] A fourth aspect of this application provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of any of the methods described in the first aspect. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A flowchart illustrating a data analysis method provided in an embodiment of this application;

[0046] Figure 2 A flowchart illustrating another data analysis method provided in an embodiment of this application;

[0047] Figure 3 A flowchart illustrating another data analysis method provided in an embodiment of this application;

[0048] Figure 4 A flowchart illustrating another data analysis method provided in an embodiment of this application;

[0049] Figure 5 A flowchart illustrating another data analysis method provided in an embodiment of this application;

[0050] Figure 6 A flowchart illustrating another data analysis method provided in an embodiment of this application;

[0051] Figure 7 A structural block diagram of a data analysis device provided in an embodiment of this application;

[0052] Figure 8 This is a hardware structure diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0053] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0054] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0055] With the development of the internet and the continuous maturation of digital technologies, the application and services of big data are deepening and expanding in scale. Faced with technical pressures and resource constraints in data management and analysis, the use of machine learning and training for massive data mining has significantly reduced labor costs. However, improving the accuracy of data analysis during model-based data mining and analysis remains a pressing technical challenge in this field.

[0056] Therefore, this application provides a data analysis method, including as follows: Figure 1 The steps 110-130 are shown. Steps 110 and 120 do not have a specific execution order and can be executed simultaneously.

[0057] Step 110: Obtain the subject set of security event documents;

[0058] The topic set includes at least one topic feature subset, which corresponds to the topic of the security event document, and the topic feature subset includes a first word feature belonging to the corresponding topic.

[0059] Security incident documents refer to security events generated by network traffic or applications, stored or represented in document form. As is understandable, each document has its own topic, and security incident documents are no exception; each security incident document also has its own topic. A topic may include one or more. For example, the topic could be "attack"; or it could be "attacker" and "victim." Thus, each security incident document has a corresponding set of topics. The elements included in the topic set are topic feature subsets, and these topic feature subsets correspond to the topics of the security incident document. For example, one topic feature subset corresponds to one topic of the security incident document. Therefore, the number of topic feature subsets matches, for example, the number of topics in the security incident document. Thus, the topic feature subsets corresponding to each topic of a security incident document constitute the topic set.

[0060] For words in a security incident document, they can be categorized into different topics within that document. Therefore, for each word in a security incident document, its word features can be assigned to a subset of topic features corresponding to its respective topic as the first word feature. In other words, the elements of the topic feature subset are the first word features. The first word feature refers to the word features belonging to the topic corresponding to that specific topic feature subset.

[0061] Optionally, words in the security incident document can be converted into word features using a word vector model. The word vectors output by the word vector model are the word features of the words. Optionally, the word vector model can be a Word2vec (word to vector) model.

[0062] For example, the set of topics for security event documents can be represented as: topic set x = {a1, a2, ..., a...} m} represents that the security event document includes m topics, and each topic corresponds to a subset of topic features. i Let a be an element in the topic set, i.e., a subset of topic features. Where 1 ≤ i ≤ m. Let a be the i-th subset of topic features in the topic set x. i ={b i1 b i2 , ..., b ir} represents the i-th topic feature subset including r first word features.

[0063] Step 120: Obtain multiple category sets from the sample library;

[0064] The category set corresponds to a category, and the category set includes second word features belonging to the corresponding category.

[0065] The sample library includes multiple category sets, with each category set corresponding to a specific category. For example, one category set corresponds to one category. A category refers to the classification of a security incident document, which can be related to the document's topic. For instance, the topic of a security incident document might include attacker and victim. The categories in the sample library could include attack categories. Therefore, when classifying a security incident document, based on its topic, it might be classified as an attack category.

[0066] The elements of the category set are the second word features. The second word features refer to the word features belonging to the category corresponding to that category set. These second word features in the category set are selected based on historical experience and are word features whose corresponding category is known.

[0067] The sample library comprises multiple category sets, which can also be understood as a total set, with the multiple category sets serving as elements within that total set. Thus, for example, the sample library can be represented as: C = {y1, y2, ..., y...} n} represents that the sample library includes n category sets, and each category set corresponds to one category. y j Let be the elements in the sample library, i.e., the category set. Where 1 ≤ j ≤ n. Let y be the j-th category set in the sample library C. j ={d j1 d j2 , ..., d js} represents that the j-th category set includes s second word features.

[0068] Step 130: Determine the target category to which the security event document belongs based on the first conditional probability of each category set under the topic set;

[0069] Conditional probability refers to the probability of an event occurring given that another event has already occurred. The first conditional probability of a category set under a topic set refers to the conditional probability of the category set given that the topic set has occurred. Since there are multiple category sets, there are multiple first conditional probabilities. The first conditional probability corresponding to each category set represents the probability that the topic set belongs to the corresponding category of that category set, that is, the probability that the security event document to which the topic set belongs belongs to the corresponding category of that category set.

[0070] For example, the various category sets y j The first conditional probability under the topic set x can be expressed as: P(y1|x), P(y2|x), ..., P(y n |x).

[0071] It can be seen that the target category to which a security event document belongs can be determined based on multiple first condition probabilities. For example, by comparing the magnitudes of multiple first condition probabilities, the category corresponding to the largest first condition probability can be determined as the target category to which the security event document belongs. That is, P(y k |x)=max{P(y1|x), P(y2|x),..., P(y n |x)},x∈y k This indicates that the security event document belongs to the category set y. k The corresponding category.

[0072] Specifically, the category set y j The first conditional probability P(y) under the topic set x j The calculation process for |x) is as follows:

[0073] When all feature attributes are conditionally independent, Bayes' theorem leads to the following derivation:

[0074]

[0075] Since the denominator P(x) in the above formula (1) is a constant among multiple first conditional probabilities, to find the largest first conditional probability, we only need to find the largest numerator P(x|y). j )P(y j That's all.

[0076] Since each feature attribute is conditionally independent, the following formula (2) is derived:

[0077]

[0078] Therefore, when comparing multiple first conditional probabilities, we can follow the above formula (2) to calculate and compare the values ​​of each category set calculated by formula (2) to know the target category of the security event document.

[0079] Step 140: Update the second word features included in the target category set corresponding to the target category using the first word features of at least a portion of the topic feature subset in the topic set.

[0080] After determining the target category to which the security incident document belongs, the second word features in the target category set can be updated using the first word features included in at least a subset of the topic features in the topic set. For example, first word features not included in the target category set can be added to the target category set, making the first word features the second word features in the target category set.

[0081] Optionally, the second word features of the target category set can be updated using the first word features included in each of the topic feature subsets of the topic set. In other words, first word features not included in the target category set from any of the topic feature subsets can be added to the target category set.

[0082] Optionally, in step 130, the first conditional probability P(y) is calculated. j When |x), the feature subsets of each topic in the category set y were calculated according to formula (2). j The third conditional probability P(a) i |y j Thus, a target subset for updating the target category set can be selected based on the third conditional probability. For example, the subset of topic features corresponding to the third conditional probability greater than a preset probability threshold can be identified as the target subset, and the first word features included in the target subset can be used to update the target category set. That is, the first word features in the target subset that are not included in the target category set are added to the target category set.

[0083] For example, if the category set y1 is determined to be the target category, as an option, all topic feature subsets in the topic set x, i.e., a1 to a1, can be utilized. m The second word features in the target category set y1 are updated using all the first word features. Alternatively, the second word features can be updated based on the third conditional probabilities P(a1|y1), P(a2|y1), ..., P(a m The value of |y1) is used to determine the third conditional probability greater than the probability threshold, for example, P(a1|y1). Then, the topic feature subset a1 is determined as the target subset, and the first word feature b in the target subset is used. 11 b 12 , ..., b 1r This is used to update the second word feature in the target category set y1. For example, the updated category set y1 is: y1 = {d} 11 d 12 , ..., d 1s b 11 b 12 , ..., b 1r}

[0084] As can be seen, the data analysis method provided in this application, after obtaining the topic set of security event documents and multiple category sets in the sample library, classifies the security event documents according to multiple first conditional probabilities to determine the category to which the security event documents belong. Subsequently, the first word features of the topic feature subset are updated to the target category set, making the second word features included in the category sets in the sample library increasingly richer. Furthermore, since the category set to be updated is determined based on the category to which the security event document belongs, the accuracy of the word features added to the target category set can be guaranteed. As the category set includes more and more high-accuracy second word features, the accuracy of classifying security event documents using the sample library will also improve. Thus, the accuracy of data analysis is improved by improving the accuracy of the sample library.

[0085] The following provides a detailed description of steps 110-140.

[0086] Regarding step 110, in some embodiments, it includes, for example... Figure 2 Steps 111-112 are shown.

[0087] Step 111: Obtain the security event document;

[0088] Step 112: Use the topic generation model to extract the topic of the security event document, and obtain the topic feature subset corresponding to the topic and the topic set.

[0089] Topic generation models are used to extract topics from security incident documents. For example, a topic generation model can be an LDA (Latent Dirichlet Allocation) model. LDA is an unsupervised model that assumes each document has one or more topics, and each topic corresponds to different keywords. Therefore, using an LDA model, the topics of a security incident document, as well as the words corresponding to each topic, can be output. Feature extraction of the topics and their corresponding words yields a subset of topic features, including the first word features. All subsets of topic features constitute the topic set.

[0090] In this embodiment, the topic of the security event document is extracted by the topic generation model, thereby obtaining a topic feature subset with its first word feature, so that the security event document can be classified according to the first conditional probability of each category set under the topic set, and the category to which the security event document belongs can be determined.

[0091] Furthermore, in some embodiments, step 111 includes as follows: Figure 3 The steps shown are as follows:

[0092] Step 1111: Capture security event data from network security devices.

[0093] Network security devices can include, but are not limited to, security testing devices and vulnerability scanning devices. Security testing devices can passively monitor network traffic and applications, while vulnerability scanning devices can actively scan network traffic and applications for vulnerabilities. Both types of network security devices can generate security event data.

[0094] The implementer of the method in this embodiment can be, for example, an electronic device that is communicatively connected to a network security device. This electronic device is equipped with a data acquisition module, which captures security event data from the network security device.

[0095] Step 1112: Perform target processing on the security event data to obtain a security event document.

[0096] The target processing includes one or more of the following: standardization processing, cleaning processing, merging processing, and storage processing.

[0097] As an alternative example, since security incident data may originate from different network security devices, these different security incident data sets may carry different information. To facilitate subsequent topic extraction and other processing, after capturing the security incident data, it can be standardized according to a custom specification. For example, electronic devices can be equipped with a data standardization module to map fields of security incident data from different sources, unifying them into standardized security incident data with a standardized structure.

[0098] As an alternative example, after capturing security incident data, the data can be filtered according to preset cleaning rules. These cleaning rules may include, but are not limited to, removing data lacking critical information, removing data from unknown sources, etc. For instance, electronic devices may be equipped with a data cleaning module for filtering security incident data according to these cleaning rules.

[0099] As an alternative example, duplicate security incident data can be merged to reduce the amount of data processing required in subsequent steps.

[0100] As an alternative example, security incident data can be stored in the form of documents. For instance, an electronic device can be equipped with a data storage module for storing security incident data. Because the security incident data is stored in document form, what is retrieved when the data is read is a security incident document.

[0101] For example, security event data captured from network security devices can be stored as a document after undergoing standardization, cleaning, and merging processes in sequence.

[0102] In this embodiment, security event data is captured from network security devices, and the sample library is updated using real network traffic data, making the second-word features in the sample library more closely resemble real-world situations. Furthermore, preprocessing the security event data improves the accuracy of subsequent topic and other feature extraction.

[0103] Based on such Figures 1-3 In any of the described embodiments, regarding the process of obtaining the first word feature, in some embodiments, for each topic of a security event document, all keywords belonging to that topic can be determined in the security event document, and the word features of all keywords can be determined as the first word feature in the topic feature subset corresponding to that topic.

[0104] In other embodiments, for each topic of the security incident document, by means of, Figure 4 The steps shown are used to obtain the first word feature in the topic feature subset corresponding to the topic.

[0105] Step 410: Obtain all keywords belonging to the topic from the security incident document.

[0106] For example, the keywords included in each topic can be output using an LDA model. For instance, by using an LDA model to extract topics from security incident documents, a probability distribution of K topics can be obtained, where each topic's probability distribution contains T keywords. Here, K and T are both positive integers.

[0107] Step 420: Calculate the similarity between each keyword and the topic.

[0108] For example, the Word2vec model can be used to convert topics and keywords into topic features and word features, and then the Word2vec model can be used to calculate the similarity between the word features and topic features of each keyword under the topic.

[0109] Step 430: Determine keywords with a similarity greater than the threshold as the first word feature belonging to the topic.

[0110] Based on the similarity between the word features of each keyword under each topic and the topic features, the first word features can be selected. For example, for the T keywords obtained in step 410, based on the similarity between the word features of the T keywords and the topic features, r keywords with similarity greater than a threshold can be selected. The word features corresponding to these r keywords are then determined as the first word features belonging to that topic, thus obtaining the topic feature subset corresponding to that topic. Where r ≤ T.

[0111] In this embodiment, keywords are filtered based on their similarity to the topic, ensuring that the topic feature subset includes word features of keywords closely related to the topic. This improves the accuracy of subsequent classification and ensures that when updating the target category set using the first word features, the word features added to the target category set have a high correlation with that category.

[0112] Regarding the sample library, in some embodiments, the sample library includes an interface for receiving updated data to update the sample library.

[0113] For example, the interface includes multiple interfaces, each for receiving different types of data.

[0114] For example, the interface includes a sample interface for receiving word features. That is, the word features are the updated data described above. The word features are used to update the second word features in a partial category set. As one example, the word features may be the first word features of at least a subset of topic features in a topic set; the category set updated by the first word features may be the target category set determined through the above embodiments. That is, in step 140, the first word features are sent to the target category set through the sample interface embedded in the sample library to update the second word features.

[0115] Of course, the word features received by the sample interface can be not only the first word features, but also word features for a partial set of categories determined by other means. This application does not limit this.

[0116] For example, the interface includes a human interface for receiving a newly added category set input by a human. That is, the newly added category set is the updated data described above. It is understood that security incident documents may have newly added categories. Therefore, the sample library can receive the newly added category set by embedding a human interface.

[0117] Thus, in this embodiment, by embedding an interface in the sample library to receive data for updating the sample library, the quality of the sample library is improved, thereby increasing the accuracy of classification using the sample library.

[0118] Once the sample library has an interface, data can be updated through the interface at any time, making the data in the sample library dynamic. Simultaneously, obtaining a security event document triggers a data analysis process. Therefore, in some scenarios, updating the sample library and classifying data using the sample library may occur simultaneously. Thus, in some embodiments, step 130 above may include, for example... Figure 5 The steps shown are as follows:

[0119] Step 131: In response to the sample library receiving the updated data through the interface, determine the updated category set;

[0120] Step 132: Calculate the second conditional probability of the updated category set under the topic set;

[0121] Step 133: Determine the target category to which the security document belongs based on the first conditional probability and the second conditional probability.

[0122] For example, if the sample library receives word features through the sample interface, the updated category set is determined to be the category set updated by the word features. For instance, if the word features are the first word features mentioned above, the updated category set is the target category set mentioned above. It is known that the elements in the updated category set have changed, so the second conditional probability of the updated category set under the topic set is calculated.

[0123] Specifically, for the updated category set, the first conditional probability before the update may have been calculated before step 131, i.e., before the updated category set is updated. In step 132, the second conditional probability after the update is calculated, and the second conditional probability overrides the first conditional probability. That is, when determining whether a security event document belongs to the updated category set, the updated second conditional probability is used for the determination.

[0124] For example, if the sample database receives a new set of categories through a manual interface, then the updated set of categories is determined to be the new set of categories. In this case, it is only necessary to calculate the second conditional probability of the new set of categories under the topic set. That is, based on the already calculated first conditional probability, the second conditional probability corresponding to the new set of categories is calculated.

[0125] Finally, based on the first conditional probability and the second conditional probability, the target category to which the security event document belongs is determined.

[0126] Before completing step 133, the sample database may be updated. To ensure classification accuracy, if the triggering condition of step 131 is met before completing the entire data analysis process, i.e., the sample database receives updated data through the interface, then the process returns to execute steps 131-133.

[0127] As can be seen, for a dynamically changing sample library, this embodiment proposes to perform secondary classification of security event documents by calculating the second conditional probability of the updated category set under the topic set during the classification process, so as to obtain more accurate data.

[0128] In some embodiments, security event documents include multiple documents. Therefore, based on any of the above embodiments, the data analysis method provided in this application may further include, for example... Figure 6 The steps shown are as follows:

[0129] Step 610: Obtain knowledge intelligence data;

[0130] Step 620: Establish the association between multiple security event documents based on the knowledge intelligence data to obtain attack chain data.

[0131] The security incident documents include multiple documents, and each document can be analyzed using any of the above embodiments. However, the above analysis process only utilizes the semantics and document information of the security incident documents themselves, without combining it with other external information. Therefore, this embodiment proposes to use knowledge intelligence data, one type of external information, to establish relationships between multiple security incident documents.

[0132] Optionally, knowledge intelligence data can be associated with the target categories of security incident documents. In this way, knowledge intelligence data is associated with the target categories corresponding to different security incident documents, forming a topological relationship, and thus multiple security incident documents are also associated through this topological relationship.

[0133] Optionally, knowledge intelligence data can be directly linked to security incident documents. In this way, knowledge intelligence data is associated with different security incident documents, and multiple security incident documents establish relationships through knowledge intelligence data.

[0134] For example, suppose there are security event documents A and B. Security event document A belongs to the category "scanning," and its content identifies the scanner as A and the scanned party as B. Security event document B belongs to the category "attack," and its content identifies the attacker as B and the attacked party as C. If there is also knowledge intelligence data about the attack phase indicating that scanning could be a step performed in the preceding stage of an attack, then security event documents A and B can be linked based on this knowledge intelligence data, forming an attack chain from A to C. Therefore, knowledge intelligence data can serve as evidence of the established relationships between multiple security event documents.

[0135] Furthermore, the established relationships and / or the attack chain from the attacker to the victim can be visualized. For example, this can be presented in the form of reports or flowcharts.

[0136] As can be seen, in this embodiment, knowledge intelligence data is introduced to establish relationships between multiple security event documents, thereby forming an attack chain from the attacker to the victim. This reveals the implicit relationships between multiple data points, yielding valuable patterns and rules, and enhancing the value of the data.

[0137] Based on any of the above embodiments, this application also provides a data analysis device, such as... Figure 7As shown, the data analysis device 700 includes:

[0138] The first acquisition module 710 is used to acquire a topic set of security event documents; the topic set includes at least one topic feature subset, the topic feature subset corresponds to the topic of the security event document, and the topic feature subset includes a first word feature belonging to the corresponding topic.

[0139] The second acquisition module 720 is used to acquire multiple category sets in the sample library; the category sets correspond to categories, and the category sets include second word features belonging to the corresponding categories;

[0140] The classification module 730 is used to determine the target category to which the security event document belongs based on the first conditional probability of the topic set under multiple category sets;

[0141] The update module 740 is used to update the second word features included in the target category set corresponding to the target category using the first word features of at least a portion of the topic feature subset in the topic set.

[0142] In some embodiments, the first acquisition module 710 is specifically used for:

[0143] Obtain the security event document;

[0144] The topics of the security event documents are extracted using a topic generation model, resulting in a topic feature subset corresponding to each topic and the topic set.

[0145] In some embodiments, the first acquisition module 710 is specifically used for:

[0146] Capture security incident data from network security devices;

[0147] The security event data is subjected to target processing to obtain security event documents; the target processing includes one or more of the following: standardization processing, cleaning processing, merging processing, and storage processing.

[0148] In some embodiments, the process of obtaining the first word feature includes:

[0149] Retrieve all keywords belonging to the topic from the security incident document;

[0150] Calculate the similarity between each keyword and the topic;

[0151] Keywords with a similarity greater than a threshold are identified as the first word features belonging to the topic.

[0152] In some embodiments, the sample library includes an interface for receiving update data; the update data is used to update the sample library.

[0153] In some embodiments, the classification module 730 is specifically used for:

[0154] In response to the sample library receiving the updated data through the interface, the updated category set is determined;

[0155] Calculate the second conditional probability of the updated category set under the topic set;

[0156] The target category to which the security document belongs is determined based on the first conditional probability and the second conditional probability.

[0157] In some embodiments, the security event document includes multiple documents; the data analysis device 700 further includes:

[0158] The third acquisition module is used to acquire knowledge intelligence data;

[0159] A module is established to create relationships between multiple security event documents based on the knowledge intelligence data, thereby obtaining attack chain data.

[0160] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0161] Based on the data analysis method described in any of the above embodiments, this application also provides, as well as... Figure 8 The diagram shows the structure of an electronic device. Figure 8 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement a data analysis method as described in any of the above embodiments.

[0162] This application also provides a computer storage medium storing a computer program, which, when executed by a processor, can be used to perform a data analysis method described in any of the above embodiments.

[0163] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0164] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0165] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0166] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0167] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0168] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A data analysis method, characterized in that, The method includes: Obtain a set of topics for security event documents; the set of topics includes at least one subset of topic features, the subset of topic features corresponds to the topic of the security event document, and the subset of topic features includes a first word feature belonging to the corresponding topic. Obtain multiple category sets from the sample library; the category sets correspond to categories, and the category sets include second word features belonging to the corresponding categories; The target category to which the security event document belongs is determined based on the first conditional probability of each category set under the topic set; Using the first word features of at least a subset of the topic features in the topic set, update the second word features included in the target category set corresponding to the target category; Acquire knowledge intelligence data; associate the knowledge intelligence data with the target category of the security event document to form a topological relationship, so that multiple security events can be associated through the topological relationship to obtain the attack chain data of the attacker to the attacked party; Determining the target category to which the security event document belongs based on the first conditional probability of each category set under the topic set includes: In response to the sample library receiving updated data via an interface, an updated category set is determined; a second conditional probability of the updated category set under the topic set is calculated; based on the first conditional probability and the second conditional probability, the target category to which the security event document belongs is determined; if the updated data is the first word feature received by the sample interface, then the updated category set is the target category set; the second conditional probability is the probability of the target category set under the topic set, and the second conditional probability overrides the first conditional probability; the target category is determined by the second conditional probability; if the updated data is a newly added category set received by a manual interface, then the updated category set is the newly added category set; the second conditional probability is the probability of the newly added category set under the topic set; the target category is determined by the second conditional probability and the first conditional probability.

2. The method according to claim 1, characterized in that, The set of topics for obtaining security event documents includes: Obtain the security event document; The topics of the security event documents are extracted using a topic generation model, resulting in a topic feature subset corresponding to each topic and the topic set.

3. The method according to claim 2, characterized in that, The process of obtaining the security event document includes: Capture security incident data from network security devices; The security event data is subjected to target processing to obtain security event documents; the target processing includes one or more of the following: standardization processing, cleaning processing, merging processing, and storage processing.

4. The method according to any one of claims 1-3, characterized in that, The process of obtaining the first word feature includes: Retrieve all keywords belonging to the topic from the security incident document; Calculate the similarity between each keyword and the topic; Keywords with a similarity greater than a threshold are identified as the first word features belonging to the topic.

5. The method according to claim 1, characterized in that, The sample database includes an interface for receiving updated data; the updated data is used to update the sample database.

6. A data analysis device, characterized in that, The apparatus is used to perform the method as described in claim 1, comprising: The first acquisition module is used to acquire a set of topics for security event documents; the set of topics includes at least one subset of topic features, the subset of topic features corresponds to the topic of the security event document, and the subset of topic features includes a first word feature belonging to the corresponding topic; The second acquisition module is used to acquire multiple category sets in the sample library; the category set corresponds to a category, and the category set includes second word features belonging to the corresponding category; The classification module is used to determine the target category to which the security event document belongs based on the first conditional probability of the topic set under multiple category sets; The update module is used to update the second word features included in the target category set corresponding to the target category using the first word features of at least a subset of the topic features in the topic set; The third acquisition module is used to acquire knowledge intelligence data; A module is established to associate knowledge intelligence data with the target categories of the security event documents to form a topological relationship, so that multiple security events can be associated through the topological relationship to obtain the attack chain data of the attacker to the attacked party.

7. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store processor-executable instructions; When the processor invokes the executable instructions, it implements the operation of any one of the methods described in claims 1-5.

8. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-5.

Images