Text classification method, device and electronic equipment
By calculating the character transfer probability interval for text classification, the problem of junk text interference in terminal detection is solved, and the accuracy and efficiency of identification of unknown threats are improved.
Patent Information
- Application Number
- CN202111620184.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-28
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-12-28
AI Technical Summary
Existing network-side detection is difficult to identify unknown threats, especially advanced persistent threats (APT) attacks, and terminal detection is inaccurate and inefficient in text classification due to excessive spam text and interfering information.
By obtaining the target storage path information of the target text and the historical storage path information in the log file, calculate the character transfer probability interval, and perform text classification to improve accuracy and avoid interference from garbage text.
It improves the accuracy of text classification, enhances the detection ability of unknown threats, reduces the impact of junk text interference, and improves detection efficiency.
Smart Images

Figure CN114398887B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of network security technology, and in particular to a text classification method, device and electronic device. Background Art
[0002] In recent years, with the rapid development of IoT technology, office LANs composed of terminals, servers and IoT entities are facing a large number of external attacks and internal virus threats. In order to effectively avoid the damage caused by unknown threats, it is necessary to accurately detect unknown threats in advance.
[0003] At present, the existing feature matching used in network-side detection cannot effectively identify unknown threats. In addition, due to the increase in the number of features in the feature library, more computing resources are required, and there are problems such as encryption of feature data, which makes it difficult to detect advanced persistent threat (APT) attacks with super-strong adversarial capabilities.
[0004] It can be seen that if only traditional network-side detection is used, once the network-side detection fails to detect the unknown threat, the defense system will be breached by the unknown threat. Therefore, as one of the effective ways to detect unknown threats, terminal detection is essential.
[0005] For example, in daily life, terminal detection collects and analyzes similar behaviors that appear in terminal logs, and then determines the maliciousness based on the behavior, so as to comprehensively judge the threat events and ultimately identify and recognize the threats, such as identifying the family of malicious samples and the attack organization information.
[0006] However, when using the above terminal detection and analyzing the terminal log to detect unknown threats, the terminal log contains a large amount of junk text and interference information, which causes valid information to be submerged and makes it difficult to capture unknown threats in a timely manner. Therefore, the above method will reduce the detection accuracy and efficiency of unknown threats on the terminal. Summary of the invention
[0007] The present application provides a text classification method, device and electronic device to improve the accuracy of text classification.
[0008] In a first aspect, an embodiment of the present application provides a text classification method, the method comprising:
[0009] Obtain the target text sent by the business data party, and record the target storage path information of the target text; wherein the target storage path information includes: at least one target path node and its respective target node name.
[0010] Obtaining the historical storage path information corresponding to each historical text recorded in the log file; wherein each historical storage path information includes: at least one historical path node and its respective historical node name;
[0011] If at least one unique node name different from each historical node name is selected from at least one target node name obtained, the character transition probability of each at least one unique node name is determined respectively; wherein each character transition probability is obtained based on the transition probability of each two adjacent characters in the character string of the corresponding unique node name.
[0012] Based on the character transition probability interval to which the at least one obtained character transition probability belongs, text classification is performed on the target text.
[0013] In a second aspect, an embodiment of the present application further provides a text classification device, the device comprising:
[0014] The recording module is used to obtain the target text sent by the business data party and record the target storage path information of the target text; wherein the target storage path information includes: at least one target path node and its respective target node name.
[0015] The acquisition module is used to acquire the historical storage path information corresponding to each historical text recorded in the log file; wherein each historical storage path information includes: at least one historical path node and its respective historical node name.
[0016] A processing module is used to determine the character transition probability of at least one unique node name if at least one unique node name different from each historical node name is selected from at least one obtained target node name; wherein each character transition probability is obtained based on the transition probability of each two adjacent characters in the character string of the corresponding unique node name.
[0017] The classification module is used to classify the target text based on the character transition probability interval to which the at least one character transition probability belongs.
[0018] In an optional embodiment, before obtaining the target text sent by the business data party: obtain at least one target cluster; wherein each target cluster is obtained by secondary clustering of a set unknown sample set, and each unknown sample contained in the unknown sample set is sent by the business data party.
[0019] For at least one target cluster obtained, the following operations are performed respectively: the characteristic behaviors corresponding to each unknown sample contained in a target cluster are determined respectively, and the characteristic similarities between the characteristic behaviors and other unknown samples are determined, and the unknown samples whose characteristic similarities meet the preset similarity conditions are selected and saved as candidate texts.
[0020] When obtaining the target text sent by the business data party, the acquisition module is specifically used to:
[0021] From the saved candidate texts, select a candidate text as the target text.
[0022] In an optional embodiment, when acquiring at least one target cluster, the acquisition module is specifically used to:
[0023] Based on the feature quantity interval to which the initial feature behavior quantity corresponding to each temporary cluster belongs, at least one candidate cluster is obtained; wherein each temporary cluster is obtained by preliminary clustering of the unknown sample set.
[0024] For at least one candidate cluster, perform the following steps respectively:
[0025] If each unknown sample contained in a candidate cluster is not associated with a corresponding label, then the candidate cluster is taken as the target cluster.
[0026] If each unknown sample included in a candidate cluster is associated with at least one label, the number of label types corresponding to the at least one label is determined, and when it is determined that the number of label types belongs to a set label type number interval, a candidate cluster is used as a target cluster.
[0027] In an optional embodiment, when obtaining at least one candidate cluster based on the feature quantity interval to which the initial feature behavior quantity corresponding to each temporary cluster belongs, the acquisition module is specifically used to:
[0028] The initial characteristic behavior quantities corresponding to each temporary cluster are obtained respectively, and at least one candidate characteristic behavior quantity belonging to a set characteristic quantity interval is selected.
[0029] For each temporary cluster corresponding to at least one candidate feature behavior number, labels are added to each unknown sample contained in each temporary cluster, and each temporary cluster corresponding to each candidate feature behavior number is used as a candidate cluster.
[0030] In an optional embodiment, when selecting at least one unique node name different from each historical node name, the processing module is specifically configured to:
[0031] Obtain a white sample data set and a black sample data set in a log file; wherein the white sample data set includes: at least one normal sample and its corresponding normal storage path information, each normal storage path information includes: at least one normal path node and its respective historical node name, and the black sample data set includes: at least one abnormal sample and its corresponding abnormal storage path information, each abnormal storage path information includes: at least one abnormal path node and its respective historical node name.
[0032] Based on the black and white sample similarity evaluation values corresponding to each of the at least one target node name, at least one unique node name that is different from each historical node name is selected from the at least one target node name; wherein the black and white sample similarity evaluation value represents: the probability that the corresponding target node name belongs to the black sample data set and the white sample data set.
[0033] In an optional embodiment, when selecting at least one unique node name different from each historical node name from at least one target node name based on the black and white sample similarity evaluation values corresponding to each target node name, the processing module is specifically configured to:
[0034] For at least one target node name, perform the following operations:
[0035] Based on the name similarity between a target node name and each historical node name, the name similarity that meets the preset first probability condition is selected as the first probability that a target node name belongs to the white sample data set, and the name similarity that meets the preset second probability condition is selected as the second probability that a target node name belongs to the black sample data set.
[0036] Based on the first probability and the second probability, a black-and-white sample similarity evaluation value of a target node name is determined.
[0037] When it is determined that the similarity evaluation value of the black and white samples does not belong to the set similarity evaluation value interval, a target node name is used as a unique node name.
[0038] In an optional embodiment, when the target text is classified based on the character transition probability interval to which the at least one character transition probability is obtained, the classification module is specifically configured to:
[0039] If there is a character transition probability that is not greater than a set first character transition probability threshold value among at least one character transition probability, the target text is determined as junk text.
[0040] If at least one character transition probability is greater than the first character transition probability threshold, and there is a character transition probability less than the set second character transition probability threshold, the text type of the target text is determined by the non-meaningful word discrimination model.
[0041] If at least one character transition probability is greater than the second character transition probability threshold, the target text is determined to be normal text.
[0042] In an optional embodiment, after classifying the target text based on the character transition probability interval to which the at least one character transition probability belongs, the classification module is further configured to:
[0043] For each normal text obtained, perform the following operations respectively:
[0044] Get the characteristic behavior and label of a normal text.
[0045] When determining that a characteristic behavior of a normal text belongs to a malicious behavior, it is determined that at least one unknown sample having the same label as the normal text is a malicious sample.
[0046] According to a third aspect, an electronic device is provided, the electronic device comprising:
[0047] Memory, used to store computer instructions.
[0048] The processor is used to read computer instructions and execute the text classification method as described in the first aspect.
[0049] According to a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the text classification method as described in the first aspect.
[0050] According to a fifth aspect, a computer program product is provided. When the computer program product is called by a computer, the computer executes the text classification method according to the first aspect.
[0051] The text classification method provided by the embodiment of the present application is based on the target storage path information of the target text and the historical storage path information corresponding to each historical text recorded in the log file. When it is determined that there is at least one unique node name in the target storage path information, the character transition probability interval to which the character transition probability of each corresponding unique node name belongs is determined respectively, thereby completing the text classification of the target text. In this way, the target text is classified according to the character transition probability interval to which the character transition probability of each unique node name belongs in the target storage path information, avoiding the technical defects of a large amount of junk text and interference information in the terminal log when detecting unknown threats based on terminal log analysis, and improving the accuracy of text classification. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 A text classification system applicable to the embodiments of the present application is exemplified;
[0053] Figure 2 The present invention exemplifies a flow chart of a method for obtaining a target text sent by a business data party provided in an embodiment of the present application;
[0054] Figure 3 A schematic diagram of a method for obtaining a target cluster provided in an embodiment of the present application is exemplified;
[0055] Figure 4 A logical schematic diagram of selecting a candidate cluster provided by an embodiment of the present application is exemplified;
[0056] Figure 5 A logic diagram for selecting candidate texts provided in an embodiment of the present application is exemplified;
[0057] Figure 6 The present application embodiment provides a method based on Figure 2 Schematic diagram of the
[0058] Figure 7 A flowchart of a text classification method for a target text provided by an embodiment of the present application is exemplified;
[0059] Figure 8 A logic diagram of text classification provided by an embodiment of the present application is exemplified;
[0060] Fig. 9 The present application embodiment provides a method based on Figure 7 Schematic diagram of the
[0061] Fig.10 A logic diagram of a model training provided by an embodiment of the present application is exemplarily shown;
[0062] Fig.11 The following is a schematic diagram showing the structure of a text classification device provided in an embodiment of the present application;
[0063] Fig.12 The schematic diagram of the structure of an electronic device provided in an embodiment of the present application is exemplified. DETAILED DESCRIPTION
[0064] In order to improve the accuracy of text classification, in an embodiment of the present application, based on the target storage path information and the historical storage path information corresponding to each historical text recorded in the log file, when it is determined that there is at least one unique node name in the target storage path information, the character transition probability intervals to which the character transition probabilities of the corresponding unique node names belong are determined respectively, thereby completing the text classification of the target text.
[0065] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are first explained below.
[0066] (1) APT attack: refers to a persistent and effective attack activity launched by an organization against a specific target. This type of attack activity is highly concealed and targeted, and usually uses various means such as infected media, supply chains, and social engineering to implement advanced, persistent, and effective threats and attacks.
[0067] (2) Wazuh Platform: is a free, open source, enterprise-ready security monitoring solution for threat detection, integrity monitoring, incident response, and compliance. The Wazuh server decodes and uses an analysis engine to perform rule checks on received events. By using rules, specific patterns in the decoded log records can be identified, which may trigger alerts and may even invoke automatic countermeasures (active response), such as Internet Protocol (IP) bans on firewalls. Events that trigger rules are added with warning data, such as rule ID and rule name.
[0068] (3) N-gram: It is a language model. A language model is a probability-based discriminant model. Its input is a sentence (i.e., a sequence of words), and its output is the probability of the sentence, i.e., the joint probability of these words.
[0069] (4) Term Frequency-Inverse Document Frequency (TFIDF): It is a commonly used weighting technique for information retrieval and text mining, which is used to evaluate the importance of a word to a document set or a document in a corpus. The importance of a word increases in direct proportion to the number of times it appears in a document, but decreases in inverse proportion to the frequency of its appearance in the corpus.
[0070] (5) Spark Streaming: It can be used to process large-scale, high-throughput, fault-tolerant real-time data streams. It supports reading data from a variety of data sources, such as Kafka, Flume, or TCP Socket.
[0071] (6) Kafka: It provides a unified, high-throughput, low-latency platform for processing real-time data. It can efficiently process real-time streaming log data and integrate with Storm, HBase, and Spark.
[0072] (7) VT: The full name is Virus Total. It is a world-class virus scanning and query platform that provides free suspicious file analysis services. It uses multiple anti-virus engines to detect uploaded files to determine whether the files are infected by viruses, worms, Trojans, and various types of malware.
[0073] It should be noted that the above-mentioned naming method of technical terms is only an example, and the embodiments of the present application do not limit the naming method of the above-mentioned technical terms.
[0074] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0075] It should be noted that in the description of this application, "multiple" is understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. A and B are connected, which can represent two situations: A and B are directly connected and A and B are connected through C. In addition, in the description of this application, words such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.
[0076] Figure 1 The system architecture diagram applicable to the embodiment of the present application is shown as an example. Figure 1 As shown, the system architecture includes: a server 101 and terminal devices (102a, 102b). The server 101 and the terminal devices (102a, 102b) can exchange information via wireless communication or wired communication.
[0077] Exemplarily, the server 101 can access the network through cellular mobile communication technology, thereby communicating with the terminal devices (102a, 102b), and the cellular mobile communication technology, for example, includes the fifth generation mobile communication (5th Generation Mobile Networks, 5G) technology.
[0078] Optionally, the server 101 may access the network via short-range wireless communication to communicate with the terminal devices (102a, 102b), wherein the short-range wireless communication method includes, for example, Wireless Fidelity (Wi-Fi) technology.
[0079] The embodiment of the present application does not limit the number of servers and other devices mentioned above. Figure 1 Only one server is used as an example for description.
[0080] Server 101 is used to obtain the target text sent by the business data party and record the target storage path information of the target text; obtain the historical storage path information corresponding to each historical text recorded in the log file; if at least one unique node name different from each historical node name is selected from the at least one target node name obtained, then determine the character transition probability of each of the at least one unique node name; based on the character transition probability interval to which the at least one character transition probability obtained belongs, perform text classification on the target text.
[0081] It should be noted that in the embodiment of the present application, in order to improve the accuracy of text classification, a Wazuh monitoring system and a machine learning real-time detection solution are introduced in the server to collect terminal logs in a real terminal device environment, and use rule-based and machine learning-based log analysis methods in parallel to capture terminal threats.
[0082] Specifically, the Wazuh monitoring system uses a serial rule-based method to group-match the collected host logs with the established Wazuh rule base, and reports the hit parts in real time; the machine learning real-time detection solution consists of five steps: log collection, filtering, annotation, model analysis, and real-time analysis. First, the necessary terminal behavior logs are collected, and then the event differentiation processing of the native log text is performed. The log filtering is divided into three modules: general path screening, garbled code recognition model, and garbled code classification model. For the collected logs, three processes of log filtering are performed to remove redundant information and junk information in the logs. Among them, the annotation first uses the N-Gram and TF-IDF methods to concatenate the text features of word segmentation and non-word segmentation, then vectorize the concatenated text data, and then automatically process the logs of unknown families. After the three processes of clustering, screening, and expert verification, the one-to-one relationship between the sample logs and the sample labels is obtained, and finally the data is annotated with family labels; the model analysis is divided into three modules: data clustering, binary classification model training and deployment, and multi-classification model training and deployment. Data clustering is used to process unknown family samples, including integrated clustering and common behavior extraction modules. The training and deployment of the model uses the data set after text vectorization; during real-time analysis, Spark Streaming is applied to receive the log data sent by Kafka in real time for window matching.
[0083] The terminal device (102a, 102b) is a device that can provide voice and / or data connectivity to users, including handheld terminal devices with wireless connection functions, vehicle-mounted terminal devices, etc.
[0084] Exemplarily, the terminal device can be: a mobile phone, a tablet computer, a laptop computer, a PDA, a mobile Internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal device in industrial control, a wireless terminal device in unmanned driving, a wireless terminal device in a smart grid, a wireless terminal device in transportation safety, a wireless terminal device in a smart city, or a wireless terminal device in a smart home, etc.
[0085] Further, based on the above system architecture, the target text for text classification is obtained, see Figure 2 As shown, in the embodiment of the present application, the method flow for obtaining the target text sent by the business data party includes the following specific steps:
[0086] S201: Acquire at least one target cluster.
[0087] In a possible implementation, when executing step S201, the server first determines at least one candidate cluster based on the feature quantity interval to which the initial feature behavior quantity corresponding to each temporary cluster belongs, and then screens out candidate clusters that meet the set label type quantity interval according to the number of label types corresponding to each candidate cluster, and finally selects the candidate cluster that meets the set label type quantity interval as the target cluster, see Figure 3 As shown, in the embodiment of the present application, the method flow for obtaining the target cluster includes the following specific steps:
[0088] S2011: Obtain at least one candidate cluster based on the feature quantity interval to which the initial feature behavior quantity corresponding to each temporary cluster belongs.
[0089] Specifically, when executing step S2011, the server respectively obtains the initial feature behavior quantity corresponding to each temporary cluster, selects at least one candidate feature behavior quantity belonging to the set feature quantity interval, adds labels to each unknown sample contained in the temporary cluster corresponding to at least one candidate feature behavior quantity, and takes each temporary cluster corresponding to the candidate feature behavior quantity as a candidate cluster, wherein each temporary cluster is obtained by preliminary clustering of the unknown sample set.
[0090] For example, see Figure 4As shown, after the server obtains the unknown sample set, it performs preliminary clustering on the unknown sample set to obtain a preliminary clustering result consisting of multiple temporary clusters, and then selects at least one candidate feature behavior quantity belonging to the set feature quantity interval based on the initial feature behavior quantity corresponding to each temporary cluster and the set feature quantity interval, and takes the temporary cluster corresponding to each candidate feature behavior quantity as a candidate cluster, wherein the first feature quantity threshold is the upper limit value of the set feature quantity interval, and the second feature quantity threshold is the lower limit value of the set feature quantity interval.
[0091] It should be noted that if the initial number of characteristic behaviors corresponding to a temporary cluster does not belong to the set characteristic quantity interval, and the temporary characteristic behavior number of the temporary cluster is greater than the first characteristic quantity threshold, the temporary cluster needs to be re-clustered; if the number of characteristic behaviors contained in the temporary cluster is less than the second characteristic quantity threshold, the temporary cluster needs to be discarded. Optionally, the unknown samples contained in the temporary cluster still need to be clustered again to avoid malicious samples in the temporary cluster that cannot be detected.
[0092] For example, assuming that the set feature quantity interval is [2, 20], and the total number of feature behaviors of the unknown sample set is 200. If the initial feature behavior number of a temporary cluster is 8, which belongs to the set feature quantity interval, then the temporary cluster can be directly used as a candidate cluster; if the initial feature behavior number of a temporary cluster is 25, which does not belong to the set feature quantity interval, but is greater than the first feature quantity threshold, then the temporary cluster needs to be re-clustered to reduce the initial feature behavior number of the temporary cluster, and then determine again whether it can be used as a candidate cluster; if the initial feature behavior number of a temporary cluster is 1, which does not belong to the set feature quantity interval and is less than the first feature quantity threshold, then the temporary cluster can be discarded.
[0093] Furthermore, after obtaining at least one candidate cluster, the server uses the non-gold standard family soft label from VT to add labels to the unknown samples contained in each candidate cluster. It is worth noting that even if the labeling operation is performed on the unknown samples contained in each candidate cluster, not every unknown sample can be labeled.
[0094] S2012: If each unknown sample contained in a candidate cluster is not associated with a corresponding label, then a candidate cluster is used as a target cluster; if each unknown sample contained in a candidate cluster is associated with at least one label, the number of label types corresponding to the at least one label is determined, and when it is determined that the number of label types belongs to the set label type number range, then a candidate cluster is used as a target cluster.
[0095] Exemplarily, when executing step S2012, assuming that the set label type number interval is [1, 5], after the server obtains at least one candidate cluster, it performs the following steps for at least one candidate cluster: Based on the label situation corresponding to each candidate cluster, it can be determined whether the corresponding candidate cluster can be used as the target cluster, where the label situation includes whether there is a label and the number of label types. Taking 5 candidate clusters, each candidate cluster contains 100 unknown samples, as an example, the label situation corresponding to each candidate cluster is shown in Table 1:
[0096] Table 1
[0097] Candidate clusters Cad.Clu.1 Cad.Clu.2 Cad.Clu.3 Cad.Clu.4 Cad.Clu.5 Is there a tag? no yes yes no yes Number of label types 0 2 5 0 8
[0098] It can be seen from the above table that the server can determine whether the corresponding candidate cluster can be used as a target cluster based on the label situation corresponding to each candidate cluster. For example, taking candidate cluster Cad.Clu.1 as an example, the 100 unknown samples contained in candidate cluster Cad.Clu.1 are not labeled, so the candidate cluster can be directly used as a target cluster; taking candidate cluster Cad.Clu.2 as an example, the 100 unknown samples contained in candidate cluster Cad.Clu.2 have a corresponding number of label types of 2, which belongs to the set label type number interval, so the candidate cluster can be used as the target cluster. Optionally, if there are unknown samples without associated labels among the 100 unknown samples contained in the candidate cluster, corresponding labels can be added according to the similarity between the unknown samples without associated labels and the unknown samples with associated labels; taking the 100 unknown samples contained in candidate cluster Cad.Clu.2 as an example, the corresponding number of label types is 8, it can be known that the 100 unknown samples contained in the candidate cluster have labels, and the number of label types does not belong to the set label type number interval, so the candidate cluster can be discarded. Optionally, the position samples contained in the candidate cluster still need to be clustered again to avoid the existence of malicious samples in the candidate cluster that cannot be detected.
[0099] S202: Determine the feature behaviors corresponding to each unknown sample contained in a target cluster and the feature similarity between the feature behaviors and other unknown samples, select the unknown samples whose feature similarity meets the preset similarity condition, and save them as candidate texts.
[0100] For details, see Figure 5 As shown, when executing step S202, after obtaining at least one target cluster, the server performs the following operations for the at least one target cluster obtained: according to the behavioral feature values of the characteristic behaviors corresponding to each unknown sample contained in a target cluster, the feature similarity between the characteristic behavior of the corresponding unknown sample and the characteristic behavior of other unknown samples can be determined respectively, and the unknown samples whose feature similarity meets the preset similarity condition are screened out, and the unknown samples are saved as candidate texts.
[0101] For example, taking a target cluster containing three unknown samples, namely, unknown sample A, unknown sample B and unknown sample C, the server can obtain the similarity between the characteristic behaviors of unknown sample A and unknown sample B as 85%, the similarity between the characteristic behaviors of unknown sample A and unknown sample C as 93%, and the similarity between the characteristic behaviors of unknown sample B and unknown sample C as 87% based on the behavioral feature values of the characteristic behaviors corresponding to the three unknown samples. It is easy to know that the similarity between unknown sample C and unknown sample A and unknown sample B is relatively high, so unknown sample C can be used as the candidate file corresponding to the target cluster and saved in the candidate text library.
[0102] It should be noted that there is at least one unknown sample in the target cluster, and its similarity with more than 60% of other unknown samples in the target cluster is greater than 80%. If this condition is not met, the candidate cluster is discarded and cannot be used as the target cluster; further, there is at least one unknown sample in the target cluster, and the sum of its similarities with other unknown samples in the target cluster is greater than a certain threshold, which is 0.8×the number of unknown samples in the target cluster, and the unknown sample corresponding to the highest sum of similarities with other unknown samples is used as the Hash of the target cluster, that is, the candidate text.
[0103] Figure 6 The following is an exemplary diagram of a logic diagram of obtaining a target text sent by a business data party provided in an embodiment of the present application, such as Figure 6 As shown, the server can arbitrarily select a target cluster from the various target clusters, and obtain the feature similarity between each unknown sample in the target cluster and other unknown samples in turn, and then combine the preset similarity conditions to select the unknown samples that meet the preset similarity conditions from the various unknown samples in the target cluster as candidate texts.
[0104] Further, based on the above-mentioned pre-operation processing, a plurality of candidate texts are obtained, and any one of the candidate texts is selected as the target text from the stored candidate texts. Figure 7 As shown, in the embodiment of the present application, the text classification method process for the target text has the following specific steps:
[0105] S701: Acquire the target text sent by the business data party, and record the target storage path information of the target text.
[0106] Specifically, when executing step S701, the server can obtain the target text sent by the business data party, and based on the feature extraction algorithm of the storage path information, obtain the target storage path information of the target text from the original database, wherein the target storage path information includes: at least one target path node and its respective target node name.
[0107] Exemplarily, the original database contains multiple storage path information. The server performs feature matching on the text identifiers corresponding to the multiple storage path information in the original database based on the text identifier of the target text, and screens out the target storage path information corresponding to the target text. Taking 4 storage path information as an example, each storage path information and its corresponding text identifier are shown in Table 2:
[0108] Table 2
[0109] Storage path information CR.M1 CR.M2 CR.M3 CR.M4 Text logo W.B1 W.B2 W.B3 W.B4
[0110] As can be seen from the above table, the server can determine the corresponding storage path information according to the text identifiers corresponding to each storage path information. For example, if the text identifier of the target text is W.B2, the server determines that the target storage path information corresponding to the target text is storage path information CR.M2 according to the text identifier W.B2, and so on.
[0111] Furthermore, the server may obtain at least one target path node and its respective target node name contained in the target storage path information corresponding to the target text. For example, assuming the target storage path information of the target text is: "C:\\Windows\\System32", wherein the node names of the respective target path nodes are: "C", "Windows", "System32" in sequence.
[0112] S702: Obtain the historical storage path information corresponding to each historical text recorded in the log file.
[0113] Specifically, when executing step S702, after obtaining the target text sent by the business data party and recording the target storage path information of the target text, the server can obtain each historical text recorded in the log file, and based on the data type of the storage path information, obtain the historical storage path information corresponding to each historical text from the original database, wherein each historical storage path information includes: at least one historical path node and its respective historical node name.
[0114] Exemplarily, the server can obtain the historical storage path information corresponding to each historical text recorded in the log file according to the data type of the storage path information, and then determine the historical path nodes and their respective historical node names contained in each historical storage path information. For example, the server can obtain the historical storage path information of historical text 1: "C:\\Windows\\System32\\HKLM\\SOFTWARE\\SpeedRun", and the historical path information of historical text 2: "C:\\Windows\\System32\\HKLM\\SOFTWARE\\TrickBot" from the original database based on the data type of the storage path information.
[0115] Furthermore, the server can obtain the node names of each historical path node of history file 1, which are: "C", "Windows", "System32", "HKLM", "SOFTWARE", "SpeedRun", and the node names of each historical path node of history file 2, which are: "C", "Windows", "System32", "HKLM", "SOFTWARE", "TrickBot".
[0116] It should be noted that the path information "C:\\Windows\\System32\\HKLM\\SOFTWARE" is part of the historical storage information of historical text 1 and the historical storage information of historical text 2. The server can use the path information "C:\\Windows\\System32\\HKLM\\SOFTWARE" as the general path information, thereby focusing on the unique node names "TrickBot" and "SpeedRun" using the N-Gram and TFIDF word segmentation and non-word segmentation text feature concatenation method to convert the log content into a numerical vector (word-word vector), and then obtain the character transfer probability corresponding to "TrickBot" and "SpeedRun" respectively.
[0117] S703: If at least one unique node name different from each historical node name is selected from the at least one obtained target node name, character transition probabilities of the at least one unique node name are determined respectively.
[0118] In one possible implementation, when executing step S703, after obtaining the historical storage path information corresponding to each historical text recorded in the log file, and at least one target node name, the server selects at least one unique node name that is different from each historical node name from each target node name based on the white sample data set and the black sample data set in the log file, and the black and white sample similarity evaluation values corresponding to each target node name.
[0119] It should be noted that the white sample data set includes: at least one normal sample and its corresponding normal storage path information, each normal storage path information includes: at least one normal path node and its respective historical node name, and the black sample data set includes: at least one abnormal sample and its corresponding abnormal storage path information, each abnormal storage path information includes: at least one abnormal path node and its respective historical node name; the black and white sample similarity evaluation value represents: the probability that the corresponding target node name belongs to the black sample data set and the white sample data set.
[0120] Exemplarily, taking 1 target node name, 5 historical node names in the white sample data set, and 5 historical node names in the black sample data set as examples, the name similarities corresponding to each historical node name and the target node name are shown in Table 3, among which historical node names 1 to 5 belong to the white sample data set, and historical node names 6 to 10 belong to the black sample data set.
[0121] Table 3
[0122]
[0123] Based on the above table, the server selects the name similarity that meets the preset first probability condition as the first probability that the target node name belongs to the white sample data set based on the name similarity between the target node name and each historical node name, and selects the name similarity that meets the preset second probability condition as the second probability that the target node name belongs to the black sample data set.
[0124] For example, assuming that the first probability condition is the maximum value of the name similarity between the target node name and the historical node name in the white sample data set, the server can use the name similarity of 93% between the target name and the historical node name 5 as the first probability that the target node name belongs to the white sample data set, and record it as a. Similarly, assuming that the second probability condition is the maximum value of the name similarity between the target node name and the historical node name in the black sample data set, the server can use the name similarity of 94% between the target name and the historical node name 7 as the second probability that the target node name belongs to the black sample data set, and record it as b.
[0125] Furthermore, the server may obtain the black-and-white sample similarity evaluation value corresponding to the corresponding target node name based on the black-and-white sample similarity evaluation value calculation formula. The specific calculation formula is as follows:
[0126]
[0127] Among them, Score represents the similarity evaluation value of black and white samples; a represents the first probability that the target node name belongs to the white sample data set; b represents the second probability that the target node name belongs to the black sample data set; and Score is selected to belong to [0.01, 1] as the set similarity evaluation value interval.
[0128] For example, assuming that the target name belongs to the white sample data set according to its first probability a and the black sample data set according to its second probability b, the black and white sample similarity evaluation value of the target name is obtained based on the above black and white sample similarity evaluation value calculation formula. If the black and white sample similarity evaluation value of the target name is 0.004, which does not belong to the similarity evaluation value interval [0.01, 1], it can be determined that the target node name can be used as a unique node name; if the black and white sample similarity evaluation value of the target name is 0.52, which belongs to the similarity evaluation value interval [0.01, 1], it can be determined that the target node name can be used as a general node name and no further processing is required.
[0129] Furthermore, after obtaining at least one unique node name, the server can determine the character transition probability of each unique node name based on the character transition probability calculation formula, wherein each character transition probability is obtained based on the transition probability of each two adjacent characters in the character string of the corresponding unique node name.
[0130] Optionally, in the embodiment of the present application, the server may apply the character-level bag-of-words text features to perform binary classification prediction for a single word, and combine it with the Cybersecurity-related corpus to calculate the probability distribution of transitions between letters in the unique node name string. The character transition probability of row i and column j on the Markov matrix is calculated as follows:
[0131]
[0132] Among them, P ij Represents the character transition probability of row i and column j on the Markov matrix; Count ij Indicates the number of times the i-th character is transferred to the j-th character; Count i Indicates the number of times character i appears in the character shift.
[0133] From the above formula, we can see that the character transition probability of word M with a character length of n is:
[0134] P(M)=P(M3|M1M2)*P(M4|M2M3)*…*P(M i+2 |M (i+1) M i )*…
[0135] *P(M n |M n-2 M n-1 )
[0136] Among them, P(M) represents the character transition probability of word M with a character length of n, where M i represents the i-th character in word M; P(M i+2 |M (i+1) M i ) represents the character transition probability from the i-th and i+1-th characters in word M to the i+2-th character.
[0137] S704: Based on the character transition probability interval to which the at least one obtained character transition probability belongs, perform text classification on the target text.
[0138] For details, see Figure 8 As shown, when executing step S704, after the server determines the character transition probability of each of at least one unique node name respectively, if among the character transition probabilities, there is a character transition probability that is not greater than the set first character transition probability threshold, the target text is determined as junk text; if all character transition probabilities are greater than the first character transition probability threshold, and there is a character transition probability that is less than the set second character transition probability threshold, the text type of the target text is determined through the non-meaningful word discrimination model; if all character transition probabilities are greater than the second character transition probability threshold, the target text is determined as normal text.
[0139] Exemplarily, assuming that the first character transfer probability threshold is 60%, the second character transfer probability threshold is 90%, and the target storage path information contains two unique node names: Un.Node.Na1 and Un.Node.Na2, and their corresponding character transfer probabilities are 92% and 95%, respectively. It is easy to know that both character transfer probabilities are greater than the second character transfer probability threshold, then the target text corresponding to the target storage path information can be treated as normal text.
[0140] Similarly, if there are two unique node names: Un.Node.Na1 and Un.Node.Na2, and their corresponding character transfer probabilities are 68% and 95% respectively, it is easy to know that the two character transfer probabilities are greater than the first character transfer probability threshold, and there is a character transfer probability less than the set second character transfer probability threshold, then the text type of the target text is determined through the non-meaningful word discrimination model; if the corresponding character transfer probabilities are 46% and 27% respectively, it is easy to know that the two character transfer probabilities are less than the first character transfer probability threshold, then the target text corresponding to the target storage path information can be regarded as junk text.
[0141] Optionally, in an embodiment of the present application, when obtaining the character transition probability of each unique node name, the server may add multiple state transition penalty items, that is, for uncommon state transition phenomena, its probability value needs to be appropriately reduced:
[0142] Case 1: The character transfer from letter to number or number to letter exceeds 2 times. For each time exceeding the limit, P'=P*k (k<1), where k is the transfer penalty factor.
[0143] Case 2: When the trigram is shifted, if three consecutive characters are all numbers more than three times, then each time it exceeds the limit, P'=P*k (k<1), where k is the shift penalty factor.
[0144] Fig. 9 A logic diagram of text classification for a target text provided by an embodiment of the present application is exemplarily shown, such as Fig. 9 As shown, the server determines at least one unique node name in the target storage path information based on the target storage path information of the target text and the historical storage path information corresponding to each historical text in the log text, and then obtains the character transfer probability corresponding to each unique node name, thereby performing text classification of the target text.
[0145] Further, based on the above method steps, the server performs text classification on the target text based on the character transition probability interval to which the at least one character transition probability obtained belongs. After the text classification, the server performs the following operations for each normal text obtained: obtaining a characteristic behavior of a normal text and its label; when determining that the characteristic behavior of a normal text belongs to a malicious behavior, determining that at least one unknown sample with the same label as a normal text is a malicious sample.
[0146] Optionally, if normal text has no label, then the unknown samples whose feature similarity with the normal text is greater than the set feature similarity threshold can be classified as normal text. When it is determined that the characteristic behavior of a normal text is malicious behavior, all unknown samples of this type are regarded as malicious samples.
[0147] Fig.10 A logical diagram of a model training provided by an embodiment of the present application is exemplarily shown, such as Fig.10 As shown in the figure, the server uses a lightweight system-layer log tool to record the terminal behavior in the real environment, transmits the log to the Spark Streaming data processing pipeline through the Kafka stream, and then inputs the concatenated feature vectors corresponding to the samples obtained after multi-layer processing into the binary classification model and the multi-classification model, and the text type and label of the target text; further, according to the characteristic behavior or label of the target text, it can be determined whether the target text is a terminal threat or malicious sample, and what kind of malicious sample it belongs to.
[0148] The text classification method provided by the embodiment of the present application is based on the target storage path information of the target text and the historical storage path information corresponding to each historical text recorded in the log file. When it is determined that there is at least one unique node name in the target storage path information, the character transition probability interval to which the character transition probability of each corresponding unique node name belongs is determined respectively, thereby completing the text classification of the target text. In this way, the target text is classified according to the character transition probability interval to which the character transition probability of each unique node name belongs in the target storage path information, avoiding the technical defects of a large amount of junk text and interference information in the terminal log when detecting unknown threats based on terminal log analysis, and improving the accuracy of text classification.
[0149] Based on the same technical concept, the embodiment of the present application also provides a text classification device, which can implement the above method flow of the embodiment of the present application.
[0150] Fig.11 The following is a schematic diagram showing the structure of a text classification device provided in an embodiment of the present application. Fig.11 As shown, the text classification device includes: a recording module 1101, an acquisition module 1102, a processing module 1103 and a classification module 1104, wherein:
[0151] The recording module 1101 is used to obtain the target text sent by the business data party and record the target storage path information of the target text; wherein the target storage path information includes: at least one target path node and its respective target node name.
[0152] The acquisition module 1102 is used to acquire the historical storage path information corresponding to each historical text recorded in the log file; wherein each historical storage path information includes: at least one historical path node and its respective historical node name.
[0153] Processing module 1103 is used to determine the character transition probability of at least one unique node name if at least one unique node name different from each historical node name is selected from at least one obtained target node name; wherein each character transition probability is obtained based on the transition probability of each two adjacent characters in the character string of the corresponding unique node name.
[0154] The classification module 1104 is used to classify the target text based on the character transition probability interval to which the at least one character transition probability belongs.
[0155] In an optional embodiment, before obtaining the target text sent by the business data party: obtain at least one target cluster; wherein each target cluster is obtained by secondary clustering of a set unknown sample set, and each unknown sample contained in the unknown sample set is sent by the business data party.
[0156] For at least one target cluster obtained, the following operations are performed respectively: the characteristic behaviors corresponding to each unknown sample contained in a target cluster are determined respectively, and the characteristic similarities between the characteristic behaviors and other unknown samples are determined, and the unknown samples whose characteristic similarities meet the preset similarity conditions are selected and saved as candidate texts.
[0157] When acquiring the target text sent by the business data party, the acquisition module 1102 is specifically used to:
[0158] From the saved candidate texts, select a candidate text as the target text.
[0159] In an optional embodiment, when acquiring at least one target cluster, the acquisition module 1102 is specifically configured to:
[0160] Based on the feature quantity interval to which the initial feature behavior quantity corresponding to each temporary cluster belongs, at least one candidate cluster is obtained; wherein each temporary cluster is obtained by preliminary clustering of the unknown sample set.
[0161] For at least one candidate cluster, perform the following steps respectively:
[0162] If each unknown sample contained in a candidate cluster is not associated with a corresponding label, then the candidate cluster is taken as the target cluster.
[0163] If each unknown sample included in a candidate cluster is associated with at least one label, the number of label types corresponding to the at least one label is determined, and when it is determined that the number of label types belongs to a set label type number interval, a candidate cluster is used as a target cluster.
[0164] In an optional embodiment, when obtaining at least one candidate cluster based on the feature quantity interval to which the initial feature behavior quantity corresponding to each temporary cluster belongs, the acquisition module 1102 is specifically configured to:
[0165] The initial characteristic behavior quantities corresponding to each temporary cluster are obtained respectively, and at least one candidate characteristic behavior quantity belonging to a set characteristic quantity interval is selected.
[0166] For each temporary cluster corresponding to at least one candidate feature behavior number, labels are added to each unknown sample contained in each temporary cluster, and each temporary cluster corresponding to each candidate feature behavior number is used as a candidate cluster.
[0167] In an optional embodiment, when selecting at least one unique node name different from each historical node name, the processing module 1103 is specifically used to:
[0168] Obtain a white sample data set and a black sample data set in a log file; wherein the white sample data set includes: at least one normal sample and its corresponding normal storage path information, each normal storage path information includes: at least one normal path node and its respective historical node name, and the black sample data set includes: at least one abnormal sample and its corresponding abnormal storage path information, each abnormal storage path information includes: at least one abnormal path node and its respective historical node name.
[0169] Based on the black and white sample similarity evaluation values corresponding to each of the at least one target node name, at least one unique node name that is different from each historical node name is selected from the at least one target node name; wherein the black and white sample similarity evaluation value represents: the probability that the corresponding target node name belongs to the black sample data set and the white sample data set.
[0170] In an optional embodiment, when selecting at least one unique node name different from each historical node name from at least one target node name based on the black and white sample similarity evaluation values corresponding to each target node name, the processing module 1103 is specifically configured to:
[0171] For at least one target node name, perform the following operations:
[0172] Based on the name similarity between a target node name and each historical node name, the name similarity that meets the preset first probability condition is selected as the first probability that a target node name belongs to the white sample data set, and the name similarity that meets the preset second probability condition is selected as the second probability that a target node name belongs to the black sample data set.
[0173] Based on the first probability and the second probability, a black-and-white sample similarity evaluation value of a target node name is determined.
[0174] When it is determined that the similarity evaluation value of the black and white samples does not belong to the set similarity evaluation value interval, a target node name is used as a unique node name.
[0175] In an optional embodiment, when the target text is classified based on the character transition probability interval to which the at least one character transition probability belongs, the classification module 1104 is specifically configured to:
[0176] If there is a character transition probability that is not greater than a set first character transition probability threshold value among at least one character transition probability, the target text is determined as junk text.
[0177] If at least one character transition probability is greater than the first character transition probability threshold, and there is a character transition probability less than the set second character transition probability threshold, the text type of the target text is determined by the non-meaningful word discrimination model.
[0178] If at least one character transition probability is greater than the second character transition probability threshold, the target text is determined to be normal text.
[0179] In an optional embodiment, after classifying the target text based on the character transition probability interval to which the at least one character transition probability belongs, the classification module 1104 is further configured to:
[0180] For each normal text obtained, perform the following operations respectively:
[0181] Get the characteristic behavior and label of a normal text.
[0182] When determining that a characteristic behavior of a normal text belongs to a malicious behavior, it is determined that at least one unknown sample having the same label as the normal text is a malicious sample.
[0183] Based on the same technical concept, the embodiment of the present application also provides an electronic device, which can implement the method flow provided in the above embodiment of the present application. In one embodiment, the electronic device can be a server, or a terminal device or other electronic device.
[0184] Fig.12The following is a schematic diagram showing the structure of an electronic device provided by an embodiment of the present application. Fig.12 As shown, the electronic device may include:
[0185] At least one processor 1201, and a memory 1202 connected to the at least one processor 1201. The specific connection medium between the processor 1201 and the memory 1202 is not limited in the embodiment of the present application. Fig.12 In the example, the processor 1201 and the memory 1202 are connected via the bus 1200. The bus 1200 is Fig.12 The connections between other components are shown in bold lines, and are not intended to be limiting. The bus 1200 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.12 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 1201 can also be called a controller, and there is no limitation on the name.
[0186] In the embodiment of the present application, the memory 1202 stores instructions that can be executed by at least one processor 1201. The at least one processor 1201 can execute a text classification method discussed above by executing the instructions stored in the memory 1202. The processor 1201 can implement Fig.11 The functions of each module in the device shown.
[0187] Among them, the processor 1201 is the control center of the device, which can use various interfaces and lines to connect the various parts of the entire control device, and monitor the device as a whole by running or executing instructions stored in the memory 1202 and calling the data stored in the memory 1202, the various functions of the device and processing data.
[0188] In one possible design, the processor 1201 may include one or more processing units, and the processor 1201 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 1201. In some embodiments, the processor 1201 and the memory 1202 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.
[0189] Processor 1201 can be a general-purpose processor, such as a CPU (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of a text classification method disclosed in the embodiments of the present application can be directly embodied as a hardware processor, or can be executed by a combination of hardware and software modules in the processor.
[0190] Memory 1202, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. Memory 1202 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), programmable read-only memory (Programmable Read Only Memory, PROM), read-only memory (Read Only Memory, ROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic memory, disk, optical disk, etc. Memory 1202 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 1202 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.
[0191] By programming the processor 1201, the code corresponding to the text classification method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 7 The steps of a text classification method in the embodiment shown are as follows: How to design and program the processor 1201 is a technique known to those skilled in the art and will not be described in detail here.
[0192] Based on the same inventive concept, an embodiment of the present application further provides a storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes a text classification method discussed above.
[0193] In some possible implementations, various aspects of a text classification method provided by the present application may also be implemented in the form of a program product, which includes program code. When the program product is run on an apparatus, the program code is used to enable the control device to execute the steps of a text classification method according to various exemplary implementations of the present application described above in this specification.
[0194] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0195] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0196] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0197] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0198] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A text classification method, characterized in that: include: Acquire the target text sent by the business data party, and record the target storage path information of the target text; wherein the target storage path information includes: at least one target path node and its respective target node name; Obtaining the historical storage path information corresponding to each historical text recorded in the log file; wherein each historical storage path information includes: at least one historical path node and its respective historical node name; If at least one unique node name different from each historical node name is selected from the at least one target node name obtained, the character transition probability of each of the at least one unique node name is determined respectively; wherein each character transition probability is obtained based on the transition probability of each two adjacent characters in the character string of the corresponding unique node name; Based on the character transition probability interval to which the at least one obtained character transition probability belongs, text classification is performed on the target text.
2. The method according to claim 1, characterized in that Before obtaining the target text sent by the business data party: Acquire at least one target cluster; wherein each target cluster is obtained by secondary clustering of a set unknown sample set, and each unknown sample contained in the unknown sample set is sent by a business data party; For at least one target cluster obtained, the following operations are performed respectively: determining the feature behaviors corresponding to each unknown sample contained in a target cluster and the feature similarities between the feature behaviors of other unknown samples, selecting the unknown samples whose feature similarities meet the preset similarity conditions, and saving them as candidate texts; The target text sent by the business data acquisition party includes: From the stored candidate texts, a candidate text is selected as the target text.
3. The method according to claim 2, characterized in that The acquiring at least one target cluster comprises: Based on the feature quantity interval to which the initial feature behavior quantity corresponding to each temporary cluster belongs, at least one candidate cluster is obtained; wherein each temporary cluster is obtained by preliminary clustering of the unknown sample set; For the at least one candidate cluster, the following steps are performed respectively: If each unknown sample contained in a candidate cluster is not associated with a corresponding label, the candidate cluster is used as the target cluster; If each unknown sample included in the candidate cluster is associated with at least one label, the number of label types corresponding to the at least one label is determined, and when it is determined that the number of label types belongs to a set label type number interval, the candidate cluster is used as the target cluster.
4. The method according to claim 3, characterized in that The obtaining of at least one candidate cluster based on the feature quantity interval to which the initial feature behavior quantity corresponding to each temporary cluster belongs includes: Obtaining the initial characteristic behavior quantity corresponding to each temporary cluster respectively, and selecting at least one candidate characteristic behavior quantity belonging to the set characteristic quantity interval; Labels are added to the unknown samples contained in the temporary clusters corresponding to the at least one candidate feature behavior quantity, and each temporary cluster corresponding to the candidate feature behavior quantity is used as a candidate cluster.
5. The method according to any one of claims 1 to 4, characterized in that The selecting of at least one unique node name different from each historical node name includes: Obtain a white sample data set and a black sample data set in a log file; wherein the white sample data set includes: at least one normal sample and its corresponding normal storage path information, each normal storage path information includes: at least one normal path node and its respective historical node name, and the black sample data set includes: at least one abnormal sample and its corresponding abnormal storage path information, each abnormal storage path information includes: at least one abnormal path node and its respective historical node name; Based on the black and white sample similarity evaluation values corresponding to each of the at least one target node name, at least one unique node name that is different from the historical node names is selected from the at least one target node name; wherein the black and white sample similarity evaluation values represent: the probability that the corresponding target node name belongs to the black sample data set and the white sample data set.
6. The method according to claim 5, characterized in that The selecting, from the at least one target node name, at least one unique node name different from the respective historical node names based on the black-and-white sample similarity evaluation values corresponding to the at least one target node name, comprises: For the at least one target node name, perform the following operations respectively: Based on the name similarities between the one target node name and the names of the various historical nodes, a name similarity that satisfies a preset first probability condition is selected as a first probability that the one target node name belongs to the white sample data set, and a name similarity that satisfies a preset second probability condition is selected as a second probability that the one target node name belongs to the black sample data set; Based on the first probability and the second probability, determining a black-and-white sample similarity evaluation value of the target node name; When it is determined that the black-and-white sample similarity evaluation value does not belong to the set similarity evaluation value interval, the one target node name is used as a unique node name.
7. The method according to any one of claims 1 to 4, characterized in that The step of classifying the target text based on the character transition probability interval to which the at least one character transition probability belongs includes: If there is a character transition probability that is not greater than a set first character transition probability threshold value among the at least one character transition probability, the target text is determined as junk text; If the at least one character transition probability is greater than the first character transition probability threshold, and there is a character transition probability less than a set second character transition probability threshold, determining the text type of the target text through a non-meaningful word discrimination model; If the at least one character transition probability is greater than the second character transition probability threshold, the target text is determined to be normal text.
8. The method according to any one of claims 1 to 4, characterized in that After classifying the target text based on the character transition probability interval to which the at least one character transition probability belongs, the method further includes: For each normal text obtained, perform the following operations respectively: Get the characteristic behavior and label of a normal text; When it is determined that the characteristic behavior of the normal text belongs to malicious behavior, it is determined that at least one unknown sample having the same label as the normal text is a malicious sample.
9. A text classification device, characterized in that: include: A recording module, used to obtain the target text sent by the business data party, and record the target storage path information of the target text; wherein the target storage path information includes: at least one target path node and its respective target node name; The acquisition module is used to acquire the historical storage path information corresponding to each historical text recorded in the log file; wherein each historical storage path information includes: at least one historical path node and its respective historical node name; A processing module, configured to determine the character transition probabilities of the at least one unique node name respectively if at least one unique node name different from each historical node name is selected from the at least one obtained target node name; wherein each character transition probability is obtained based on the transition probability of each two adjacent characters in the character string of the corresponding unique node name; The classification module is used to classify the target text based on the character transition probability interval to which the at least one character transition probability belongs.
10. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.
