A log classification method and computing device

Abstract

Embodiments of the present application disclose a log packet classification method and a computing device, which are used for classifying a to-be-classified log packet. In the present application, the computing device first determines a plurality of feature types of the to-be-classified log packet, and then determines a plurality of model parameters of the to-be-classified log packet based on the plurality of feature types. Next, the computing device calculates similarities between the to-be-classified log packet and each of K log packet types based on the plurality of model parameters, and obtains K similarities. Finally, the computing device determines a log packet type to which the to-be-classified log packet belongs from the K log packet types according to the K similarities. Thus, the to-be-classified log packet that does not conform to any rule of any of the K log packet types can also be classified.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, and in particular to a log classification method and a computing device. BACKGROUND

[0002] With the development of data center business, a large amount of log packages are generated every day, and different log packages record different fault information. The log packages can be processed by analysis and high-value information can be mined. In order to improve the efficiency of analyzing and processing log packages, it is generally necessary to first classify the log packages, and different analysis and processing logics are performed on different types of log packages. Therefore, it is very important to accurately and quickly classify log packages.

[0003] Currently, the industry classifies log packages based on preset rules, and the preset rules include one or more feature types. The multiple feature types of the log package are compared with the one or more feature types of the preset rule. For the log package that fully meets all the feature types of the preset rule, the log package meets the preset rule and is classified into the log package type corresponding to the preset rule. However, for the log package that does not meet any preset rule, this classification method cannot classify the log package. SUMMARY

[0004] Embodiments of the present application provide a log package classification method and a computing device for classifying a to-be-classified log package.

[0005] The first aspect of the present application provides a log package classification method. The computing device first determines the multiple feature types of the to-be-classified log package, and then determines the multiple model parameters of the to-be-classified log package based on the multiple feature types. Next, the computing device calculates the similarity between the to-be-classified log package and each log package type in K log package types based on the multiple model parameters, obtains K similarities, and finally determines the log package type to which the to-be-classified log package belongs from the K log package types according to the K similarities. Therefore, for the to-be-classified log package that does not meet any rule of any log package type in the K log package types, the to-be-classified log package can also be classified.

[0006] In some possible implementation manners, the computing device can determine the highest value similarity in the K similarities, and determine the corresponding log package type of the highest value similarity in the K log package types, to obtain the log package type to which the to-be-classified log package belongs, so as to obtain the most possible log package type to which the to-be-classified log package belongs.

[0007] In some possible implementation manners, the computing device can determine the feature values of each feature in the multiple features in the to-be-classified log package, obtain the multiple feature values of the to-be-classified log package, and determine the corresponding feature type based on each feature in the multiple feature values, so as to obtain the multiple feature types to which the to-be-classified log package belongs.

[0008] In some possible implementation manners, the plurality of features include data size, an uploading person, a keyword, and / or a first-level directory structure.

[0009] In some possible implementation manners, the computing device can obtain M log packages, each of which belongs to one log package type of the K log package types, M being a positive integer, and determine a number of log packages of the K log package types in which a feature i belongs to a feature type ij as a corresponding model parameter N(x=ij|y=k), the feature i being one of the plurality of features, the feature type ij being one of a plurality of feature types to which the feature i belongs, i∈I, ij∈J i , I being a set of the plurality of features, J i being a set of the plurality of feature types of the feature i, and k=1, 2,..., K, so as to obtain the model parameter N(x=ij|y=k) for determining a log package type of a log to be classified.

[0010] In some possible implementation manners, the computing device can determine log package features based on the M log packages, the log package features including one or more feature types, and then, before determining the corresponding model parameter of each feature type of the plurality of feature types, the computing device can determine, based on the plurality of feature types of the log package to be classified, that the log package to be classified meets at least one feature type in the log package features, so as to screen the log package to be classified and improve classification efficiency.

[0011] In some possible implementation manners, the computing device can obtain the corresponding model parameter of each feature type of the plurality of feature types of the k-th log package type, obtain a plurality of model parameters corresponding to the k-th log package type, k=1, 2,..., K, and then bring the plurality of model parameters of the log package to be classified and the plurality of model parameters corresponding to the k-th log package type into a preset Naive Bayes formula, so as to obtain a similarity between the log package to be classified and the k-th log package type.

[0012] In some possible implementation manners, the Naive Bayes formula is as follows:

[0013]

[0014] wherein N cn represents a sum of the plurality of model parameters corresponding to the cn-th log package type, N k represents a sum of the plurality of model parameters corresponding to the k-th log package type, I represents the plurality of features, J i represents a set of feature types of the feature i of the log package to be classified, λ, s iN(x=ij|y=cn) represents a model parameter of a feature i belonging to a feature type ij of a cn th log package, cn=1, 2, …, K, so as to calculate the similarity between the log package to be classified and the k th log package type.

[0015] The second aspect of the present application provides a computing device, comprising:

[0016] a processing module configured to determine a plurality of feature types of a log package to be classified;

[0017] The processing module is further configured to determine a model parameter corresponding to each feature type in the plurality of feature types, to obtain a plurality of model parameters of the log package to be classified.

[0018] The processing module is further configured to calculate a similarity between the log package to be classified and each log package type in K log package types based on the plurality of model parameters of the log package to be classified, to obtain K similarities.

[0019] The processing module is further configured to determine a log package type to which the log package to be classified belongs from the K log package types according to the K similarities.

[0020] In some possible implementation manners, the processing module is specifically configured to:

[0021] determine a highest value similarity in the K similarities;

[0022] determine a log package type corresponding to the highest value similarity in the K log package types, to obtain the log package type to which the log package to be classified belongs.

[0023] In some possible implementation manners, the processing module is specifically configured to:

[0024] determine a feature value of each feature in a plurality of features in the log package to be classified, to obtain a plurality of feature values of the log package to be classified;

[0025] determine a feature type corresponding to each feature based on the plurality of feature values, to obtain the plurality of feature types of the log package to be classified.

[0026] In some possible implementation manners, the plurality of features include a data size, an uploader, a keyword, and / or a first-level directory structure.

[0027] In some possible implementation manners, the processing module is further configured to:

[0028] obtain M log packages, each log package in the M log packages belonging to a log package type in the K log package types, M being a positive integer;

[0029] determine a number of log packages of the feature i belonging to the feature type ij in the kth log package type of the K log package types as a corresponding model parameter N(x=ij|y=k), the feature i being one of the plurality of features, the feature type ij being one of the plurality of feature types to which the feature i belongs, i∈I, ij∈J, k=1,2,…,K i , I being a set of the plurality of features, J i being a set of the plurality of feature types of the feature i, k=1,2,…,K.

[0030] In some possible implementation manners, the processing module is further configured to:

[0031] determine log package features based on the M log packages, the log package features comprising one or more feature types;

[0032] determine, based on the plurality of feature types of the log package to be classified, that the log package to be classified conforms to at least one feature type in the log package features.

[0033] In some possible implementation manners, the processing module is specifically configured to:

[0034] obtain model parameters corresponding to each feature type in the plurality of feature types of the kth log package type, to obtain a plurality of model parameters corresponding to the kth log package type, k=1,2,…,K;

[0035] input the plurality of model parameters of the log package to be classified and the plurality of model parameters corresponding to the kth log package type into a preset Naive Bayes formula, to obtain a similarity between the log package to be classified and the kth log package type.

[0036] In some possible implementation manners, the Naive Bayes formula is as follows:

[0037]

[0038] wherein, N cn represents a sum of the plurality of model parameters corresponding to the cnth log package type, N k represents a sum of the plurality of model parameters corresponding to the kth log package type, I represents the plurality of features, J i represents a set of feature types of the feature i of the log package to be classified, λ, s i are preset constant parameters, N(x=ij|y=cn) represents a model parameter of the feature i belonging to the feature type ij in the cnth log package, cn=1,2,…,K.

[0039] The third aspect of the present application provides a computer readable storage medium, the computer readable storage medium stores instructions, when the instructions run on a computer, the computer executes the method in any one of the first aspect.

[0040] The fourth aspect of the present application provides a computer program product, which comprises computer execution instructions stored in a computer readable storage medium; at least one processor of a device can read the computer execution instructions from the computer readable storage medium, and the at least one processor executes the computer execution instructions to enable the device to implement the method provided in the first aspect or any possible implementation manner of the first aspect.

[0041] The fifth aspect of the present application provides a communication apparatus, which can comprise at least one processor, a memory and a communication interface. The at least one processor is coupled with the memory and the communication interface. The memory is configured to store instructions, the at least one processor is configured to execute the instructions, and the communication interface is configured to communicate with other communication apparatus under the control of the at least one processor. The instructions, when executed by the at least one processor, enable the at least one processor to execute the method in the first aspect or any possible implementation manner of the first aspect.

[0042] The sixth aspect of the present application provides a chip system, which comprises a processor configured to support the functions involved in the first aspect or any possible implementation manner of the first aspect.

[0043] In a possible design, the chip system can further comprise a memory configured to store necessary program instructions and data. The chip system can be composed of a chip, or can comprise the chip and other discrete devices.

[0044] The technical effects brought by the second to sixth aspects or any possible implementation manner thereof can be referred to the technical effects brought by the first aspect or different possible implementation manners of the first aspect, which will not be described herein. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 A flowchart of a log packet classification method provided by an embodiment of the present application;

[0046] Figure 2-1 A flowchart of generating a plurality of model parameters based on M log packets provided by an embodiment of the present application;

[0047] Figure 2-2 A schematic diagram of a directory structure in the present application;

[0048] Figure 3 Another flowchart of a log packet classification method provided by an embodiment of the present application;

[0049] Figure 4 A structural schematic diagram of a computing device provided by an embodiment of the present application;

[0050] Figure 5 FIG. 1 is a structural schematic diagram of a communication device according to an embodiment of the present application. DETAILED DESCRIPTION

[0051] The embodiments of the present application provide a log package classification method and a computing device, which are used for classifying a to-be-classified log package.

[0052] The embodiments of the present application will be described below with reference to the drawings.

[0053] The terms "first", "second", and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the terms thus used can be interchanged under appropriate circumstances, and are merely used to distinguish between similar objects in the description of the embodiments of the present application. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that a process, method, system, product or device that includes a list of elements does not necessarily limit those elements, but can include other elements not clearly listed or inherent to such a process, method, product or device.

[0054] A log package is a file used to record events occurring in the running of an operating system or other application software or messages between different users of communication software. The log package can be used for tracking, locating errors, debugging and analyzing code, and for monitoring the performance of an application program. A technician can determine the running state of an operating system or application software by events recorded in the log package. When an operating system or application program has an error or crashes, the problem existing in the operating system or application program can be found by analyzing the log package.

[0055] With the development of data center business, a large amount of log packages are generated every day, and different log packages record different fault information. By analyzing and processing the log packages, high-value information can be mined. In order to improve the efficiency of analyzing and processing the log packages, it is generally necessary to first classify the log packages, and perform different analysis and processing logics on different types of log packages. Therefore, it is very important to accurately and quickly classify the log packages.

[0056] Currently, the industry generally sets multiple preset rules, each preset rule corresponds to one or more feature types, if the to-be-classified log package meets all feature types of a preset rule, the to-be-classified log package is classified as the log package type corresponding to the preset rule. For example, the preset rule corresponding to the log package type A includes: 1, the uploader is Frank; 2, the data size of the log package is 0-10 megabytes (M). Then, if the uploader of the log package A is Frank, and the data size of the log package is 5M, the log package A meets the preset rule corresponding to the log package type A, that is, the log package A is classified as the log package type A. If the uploader of the log package B is Chris, or the data size of the log package B is 20M, the log package B does not meet the preset rule corresponding to the log package type A, and the log package B does not belong to the log package type A.

[0057] However, for the log package that does not meet any preset rule, the classification method cannot be used for classification. For example, it is known that the log package types A / B / C / D, each log package type has a corresponding preset rule, and each preset rule includes one or more feature types. If the log package C does not meet any preset rule, the classification method cannot classify the log package C.

[0058] Therefore, the present application proposes a log package classification method. In the present application, the computing device first determines the multiple feature types of the to-be-classified log package, and then determines the multiple model parameters of the to-be-classified log package based on the multiple feature types. Next, the computing device calculates the similarity between the to-be-classified log package and each log package type in the K log package types based on the multiple model parameters, obtains K similarities, and finally determines the log package type to which the to-be-classified log package belongs from the K log package types according to the K similarities. Then, for the to-be-classified log package that does not meet any rule of any log package type in the K log package types, the classification can also be performed.

[0059] In the embodiments of the present application, the multiple model parameters are determined based on the M log packages in embodiment one, the multiple model parameters corresponding to the to-be-classified log package are determined in embodiment two, and the log package type to which the to-be-classified log package belongs is determined based on the multiple model parameters corresponding to the to-be-classified log package.

[0060] Specifically, as shown in Figure 1 After the computing device obtains the M log packages, the log package type of each log package in the M log packages can be determined in a manual labeling manner, the number of log packages of each log package type in the M log packages is obtained, and the feature type and the corresponding model parameter N(x=ij|y=k), k=1, 2, …, K of each log package in the M log packages are determined. In addition, the computing device can also determine the log package features based on the M log packages.

[0061] Then, after the computing device obtains the log package to be classified, the computing device first determines the feature values of the plurality of features of the log package to be classified, and determines the corresponding feature types, obtains the plurality of feature types, and determines whether the log package to be classified needs to be classified based on the plurality of feature types and the log package features. If the log package to be classified needs to be classified, the computing device determines the plurality of model parameters of the log package to be classified based on the plurality of feature types of the log package to be classified, and inputs the plurality of model parameters of the log package to be classified and the plurality of model parameters corresponding to each log package type in the K log package types into the Naive Bayes formula to calculate the similarity between the log package to be classified and each log package type in the K log package types, obtains the K similarities, and finally determines the log package type to which the log package to be classified belongs from the K log package types according to the K similarities. Then, the log package to be classified which does not meet any rule of any log package type in the K log package types can also be classified.

[0062] Referring to Figure 2-1 As shown in the figure, the log package classification method provided by the embodiment one of the present application mainly includes the following steps:

[0063] 201. The computing device obtains M log packages.

[0064] In some possible implementation manners, the M log packages are log packages generated in one or more time periods in the past, the M log packages record events occurring in the running of an operating system or other application software or messages transmitted between different users of communication software in one or more time periods in the past, and M is a positive integer.

[0065] For example, M = 10000, that is, the inventory of log packages is 10000, of which 2000 log packages record events occurring in the operating system, 3000 log packages record messages between different users in the communication software, and 5000 log packages record events occurring in each application software based on the operating system except the communication software.

[0066] 202. The computing device obtains the log package type of each log package in the M log packages.

[0067] In the embodiments of the present application, the M log packages can be inventory log packages that have been labeled with log package types, that is, each log package in the M log packages belongs to one log package type of the K log package types, and M is a positive integer. For example, the computing device can be preconfigured with K log package types (K is a positive integer), and when the M log packages are obtained, the log package type to which each log package in the M log packages belongs can be determined by manual labeling. For example, the K log package types include a first log package type, a second log package type, a third log package type, and a Kth log package type. For a log package A in the M log packages, the log package A can be determined to belong to the first log package type by manual labeling. It should be noted that a log package can only belong to one log package type. For example, if the log package A belongs to the first log package type, the log package A does not belong to the second log package type, the third log package type, or the Kth log package type.

[0068] For example, the first log package type is an event occurring in the running of an operating system, and the second log package type is a message transmitted between different users of communication software, which is not limited here.

[0069] 203. The computing device determines the number of log packages of each log package type in the K log package types according to the log package type of each log package in the M log packages.

[0070] In the embodiments of the present application, after the above manual labeling, the number of log packages corresponding to each log package type in the K log package types is obtained. For example, M = 10000, that is, the computing device obtains 10000 log packages, and the 10000 log packages can be divided into 4 log package types, that is, K = 4, which are a first log package type, a second log package type, a third log package type, and a fourth log package type. The number of log packages of the first log package type is 2500, the number of log packages of the second log package type is 3000, the number of log packages of the third log package type is 1000, and the number of log packages of the fourth log package type is 3500.

[0071] 204. The computing device determines the number of log packages in which the feature value of the feature i in the M log packages belonging to the kth log package type belongs to the feature type ij, k = 1, 2, …, K.

[0072] It should be noted that each log package can have multiple features, where I represents a set of multiple features, and i ∈ I. For example, the multiple features are 4 features, including data size (feature 1), keywords (feature 2), upload person (feature 3), and / or first-level directory structure (feature 4).

[0073] Each feature of the log package has a feature value. For example, the data size interval (feature 1) of the log package A is 5M, i.e., the feature value of feature 1 of the log package A is 5M.

[0074] Each feature can have multiple feature types. For example, the data size (feature 1) has 4 feature types, which are (0MB, 10MB) (feature type 11), (10MB, 50MB) (feature type 12), (50MB, 200MB) (feature type 13) and (200MB, 1000MB) (feature type 14); the keyword (feature 2) has 5 feature types, which are “security information” (feature type 21), “system login” (feature type 22), “network connection” (feature type 23), “fault” (feature type 24) and “load balancing” (feature type 25); the uploader (feature 3) has 4 feature types, which are Benjamin (feature type 31), Hunter (feature type 32), Gordon (feature type 33) and Frank (feature type 34); and the first-level directory structure (feature 4) has 2 feature types, which are syslog (feature type 41) and math (feature type 42) (for example Figure 1

[0075] Different feature values can belong to the same or different feature types. Then, the feature value of feature 1 of the log package A belongs to (0MB, 10MB) (feature type 11); the feature value of feature 1 of the log package B = 8M, which also belongs to (0MB, 10MB) (feature type 11); and the feature value of feature 1 of the log package C = 15M, which belongs to (10MB, 50MB) (feature type 12).

[0076] Then, the computing device can obtain the feature values of each feature of the multiple features of a log package, obtain multiple feature values, and then the computing device can determine the feature types to which each feature value in the multiple feature values belongs, to obtain the multiple feature types of the log package.

[0077] For example, the feature values and the feature types to which the feature values belong of the 4 features of the log package A are as follows: the feature value of the data size interval (feature 1) is 5M (belongs to feature type 11); the feature value of the keyword (feature 2) is “security information” (belongs to feature type 21); the uploader (feature 3) is Benjamin (belongs to feature type 21); and the first-level directory structure (feature 4) is syslog (belongs to feature type 41).

[0078] ​In the embodiments of the present application, after the computing device determines the number of log packages of the kth log package type according to the determination of the log package type of each log package in the M log packages, the computing device can determine the number of log packages of feature i belonging to the kth log package type, which belongs to the feature type ij, where k = 1, 2, …, K. Wherein, ij∈J i , Ji is a set of multiple feature types of feature i, and the feature type ij is a feature type of feature i.

[0079] In the above example, if K = 4, k = 1, and M = 10000, then the number of log packages of the kth log package type in the M log packages is 2500. If i = 1 and ij = 12, the computing device can determine the number of log packages in which the data size interval (feature 1) belongs to (10MB, 50MB) (feature type 12) in the 2500 log packages. Then the computing device can determine that in the 2500 log packages:

[0080] The number of log packages in which the data size interval (feature 1) belongs to (0MB, 10MB) (feature type 11); the number of log packages in which the data size interval (feature 1) belongs to (10MB, 50MB) (feature type 12); the number of log packages in which the data size interval (feature 1) belongs to (50MB, 200MB) (feature type 13); the number of log packages in which the data size interval (feature 1) belongs to (200MB, 1000MB) (feature type 14);

[0081] The number of log packages in which the keyword (feature 2) belongs to “security information” (feature type 21); the number of log packages in which the keyword (feature 2) belongs to “system login” (feature type 22); the number of log packages in which the keyword (feature 2) belongs to “network connection” (feature type 23); the number of log packages in which the keyword (feature 2) belongs to “fault” (feature type 24); the number of log packages in which the keyword (feature 2) belongs to “load balancing” (feature type 25);

[0082] The number of log packages in which the uploader (feature 3) belongs to Benjamin (feature type 31); the number of log packages in which the uploader (feature 3) belongs to Hunter (feature type 32); the number of log packages in which the uploader (feature 3) belongs to Gordon (feature type 33); the number of log packages in which the uploader (feature 3) belongs to Frank (feature type 34);

[0083] The number of log packages in which the first-level directory structure (feature 4) belongs to syslog (feature type 41); the number of log packages in which the first-level directory structure (feature 4) belongs to math (feature type 42).

[0084] 205. The computing device determines the corresponding model parameters N (x=ij|y=k), k=1,2,…,K, based on the number of log packets in M ​​log packets whose feature i belongs to the kth log packet type and whose feature type ij belongs to the feature type.

[0085] In some possible implementations, the computing device can directly use the number of log packets in the M log packets whose feature i belongs to feature type ij and which belongs to feature type k in the k-th log packet type as the corresponding model parameter N(x = ij | y = k). In other possible implementations, the computing device can perform a certain conversion based on the number of log packets in the K-th log packet type whose feature i belongs to feature type ij and which is used as the corresponding model parameter N(x = ij | y = k). For example, dividing the number of log packets in the k-th log packet type whose feature i belongs to feature type j by 1000 and rounding down yields the corresponding model parameter N(x = ij | y = k). For instance, N(x = 11 | y = 2) represents the model parameter corresponding to feature 1 belonging to feature type 1 in the 2nd log packet type.

[0086] 206. The computing device determines the characteristics of the log packets based on multiple feature types of each log packet in the M log packets.

[0087] It should be noted that log packet features are used to determine whether a log packet needs to be classified. If a log packet has these features, it is determined that the log packet is a log packet to be classified, that is, it needs to be classified; otherwise, it is determined that the log packet does not need to be classified.

[0088] In some possible implementations, if at least one of the M log packets contains a feature that belongs to a certain feature type, then that feature type belongs to the log packet's features. Otherwise, if no feature of any of the M log packets belongs to a certain feature type, then that feature type does not belong to the log packet's features.

[0089] For example, if the number of log packets in M ​​log packets whose feature 1 belongs to feature type 13 is 100, then the log packet feature includes feature type 13; if the number of log packets in M ​​log packets whose feature 1 belongs to feature type 13 is 0, then the log packet feature does not include feature type 13.

[0090] Exemplarily, the log package features can include feature type 11, feature type 12, feature type 13, feature type 21, feature type 22, feature type 23, feature type 24, feature type 25, feature type 31, feature type 32, feature type 33, feature type 34, feature type 41, and feature type 42, but not include feature type 14, by determining, by the computing device, the log package features based on the plurality of feature types of each of the M log packages.

[0091] It should be noted that step 206 is optional.

[0092] In the embodiment of the present application, the computing device determines the corresponding model parameters N(x=ij|y=k), k=1, 2, …, K, by obtaining the M log packages and determining the number of log packages of the feature i belonging to the k-th log package type and belonging to the feature type ij, so that the similarity between the to-be-classified log package and the K log package types can be calculated by the model parameters subsequently, to classify the to-be-classified log package. In addition, the computing device can also determine the log package features based on the M log packages, so that it can be determined subsequently whether the to-be-classified log package needs to be classified, to improve the classification efficiency of the log package.

[0093] Referring to Figure 3 The log package classification method provided in the second embodiment of the present application mainly includes the following steps:

[0094] 301. The computing device obtains a to-be-classified log package.

[0095] In the embodiment of the present application, the to-be-classified log package can be a log package generated by the computing device in real time. Exemplarily, the to-be-classified log package is a file of events occurring in real time during the running of an operating system or other application software or messages between different users of communication software.

[0096] 302. The computing device extracts feature values of a plurality of features of the to-be-classified log package, to obtain a plurality of feature values of the to-be-classified log package.

[0097] In some possible implementation manners, the computing device can extract the feature values of each of the plurality of features of the to-be-classified log package. For example, the plurality of features can include data size (feature 1), keywords in the name (feature 2), upload person (feature 3), and first-level directory structure (feature 4).

[0098] For example, the data size (feature 1) of the log package to be classified is 5M (feature value), the name (feature 2) of the log package is "information of security information and system login and network connection", the keywords in the name are "security information", "system login" and "network connection" (feature values), the uploader (feature 3) is Benjamin (feature value), and the first-level directory structure (feature 4) includes math (feature value).

[0099] In some possible implementations, the computing device can extract the keywords in the name of the log package to be classified based on a preset segmentation library. The preset segmentation library includes a plurality of keywords. If the name of the log package to be classified has a keyword in the preset segmentation library, the computing device can extract the keyword from the name of the log package to be classified as the feature value of the feature of the log package to be classified.

[0100] For example, the preset segmentation library includes the following keywords: "security information", "system login", "network connection", "failure" and "load balancing". Then, in the name "information of security information and system login and network connection" of the log package to be classified, the keywords "security information", "system login" and "network connection" are included. Therefore, the computing device extracts the keywords "security information", "system login" and "network connection" from the name of the log package to be classified as the feature values of the feature of the log package to be classified.

[0101] 303. The computing device determines whether the log package to be classified needs to be classified based on the log package feature.

[0102] Continuing the foregoing example, the log package feature can include the feature type 11, the feature type 12, the feature type 13, the feature type 21, the feature type 22, the feature type 23, the feature type 24, the feature type 25, the feature type 31, the feature type 32, the feature type 33, the feature type 34, the feature type 41 and the feature type 42, but not the feature type 14. Then, if the data size of the log package to be classified is 5M, the feature type 11 is satisfied, the log package feature is met, and the log package to be classified needs to be classified. If the data size of the log package to be classified is 500M, the feature type 14 is satisfied, the log package feature is not met, and the log package to be classified does not need to be classified.

[0103] It should be noted that step 203 is optional.

[0104] 304. The computing device determines the feature type to which each feature value in the plurality of feature values of the log package to be classified belongs, and obtains a plurality of feature types of the log package to be classified.

[0105] Continuing the above example, if the log package to be classified is log package A, if the data size (feature 1) in the log package to be classified is 5M (feature value), it belongs to feature type 11; if the keywords (feature 2) in the name in the log package to be classified are "system login" (feature value) and "network connection" (feature value) (based on finding corresponding nouns in the preset segmentation library), it belongs to feature type 22 and feature type 24, respectively; if the upload person (feature 3) in the log package to be classified is Benjamin (feature value), it belongs to feature type 31; and if the first layer directory structure (feature 4) in the log package to be classified is math (feature value), it belongs to feature type 42.

[0106] 305. The computing device determines the model parameters corresponding to each of the plurality of feature types of the log package to be classified, to obtain a plurality of model parameters of the log package to be classified.

[0107] Continuing the above example, N(x=ij|y=k) represents the model parameter corresponding to the feature type ij of the feature value of feature i in the kth log package type. The feature value 1 of feature 1 of the log package to be classified belongs to feature type 11, the feature value 2 of feature 2 belongs to feature type 22 and feature type 24, the feature value 3 of feature 3 belongs to feature type 31, and the feature value 4 of feature 4 belongs to feature type 42. Therefore, the plurality of model parameters corresponding to the log package to be classified are N(x=11|y=k), N(x=22|y=k), N(x=24|y=k), N(x=31|y=k), and N(x=42|y=k), where k=1, 2, …, K.

[0108] 306. The computing device calculates the similarity between the log package to be classified and each of the K log package types based on the plurality of model parameters of the log package to be classified and the plurality of model parameters corresponding to each of the K log package types, to obtain K similarities.

[0109] In some possible implementations, the computing device can calculate the sum of the plurality of model parameters corresponding to the cnth log package type, and then input the plurality of model parameters of the log package to be classified and the sum of the plurality of model parameters corresponding to the cnth log package type into a preset Naive Bayes formula to obtain the similarity between the log package to be classified and the cnth log package type.

[0110] Continuing the above example, the plurality of model parameters of the log package to be classified are N(x=11|y=k), N(x=22|y=k), N(x=24|y=k), N(x=31|y=k), and N(x=42|y=k), where k=1, 2, …, K, and the sum of the plurality of model parameters of the cnth log package type is N cn N(x=11|y=k), N(x=22|y=k), N(x=24|y=k), N(x=31|y=k), and N(x=42|y=k), where k=1, 2, …, K, and the sum of the plurality of model parameters of the cnth log package type is N cnand N(x=22|y=k), N(x=24|y=k), N(x=31|y=k), N(x=42|y=k), where k = 1, 2, …, K are put into the following naive Bayes classification formula to obtain the similarity f(cn):

[0111]

[0112] where N cn represents the sum of the plurality of model parameters corresponding to the cnth log packet type, N k represents the sum of the plurality of model parameters corresponding to the kth log packet type, I represents the plurality of features, J i represents the set of feature types of the feature i of the log packet to be classified, λ, s i are preset constant parameters, and N(x=ij|y=cn) represents the model parameter of the feature i of the cnth log packet belonging to the feature type ij, cn = 1, 2, …, K.

[0113] Continuing with the above example, J1 = {1}, J2 = {2, 4}, J3 = {1}, and J4 = {2}. Through the naive Bayes classification formula, the similarity of the log packet to be classified with the K log packet types can be obtained, that is, K similarities.

[0114] 307. The computing device determines the log packet type to which the log packet to be classified belongs from the K log packet types according to the K similarities.

[0115] In some possible implementation manners, the computing device can take the highest value similarity among the K similarities, and take the log packet type corresponding to the highest value similarity among the K log packet types as the log packet type to which the log packet to be classified belongs.

[0116] For example, the computing device determines the similarities of the log packet to be classified with the 1st log packet type, the 2nd log packet type, …, and the Kth log packet type, to obtain similarities 1, 2, …, K. By comparing the similarities 1, 2, …, K, the log packet type corresponding to the highest value similarity is taken as the log packet type of the log packet to be classified.

[0117] In some possible implementation manners, the computing device can take the similarity higher than a preset value among the K similarities, and take the log packet type corresponding to the similarity higher than the preset value among the K log packet types as the log packet type to which the log packet to be classified belongs. Here, the preset value is not limited.

[0118] In the present application, the computing device first determines a plurality of feature types of the log package to be classified, and then determines a plurality of model parameters of the log package to be classified based on the plurality of feature types. Next, the computing device calculates the similarity of the log package to be classified to each log package type in the K log package types based on the plurality of model parameters, to obtain K similarities, and finally determines the log package type to which the log package to be classified belongs from the K log package types according to the K similarities. Therefore, for any log package to be classified that does not meet any rule of any log package type in the K log package types, the log package can also be classified.

[0119] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0120] In order to better implement the above-mentioned scheme of the embodiments of the present application, the related device for implementing the above-mentioned scheme is also provided below.

[0121] Please refer to Figure 4 As shown in the figure, the computing device 400 provided by the embodiments of the present application can include a processing module 401, which is used to perform the following steps:

[0122] determining a plurality of feature types of the log package to be classified, determining the model parameters corresponding to each feature type in the plurality of feature types, to obtain a plurality of model parameters of the log package to be classified, calculating the similarity of the log package to be classified to each log package type in the K log package types based on the plurality of model parameters of the log package to be classified, to obtain K similarities, and determining the log package type to which the log package to be classified belongs from the K log package types according to the K similarities.

[0123] In some possible implementation ways, the processing module 401 is specifically used for: determining the highest value similarity in the K similarities; determining the log package type corresponding to the highest value similarity in the K log package types, to obtain the log package type to which the log package to be classified belongs.

[0124] In some possible implementation ways, the processing module 401 is specifically used for: determining the feature value of each feature in the plurality of features in the log package to be classified, to obtain a plurality of feature values of the log package to be classified; determining the feature type corresponding to each feature based on the plurality of feature values, to obtain the plurality of feature types of the log package to be classified.

[0125] In some possible implementation manners, the plurality of features include data size, an uploading person, a keyword, and / or a first-level directory structure.

[0126] In some possible implementation manners, the computing device 400 further includes:

[0127] The obtaining module 402 is configured to obtain M log packages, each of the M log packages belonging to one log package type of the K log package types, M being a positive integer.

[0128] The processing module 401 is further configured to determine, as a corresponding model parameter N(x=ij|y=k), a number of log packages of a feature i belonging to a feature type ij in a kth log package type of the K log package types, the feature i being one of the plurality of features, the feature type ij being one of a plurality of feature types to which the feature i belongs, i∈I, ij∈J i , I being a set of the plurality of features, J i being a set of the plurality of feature types of the feature i, and k=1, 2, …, K.

[0129] In some possible implementation manners, the processing module 401 is further configured to: determine log package features based on the M log packages, the log package features including one or more feature types; and determine, based on the plurality of feature types of the log package to be classified, that the log package to be classified conforms to at least one feature type in the log package features.

[0130] In some possible implementation manners, the processing module 401 is specifically configured to: obtain a model parameter corresponding to each feature type in a plurality of feature types of the kth log package type, to obtain a plurality of model parameters corresponding to the kth log package type, k=1, 2, …, K; and input the plurality of model parameters of the log package to be classified and the plurality of model parameters corresponding to the kth log package type into a preset Naive Bayes formula, to obtain a similarity between the log package to be classified and the kth log package type.

[0131] In some possible implementation manners, the Naive Bayes formula is as follows:

[0132]

[0133] wherein N cn represents a sum of the plurality of model parameters corresponding to the cnth log package type, N k represents a sum of the plurality of model parameters corresponding to the kth log package type, I represents the plurality of features, J i represents a set of feature types of the feature i of the log package to be classified, λ, s iFor the preset constant parameters, N(x=ij|y=cn) represents the model parameters of feature i belonging to feature type ij in the cn-th log packet, where cn=1,2,…,K.

[0134] It should be noted that the information interaction and execution process between the modules / units of the above-mentioned device are based on the same concept as the method embodiments of this application, and the resulting technical effects are the same as those of the method embodiments of this application. For details, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0135] This application also provides a computer storage medium storing a program that performs some or all of the steps described in the above method embodiments.

[0136] The following describes another communication device provided in the embodiments of this application. Please refer to [link to relevant documentation]. Figure 5 As shown, the communication device 500 includes:

[0137] The system comprises a receiver 501, a transmitter 502, a processor 503, and a memory 504. In some embodiments of this application, the receiver 501, transmitter 502, processor 503, and memory 504 may be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0138] Memory 504 may include read-only memory and random access memory, and provides instructions and data to processor 503. A portion of memory 504 may also include non-volatile random access memory (NVRAM). Memory 504 stores operating system and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof. The operation instructions may include various operation instructions for implementing various operations. The operating system may include various system programs for implementing various basic business functions and handling hardware-based tasks.

[0139] Processor 503 controls the operation of communication device 500. Processor 503 can also be called a central processing unit (CPU). In specific applications, the various components of communication device 500 are coupled together through a bus system. This bus system includes not only a data bus but also a power bus, control bus, and status signal bus. However, for clarity, all buses are referred to as a bus system in the diagram.

[0140] The method disclosed in the embodiments of the present application can be applied to the processor 503 or implemented by the processor 503. The processor 503 can be an integrated circuit chip having a signal processing capability. In the implementation process, the steps of the above method can be completed by hardware integrated logic circuit or software form instructions in the processor 503. The processor 503 described above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The disclosed methods, steps and logic block diagrams in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor or the like. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory 504, and the processor 503 reads the information in the memory 504, and combines the hardware to complete the steps of the above method.

[0141] The receiver 501 can be used to receive input digital or character information, and generate signal input related to relevant settings and function control. The transmitter 502 can include a display device such as a display screen, and the transmitter 502 can be used to output digital or character information through an external interface.

[0142] In the embodiments of the present application, the processor 503 is used to execute the above-mentioned log packet classification method.

[0143] In another possible design, when the communication apparatus 500 is a chip, the chip includes a processing unit, for example, a processor, and a communication unit, for example, an input / output interface, a pin, or a circuit, etc. The processing unit can execute computer-executed instructions stored in a storage unit, so that the chip in the terminal performs the wireless reporting information sending method in any one of the first aspects. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit can also be a storage unit outside the chip in the terminal, such as a read-only memory (ROM) or another type of static storage device that can store static information and instructions, a random access memory (RAM), etc.

[0144] The processor mentioned in any one of the above can be a general central processor, a microprocessor, an ASIC, or one or more integrated circuits for controlling execution of programs of the above method.

[0145] It should be further noted that the apparatus embodiments described above are merely illustrative, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment. In addition, the connection relationship between the modules in the apparatus embodiment provided in the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0146] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and necessary general hardware, and of course can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. Generally, functions completed by computer programs can be easily implemented by corresponding hardware, and specific hardware structures for implementing the same function can also be various, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better embodiment. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application.

[0147] In the embodiments described above, the entire or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, the entire or part of the embodiments can be implemented in the form of a computer program product.

[0148] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the entire or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

Claims

1. A log packet classification method, characterized in that, include: The computing device determines multiple feature types of the log packets to be classified; The feature type indicates the range or category of the feature values ​​of the log package to be classified. The computing device determines the model parameters corresponding to each of the multiple feature types to obtain multiple model parameters of the log package to be classified. The computing device calculates the similarity between the log packet to be classified and each of the K log packet types based on multiple model parameters of the log packet to be classified, and obtains K similarity scores. The computing device determines the log packet type to which the log packet to be classified belongs from the K log packet types based on the K similarity scores; The method further includes: The computing device acquires M log packets, each of the M log packets belonging to one of the K log packet types, where M is a positive integer; The computing device determines the number of log packets in the k-th log packet type whose feature i belongs to feature type ij, and uses this number as the corresponding model parameter N (x=ij|y=k). Feature i is one of the plurality of features, and feature type ij is one of the plurality of feature types to which feature i belongs, i∈I, ij∈J. i I is the set of the multiple features, J i Let k be the set of multiple feature types of feature i, where k = 1, 2, ..., K.

2. The method according to claim 1, characterized in that, The computing device determines the log packet type to which the log packet to be classified belongs based on the K similarities, including: The computing device determines the highest similarity value among the K similarities; The computing device determines the log packet type corresponding to the highest similarity value among the K log packet types, and obtains the log packet type to which the log packet to be classified belongs.

3. The method according to claim 1 or 2, characterized in that, The computing device determines multiple feature types of the log packets to be classified, including: The computing device determines the feature value of each feature among multiple features in the log packet to be classified, and obtains multiple feature values ​​of the log packet to be classified. The computing device determines the corresponding feature type based on each feature among the plurality of feature values, thereby obtaining the plurality of feature types of the log package to be classified.

4. The method according to any one of claims 1-3, characterized in that, The features include data size, uploader, keywords, and / or first-level directory structure.

5. The method according to claim 1, characterized in that, After the computing device acquires M log packets, it also includes: The computing device determines log packet characteristics based on the M log packets, and the log packet characteristics include one or more feature types; Before the computing device determines the model parameters corresponding to each of the plurality of feature types, the method further includes: The computing device determines that the log packet to be classified conforms to at least one of the log packet features based on multiple feature types of the log packet to be classified.

6. The method according to any one of claims 1-5, characterized in that, The computing device calculates the similarity between the log packet to be classified and each of the K log packet types based on multiple model parameters of the log packet to be classified, and obtains K similarity scores, including: The computing device obtains the model parameters corresponding to each feature type among the multiple feature types of the k-th log packet type, and obtains multiple model parameters corresponding to the k-th log packet type, k=1,2,…,K; The computing device inputs multiple model parameters of the log packet to be classified and multiple model parameters corresponding to the kth log packet type into a preset Naive Bayes formula to obtain the similarity between the log packet to be classified and the kth log packet type.

7. The method according to claim 6, characterized in that, The Naive Bayes formula is as follows: ； Where, N cn N represents the sum of multiple model parameters corresponding to the nth log packet type. k I represents the sum of multiple model parameters corresponding to the k-th log packet type, and J represents the multiple features. i Let λ and s represent the set of feature types of feature i in the log packet to be classified. i For the preset constant parameters, N(x=ij|y=cn) represents the model parameters of feature i belonging to feature type ij in the cn-th log packet, where cn=1,2,…,K.

8. A computing device, characterized in that, include: The processing module is used to determine the multiple feature types to which the log packets to be classified belong; The feature type indicates the range or category of the feature values ​​of the log package to be classified. The processing module is also used to determine the corresponding model parameters based on each of the multiple feature types to obtain multiple model parameters of the log package to be classified. The processing module is also used to calculate the similarity between the log package to be classified and each of the K log package types based on multiple model parameters of the log package to be classified, so as to obtain K similarity scores. The processing module is further configured to determine the log packet type to which the log packet to be classified belongs from the K log packet types based on the K similarities; The processing module is further configured to acquire M log packets, each of which belongs to one of the K log packet types. M is a positive integer that determines the number of log packets in the k-th log packet type whose feature i belongs to feature type ij, and serves as the corresponding model parameter N (x=ij|y=k). Feature i is one of the plurality of features, and feature type ij is one of the plurality of feature types to which feature i belongs, i∈I, ij∈J. i I is the set of the multiple features, J i Let k be the set of multiple feature types of feature i, where k = 1, 2, ..., K.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that causes a computer device to perform the method as described in any one of claims 1-7.

Images