A security log analysis method based on multi-source label weak supervision learning

Abstract

The application relates to the technical field of network security, and discloses a security log analysis method based on multi-source label weak supervision learning, which comprises the following steps: obtaining similar log samples in a label log data set, and combining the similar log samples into a sub-sample set; taking the same type of each log sample in the sub-sample set as a target, updating the comparison weight corresponding to each label of the log sample in the sub-sample set through a target classifier, and obtaining a weight value matrix; the comparison weight is used for indicating the difference distance between the candidate label information of the log sample and other predicted labels; generating a target label corresponding to a target sample log based on the weight value matrix, and training a log analysis model according to the target sample log and the target label; wherein the trained log analysis model is used for analyzing security logs. The application can improve the accuracy of security log analysis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and particularly relates to a security log analysis method based on multi-source label weak supervision learning. BACKGROUND

[0002] When detecting or analyzing security logs, some high-value labeled data is often needed for model construction and learning. However, in reality, especially in network security business scenarios, it is extremely difficult to obtain real attack label data. This is limited by the complexity of the attack process and the complexity of the network environment and other factors. Therefore, the detection and analysis of network security often rely on traditional expert experience, rules or knowledge collision, but these often cannot cope with complex attack scenarios or unknown attack types. Even if deep learning or machine learning methods are used, they are based on a large amount of high-value label data to obtain better results.

[0003] Therefore, there is an urgent need for a method for security log analysis using limited labels or weak value labels. SUMMARY

[0004] Therefore, the present application provides a security log analysis method based on multi-source label weak supervision learning to improve the accuracy of security log analysis.

[0005] In a first aspect, the present application provides a security log analysis method based on multi-source label weak supervision learning, which comprises: obtaining similar log samples in a label log data set, and combining the similar log samples into a sub-sample set; updating the comparison weight corresponding to each label of the log samples in the sub-sample set through a target classifier, taking the same type of each log sample in the sub-sample set as the target, to obtain a weight value matrix; the comparison weight is used to indicate the difference distance between the candidate label information of the log sample and other predicted labels; generating a target label corresponding to a target sample log based on the weight value matrix, and training a log analysis model according to the target sample log and the target label; wherein the trained log analysis model is used for analyzing security logs.

[0006] In an optional implementation, before updating the comparison weight corresponding to each label of the log samples in the sub-sample set through the target classifier, it further comprises: obtaining each log sample in the label log data set and the label information corresponding to each log sample; obtaining the comparison weight corresponding to the candidate label information according to the difference distance between the candidate label information of each log sample and other predicted labels.

[0007] In an optional implementation, the difference distance between the candidate label information of the log sample and other predicted labels is obtained by the following formula: S is the number of labels of the log sample; wherein L i,k is a weak label L k The label result of the log sample X i is C k (X i ) is the prediction result of the label result of the log sample X i by the weak classifier C k .

[0008] In an optional implementation, the method further comprises: for each log sample, detecting by the plurality of weak classifiers, obtaining a plurality of weak labels corresponding to each log sample, and taking the plurality of weak labels as the label information.

[0009] In an optional implementation, the obtaining of the similar log samples in the labeled log dataset comprises: for a target log sample, obtaining a text vector of the target log sample; obtaining a text vector distance between each log sample in the labeled log dataset and the target log sample; and according to the text vector distance between each log sample and the target log sample, screening a plurality of log samples, and taking the plurality of log samples and the target log sample as the similar log samples.

[0010] In a second aspect, the present application provides a security log analysis device based on multi-source label weak supervision learning, the device comprising: a sample acquisition module, configured to obtain similar log samples in a labeled log dataset, and combine the similar log samples into a sub-sample set; a weight determination module, configured to take the same type of each log sample in the sub-sample set as a target, update a contrast weight corresponding to each label of the log sample in the sub-sample set by a target classifier, and obtain a weight value matrix; the contrast weight is used to indicate a difference distance between candidate label information of the log sample and other predicted labels; a model training module, configured to generate a target label corresponding to a target sample log based on the weight value matrix, and train a log analysis model according to the target sample log and the target label; wherein the trained log analysis model is used to analyze security logs.

[0011] In an optional implementation, the weight determination module further comprises: a label acquisition unit, configured to obtain each log sample in the labeled log dataset and each label information corresponding to each log sample; and a weight acquisition unit, configured to obtain a contrast weight corresponding to the candidate label information according to the difference distance between the candidate label information of each log sample and other predicted labels.

[0012] In an optional implementation, the weight acquisition unit is further configured to: the difference distance between the candidate label information of the log sample and other predicted labels is obtained by the following formula: S is the number of labels of the log sample; wherein Li,k is a weak label L k the label result of the log sample X i , C k (X i ) is a weak classifier C k the predicted result of the label result of the log sample X i .

[0013] In an optional implementation, the apparatus further includes a sample detection module configured to, for each log sample, detect by the plurality of weak classifiers, obtain a plurality of weak labels corresponding to each log sample, and take the plurality of weak labels as the label information.

[0014] In an optional implementation, the sample acquisition module includes a first sample acquisition unit configured to, for a target log sample, acquire a text vector of the target log sample; a second sample acquisition unit configured to acquire a text vector distance between each log sample in the label log data set and the target log sample; and a third sample acquisition unit configured to, according to the text vector distance between each log sample and the target log sample, filter out a plurality of log samples, and take the plurality of log samples and the target log sample as similar log samples.

[0015] In a third aspect, the present application provides a computer device, including a memory and a processor, the memory and the processor are communicatively connected with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the security log analysis method based on multi-source label weak supervision learning of the first aspect or any of the corresponding embodiments thereof.

[0016] In a fourth aspect, the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the security log analysis method based on multi-source label weak supervision learning of the first aspect or any of the corresponding embodiments thereof.

[0017] In a fifth aspect, the present application provides a computer program product, which includes computer instructions, and the computer instructions are used to make a computer execute the security log analysis method based on multi-source label weak supervision learning of the first aspect or any of the corresponding embodiments thereof.

[0018] The technical scheme provided in the present application can include the following beneficial effects:

[0019] Similar log samples are obtained in the label log data set, a data basis with similar features is provided, and similar log samples are combined into a sub-sample set. The same type of each log sample in the sub-sample set is taken as the target, the labels in the sub-sample set are compared by a target classifier, and the comparison weight is updated according to the difference distance between the labels to identify more accurate labels. The comparison weight is used to indicate the difference distance between the candidate label information of the log sample and other predicted labels, as a reference for the model to adjust the weight during training. The target label corresponding to the target sample log is generated based on the weight value matrix, and the log analysis model is trained according to the target sample log and the target label to improve the prediction ability of the model. The trained log analysis model is used to analyze the security log.

[0020] The above scheme uses multi-source weak value labels for security log analysis by obtaining similar log samples, combining a sub-sample set, updating comparison weights, generating target labels, and training a log analysis model, thereby improving the accuracy of security log analysis. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0022] Figure 1 is a flow diagram of a security log analysis method based on multi-source label weak supervision learning according to an embodiment of the present application;

[0023] Figure 2 is a flow diagram of a label comparison weight corresponding method according to an optional embodiment of the present application;

[0024] Figure 3 is a structural block diagram of a security log analysis device based on multi-source label weak supervision learning according to an embodiment of the present application;

[0025] Figure 4 is a hardware structure diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0027] In the network attack scenario, various high-value label data is difficult to obtain, and limited labels or weak-value labels need to be used for security log analysis.

[0028] For example, some logs with rule qualitative labels can be formed based on existing detection rules (ids rules, packet detection rules), but there are a large number of false positives due to the complexity of attacks or the timeliness of rule updates. However, for some known or traditional attack means, there is still some detection value, and the labels generated by these detection rules can be used as weak-value labels.

[0029] In addition, some knowledge or threat intelligence can also provide some collision hit label information, such as for an attack source IP, if there is threat intelligence information, the attack log can be labeled as malicious to some extent, but one is that threat intelligence is difficult to obtain, and two is that the iterative update timeliness is high, so it can also be regarded as weak-value label information.

[0030] According to the embodiments of the present application, a security log analysis method based on multi-source label weak supervision learning is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] In the present embodiment, a security log analysis method based on multi-source label weak supervision learning is provided, Figure 1 is a flowchart of a security log analysis method based on multi-source label weak supervision learning according to the embodiments of the present application, as Figure 1 shown, the flow includes the following steps:

[0032] Step S101, obtaining similar log samples in the label log data set, and combining the similar log samples into a sub-sample set.

[0033] The label log data set is a data set including a plurality of labeled log samples. The log sample includes a plurality of fields such as source IP, destination IP, port number, protocol, log time, Pyload information, etc. Similar log samples in the data set are identified based on the fields of the log sample. The identified similar log samples are combined together to form a sub-sample set.

[0034] In step S102, the target classifier is used to update the contrast weight corresponding to each label of the log samples in the sub-sample set, so as to obtain a weight value matrix, with the same type of each log sample in the sub-sample set as the target. The contrast weight is used to indicate the difference distance between the candidate label information of the log sample and other predicted labels.

[0035] The target classifier is a model used for classifying log samples, which can be a neural network, a support vector machine, a decision tree, etc. The target classifier is used to predict the log samples in the sub-sample set and calculate the difference distance between the candidate label information of each sample and other predicted labels. The difference distance is used as the contrast weight to represent the similarity between different labels. The update of the contrast weight is an iterative process. In each iteration, the weight value is updated according to the current classification result and the difference distance. After multiple iterations, the final weight value matrix is obtained. The weight value matrix includes the contrast weight value corresponding to each label of the log samples in each sub-sample set, which reflects the effective weight value of different labels in the model.

[0036] In step S103, the target label corresponding to the target sample log is generated based on the weight value matrix, and the log analysis model is trained according to the target sample log and the target label. The trained log analysis model is used for analyzing security logs.

[0037] The target label corresponding to the target sample log is generated based on the weight value matrix. The log analysis model is trained using the target sample log and the target label. After training, the log analysis model is used to analyze new security logs. The new log sample is input into the model, and the prediction result of the model is obtained.

[0038] In summary, by obtaining similar log samples, combining sub-sample sets, updating contrast weights, generating target labels, and training log analysis models, the multi-source weak value labels are used for security log analysis, and the accuracy of security log analysis is improved.

[0039] In an optional embodiment, Figure 2 The flowchart of the label contrast weight corresponding method according to the optional embodiment of the present application is shown in FIG. 2. The flowchart includes the following steps:

[0040] Step a11: Obtain each log sample in the labeled log dataset, and the label information corresponding to each log sample.

[0041] Step a12: Based on the difference distance between the candidate label information of each log sample and other predicted labels, obtain the comparison weight corresponding to the candidate label information.

[0042] Candidate labels may be generated by different detection methods or rules. Predicted labels are the labels obtained after predicting log samples using a certain method (such as a machine learning model). Predicted labels may be the same as candidate labels.

[0043] Calculate the difference distance between the candidate label information and other predicted labels for each log sample. Based on the calculated difference distance, assign a comparison weight to each candidate label information. Optionally, the smaller the difference distance, the higher the consistency between the candidate label and the predicted label, and the greater the comparison weight of the candidate label; conversely, the larger the difference distance, the smaller the comparison weight of the candidate label.

[0044] In an optional implementation, the difference distance between the candidate label information of the log sample and other predicted labels in step a12 above is obtained by the following formula:

[0045] S is the number of labels in the log sample.

[0046] Where L i,k It is a weak tag L k In log sample X i The label results, C k (X i ) is a weak classifier C k In log sample X i The predicted results of the label results.

[0047] In one optional implementation, for each log sample, several weak classifiers are used for detection to obtain several weak labels corresponding to each log sample, and these weak labels are used as label information.

[0048] For each log sample, several selected weak classifiers are used for detection. Each weak classifier classifies the log sample based on its own algorithm and features, and provides a classification result (i.e., a weak label). The weak labels of the several weak classifiers corresponding to each log sample are collected as the label information for that log sample.

[0049] In an optional implementation, the process of step S101 above includes the following steps:

[0050] Step S1011: For the target log sample, obtain the text vector of the target log sample.

[0051] Text vectors are a method of converting text data into numerical form, enabling computers to process and understand the text content. Converting target log samples into text vectors allows for comparison and computation within a numerical space.

[0052] Step S1012: Obtain the text vector distance between each log sample in the labeled log dataset and the target log sample.

[0053] A labeled log dataset is a collection of log samples with known labels or categories, used to compare with target log samples to find similarities. Text vector distance is a measure of how close two text vectors are in a multidimensional space.

[0054] Step S1013: Based on the text vector distance between each log sample and the target log sample, select several log samples and use these log samples and the target log sample as similar log samples.

[0055] Log samples in the labeled log dataset are sorted based on text vector distance, and the samples with the smallest distances are selected. These selected log samples are grouped together for subsequent analysis or label assignment processes.

[0056] The following provides a detailed description of an embodiment of a security log analysis method based on multi-source label weak supervision learning provided by the present invention:

[0057] Step 1: Multi-source tag processing

[0058] Given a security log entry x, it typically contains the following fields, as shown in Table 1:

[0059] Table 1 Security Log x Field Information Table

[0060]

[0061]

[0062] This embodiment assumes that each original detection method corresponds to a weak label. For example, in machine learning, the detection result of a simple classifier is the output label of the machine learning algorithm. This embodiment uses three weak labels as an example for illustration. Therefore, for the original data, we can obtain:

[0063] X|L1,L2,L3

[0064] Where X is the original log, and L1, L2, L3 correspond to three weak label results, which are respectively associated with a weak classifier C1, C2, C3.

[0065] Step 2: Multi-source tag value comparison learning

[0066] The quality of weak labels varies, and the purpose of this step is to learn the contrast effect between labels, thereby assigning a contrast weight to each label. Based on the predictions, a corresponding predicted label is predicted, and the distance between the predicted labels of all samples at each source and the predicted labels of other sources is calculated. Here, it is assumed that the number of original log samples is N.

[0067] 1) Calculate the distance between the current source i and the predicted labels of other classifiers.

[0068] S represents the number of corresponding source tags;

[0069] Where L i,k It is a weak tag L k In log sample X i The label results, C k (X i ) is a weak classifier C k In log sample X i The predicted label results are shown above. The results reflect the distance information of each source label on each original log sample.

[0070] 2) Based on the calculated distance, it can be determined that the closer the distance, the better the quality of the source tag corresponding to the log sample.

[0071] Step 3: Tag Fusion Learning

[0072] Furthermore, label fusion learning is performed based on the distance learned above as weight information. For each sample, pseudo-labels are generated by combining the previously learned weight information. After generating pseudo-labels, this information can be used to build a classification model, and its classification loss can be calculated. The weight information can then be adjusted through forward feedback of the loss. Specifically:

[0073] 1) Calculate X for each log sample i The embedding representation vector of the pyload information attribute can be obtained from a regular text embedding vector.

[0074] 2) Obtain its K nearest neighbor samples through vector distance. The distance function can be the cosine distance function.

[0075] 3) Based on the K neighbor samples, add X i This itself forms a new subsample. This subsample forms a subset with respect to X. iSample subclasses with similar characteristics. In this context, similar log loads always have the same label information and should have similar values ​​in the weight matrix. If the weight matrix does not have this similarity or the difference is large, it indicates that the original source information has strong randomness. The difference can be eliminated by adjusting the weights during training, thereby achieving the purpose of weight value fusion learning.

[0076] 4) Further, using the aforementioned weight information, subsample training data is constructed, wherein the labels are obtained by weighting the source labels based on the weight information;

[0077] 5) Construct a classifier, and use the cross-entropy loss function as the loss function. Cross-entropy loss is a commonly used loss function in deep learning, especially suitable for classification problems. It measures the difference between the model's predicted probability distribution and the true label.

[0078] 6) All log samples undergo the same processing, outputting the training process. Through iterative training, the final learned weight matrix D is obtained. ′ (N*S).

[0079] Step 4: Construct the fusion model

[0080] D ′ (N*S) represents the effective weight values ​​that can be simultaneously expressed for the current sample across all source labels. Each value in the matrix represents the contribution and value of the current source label. The final label generated by this matrix can be used as a high-value label for the true label, such as to build a global classifier. Because this classifier is trained using the value labels of the fused weight matrix, rather than a simple weighted average, it has more accurate and effective information, resulting in a more accurate fusion model. Based on the global classifier, it can be used for the detection and analysis of new security logs to obtain classification results.

[0081] This invention integrates and analyzes existing weak value labels or weak classification results to obtain labels with higher value than the original weak value labels. Based on these labels, global training can be performed to obtain a more accurate global classifier.

[0082] This embodiment also provides a security log analysis device based on multi-source label weak supervision learning. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0083] This embodiment provides a security log analysis device based on multi-source label weak supervision learning, such as... Figure 3 As shown, it includes:

[0084] The sample acquisition module 301 is used to acquire similar log samples from the labeled log dataset and combine the similar log samples into a subsample set;

[0085] The weight determination module 302 is used to update the contrast weights corresponding to each label of the log samples in the subsample set with the goal of all log samples in the subsample set having the same type, and obtain a weight value matrix by using a target classifier; the contrast weights are used to indicate the difference distance between the candidate label information of the log sample and other predicted labels.

[0086] The model training module 303 is used to generate target labels corresponding to target sample logs based on the weight value matrix, and to train the log analysis model based on the target sample logs and target labels; wherein, the trained log analysis model is used to analyze security logs.

[0087] In an optional implementation, the weight determination module 302 further includes:

[0088] The label acquisition unit is used to acquire each log sample in the label log dataset, as well as the label information corresponding to each log sample.

[0089] The weight acquisition unit is used to obtain the comparison weight corresponding to the candidate label information based on the difference distance between the candidate label information of each log sample and other predicted labels.

[0090] In an optional implementation, the weight acquisition unit is further configured to:

[0091] The difference distance between the candidate label information of the log sample and other predicted labels is obtained by the following formula:

[0092] S is the number of labels in the log sample.

[0093] Where L i,k It is a weak tag L k In log sample X i The label results, C k (X i ) is a weak classifier C k In log sample X i The predicted results of the label results.

[0094] In one alternative embodiment, the apparatus further includes:

[0095] The sample detection module is used to detect each log sample using several weak classifiers, obtain several weak labels corresponding to each log sample, and use these weak labels as label information.

[0096] In one optional implementation, the sample acquisition module 301 includes:

[0097] The first sample acquisition unit is used to acquire the text vector of the target log sample.

[0098] The second sample acquisition unit is used to obtain the text vector distance between each log sample in the labeled log dataset and the target log sample;

[0099] The third sample acquisition unit is used to filter out several log samples based on the text vector distance between each log sample and the target log sample, and to use several log samples and the target log sample as similar log samples.

[0100] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0101] In this embodiment, a security log analysis device based on multi-source label weak supervision learning is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit), a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0102] This invention also provides a computer device having the above-described features. Figure 3 The diagram shows a security log analysis device based on multi-source label weak supervision learning.

[0103] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 4As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 4 Take a processor 10 as an example.

[0104] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0105] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0106] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0107] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0108] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means.Figure 4 Taking the example of a connection between China and Israel via a bus.

[0109] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0110] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0111] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0112] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A security log analysis method based on multi-source label weakly supervised learning, characterized in that, The method includes: Obtain similar log samples from the labeled log dataset and combine the similar log samples into a subsample set; With the goal of ensuring that all log samples in the subsample set are of the same type, a target classifier updates the contrast weights corresponding to each label of the log samples in the subsample set, obtaining a weight matrix. The contrast weights indicate the difference distance between the candidate label information of the log sample and other predicted labels. This difference distance represents the similarity between different labels. The log sample has multiple labels. The difference distance between the candidate label information of the log sample and other predicted labels is obtained using the following formula: Where i=1,…,N, j=1,2,…,S, and S is the number of labels for the log samples; i represents the log sample number, and j represents the source number for the current analysis. It is a weak label In log samples The label results It is a weak classifier In log samples The prediction results of the label results; the smaller the difference distance, the better the quality of the source label corresponding to the log sample for which the difference distance is calculated; the difference distance is used as weight information to perform label fusion learning and obtain a weight value matrix; Based on the weight value matrix, target labels corresponding to the target sample logs are generated, and the log analysis model is trained based on the target sample logs and target labels. The trained log analysis model is used to analyze security logs. The weight value matrix represents the effective weight values ​​of different labels in the log analysis model. The effective weight values ​​represent the contribution and value of the current source label. The target label is generated based on the weight value matrix and is used as the high-value label of the current source label.

2. The method according to claim 1, characterized in that, Before updating the contrast weights corresponding to each label of the log samples in the subsample set using the target classifier, the method further includes: Obtain each log sample in the labeled log dataset, and the label information corresponding to each log sample; The comparison weights corresponding to the candidate label information are obtained based on the difference distance between the candidate label information and other predicted labels of each log sample.

3. The method according to claim 2, characterized in that, The method further includes: For each log sample, several weak classifiers are used for detection to obtain several weak labels corresponding to each log sample, and these weak labels are used as label information.

4. The method according to claim 1 or 2, characterized in that, Obtaining similar log samples from the labeled log dataset includes: For the target log sample, obtain the text vector of the target log sample; Obtain the text vector distance between each log sample in the labeled log dataset and the target log sample; Based on the text vector distance between each log sample and the target log sample, several log samples are selected, and these log samples and the target log sample are considered as similar log samples.

5. A security log analysis device based on multi-source label weakly supervised learning, characterized in that, The device includes: The sample acquisition module is used to acquire similar log samples from the labeled log dataset and combine the similar log samples into a subsample set; The weight determination module, aiming for all log samples in the subsample set to be of the same type, updates the comparison weights corresponding to each label of the log samples in the subsample set using a target classifier to obtain a weight value matrix. The comparison weights indicate the difference distance between the candidate label information of the log sample and other predicted labels, representing the similarity between different labels. The log sample has multiple labels. The difference distance between the candidate label information of the log sample and other predicted labels is obtained using the following formula: Where i=1,…,N, j=1,2,…,S, and S is the number of labels for the log samples; i represents the log sample number, and j represents the source number for the current analysis. It is a weak label In log samples The label results It is a weak classifier In log samples The prediction results of the label results; the smaller the difference distance, the better the quality of the source label corresponding to the log sample for which the difference distance is calculated; the difference distance is used as weight information to perform label fusion learning and obtain a weight value matrix; The model training module is used to generate target labels corresponding to target sample logs based on the weight value matrix, and to train the log analysis model based on the target sample logs and target labels. The trained log analysis model is used to analyze security logs. The weight value matrix represents the effective weight values ​​of different labels in the log analysis model. The effective weight values ​​represent the contribution and value of the current source label. The target label is generated based on the weight value matrix and is used as the high-value label of the current source label.

6. The apparatus according to claim 5, characterized in that, The weight determination module is also used for: Obtain each log sample in the labeled log dataset, and the label information corresponding to each log sample; The comparison weights corresponding to the candidate label information are obtained based on the difference distance between the candidate label information and other predicted labels of each log sample.

7. A computer device, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the security log analysis method based on multi-source label weak supervision learning as described in any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the security log analysis method based on multi-source label weak supervision learning as described in any one of claims 1 to 4.

9. A computer program product, characterized in that, Includes computer instructions for causing a computer to execute the security log analysis method based on multi-source label weakly supervised learning as described in any one of claims 1 to 4.

Images