Real-time Behavioral Safety Baseline Data Denoising Method and Device for Safety Analysis

Abstract

The present invention provides a real-time behavioral safety baseline data noise reduction method and device for safety analysis; wherein, the method includes: obtaining a behavior data set to be denoised; wherein, the behavior data set to be denoised includes at least one piece of behavior data; performing data noise reduction on each piece of behavior data in the behavior data set to be denoised according to a data noise reduction strategy to obtain a denoised behavior data set. The present invention can eliminate data noise in the data for behavioral safety baseline learning, improve the accuracy of the data for behavioral safety baseline learning, and reduce the false alarm and error rate of behavioral safety baseline calculation.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular, to a real-time behavior security baseline data noise reduction method and device for security analysis. Background Art

[0002] With the development of technology and the spread of knowledge, the methods and numbers of network attacks have also increased significantly. Various new attack means emerge in an endless stream, bringing great challenges and pressures to security analysts and products. Traditional security analysis and detection means are based on prior knowledge and use features to perform security detection on network data and logs. This method can deal with known attack methods, but the detection efficiency for unknown and new attack methods is low and cannot adapt to the current severe network security situation. In recent years, with the development of machine learning and the rise of real-time computing frameworks, behavior-based security analysis methods have begun to be increasingly applied to various security products.

[0003] In related technologies, behavior-based security analysis methods use machine learning methods to learn network data and logs, statistically summarize the behavior characteristics of users and entities, and can be well used for abnormal behavior analysis and detection through the learned behavior security baseline, achieving effects that many feature-based methods cannot achieve. However, there is data noise in the data for behavior security baseline learning, which makes the accuracy of the data for behavior security baseline learning low, resulting in a high false alarm and error rate in the calculation of the behavior security baseline. Summary of the Invention

[0004] The present invention provides a real-time behavior security baseline data noise reduction method and device for security analysis.

[0005] The present invention provides a real-time behavior security baseline data noise reduction method for security analysis, and the method includes:

[0006] Obtain a behavior data set to be denoised; wherein, the behavior data set to be denoised includes at least one piece of behavior data;

[0007] Perform data noise reduction on each piece of behavior data in the behavior data set to be denoised according to a data noise reduction strategy to obtain a denoised behavior data set.

[0008] According to the real-time behavior security baseline data noise reduction method for security analysis provided by the present invention, the data noise reduction strategy includes at least one of the following:

[0009] The first strategy is used to represent comparing the data to be denoised corresponding to the behavior data with the behavior data set in the current period;

[0010] The second strategy is used to represent comparing the data to be denoised corresponding to the behavior data with the behavior data set in the previous period;

[0011] The third strategy is used to represent comparing the data value to be denoised corresponding to the behavior data with the first expression.

[0012] According to a real-time behavior safety baseline data denoising method for security analysis provided by the present invention, data denoising is performed on each behavior data in the behavior data set to be denoised according to the data denoising strategy, and a denoised behavior data set is obtained, including:

[0013] When the data denoising strategy includes the first strategy, calculate the current cycle data denoising model according to the behavior data set to be denoised;

[0014] Traverse all behavior data in the behavior data set to be denoised. For each piece of behavior data, execute the second expression to calculate the data value to be denoised, and calculate the calculated data value to be denoised with the current cycle data denoising model. When the calculation result indicates that the current behavior data is not noise, save the current behavior data to the denoised behavior data set.

[0015] According to a real-time behavior safety baseline data denoising method for security analysis provided by the present invention, data denoising is performed on each behavior data in the behavior data set to be denoised according to the data denoising strategy, and a denoised behavior data set is obtained, including:

[0016] When the data denoising strategy includes the second strategy, calculate the current cycle data denoising model according to the behavior data set to be denoised;

[0017] Judge whether there is a previous cycle data denoising model;

[0018] When there is a previous cycle data denoising model, traverse all behavior data in the behavior data set to be denoised. For each piece of behavior data, execute the second expression to calculate the data value to be denoised, and calculate the calculated data value to be denoised with the previous cycle data denoising model. When the calculation result indicates that the current behavior data is not noise, save the current behavior data to the denoised behavior data set;

[0019] When there is no previous cycle data denoising model, traverse all behavior data in the behavior data set to be denoised. For each piece of behavior data, execute the second expression to calculate the data value to be denoised, and calculate the calculated data value to be denoised with the current cycle data denoising model. When the calculation result indicates that the current behavior data is not noise, save the current behavior data to the denoised behavior data set.

[0020] A real-time behavioral safety baseline data denoising method for security analysis provided by the present invention, which denoises each behavioral data in the to-be-denoised behavioral dataset according to a data denoising strategy to obtain a denoised behavioral dataset, includes:

[0021] Traverse all behavioral data in the to-be-denoised behavioral dataset, execute a second expression for each piece of behavioral data to calculate the to-be-denoised data value, input the calculated to-be-denoised data value into the first expression for Boolean calculation, and when the calculation result indicates that the current behavioral data is not noise, save the current behavioral data to the denoised behavioral dataset.

[0022] A real-time behavioral safety baseline data denoising method for security analysis provided by the present invention, which calculates the current cycle data denoising model according to the to-be-denoised behavioral dataset, includes:

[0023] When the behavioral data in the to-be-denoised behavioral dataset is numerical data, traverse all behavioral data in the to-be-denoised behavioral dataset, execute a second expression for each piece of behavioral data to calculate the to-be-denoised data value, and save the calculated to-be-denoised data value to the denoised behavioral data value set; calculate the average value and standard deviation of the denoised behavioral data value set; determine the distance value according to the data noise value probability; determine the minimum non-noise data value and the maximum non-noise data value according to the average value, the standard deviation and the distance value.

[0024] A real-time behavioral safety baseline data denoising method for security analysis provided by the present invention, which calculates the current cycle data denoising model according to the to-be-denoised behavioral dataset, includes:

[0025] When the behavioral data in the to-be-denoised behavioral dataset is non-numerical data, traverse all behavioral data in the to-be-denoised behavioral dataset, execute a second expression for each piece of behavioral data to calculate the to-be-denoised data value, and save the calculated to-be-denoised data value to the denoised behavioral data value set;

[0026] Traverse the denoised behavioral data value set, and query whether the to-be-denoised data value exists in the value frequency table frequencyMap for each to-be-denoised data value; if it does not exist, save a record <to-be-denoised data value, 1> to frequencyMap; if it exists, update <to-be-denoised data value, frequency value> in frequencyMap to <to-be-denoised data value, frequency value + 1>;

[0027] Traverse the frequencyMap. For each <data value to be denoised, frequency value> record in the frequencyMap, query whether the <frequency value> exists in the temporary table tmpMap; if it does not exist, save <frequency value, quantity = frequency value> to tmpMap; if it exists, update the <frequency value, quantity> in tmpMap to <frequency value, quantity + frequency value>.

[0028] Determine the data value threshold to be denoised based on the number of data values to be denoised and the noise value probability in the denoising behavior data value set; obtain the initial value of the variable dataCount, and sort tmpMap in descending order of the frequency value in the <frequency value, quantity> record; traverse the tmpMap table, and for each <frequency value, quantity> record in the table, determine whether dataCount is greater than or equal to the data value threshold to be denoised; if so, set the minimum frequency of non-noise data minFrequency = the frequency value in the <frequency value, quantity> record; if not, update dataCount = dataCount + the quantity value in the <frequency value, quantity> record, and determine whether the updated dataCount is greater than or equal to the data value threshold to be denoised.

[0029] The present invention also provides a real-time behavioral safety baseline data denoising device for security analysis, and the device includes:

[0030] An acquisition module, configured to acquire a data set of behaviors to be denoised; wherein, at least one behavior data is included in the data set of behaviors to be denoised.

[0031] A data denoising module, configured to perform data denoising on each behavior data in the data set of behaviors to be denoised according to a data denoising strategy, so as to obtain a denoised behavior data set.

[0032] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the real-time behavioral safety baseline data denoising method for security analysis as described in any one of the above are implemented.

[0033] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the real-time behavioral safety baseline data denoising method for security analysis as described in any one of the above are implemented.

[0034] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the real-time behavioral safety baseline data denoising method for security analysis as described in any one of the above are implemented.

[0035] The real-time behavioral safety baseline data noise reduction method for security analysis provided by the present invention obtains a dataset of behaviors to be denoised, and performs data noise reduction on each behavior data in the dataset of behaviors to be denoised according to a data noise reduction strategy, so as to obtain a dataset of denoised behaviors, realizing the elimination of data noise in the data for learning the behavioral safety baseline, being able to improve the accuracy of the data for learning the behavioral safety baseline, and reducing the false alarm and error rates in the calculation of the behavioral safety baseline. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0037] Figure 1 is one of the schematic flowcharts of the real-time behavioral safety baseline data noise reduction method for security analysis provided by the present invention;

[0038] Figure 2 is the second schematic flowchart of the real-time behavioral safety baseline data noise reduction method for security analysis provided by the present invention;

[0039] Figure 3 is the schematic flowchart of the calculation process of the data noise reduction calculator provided by the present invention;

[0040] Figure 4 is the schematic flowchart of the calculation process of the data noise reduction model calculator provided by the present invention;

[0041] Figure 5 is the schematic flowchart of the construction process of the numerical data noise reduction model provided by the present invention;

[0042] Figure 6 is the schematic flowchart of the noise reduction calculation process of the numerical data noise reduction model provided by the present invention;

[0043] Figure 7 is the schematic flowchart of the construction process of the non-numerical data noise reduction model provided by the present invention;

[0044] Figure 8 is the schematic flowchart of the noise reduction calculation process of the non-numerical data noise reduction model provided by the present invention;

[0045] Figure 9 is the schematic structural diagram of the real-time behavioral safety baseline data noise reduction device for security analysis provided by the present invention;

[0046] Figure 10 is the schematic structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Apparently, the described embodiments are some, rather than all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0048] The real-time behavior security baseline data noise reduction method for security analysis provided by the present invention will be described in detail below in conjunction with the accompanying drawings, through some embodiments and their application scenarios.

[0049] The present invention provides a real-time behavior security baseline data noise reduction method for security analysis. This real-time behavior security baseline data noise reduction method for security analysis can be applied to the scenarios of security analysis and detection for coping with network attacks. By obtaining a behavior data set to be noise-reduced, data noise reduction is performed on each behavior data in the behavior data set to be noise-reduced according to a data noise reduction strategy, so as to obtain a noise-reduced behavior data set, realizing the elimination of data noise in the data for behavior security baseline learning, being able to improve the accuracy of the data for behavior security baseline learning, and reducing the false alarm and error rate of behavior security baseline calculation.

[0050] Figure 1 is a schematic flowchart of the real-time behavior security baseline data noise reduction method for security analysis provided by the present invention. As Figure 1 shown, this method includes step 101-step 102, wherein:

[0051] Step 101, obtaining a behavior data set to be noise-reduced; wherein, the behavior data set to be noise-reduced includes at least one piece of behavior data;

[0052] Step 102, performing data noise reduction on each behavior data in the behavior data set to be noise-reduced according to a data noise reduction strategy to obtain a noise-reduced behavior data set.

[0053] It should be noted that the real-time behavior security baseline data noise reduction method for security analysis provided by the present invention can be applied to the scenarios of security analysis and detection for coping with network attacks. The execution subject of this method can be a real-time behavior security baseline data noise reduction device for security analysis, such as an electronic device, or a control module in the real-time behavior security baseline data noise reduction device for security analysis for executing the real-time behavior security baseline data noise reduction method for security analysis. The electronic device can include a mobile phone, a tablet computer, or a desktop computer.

[0054] The real-time behavioral safety baseline data denoising method for security analysis provided by the present invention obtains a dataset of behaviors to be denoised, and performs data denoising on each behavior data in the dataset of behaviors to be denoised according to a data denoising strategy, so as to obtain a dataset of denoised behaviors, realizing the elimination of data noise in the data for behavioral safety baseline learning, and capable of improving the accuracy of the data for behavioral safety baseline learning and reducing the false alarm and error rates in the calculation of the behavioral safety baseline.

[0055] Optionally, the present invention provides behavioral safety baseline data denoising parameters; specifically, the behavioral safety baseline data denoising parameters include at least one of the following:

[0056] 1) Data to be denoised (data)

[0057] Define the acquisition method of the data to be denoised, and calculate the data to be denoised from the input behavioral data through a second expression (i.e., the calculation expression of parameter data). The following is an example: data = fields["Number of user accesses to the website per minute"].

[0058] 2) Denoised data type (type)

[0059] Define the data type of the data to be denoised. type = "number" indicates that the data to be denoised is of numerical type; type = "other" indicates that the data to be denoised is of non-numerical type.

[0060] 3) Data denoising strategy (strategy)

[0061] In practice, the data denoising strategy may include at least one of the following:

[0062] (a) The first strategy, which is used to represent comparing the data to be denoised corresponding to the behavioral data with the dataset of behaviors in the current period.

[0063] Specifically, the first strategy can be denoted as strategy = "current", indicating that based on the first strategy, the data to be denoised is compared with the current period data denoising model corresponding to the dataset of behaviors in the current period to determine whether the behavioral data is noise. In practice, the behavioral data may include network data and / or logs, etc.

[0064] (b) The second strategy, which is used to represent comparing the data to be denoised corresponding to the behavioral data with the dataset of behaviors in the previous period.

[0065] Specifically, the second strategy can be denoted as strategy = "prev", indicating that based on the second strategy, the data to be denoised is compared with the previous period data denoising model corresponding to the dataset of behaviors in the previous period to determine whether the behavioral data is noise.

[0066] (c) The third strategy is used to represent the comparison of the data value to be denoised corresponding to the behavior data with the first expression.

[0067] Specifically, the third strategy can be denoted as strategy = "expression", indicating the use of the first expression (i.e., the parameter expression boolean calculation expression), that is, the specified denoising calculation method for data denoising. At this time, the parameter of the specified data denoising method (expression) needs to be specified.

[0068] 4) Specified data denoising method (expression)

[0069] When strategy = "expression", it is necessary to specify the data denoising calculation method. Example of use: expression = fields["Number of user accesses per minute on the website"] > 100, indicating that when the field value fields["Number of user accesses per minute on the website"] > 100 in the input behavior data to be denoised, it is determined that the input behavior data to be denoised is noise data.

[0070] 5) Data noise probability (probability)

[0071] Define the data noise value probability, which represents the probability of noise occurrence in the dataset. The larger the value, the higher the noise probability, and the smaller the value, the lower the noise probability. 0 < data noise probability <= 1. This parameter needs to be specified when strategy = "current / prev".

[0072] Example of use: denoise(data = fields["Number of user accesses per minute on the website"], type = "number", strategy = "current"), indicating that data denoising is performed on the input behavior data to be denoised. The denoised data is the field value "Number of user accesses per minute on the website", the type of data to be denoised is a numerical value, and the data denoising strategy is to compare with the current cycle dataset.

[0073] Next, based on the behavior security baseline data denoising parameters, the implementation method of data denoising for each behavior data in the to-be-denoised behavior dataset provided by the present invention will be described.

[0074] The implementation method of data denoising for each behavior data in the to-be-denoised behavior dataset according to the data denoising strategy to obtain the denoised behavior dataset may include at least one of the following:

[0075] Method 1: When the data noise reduction strategy includes the first strategy, calculate the current cycle data noise reduction model according to the behavior dataset to be noise-reduced; traverse all behavior data in the behavior dataset to be noise-reduced, and for each piece of behavior data, execute the second expression to calculate the data value to be noise-reduced, and calculate the calculated data value to be noise-reduced with the current cycle data noise reduction model. When the calculation result indicates that the current behavior data is not noise, save the current behavior data to the noise-reduced behavior dataset. Optionally, save the current cycle data noise reduction model.

[0076] Method 2: When the data noise reduction strategy includes the second strategy, calculate the current cycle data noise reduction model according to the behavior dataset to be noise-reduced; determine whether there is a previous cycle data noise reduction model:

[0077] When there is a previous cycle data noise reduction model, traverse all behavior data in the behavior dataset to be noise-reduced, and for each piece of behavior data, execute the second expression to calculate the data value to be noise-reduced, and calculate the calculated data value to be noise-reduced with the previous cycle data noise reduction model. When the calculation result indicates that the current behavior data is not noise, save the current behavior data to the noise-reduced behavior dataset;

[0078] Or, when there is no previous cycle data noise reduction model, traverse all behavior data in the behavior dataset to be noise-reduced, and for each piece of behavior data, execute the second expression to calculate the data value to be noise-reduced, and calculate the calculated data value to be noise-reduced with the current cycle data noise reduction model. When the calculation result indicates that the current behavior data is not noise, save the current behavior data to the noise-reduced behavior dataset. Optionally, save the current cycle data noise reduction model.

[0079] Method 3: When the data noise reduction strategy includes the third strategy, traverse all behavior data in the behavior dataset to be noise-reduced, and for each piece of behavior data, execute the second expression to calculate the data value to be noise-reduced, and input the calculated data value to be noise-reduced into the first expression for Boolean calculation. When the calculation result indicates that the current behavior data is not noise, save the current behavior data to the noise-reduced behavior dataset.

[0080] The implementation methods of calculating the current cycle data noise reduction model according to the behavior dataset to be noise-reduced in Method 1 and Method 2 of the present invention may include at least one of the following:

[0081] 1) When the behavioral data in the behavioral dataset to be noise-reduced is numerical data, traverse all the behavioral data in the behavioral dataset to be noise-reduced. For each piece of behavioral data, execute the second expression to calculate the data value to be noise-reduced, and save the calculated data value to be noise-reduced into the noise-reduced behavioral data value set; calculate the average value and standard deviation of the noise-reduced behavioral data value set; determine the distance value according to the data noise value probability; determine the minimum non-noise data value and the maximum non-noise data value according to the average value, the standard deviation and the distance value.

[0082] 2) When the behavioral data in the behavioral dataset to be noise-reduced is non-numerical data, traverse all the behavioral data in the behavioral dataset to be noise-reduced. For each piece of behavioral data, execute the second expression to calculate the data value to be noise-reduced, and save the calculated data value to be noise-reduced into the noise-reduced behavioral data value set;

[0083] Traverse the noise-reduced behavioral data value set. For each data value to be noise-reduced, query whether the data value to be noise-reduced exists in the value frequency table (frequencyMap); if it does not exist, save a record <data value to be noise-reduced, 1> into the frequencyMap; if it exists, update <data value to be noise-reduced, frequency value> in the frequencyMap to <data value to be noise-reduced, frequency value + 1>; traverse the frequencyMap. For each <data value to be noise-reduced, frequency value> record in the frequencyMap, query whether <frequency value> exists in the temporary table (tmpMap); if it does not exist, save <frequency value, quantity = frequency value> into the tmpMap; if it exists, update <frequency value, quantity> in the tmpMap to <frequency value, quantity + frequency value>.

[0084] Determine the threshold of the data value to be noise-reduced (nonNoiseDataThreshold) according to the number of data values to be noise-reduced in the noise-reduced behavioral data value set and the noise value probability; obtain the initial value of the variable dataCount, and sort the tmpMap in descending order of the frequency value in the record <frequency value, quantity>; traverse the tmpMap table. For each <frequency value, quantity> record in the table, judge whether the dataCount is greater than or equal to the threshold of the data value to be noise-reduced; if so, set the minimum non-noise data frequency minFrequency = the frequency value in the record <frequency value, quantity>; if not, update the dataCount = dataCount + the quantity value in the record <frequency value, quantity>, and judge whether the updated dataCount is greater than or equal to the threshold of the data value to be noise-reduced.

[0085] The data noise reduction process in the present invention is shown in Figure 2 , Figure 2It is a schematic diagram of the data noise reduction process provided by the present invention. The data noise reduction process includes steps 201 - 203, where:

[0086] Step 201, input <dataset of behaviors to be noise-reduced>;

[0087] Step 202, use a data noise reduction calculator to perform data noise reduction on the input <dataset of behaviors to be noise-reduced> according to the configured parameters;

[0088] Step 203, output the calculation result <dataset of noise-reduced behaviors> to the lower-level calculator.

[0089] Here, the calculation method of the data noise reduction calculator will be described in conjunction with the accompanying drawings.

[0090] The variables involved in the data noise reduction calculator are: the data noise reduction model of the previous cycle (prevModel); the definition of the variable prevModel is: save the data noise reduction model of the previous cycle. The variable prevModel is used when strategy = "prev".

[0091] Figure 3 It is a schematic diagram of the calculation process of the data noise reduction calculator provided by the present invention. As Figure 3 shown, it includes steps 301 - 311, where:

[0092] Step 301, input <dataset of behaviors to be noise-reduced>;

[0093] Step 302, determine whether the parameter strategy == "expression"? If so, execute step 303; if not, execute step 304;

[0094] Step 303, traverse all behavior data in the <dataset of behaviors to be noise-reduced>, calculate the value to be noise-reduced for each piece of behavior data using the parameter data calculation expression, then input the calculated value to be noise-reduced into the parameter expression boolean calculation expression. If the calculation result of the expression parameter boolean calculation expression is true, add the current behavior data to the <dataset of noise-reduced behaviors>. After the traversal is completed, execute step 311;

[0095] Step 304, the data noise reduction model calculator calculates the data noise reduction model according to the <dataset of behaviors to be noise-reduced>;

[0096] Step 305, determine whether the parameter strategy == "prev". If so, execute step 306; if not, execute step 309;

[0097] Step 306, determine whether the variable prevModel has a value. If so, execute step 307; if not, execute step 308;

[0098] Step 307: Set the current data noise reduction model to the value saved in variable prevModel, and save the calculated data noise reduction model to variable prevModel, then execute Step 310;

[0099] Step 308: Set the current data noise reduction model to the calculated data noise reduction model, and save the calculated data noise reduction model to variable prevModel, then execute Step 310;

[0100] Step 309: Set the current data noise reduction model to the calculated data noise reduction model, then execute Step 310;

[0101] Step 310: Traverse all the behavior data in the <dataset of behaviors to be noise-reduced>. For each piece of behavior data, execute the parameter data calculation expression to calculate the value to be noise-reduced, and then calculate the calculated value to be noise-reduced with the current data noise reduction model. If it is not noise, add the current behavior data to the <dataset of noise-reduced behaviors>, then execute Step 311;

[0102] Step 311: Output the <dataset of noise-reduced behaviors>.

[0103] Here, for the calculation process of the data noise reduction model calculator in Step 304, refer to Figure 4 , Figure 4 which is the schematic diagram of the calculation process of the data noise reduction model calculator provided by the present invention, including Step 401 - Step 405, where:

[0104] Step 401: Input the <dataset of behaviors to be noise-reduced>;

[0105] Step 402: Determine whether the parameter type == "number"? If so, execute Step 403; if not, execute Step 404;

[0106] Step 403: Use the numerical data noise reduction model calculator to construct the data noise reduction model;

[0107] Step 404: Use the non-numerical data noise reduction model calculator to construct the data noise reduction model;

[0108] Step 405: Output the data noise reduction model.

[0109] The calculation methods of the numerical data noise reduction model and the non-numerical data noise reduction model will be described below respectively.

[0110] I. Calculation Method of Numerical Data Noise Reduction Model

[0111] The variables involved in the numerical data noise reduction model include:

[0112] a) The minimum value (min) of non-noise data. If the data value to be denoised < min, then the data value to be denoised is noise;

[0113] b) The maximum value (max) of non-noise data. If the data value to be denoised > max, then the data value to be denoised is noise.

[0114] Figure 5 is a schematic diagram of the construction process of the numerical data denoising model provided by the present invention. As Figure 5 shown, it includes step 501 - step 506, where:

[0115] Step 501, input <the dataset of behaviors to be denoised>;

[0116] Step 502, traverse all the behavior data in <the dataset of behaviors to be denoised>. For each piece of behavior data, calculate the value of the parameter data calculation expression, and add the value to <the set of denoised behavior data values>;

[0117] Step 503, calculate the average value of <the set of denoised behavior data values>;

[0118] Step 504, calculate the standard deviation of <the set of denoised behavior data values>;

[0119] Step 505, query the distance value corresponding to the probability value in the normal distribution probability table according to the value <1 - parameter probability>;

[0120] Step 506, set the variable min = average value - standard deviation * distance value; set the variable max = average value + standard deviation * distance value.

[0121] On this basis, the denoising calculation process based on the numerical data denoising model is as follows:

[0122] Figure 6 is a schematic diagram of the denoising calculation process of the numerical data denoising model provided by the present invention. As Figure 6 shown, it includes step 601 - step 603, where:

[0123] Step 601, input <the dataset of behaviors to be denoised>;

[0124] Step 602, traverse all the behavior data in <the dataset of behaviors to be denoised>. For each piece of behavior data, execute the parameter data calculation expression to calculate the data value to be denoised. If min <= the calculated data value to be denoised <= max, then add the current behavior data to be denoised to <the dataset of denoised behaviors>;

[0125] Step 603, output <the dataset of denoised behaviors>.

[0126] II. Non-numerical data denoising model calculation method

[0127] The variables involved in the non-numerical data noise reduction model include:

[0128] a) The minimum frequency (minFrequency) of non-noise data. If the data value to be noise-reduced < minFrequency, then the data value to be noise-reduced is noise.

[0129] b) The value frequency table (frequencyMap), which is statistically obtained from the data values to be noise-reduced. The frequency corresponding to the <data value to be noise-reduced> can be obtained through the expression frequencyMap[<data value to be noise-reduced>]. If frequencyMap[<data value to be noise-reduced>] < minFrequency, it means that the data value to be noise-reduced is noise.

[0130] Figure 7 It is a schematic diagram of the non-numerical data noise reduction model construction process provided by the present invention. As Figure 7 shown, it includes steps 701 - step 710, where:

[0131] Step 701, input the <dataset of behaviors to be noise-reduced>;

[0132] Step 702, traverse all the behavior data in the <dataset of behaviors to be noise-reduced>, calculate the value of the parameter data calculation expression for each piece of behavior data, and add the value to the <set of noise-reduced behavior data values>;

[0133] Step 703, traverse the <set of noise-reduced behavior data values>, and for each piece of noise-reduced data value, query in the frequency table frequencyMap whether the <noise-reduced data value> exists. If it does not exist, then save <noise-reduced data value, 1> to the table. If it exists, then update the <noise-reduced data value, frequency value> in frequencyMap to <noise-reduced data value, frequency value + 1>;

[0134] Step 704, create a new temporary table tmpMap, traverse the frequencyMap table, and for each <noise-reduced data value, frequency value> record in the table, query in the temporary table tmpMap whether the <frequency value> exists. If it does not exist, then save <frequency value, quantity = frequency value> to the table. If it exists, then update the <frequency value, quantity> in tmpMap to <frequency value, quantity + frequency value>;

[0135] Step 705, calculate nonNoiseDataThreshold = the number of data in the dataset of behaviors to be noise-reduced * (1 - parameter probability);

[0136] Step 706, set the variable dataCount = 0, and sort the tmpMap table in descending order according to the frequency value in the record <frequency value, quantity>;

[0137] Step 707, traverse the tmpMap table, and for each <frequency value, quantity> record in the table, execute Step 708;

[0138] Step 708, for each record, determine whether dataCount >= nonNoiseDataThreshold? If yes, execute Step 710, if not, execute Step 709;

[0139] Step 709, update dataCount = dataCount + the quantity value in the record <frequency value, quantity>, and continue to execute Step 708;

[0140] Step 710, set minFrequency = the frequency value in the record <frequency value, quantity>, and then end the calculation.

[0141] On this basis, the noise reduction calculation process based on the non-numerical data noise reduction model is as follows:

[0142] Figure 8 It is a schematic diagram of the noise reduction calculation process of the non-numerical data noise reduction model provided by the present invention. As Figure 8 shown, it includes Step 801 - Step 806, where:

[0143] Step 801, input <dataset of behaviors to be noise-reduced>;

[0144] Step 802, traverse all the behavior data in the <dataset of behaviors to be noise-reduced>, and for each piece of behavior data, execute the parameter data calculation expression to calculate the data value to be noise-reduced;

[0145] Step 803, for each data value to be noise-reduced, query whether there is a record <calculated data value to be noise-reduced, frequency value> in the frequencyMap. If there is, execute Step 804, if not, execute Step 805;

[0146] Step 804, if the current frequency value > minFrequency, add the current behavior data to the <dataset of noise-reduced behaviors>. If the traversal is completed, execute Step 806, if not, execute Step 803;

[0147] Step 805, if 0 > minFrequency, add the current behavior data to the <dataset of noise-reduced behaviors>; if the traversal is completed, execute Step 806, if not, execute Step 803;

[0148] Step 806, output the <dataset of noise-reduced behaviors>.

[0149] The real-time behavioral safety baseline data noise reduction device for security analysis provided by the present invention will be described below. The real-time behavioral safety baseline data noise reduction device for security analysis described below can be correspondingly referred to the real-time behavioral safety baseline data noise reduction method for security analysis described above.

[0150] Figure 9 FIG. is a schematic structural diagram of the real-time behavioral safety baseline data noise reduction device for security analysis provided by the present invention. As Figure 9 shown, the real-time behavioral safety baseline data noise reduction device 900 for security analysis includes: an acquisition module 901 and a data noise reduction module 902; wherein,

[0151] The acquisition module 901 is configured to acquire a behavior data set to be noise-reduced; wherein, at least one behavior data is included in the behavior data set to be noise-reduced;

[0152] The data noise reduction module 902 is configured to perform data noise reduction on each behavior data in the behavior data set to be noise-reduced according to a data noise reduction strategy, so as to obtain a noise-reduced behavior data set.

[0153] The real-time behavioral safety baseline data noise reduction device for security analysis provided by the present invention realizes data noise elimination for the data of behavior safety baseline learning by acquiring a behavior data set to be noise-reduced and performing data noise reduction on each behavior data in the behavior data set to be noise-reduced according to a data noise reduction strategy, can improve the accuracy of the data for behavior safety baseline learning, and reduce the false alarm and error rate of behavior safety baseline calculation.

[0154] Optionally, the data noise reduction strategy includes at least one of the following:

[0155] The first strategy is used to represent comparing the data to be noise-reduced corresponding to the behavior data with the current cycle behavior data set;

[0156] The second strategy is used to represent comparing the data to be noise-reduced corresponding to the behavior data with the previous cycle behavior data set;

[0157] The third strategy is used to represent comparing the value of the data to be noise-reduced corresponding to the behavior data with a first expression.

[0158] Optionally, the data noise reduction module 902 is specifically configured to:

[0159] When the data noise reduction strategy includes the first strategy, calculate a current cycle data noise reduction model according to the behavior data set to be noise-reduced;

[0160] Traverse all the behavior data in the behavior data set to be denoised. For each piece of behavior data, execute the second expression to calculate the data value to be denoised, and calculate the calculated data value to be denoised with the current cycle data denoising model. When the calculation result indicates that the current behavior data is not noise, save the current behavior data to the denoised behavior data set.

[0161] Optionally, the data denoising module 902 is specifically configured to:

[0162] In the case where the data denoising strategy includes the second strategy, calculate the current cycle data denoising model according to the behavior data set to be denoised;

[0163] Determine whether there is a previous cycle data denoising model;

[0164] When there is a previous cycle data denoising model, traverse all the behavior data in the behavior data set to be denoised. For each piece of behavior data, execute the second expression to calculate the data value to be denoised, and calculate the calculated data value to be denoised with the previous cycle data denoising model. When the calculation result indicates that the current behavior data is not noise, save the current behavior data to the denoised behavior data set;

[0165] When there is no previous cycle data denoising model, traverse all the behavior data in the behavior data set to be denoised. For each piece of behavior data, execute the second expression to calculate the data value to be denoised, and calculate the calculated data value to be denoised with the current cycle data denoising model. When the calculation result indicates that the current behavior data is not noise, save the current behavior data to the denoised behavior data set.

[0166] Optionally, the data denoising module 902 is specifically configured to:

[0167] In the case where the data denoising strategy includes the third strategy, traverse all the behavior data in the behavior data set to be denoised. For each piece of behavior data, execute the second expression to calculate the data value to be denoised, and input the calculated data value to be denoised into the first expression for Boolean calculation. When the calculation result indicates that the current behavior data is not noise, save the current behavior data to the denoised behavior data set.

[0168] Optionally, the data noise reduction module 902 is specifically configured to: when the behavior data in the behavior dataset to be noise-reduced is numerical data, traverse all the behavior data in the behavior dataset to be noise-reduced, perform a second expression calculation on each piece of behavior data to obtain the data value to be noise-reduced, and save the calculated data value to be noise-reduced to the noise-reduced behavior data value set; calculate the average value and standard deviation of the noise-reduced behavior data value set; determine a distance value according to the data noise value probability; and determine the minimum non-noise data value and the maximum non-noise data value according to the average value, the standard deviation, and the distance value.

[0169] Optionally, the data noise reduction module 902 is specifically configured to:

[0170] When the behavior data in the behavior dataset to be noise-reduced is non-numerical data, traverse all the behavior data in the behavior dataset to be noise-reduced, perform a second expression calculation on each piece of behavior data to obtain the data value to be noise-reduced, and save the calculated data value to be noise-reduced to the noise-reduced behavior data value set;

[0171] Traverse the noise-reduced behavior data value set, and for each data value to be noise-reduced, query whether the data value to be noise-reduced exists in the value frequency table frequencyMap; if it does not exist, save a record <data value to be noise-reduced, 1> to frequencyMap; if it exists, update <data value to be noise-reduced, frequency value> in frequencyMap to <data value to be noise-reduced, frequency value + 1>;

[0172] Traverse frequencyMap, and for each <data value to be noise-reduced, frequency value> record in frequencyMap, query whether <frequency value> exists in the temporary table tmpMap; if it does not exist, save <frequency value, quantity = frequency value> to tmpMap; if it exists, update <frequency value, quantity> in tmpMap to <frequency value, quantity + frequency value>;

[0173] Determine the threshold of the data value to be noise-reduced according to the number of data values to be noise-reduced in the noise-reduced behavior data value set and the noise value probability; obtain the initial value of the variable dataCount, and sort tmpMap in descending order of the frequency value in the record <frequency value, quantity>; traverse the tmpMap table, and for each <frequency value, quantity> record in the table, determine whether dataCount is greater than or equal to the threshold of the data value to be noise-reduced; if so, set the minimum non-noise data frequency minFrequency = the frequency value in the record <frequency value, quantity>; if not, update dataCount = dataCount + the quantity value in the record <frequency value, quantity>, and determine whether the updated dataCount is greater than or equal to the threshold of the data value to be noise-reduced.

[0174] Figure 10 is a schematic structural diagram of the electronic device provided by the present invention. As Figure 10 shown, the electronic device 1000 includes: a processor 1010, a communications interface 1020, a memory 1030, and a communication bus 1040; wherein, the processor 1010, the communications interface 1020, and the memory 1030 complete mutual communication through the communication bus 1040. The processor 1010 can call logical instructions in the memory 1030 to execute a real-time behavior security baseline data noise reduction method for security analysis. The method includes: obtaining a behavior data set to be noise-reduced; wherein, the behavior data set to be noise-reduced includes at least one piece of behavior data; performing data noise reduction on each piece of behavior data in the behavior data set to be noise-reduced according to a data noise reduction policy to obtain a noise-reduced behavior data set.

[0175] In addition, when the logical instructions in the above-mentioned memory 1030 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present invention. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc that can store program codes.

[0176] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the real-time behavior security baseline data noise reduction method for security analysis provided by the above-mentioned various methods. The method includes: obtaining a behavior data set to be noise-reduced; wherein, the behavior data set to be noise-reduced includes at least one piece of behavior data; performing data noise reduction on each piece of behavior data in the behavior data set to be noise-reduced according to a data noise reduction policy to obtain a noise-reduced behavior data set.

[0177] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a real-time behavioral security baseline data noise reduction method for security analysis provided by the above-mentioned various methods. The method includes: obtaining a behavior data set to be denoised; wherein, the behavior data set to be denoised includes at least one piece of behavior data; performing data noise reduction on each piece of behavior data in the behavior data set to be denoised according to a data noise reduction strategy to obtain a denoised behavior data set.

[0178] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0179] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time behavioral safety baseline data denoising method for safety analysis, characterized in that, The method includes: Obtaining a dataset of behavior data to be denoised; wherein, the dataset of behavior data to be denoised includes at least one piece of behavior data; Performing data denoising on each piece of behavior data in the dataset of behavior data to be denoised according to a data denoising strategy to obtain a dataset of denoised behavior data; The data denoising strategy is used to represent comparing the data to be denoised corresponding to the behavior data with the dataset of behavior data in the previous cycle; The performing data denoising on each piece of behavior data in the dataset of behavior data to be denoised according to the data denoising strategy to obtain a dataset of denoised behavior data includes: Calculating a data denoising model for the current cycle according to the dataset of behavior data to be denoised; Determining whether there is a data denoising model for the previous cycle; When there is a data denoising model for the previous cycle, traversing all the behavior data in the dataset of behavior data to be denoised, performing a second expression calculation on the data to be denoised value for each piece of behavior data, calculating the calculated data to be denoised value with the data denoising model for the previous cycle, and when the calculation result indicates that the current behavior data is not noise, saving the current behavior data to the dataset of denoised behavior data; When there is no data denoising model for the previous cycle, traversing all the behavior data in the dataset of behavior data to be denoised, performing a second expression calculation on the data to be denoised value for each piece of behavior data, calculating the calculated data to be denoised value with the data denoising model for the current cycle, and when the calculation result indicates that the current behavior data is not noise, saving the current behavior data to the dataset of denoised behavior data.

2. The method according to claim 1, wherein The calculating a data denoising model for the current cycle according to the dataset of behavior data to be denoised includes: In the case where the behavior data in the dataset of behavior data to be denoised is numerical data, traversing all the behavior data in the dataset of behavior data to be denoised, performing a second expression calculation on the data to be denoised value for each piece of behavior data, and saving the calculated data to be denoised value to a set of denoised behavior data values; calculating the average value and standard deviation of the set of denoised behavior data values; determining a distance value according to the probability of data noise value; determining the minimum value of non-noise data and the maximum value of non-noise data according to the average value, the standard deviation, and the distance value.

3. The method according to claim 1, characterized in that, The calculating a data denoising model for the current cycle according to the dataset of behavior data to be denoised includes: In the case where the behavior data in the dataset of behavior data to be denoised is non-numerical data, traversing all the behavior data in the dataset of behavior data to be denoised, performing a second expression calculation on the data to be denoised value for each piece of behavior data, and saving the calculated data to be denoised value to a set of denoised behavior data values; Traversing the set of denoised behavior data values, and querying whether the data to be denoised value exists in the value frequency table frequencyMap for each piece of data to be denoised value; if it does not exist, saving a record <data to be denoised value, 1> to frequencyMap; if it exists, updating <data to be denoised value, frequency value> in frequencyMap to <data to be denoised value, frequency value + 1>; Traverse the frequencyMap. For each record of <data value to be noise-reduced, frequency value> in the frequencyMap, query whether the <frequency value> exists in the temporary table tmpMap; if it does not exist, save <frequency value, quantity = frequency value> to tmpMap; if it exists, update <frequency value, quantity> in tmpMap to <frequency value, quantity + frequency value>. Determine the threshold of the data value to be noise-reduced according to the number of data values to be noise-reduced and the noise value probability in the noise-reduction behavior data value set; obtain the initial value of the variable dataCount, and sort tmpMap in descending order according to the frequency value in the record <frequency value, quantity>; traverse the tmpMap table, and for each record of <frequency value, quantity> in the table, judge whether dataCount is greater than or equal to the threshold of the data value to be noise-reduced; if so, set the minimum frequency minFrequency of non-noise data to the frequency value in the record <frequency value, quantity>; if not, update dataCount = dataCount + the quantity value in the record <frequency value, quantity>, and judge whether the updated dataCount is greater than or equal to the threshold of the data value to be noise-reduced.

4. A real-time behavioral safety baseline data noise reduction device for safety analysis, characterized in that, The device includes: An acquisition module, configured to acquire a data set of behaviors to be noise-reduced; wherein, the data set of behaviors to be noise-reduced includes at least one piece of behavior data. A data noise-reduction module, configured to perform data noise-reduction on each piece of behavior data in the data set of behaviors to be noise-reduced according to a data noise-reduction strategy, so as to obtain a data set of noise-reduced behaviors. The data noise-reduction strategy is used to represent comparing the data to be noise-reduced corresponding to the behavior data with the data set of behaviors in the previous cycle. The data noise-reduction module is specifically configured to: Calculate a data noise-reduction model for the current cycle according to the data set of behaviors to be noise-reduced. Judge whether there is a data noise-reduction model in the previous cycle. When there is a data noise-reduction model in the previous cycle, traverse all the behavior data in the data set of behaviors to be noise-reduced, execute the second expression for each piece of behavior data to calculate the data value to be noise-reduced, calculate the calculated data value to be noise-reduced with the data noise-reduction model in the previous cycle, and when the calculation result indicates that the current behavior data is not noise, save the current behavior data to the data set of noise-reduced behaviors. When there is no data noise-reduction model in the previous cycle, traverse all the behavior data in the data set of behaviors to be noise-reduced, execute the second expression for each piece of behavior data to calculate the data value to be noise-reduced, calculate the calculated data value to be noise-reduced with the data noise-reduction model in the current cycle, and when the calculation result indicates that the current behavior data is not noise, save the current behavior data to the data set of noise-reduced behaviors.

5. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the real-time behavior safety baseline data noise-reduction method for safety analysis according to any one of claims 1 to 3.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the real-time behavior safety baseline data noise-reduction method for safety analysis according to any one of claims 1 to 3.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the real-time behavioral safety baseline data noise reduction method for safety analysis according to any one of claims 1 to 3.

Images