A malware detection method and system based on perception discriminates drift type

Abstract

The application provides a malware detection method and system based on perception discrimination of drift type, comprising: extracting static features from each PE file to construct a high-dimensional feature vector, and dividing a data set into multiple non-overlapping time windows; clustering data of each time window, calculating the moving distance of clustering centroids between continuous windows to obtain a drift score sequence, and perceiving and discriminating the drift type; calculating a comprehensive collection score for each unlabeled sample of each window; according to the comprehensive collection score, sorting all unlabeled samples in each window in descending order of score, selecting the top m samples and submitting them to a labeling link to obtain real labels; after obtaining the real labels of the newly labeled samples, training a malware detection model using the newly labeled samples to obtain a trained malware detection model; and using the trained malware detection model to detect PE files to be detected. The application can detect malware.

Description

Technical Field

[0001] This invention relates to the field of malware detection technology, and in particular to a malware detection method and system based on perceptual discrimination of drift types. Background Technology

[0002] With the continuous evolution of network attack and defense technologies, malware is constantly upgrading in terms of technical complexity, mutation speed, and attack concealment, posing a continuous challenge to existing security defense systems. Windows, as the world's mainstream desktop operating system, has its executable PE (Portable Executable) files becoming a primary carrier for malware propagation and infection, posing a continuous threat to system security. Detection methods based on machine learning and deep learning often face the problem of model performance degradation over time after deployment in dynamic network environments. This is mainly due to concept drift: in online learning or data stream environments, the statistical characteristics behind the data undergo non-stationary changes over time, leading to a decline in the performance of predictive models trained on historical data. Here, "concept" refers to the mapping relationship between the input feature vector x and the output label y, while drift refers to the change of this mapping relationship over time, causing deviations in the association between x and y learned by the model based on the training data. Based on the different change patterns, they can be mainly divided into three types: Sudden Drift: The data distribution undergoes a sharp and significant change at a certain moment; Gradual Drift: New concepts gradually replace old concepts, and the change continues slowly over a long period of time; Recurring Drift: The data distribution state that has appeared before reappears in the future, exhibiting periodicity or repetition.

[0003] In the specific field of malware detection, the problem of concept drift is particularly prominent. This is mainly due to the continuous evolution of malware families and the emergence of new attack techniques, which leads to non-stationary changes in the distribution of data stream features over time.

[0004] Existing methods for addressing concept drift suffer from limited adaptability due to their simplistic update strategies, making them ill-suited to diverse drift types. Furthermore, the lack of adaptive mechanisms results in inefficient annotation processes and high resource consumption during model updates. In addition, their reliance on a large number of real labels further restricts their application scope. Summary of the Invention

[0005] To address the technical problems existing in the prior art, the present invention provides a method and system for detecting malware based on perceptual discrimination of drift types, the technical solution of which is as follows:

[0006] On the one hand, a malware detection method based on perceptual discrimination of drift type is provided, the method comprising:

[0007] S1. Collect multiple executable PE files, extract static features from each PE file, construct a high-dimensional feature vector, and divide the dataset composed of the high-dimensional feature vector into multiple non-overlapping time windows.

[0008] S2. Cluster the data for each time window, calculate the movement distance of the cluster centroids between consecutive windows, obtain the drift score sequence, and perceive and determine the drift type based on the drift score sequence;

[0009] S3. Based on the drift type, the sample selection strategy is dynamically adjusted through an adaptive sampler that integrates a hybrid query strategy, and a comprehensive collection score is calculated for each unlabeled sample in each window.

[0010] S4. Based on the comprehensive collection score, sort all unlabeled samples in each window in descending order of score, and select the highest-ranked sample. m Each sample is submitted to the labeling stage to obtain the real label;

[0011] S5. After obtaining the real labels of the newly labeled samples, the malware detection model is trained using the newly labeled samples. The incremental learning method is used to fine-tune the original model parameters to obtain the trained malware detection model.

[0012] S6. Using the trained malware detection model, detect the PE file to be detected and obtain the detection result of whether the PE file to be detected is malware.

[0013] Optionally, S1 specifically includes:

[0014] S11. Collect multiple PE files from public datasets or the network, wherein the multiple PE files include benign files and malicious files;

[0015] S12. Extract static features from each PE file and construct a high-dimensional feature vector. The extracted static features include: byte histogram, byte-entropy histogram, string features, general file information, PE header and optional header information, section features, import function features, export function features, and data directory features. Each sample is converted into a... 3D feature vector ;

[0016] S13. Dataset sorted by timestamp Based on fixed time interval length Divided into non-overlapping time windows, where For the sample 3D feature vectors For its corresponding timestamp, the window Defined as the set of all sample feature vectors included within this specific time interval:

[0017] .

[0018] Optionally, S2 specifically includes:

[0019] S21. For each time window The dataset within the window is processed using the K-Means clustering algorithm: indivual Dimensional data points Divided into including Cluster set of clusters In this context, the objective function is to find the optimal set of clusters. Minimize the intra-cluster sum of squares (WCSS):

[0020]

[0021] in For clusters The center of mass, Belongs to cluster Specific data points, Let represent the variable that minimizes the objective function. The initial centroids of each cluster are randomly determined, and the following steps are performed iteratively until convergence:

[0022] Assignment steps: Assign each data point to the cluster containing the centroid closest to it in Euclidean distance;

[0023] Update steps: Recalculate the centroid of each cluster. ;

[0024] Output is the current window set of centroid vectors and the number of samples in each cluster ;

[0025] S22: Calculate the centroid set and The distance between them, to quantify the continuous time window and The distribution changes between them are determined because the cluster labels generated in each K-Means run are arbitrary. Therefore, it's necessary to first establish the correspondence between the two centroid sets, using a greedy nearest neighbor matching strategy for each centroid in the current window. Find the nearest Euclidean centroid within the centroid set of the previous window. Perform a match, where:

[0026]

[0027] Calculate the weighted average centroid distance (WACD) as the drift score. :

[0028]

[0029] Weight Determined by the relative size of each cluster in the current window;

[0030] S23. The drift score is calculated between each time window. Constructing a drift fraction sequence Then, the specific types of drift are analyzed:

[0031] Mutation drift sensing discrimination: in the most recent Calculate the moving average of the historical drift score on each window. and moving standard deviation ,when When a mutation drift is detected, among which For confidence level parameters;

[0032] Progressive drift perception discrimination: Detecting the nearest Does the drift score sequence within a window exhibit a monotonically increasing trend, when the most recent The drift score of each window satisfies and At that time, it is determined that a gradual drift has occurred, in which The preset total increase threshold;

[0033] Cyclic drift sensing and discrimination: Maintaining a historical centroid library Set the current centroid vector set Compare with all centroid sets in the historical centroid library, when the following conditions are met... When a cycle drift is detected, it is determined that a cycle drift has occurred. It is a threshold used to determine whether a state is similar to a historical state.

[0034] Optionally, S3 specifically includes:

[0035] For each unlabeled sample Calculate the comprehensive collection score The result is obtained by weighted summation after normalization of the scores according to the three basic criteria:

[0036]

[0037] The scores for each basic criterion are calculated as follows:

[0038] Uncertainty score This measure assesses the uncertainty of the malware detection model's sample predictions, calculated using a minimum confidence strategy.

[0039]

[0040] in For the model to determine the category to which the sample belongs The higher the uncertainty score, the lower the model's confidence in classifying the sample.

[0041] Diversity score Used to select samples that can represent new regions in the data distribution, avoiding label redundancy, by calculating the difference between the samples and the already labeled set. The minimum distance is obtained as follows:

[0042]

[0043] in The distance is Euclidean. A higher diversity score indicates that the sample is further away from the existing labeled area and contains more information.

[0044] outlier score Used to identify isolated samples that may belong to new malware variants or noise, calculating the sample's characteristics and... Average distance of the nearest neighbors:

[0045]

[0046] in For the sample The higher the outlier score in the nearest neighbor set, the more isolated the sample is in the local neighborhood.

[0047] Where the weight vector satisfy And dynamically allocate weights based on the specific drift type. This allows for the most effective sample selection for different drift scenarios.

[0048] Optionally, the dynamic scheduling weights are determined based on the specific drift type. Specifically, it includes:

[0049] Mutation drift: When mutation drift occurs, the data distribution changes drastically, and the knowledge of the old model quickly becomes invalid. In order to fully explore the characteristics of the new distribution, the diversity is given the highest weight to select samples that can represent the overall structure of the new distribution. Outliers are given the second highest weight to capture outliers that may represent new attack patterns. Uncertainty is given the lowest weight because the uncertainty measure of the old model under the new distribution is unreliable.

[0050] Progressive drift: In such slow changes, new malware variants gradually infiltrate, and the classification confidence in the model decision boundary region is the weakest. The core strategy is to accurately optimize the decision boundary. Therefore, uncertainty indicators are given dominant weights, and annotation resources are concentrated on samples with low model prediction confidence. This improves the model's ability to distinguish progressive changes with optimal efficiency. Low diversity weights are set because the distribution changes are continuous and do not require large-scale exploration. Abnormal sample weights are set to zero to avoid wasting precious annotation budget on potentially noisy data.

[0051] Cyclic drift: When cyclic drift is detected, it means that a historical concept is reproduced. The strategy goal is to quickly recall historical knowledge and adapt to its possible changes. Therefore, uncertainty sampling is given the highest weight, and the model is given priority to identify ambiguous samples in the current reproduction distribution to adjust the existing knowledge. At the same time, a small amount of weight for diversity and outliers is retained to capture new variants that may appear in this drift.

[0052] On the other hand, a malware detection system based on perceptual drift type is provided, the system comprising:

[0053] The extraction module is used to collect multiple executable PE files, extract static features from each PE file, construct a high-dimensional feature vector, and divide the dataset composed of the high-dimensional feature vector into multiple non-overlapping time windows.

[0054] The perception and discrimination module is used to cluster the data of each time window, calculate the movement distance of the cluster centroids between consecutive windows, obtain the drift score sequence, and perceive and discriminate the drift type based on the drift score sequence.

[0055] The calculation module is used to dynamically adjust the sample selection strategy based on the drift type, through an adaptive sampler that integrates a hybrid query strategy, and calculate a comprehensive collection score for each unlabeled sample in each window.

[0056] The selection module is used to sort all unlabeled samples in each window in descending order of their comprehensive collection scores, and select the highest-ranked sample. m Each sample is submitted to the labeling stage to obtain the real label;

[0057] The training module is used to train the malware detection model using the new labeled samples after obtaining the real labels of the new labeled samples. It adopts an incremental learning method to fine-tune the original model parameters to obtain the trained malware detection model.

[0058] The detection module is used to use a trained malware detection model to detect the PE file to be detected and obtain the detection result of whether the PE file to be detected is malware.

[0059] Optionally, the extraction module is specifically used for:

[0060] Collect multiple PE files from public datasets or the internet, including both benign and malicious files;

[0061] Static features are extracted from each PE file to construct a high-dimensional feature vector. The extracted static features include: PE header information, optional header information, number of sections, section entropy value, virtual section size, import function table, export function table, and byte frequency histogram. Each sample is converted into a... 3D feature vector ;

[0062] Datasets sorted by timestamp Based on fixed time interval length Divided into non-overlapping time windows, where For the sample 3D feature vectors For its corresponding timestamp, the window Defined as the set of all sample feature vectors included within this specific time interval:

[0063] .

[0064] Optionally, the perception and discrimination module is specifically used for:

[0065] For each time window The dataset within the window is processed using the K-Means clustering algorithm: indivual Dimensional data points Divided into including Cluster set of clusters In this context, the objective function is to find the optimal set of clusters. Minimize the intra-cluster sum of squares (WCSS):

[0066]

[0067] in For clusters The center of mass, Belongs to cluster Specific data points, Let represent the variable that minimizes the objective function. The initial centroids of each cluster are randomly determined, and the following steps are performed iteratively until convergence:

[0068] Assignment steps: Assign each data point to the cluster containing the centroid closest to it in Euclidean distance;

[0069] Update steps: Recalculate the centroid of each cluster. ;

[0070] Output is the current window set of centroid vectors and the number of samples in each cluster ;

[0071] Calculate the centroid set and The distance between them, to quantify the continuous time window and The distribution changes between them are determined because the cluster labels generated in each K-Means run are arbitrary. Therefore, it's necessary to first establish the correspondence between the two centroid sets, using a greedy nearest neighbor matching strategy for each centroid in the current window. Find the nearest Euclidean centroid within the centroid set of the previous window. Perform a match, where:

[0072]

[0073] Calculate the weighted average centroid distance (WACD) as the drift score. :

[0074]

[0075] Weight Determined by the relative size of each cluster in the current window;

[0076] The drift score is calculated between each time window. Constructing a drift fraction sequence Then, the specific types of drift are analyzed:

[0077] Mutation drift sensing discrimination: in the most recent Calculate the moving average of the historical drift score on each window. and moving standard deviation ,when When a mutation drift is detected, among which For confidence level parameters;

[0078] Progressive drift perception discrimination: Detecting the nearest Does the drift score sequence within a window exhibit a monotonically increasing trend, when the most recent The drift score of each window satisfies and At that time, it is determined that a gradual drift has occurred, in which The preset total increase threshold;

[0079] Cyclic drift sensing and discrimination: Maintaining a historical centroid library Set the current centroid vector set Compare with all centroid sets in the historical centroid library, when the following conditions are met... When a cycle drift is detected, it is determined that a cycle drift has occurred. It is a threshold used to determine whether a state is similar to a historical state.

[0080] Optionally, the computing module is specifically used for:

[0081] For each unlabeled sample Calculate the comprehensive collection score The result is obtained by weighted summation after normalization of the scores according to the three basic criteria:

[0082]

[0083] The scores for each basic criterion are calculated as follows:

[0084] Uncertainty score This measure assesses the uncertainty of the malware detection model's sample predictions, calculated using a minimum confidence strategy.

[0085]

[0086] in For the model to determine the category to which the sample belongs The higher the uncertainty score, the lower the model's confidence in classifying the sample.

[0087] Diversity score Used to select samples that can represent new regions in the data distribution, avoiding label redundancy, by calculating the difference between the samples and the already labeled set. The minimum distance is obtained as follows:

[0088]

[0089] in The distance is Euclidean. A higher diversity score indicates that the sample is further away from the existing labeled area and contains more information.

[0090] outlier score Used to identify isolated samples that may belong to new malware variants or noise, calculating the sample's characteristics and... Average distance of the nearest neighbors:

[0091]

[0092] in For the sample The higher the outlier score in the nearest neighbor set, the more isolated the sample is in the local neighborhood.

[0093] Where the weight vector satisfy And dynamically allocate weights based on the specific drift type. This allows for the most effective sample selection for different drift scenarios.

[0094] Optionally, the dynamic scheduling weights are determined based on the specific drift type. Specifically, it includes:

[0095] Mutation drift: When mutation drift occurs, the data distribution changes drastically, and the knowledge of the old model quickly becomes invalid. In order to fully explore the characteristics of the new distribution, the diversity is given the highest weight to select samples that can represent the overall structure of the new distribution. Outliers are given the second highest weight to capture outliers that may represent new attack patterns. Uncertainty is given the lowest weight because the uncertainty measure of the old model under the new distribution is unreliable.

[0096] Progressive drift: In such slow changes, new malware variants gradually infiltrate, and the classification confidence in the model decision boundary region is the weakest. The core strategy is to accurately optimize the decision boundary. Therefore, uncertainty indicators are given dominant weights, and annotation resources are concentrated on samples with low model prediction confidence. This improves the model's ability to distinguish progressive changes with optimal efficiency. Low diversity weights are set because the distribution changes are continuous and do not require large-scale exploration. Abnormal sample weights are set to zero to avoid wasting precious annotation budget on potentially noisy data.

[0097] Cyclic drift: When cyclic drift is detected, it means that a historical concept is reproduced. The strategy goal is to quickly recall historical knowledge and adapt to its possible changes. Therefore, uncertainty sampling is given the highest weight, and the model is given priority to identify ambiguous samples in the current reproduction distribution to adjust the existing knowledge. At the same time, a small amount of weight for diversity and outliers is retained to capture new variants that may appear in this drift.

[0098] On the one hand, an electronic device is provided, the electronic device including a processor and a memory, the memory storing at least one instruction, the at least one instruction being loaded and executed by the processor to implement the above-described malware detection method based on perception-based drift type.

[0099] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement the above-described malware detection method based on perceptual discrimination drift type.

[0100] The beneficial effects of the technical solution provided by this invention include at least the following:

[0101] This invention proposes a malware detection method and system based on perceptual discrimination of drift types, aiming to solve the problem of detection model performance degradation caused by concept drift in data stream detection environment. It is especially suitable for dynamic scenarios such as malware detection. The method achieves efficient model adaptation with minimal annotation cost by perceptually discriminating drift types and dynamically scheduling active learning strategies. It solves the problem of poor adaptability of detection models and waste of annotation resources caused by the complexity and variability of concept drift in malware detection. Attached Figure Description

[0102] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0103] Figure 1 This is a flowchart of a malware detection method based on perception-based drift type provided by an embodiment of the present invention;

[0104] Figure 2 This is a block diagram of a malware detection system based on perception-based drift type provided in an embodiment of the present invention;

[0105] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0106] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0107] This invention provides a method for detecting malware based on perceptual drift type. This method can be implemented by an electronic device, which can be a terminal or a server. Figure 1 The diagram shown is a flowchart of the method. The processing flow may include the following steps:

[0108] S1. Collect multiple executable PE files, extract static features from each PE file, construct a high-dimensional feature vector, and divide the dataset composed of the high-dimensional feature vector into multiple non-overlapping time windows.

[0109] Optionally, S1 specifically includes:

[0110] S11. Collect multiple PE files from public datasets or the network, wherein the multiple PE files include benign files and malicious files;

[0111] S12. Extract static features from each PE file and construct a high-dimensional feature vector. The extracted static features include: byte histogram, byte-entropy histogram, string features, general file information, PE header and optional header information, section features, import function features, export function features, and data directory features. Each sample is converted into a... 3D feature vector ;

[0112] S13. Dataset sorted by timestamp Based on fixed time interval length Divided into non-overlapping time windows (time interval length) (Can be set to one week or one month), where For the sample 3D feature vectors For its corresponding timestamp, the window Defined as the set of all sample feature vectors included within this specific time interval:

[0113] .

[0114] S2. Cluster the data for each time window, calculate the movement distance of the cluster centroids between consecutive windows, obtain the drift score sequence, and perceive and determine the drift type based on the drift score sequence;

[0115] This invention identifies and characterizes concept drift by quantifying the dynamic changes of the centroids of data clusters in the feature space. Its core technology lies in using the K-Means algorithm to cluster the data in each time window, then designing a quantitative index to calculate the movement distance of the cluster centroids between consecutive windows, and finally determining the specific type of drift by analyzing the time series of this distance index.

[0116] Optionally, S2 specifically includes:

[0117] S21. For each time window The dataset within the window is processed using the K-Means clustering algorithm: indivual Dimensional data points Divided into including Cluster set of clusters In this context, the objective function is to find the optimal set of clusters. Minimize the intra-cluster sum of squares (WCSS):

[0118]

[0119] in For clusters The center of mass, Belongs to cluster Specific data points, Let represent the variable that minimizes the objective function. The initial centroids of each cluster are randomly determined, and the following steps are performed iteratively until convergence:

[0120] Assignment steps: Assign each data point to the cluster containing the centroid closest to it in Euclidean distance;

[0121] Update steps: Recalculate the centroid of each cluster. ;

[0122] Output is the current window set of centroid vectors and the number of samples in each cluster ;

[0123] S22: Calculate the centroid set and The distance between them, to quantify the continuous time window and The distribution changes between them are determined because the cluster labels generated in each K-Means run are arbitrary. Therefore, it's necessary to first establish the correspondence between the two centroid sets, using a greedy nearest neighbor matching strategy for each centroid in the current window. Find the nearest Euclidean centroid within the centroid set of the previous window. Perform a match, where:

[0124]

[0125] Calculate the weighted average centroid distance (WACD) as the drift score. :

[0126]

[0127] Weight Determined by the relative size of each cluster in the current window;

[0128] S23. The drift score is calculated between each time window. Constructing a drift fraction sequence Then, the specific types of drift are analyzed:

[0129] Mutation drift sensing discrimination: in the most recent Calculate the moving average of the historical drift score on each window. and moving standard deviation ,when When a mutation drift is detected, among which For confidence level parameters;

[0130] Progressive drift perception discrimination: Detecting the nearest Does the drift score sequence within a window exhibit a monotonically increasing trend, when the most recent The drift score of each window satisfies and At that time, it is determined that a gradual drift has occurred, in which The preset total increase threshold;

[0131] Cyclic drift sensing and discrimination: Maintaining a historical centroid library Set the current centroid vector set Compare with all centroid sets in the historical centroid library, when the following conditions are met... When a cycle drift is detected, it is determined that a cycle drift has occurred. It is a threshold used to determine whether a state is similar to a historical state.

[0132] S3. Based on the drift type, the sample selection strategy is dynamically adjusted through an adaptive sampler that integrates a hybrid query strategy, and a comprehensive collection score is calculated for each unlabeled sample in each window.

[0133] According to the drift type of the output, the present invention dynamically adjusts the sample selection strategy to efficiently obtain labeled samples for detection model updates. Its core is an adaptive sampler, which integrates a hybrid query strategy to calculate a comprehensive collection score for each unlabeled sample in each window, so as to achieve accurate response for different drift types.

[0134] Optionally, S3 specifically includes:

[0135] For each unlabeled sample Calculate the comprehensive collection score The result is obtained by weighted summation after normalization of the scores according to the three basic criteria:

[0136]

[0137] The scores for each basic criterion are calculated as follows:

[0138] Uncertainty score This measure assesses the uncertainty of the malware detection model's sample predictions, calculated using a minimum confidence strategy.

[0139]

[0140] in For the model to determine the category to which the sample belongs The higher the uncertainty score, the lower the model's confidence in classifying the sample.

[0141] Diversity score Used to select samples that can represent new regions in the data distribution, avoiding label redundancy, by calculating the difference between the samples and the already labeled set. The minimum distance is obtained as follows:

[0142]

[0143] in The distance is Euclidean. A higher diversity score indicates that the sample is further away from the existing labeled area and contains more information.

[0144] outlier score Used to identify isolated samples that may belong to new malware variants or noise, calculating the sample's characteristics and... Average distance of the nearest neighbors:

[0145]

[0146] in For the sample The higher the outlier score in the nearest neighbor set, the more isolated the sample is in the local neighborhood.

[0147] Where the weight vector satisfy And dynamically allocate weights based on the specific drift type. This allows for the most effective sample selection for different drift scenarios.

[0148] Optionally, the dynamic scheduling weights are determined based on the specific drift type. Specifically, it includes:

[0149] Mutation drift: When mutation drift occurs, the data distribution changes drastically, and the knowledge of the old model quickly becomes invalid. In order to fully explore the characteristics of the new distribution, the diversity is given the highest weight to select samples that can represent the overall structure of the new distribution. Outliers are given the second highest weight to capture outliers that may represent new attack patterns. Uncertainty is given the lowest weight because the uncertainty measure of the old model under the new distribution is unreliable.

[0150] The embodiments of the present invention can preferably be configured for mutation drift. .

[0151] Progressive drift: In such slow changes, new malware variants gradually infiltrate, and the classification confidence in the model decision boundary region is the weakest. The core strategy is to accurately optimize the decision boundary. Therefore, uncertainty indicators are given dominant weights, and annotation resources are concentrated on samples with low model prediction confidence. This improves the model's ability to distinguish progressive changes with optimal efficiency. Low diversity weights are set because the distribution changes are continuous and do not require large-scale exploration. Abnormal sample weights are set to zero to avoid wasting precious annotation budget on potentially noisy data.

[0152] The embodiments of the present invention can preferably be configured for gradual drift. .

[0153] Cyclic drift: When cyclic drift is detected, it means that a historical concept is reproduced. The strategy goal is to quickly recall historical knowledge and adapt to its possible changes. Therefore, uncertainty sampling is given the highest weight, and the model is given priority to identify ambiguous samples in the current reproduction distribution to adjust the existing knowledge. At the same time, a small amount of weight for diversity and outliers is retained to capture new variants that may appear in this drift.

[0154] The embodiments of the present invention can preferably be configured for cyclic drift. .

[0155] S4. Based on the comprehensive collection score, sort all unlabeled samples in each window in descending order of score, and select the highest-ranked sample. m Each sample is submitted to the annotation stage to obtain the real labels (where m is the preset annotation budget).

[0156] S5. After obtaining the true labels of the newly labeled samples, train the malware detection model using the new labeled samples. Employ incremental learning (Incremental learning is a model update method that, when new labeled data arrives, does not require retraining the model using all historical data, but instead incrementally updates the existing model parameters. This method can effectively adapt to data flow environments and save computational and storage resources) to fine-tune the original model parameters, resulting in a trained malware detection model.

[0157] S6. Using the trained malware detection model, detect the PE file to be detected and obtain the detection result of whether the PE file to be detected is malware.

[0158] In this embodiment of the invention, static features are first extracted from the PE file to be detected, and then input into the trained malware detection model to output the detection result of whether the PE file to be detected is malware.

[0159] like Figure 2 As shown, this embodiment of the invention also provides a malware detection system based on perceptual discrimination drift type, the system comprising:

[0160] Extraction module 210 is used to collect multiple executable PE files, extract static features from each PE file, construct a high-dimensional feature vector, and divide the dataset composed of the high-dimensional feature vector into multiple non-overlapping time windows.

[0161] The perception and discrimination module 220 is used to cluster the data of each time window, calculate the moving distance of the cluster centroid between consecutive windows, obtain the drift score sequence, and perceive and discriminate the drift type based on the drift score sequence.

[0162] The calculation module 230 is used to dynamically adjust the sample selection strategy based on the drift type by using an adaptive sampler that integrates a hybrid query strategy, and to calculate a comprehensive collection score for each unlabeled sample in each window.

[0163] Module 240 is used to sort all unlabeled samples in each window in descending order of their scores based on the comprehensive acquisition scores, and select the highest-ranked sample. m Each sample is submitted to the labeling stage to obtain the real label;

[0164] Training module 250 is used to train the malware detection model using the new labeled samples after obtaining the real labels of the new labeled samples. It adopts an incremental learning method to fine-tune the original model parameters to obtain the trained malware detection model.

[0165] The detection module 260 is used to use a trained malware detection model to detect the PE file to be detected and obtain the detection result of whether the PE file to be detected is malware.

[0166] Optionally, the extraction module is specifically used for:

[0167] Collect multiple PE files from public datasets or the internet, including both benign and malicious files;

[0168] Static features are extracted from each PE file to construct a high-dimensional feature vector. The extracted static features include: PE header information, optional header information, number of sections, section entropy value, virtual section size, import function table, export function table, and byte frequency histogram. Each sample is converted into a... 3D feature vector ;

[0169] Datasets sorted by timestamp Based on fixed time interval length Divided into non-overlapping time windows, where For the sample 3D feature vectors For its corresponding timestamp, the window Defined as the set of all sample feature vectors included within this specific time interval:

[0170] .

[0171] Optionally, the perception and discrimination module is specifically used for:

[0172] For each time window The dataset within the window is processed using the K-Means clustering algorithm: indivual Dimensional data points Divided into including Cluster set of clusters In this context, the objective function is to find the optimal set of clusters. Minimize the intra-cluster sum of squares (WCSS):

[0173]

[0174] in For clusters The center of mass, Belongs to cluster Specific data points, Let represent the variable that minimizes the objective function. The initial centroids of each cluster are randomly determined, and the following steps are performed iteratively until convergence:

[0175] Assignment steps: Assign each data point to the cluster containing the centroid closest to it in Euclidean distance;

[0176] Update steps: Recalculate the centroid of each cluster. ;

[0177] Output is the current window set of centroid vectors and the number of samples in each cluster ;

[0178] Calculate the centroid set and The distance between them, to quantify the continuous time window and The distribution changes between them are determined because the cluster labels generated in each K-Means run are arbitrary. Therefore, it's necessary to first establish the correspondence between the two centroid sets, using a greedy nearest neighbor matching strategy for each centroid in the current window. Find the nearest Euclidean centroid within the centroid set of the previous window. Perform a match, where:

[0179]

[0180] Calculate the weighted average centroid distance (WACD) as the drift score. :

[0181]

[0182] Weight Determined by the relative size of each cluster in the current window;

[0183] The drift score is calculated between each time window. Constructing a drift fraction sequence Then, the specific types of drift are analyzed:

[0184] Mutation drift sensing discrimination: in the most recent Calculate the moving average of the historical drift score on each window. and moving standard deviation ,when When a mutation drift is detected, among which For confidence level parameters;

[0185] Progressive drift perception discrimination: Detecting the nearest Does the drift score sequence within a window exhibit a monotonically increasing trend, when the most recent The drift score of each window satisfies and At that time, it is determined that a gradual drift has occurred, in which The preset total increase threshold;

[0186] Cyclic drift sensing and discrimination: Maintaining a historical centroid library Set the current centroid vector set Compare with all centroid sets in the historical centroid library, when the following conditions are met... When a cycle drift is detected, it is determined that a cycle drift has occurred. It is a threshold used to determine whether a state is similar to a historical state.

[0187] Optionally, the computing module is specifically used for:

[0188] For each unlabeled sample Calculate the comprehensive collection score The result is obtained by weighted summation after normalization of the scores according to the three basic criteria:

[0189]

[0190] The scores for each basic criterion are calculated as follows:

[0191] Uncertainty score This measure assesses the uncertainty of the malware detection model's sample predictions, calculated using a minimum confidence strategy.

[0192]

[0193] in For the model to determine the category to which the sample belongs The higher the uncertainty score, the lower the model's confidence in classifying the sample.

[0194] Diversity score Used to select samples that can represent new regions in the data distribution, avoiding label redundancy, by calculating the difference between the samples and the already labeled set. The minimum distance is obtained as follows:

[0195]

[0196] in The distance is Euclidean. A higher diversity score indicates that the sample is further away from the existing labeled area and contains more information.

[0197] outlier score Used to identify isolated samples that may belong to new malware variants or noise, calculating the sample's characteristics and... Average distance of the nearest neighbors:

[0198]

[0199] in For the sample The higher the outlier score in the nearest neighbor set, the more isolated the sample is in the local neighborhood.

[0200] Where the weight vector satisfy And dynamically allocate weights based on the specific drift type. This allows for the most effective sample selection for different drift scenarios.

[0201] Optionally, the dynamic scheduling weights are determined based on the specific drift type. Specifically, it includes:

[0202] Mutation drift: When mutation drift occurs, the data distribution changes drastically, and the knowledge of the old model quickly becomes invalid. In order to fully explore the characteristics of the new distribution, the diversity is given the highest weight to select samples that can represent the overall structure of the new distribution. Outliers are given the second highest weight to capture outliers that may represent new attack patterns. Uncertainty is given the lowest weight because the uncertainty measure of the old model under the new distribution is unreliable.

[0203] Progressive drift: In such slow changes, new malware variants gradually infiltrate, and the classification confidence in the model decision boundary region is the weakest. The core strategy is to accurately optimize the decision boundary. Therefore, uncertainty indicators are given dominant weights, and annotation resources are concentrated on samples with low model prediction confidence. This improves the model's ability to distinguish progressive changes with optimal efficiency. Low diversity weights are set because the distribution changes are continuous and do not require large-scale exploration. Abnormal sample weights are set to zero to avoid wasting precious annotation budget on potentially noisy data.

[0204] Cyclic drift: When cyclic drift is detected, it means that a historical concept is reproduced. The strategy goal is to quickly recall historical knowledge and adapt to its possible changes. Therefore, uncertainty sampling is given the highest weight, and the model is given priority to identify ambiguous samples in the current reproduction distribution to adjust the existing knowledge. At the same time, a small amount of weight for diversity and outliers is retained to capture new variants that may appear in this drift.

[0205] The malware detection system based on perception-based drift type provided in this embodiment of the invention has a functional structure that corresponds to the malware detection method based on perception-based drift type provided in this embodiment of the invention, and will not be described again here.

[0206] Figure 3 This is a schematic diagram of the structure of an electronic device 300 provided in an embodiment of the present invention. The electronic device 300 may vary considerably due to different configurations or performance. It may include one or more central processing units (CPUs) 301 and one or more memories 302. The memory 302 stores at least one instruction, which is loaded and executed by the processor 301 to implement the steps of the above-described malware detection method based on perception-based drift type.

[0207] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions that can be executed by a processor in a terminal to perform the aforementioned malware detection method based on perceptual drift type. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0208] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0209] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A malware detection method based on perceptual discrimination drift type, characterized in that, The method includes: S1. Collect multiple executable PE files, extract static features from each PE file, construct a high-dimensional feature vector, and divide the dataset composed of the high-dimensional feature vector into multiple non-overlapping time windows. S2. Cluster the data for each time window, calculate the movement distance of the cluster centroids between consecutive windows, obtain the drift score sequence, and perceive and determine the drift type based on the drift score sequence; S3. Based on the drift type, the sample selection strategy is dynamically adjusted through an adaptive sampler that integrates a hybrid query strategy, and a comprehensive collection score is calculated for each unlabeled sample in each window. S4. Based on the comprehensive collection score, sort all unlabeled samples in each window in descending order of score, and select the highest-ranked sample. m Each sample is submitted to the labeling stage to obtain the real label; S5. After obtaining the real labels of the newly labeled samples, the malware detection model is trained using the newly labeled samples. The incremental learning method is used to fine-tune the original model parameters to obtain the trained malware detection model. S6. Using the trained malware detection model, detect the PE file to be detected and obtain the detection result of whether the PE file to be detected is malware. S3 specifically includes: For each unlabeled sample Calculate the comprehensive collection score The result is obtained by weighted summation after normalization of the scores according to the three basic criteria: ； The scores for each basic criterion are calculated as follows: Uncertainty score This measure assesses the uncertainty of the malware detection model's sample predictions, calculated using a minimum confidence strategy. ； in For the model to determine the category to which the sample belongs The higher the uncertainty score, the lower the model's confidence in classifying the sample. Diversity score Used to select samples that can represent new regions in the data distribution, avoiding label redundancy, by calculating the difference between the samples and the already labeled set. The minimum distance is obtained as follows: ； in The distance is Euclidean. A higher diversity score indicates that the sample is further away from the existing labeled area and contains more information. outlier score Used to identify isolated samples that may belong to new malware variants or noise, calculating the sample's characteristics and... Average distance of the nearest neighbors: ； in For the sample The higher the outlier score in the nearest neighbor set, the more isolated the sample is in the local neighborhood. Where the weight vector satisfy And dynamically allocate weights based on the specific drift type. This allows for the most effective sample selection for different drift scenarios.

2. The method according to claim 1, characterized in that, S1 specifically includes: S11. Collect multiple PE files from public datasets or the network, wherein the multiple PE files include benign files and malicious files; S12. Extract static features from each PE file and construct a high-dimensional feature vector. The extracted static features include: byte histogram, byte-entropy histogram, string features, general file information, PE header and optional header information, section features, import function features, export function features, and data directory features. Each sample is converted into a... 3D feature vector ; S13. Dataset sorted by timestamp Based on fixed time interval length Divided into non-overlapping time windows, where For the sample 3D feature vectors For its corresponding timestamp, the window Defined as the set of all sample feature vectors included within this specific time interval: 。 3. The method according to claim 2, characterized in that, S2 specifically includes: S21. For each time window The dataset within the window is processed using the K-Means clustering algorithm: indivual Dimensional data points Divided into including Cluster set of clusters In this context, the objective function is to find the optimal set of clusters. Minimize the intra-cluster sum of squares (WCSS): ； in For clusters The center of mass, Belongs to cluster Specific data points, Let represent the variable that minimizes the objective function. The initial centroids of each cluster are randomly determined, and the following steps are performed iteratively until convergence: Assignment steps: Assign each data point to the cluster containing the centroid closest to it in Euclidean distance; Update steps: Recalculate the centroid of each cluster. ; Output is the current window set of centroid vectors and the number of samples in each cluster ; S22: Calculate the centroid set and The distance between them, to quantify the continuous time window and To address the distribution changes between centroids, first establish the correspondence between the two centroid sets, then use a greedy nearest neighbor matching strategy to match the centroids of the current window. Find the nearest Euclidean centroid within the centroid set of the previous window. Perform a match, where: ； Calculate the weighted average centroid distance (WACD) as the drift score. : ； Weight Determined by the relative size of each cluster in the current window; S23. The drift score is calculated between each time window. Constructing a drift fraction sequence Then, the specific types of drift are analyzed: Mutation drift sensing discrimination: in the most recent Calculate the moving average of the historical drift score on each window. and moving standard deviation ,when When a mutation drift is detected, among which For confidence level parameters; Progressive drift perception discrimination: Detecting the nearest Does the drift score sequence within a window exhibit a monotonically increasing trend, when the most recent The drift score of each window satisfies and At that time, it is determined that a gradual drift has occurred, in which The preset total increase threshold; Cyclic drift sensing and discrimination: Maintaining a historical centroid library Set the current centroid vector set Compare with all centroid sets in the historical centroid library, when the following conditions are met... When a cycle drift is detected, it is determined that a cycle drift has occurred. It is a threshold used to determine whether a state is similar to a historical state.

4. The method according to claim 1, characterized in that, The weights are dynamically scheduled based on the specific drift type. Specifically, it includes: Mutation drift: When mutation drift occurs, the data distribution changes drastically, and the knowledge of the old model quickly becomes invalid. In order to fully explore the characteristics of the new distribution, the diversity is given the highest weight to select samples that can represent the overall structure of the new distribution. Outliers are given the second highest weight to capture outliers that may represent new attack patterns. Uncertainty is given the lowest weight because the uncertainty measure of the old model under the new distribution is unreliable. Progressive drift: In such slow changes, new malware variants gradually infiltrate, and the classification confidence in the model decision boundary region is the weakest. The core strategy is to accurately optimize the decision boundary. Therefore, uncertainty indicators are given dominant weights, and annotation resources are concentrated on samples with low model prediction confidence. This improves the model's ability to distinguish progressive changes with optimal efficiency. Low diversity weights are set because the distribution changes are continuous and do not require large-scale exploration. Abnormal sample weights are set to zero to avoid wasting precious annotation budget on potentially noisy data. Cyclic drift: When cyclic drift is detected, it means that a historical concept is reproduced. The strategy goal is to quickly recall historical knowledge and adapt to its possible changes. Therefore, uncertainty sampling is given the highest weight, and the model is given priority to identify ambiguous samples in the current reproduction distribution to adjust the existing knowledge. At the same time, a small amount of weight for diversity and outliers is retained to capture new variants that may appear in this drift.

5. A malware detection system based on perception-based drift type discrimination, characterized in that, The system includes: The extraction module is used to collect multiple executable PE files, extract static features from each PE file, construct a high-dimensional feature vector, and divide the dataset composed of the high-dimensional feature vector into multiple non-overlapping time windows. The perception and discrimination module is used to cluster the data of each time window, calculate the movement distance of the cluster centroids between consecutive windows, obtain the drift score sequence, and perceive and discriminate the drift type based on the drift score sequence. The calculation module is used to dynamically adjust the sample selection strategy based on the drift type, through an adaptive sampler that integrates a hybrid query strategy, and calculate a comprehensive collection score for each unlabeled sample in each window. The selection module is used to sort all unlabeled samples in each window in descending order of their comprehensive collection scores, and select the highest-ranked sample. m Each sample is submitted to the labeling stage to obtain the real label; The training module is used to train the malware detection model using the new labeled samples after obtaining the real labels of the new labeled samples. It adopts an incremental learning method to fine-tune the original model parameters to obtain the trained malware detection model. The detection module is used to use a trained malware detection model to detect the PE file to be detected and obtain the detection result of whether the PE file to be detected is malware. The computing module is specifically used for: For each unlabeled sample Calculate the comprehensive collection score The result is obtained by weighted summation after normalization of the scores according to the three basic criteria: ； The scores for each basic criterion are calculated as follows: Uncertainty score This measure assesses the uncertainty of the malware detection model's sample predictions, calculated using a minimum confidence strategy. ； in For the model to determine the category to which the sample belongs The higher the uncertainty score, the lower the model's confidence in classifying the sample. Diversity score Used to select samples that can represent new regions in the data distribution, avoiding label redundancy, by calculating the difference between the samples and the already labeled set. The minimum distance is obtained as follows: ； in The distance is Euclidean. A higher diversity score indicates that the sample is further away from the existing labeled area and contains more information. outlier score Used to identify isolated samples that may belong to new malware variants or noise, calculating the sample's characteristics and... Average distance of the nearest neighbors: ； in For the sample The higher the outlier score in the nearest neighbor set, the more isolated the sample is in the local neighborhood. Where the weight vector satisfy And dynamically allocate weights based on the specific drift type. This allows for the most effective sample selection for different drift scenarios.

6. The system according to claim 5, characterized in that, The extraction module is specifically used for: Collect multiple PE files from public datasets or the internet, including both benign and malicious files; Static features are extracted from each PE file to construct a high-dimensional feature vector. The extracted static features include: PE header information, optional header information, number of sections, section entropy value, virtual section size, import function table, export function table, and byte frequency histogram. Each sample is converted into a... 3D feature vector ; Datasets sorted by timestamp Based on fixed time interval length Divided into non-overlapping time windows, where For the sample 3D feature vectors For its corresponding timestamp, the window Defined as the set of all sample feature vectors included within this specific time interval: 。 7. The system according to claim 6, characterized in that, The perception and discrimination module is specifically used for: For each time window The dataset within the window is processed using the K-Means clustering algorithm: indivual Dimensional data points Divided into including Cluster set of clusters In this context, the objective function is to find the optimal set of clusters. Minimize the intra-cluster sum of squares (WCSS): ； in For clusters The center of mass, Belongs to cluster Specific data points, Let represent the variable that minimizes the objective function. The initial centroids of each cluster are randomly determined, and the following steps are performed iteratively until convergence: Assignment steps: Assign each data point to the cluster containing the centroid closest to it in Euclidean distance; Update steps: Recalculate the centroid of each cluster. ; Output is the current window set of centroid vectors and the number of samples in each cluster ; Calculate the centroid set and The distance between them, to quantify the continuous time window and To address the distribution changes between centroids, first establish the correspondence between the two centroid sets, then use a greedy nearest neighbor matching strategy to match the centroids of the current window. Find the nearest Euclidean centroid within the centroid set of the previous window. Perform a match, where: ； Calculate the weighted average centroid distance (WACD) as the drift score. : ； Weight Determined by the relative size of each cluster in the current window; The drift score is calculated between each time window. Constructing a drift fraction sequence Then, the specific types of drift are analyzed: Mutation drift sensing discrimination: in the most recent Calculate the moving average of the historical drift score on each window. and moving standard deviation ,when When a mutation drift is detected, among which For confidence level parameters; Progressive drift perception discrimination: Detecting the nearest Does the drift score sequence within a window exhibit a monotonically increasing trend, when the most recent The drift score of each window satisfies and At that time, it is determined that a gradual drift has occurred, in which The preset total increase threshold; Cyclic drift sensing and discrimination: Maintaining a historical centroid library Set the current centroid vector set Compare with all centroid sets in the historical centroid library, when the following conditions are met... When a cycle drift is detected, it is determined that a cycle drift has occurred. It is a threshold used to determine whether a state is similar to a historical state.

8. The system according to claim 5, characterized in that, The weights are dynamically scheduled based on the specific drift type. Specifically, it includes: Mutation drift: When mutation drift occurs, the data distribution changes drastically, and the knowledge of the old model quickly becomes invalid. In order to fully explore the characteristics of the new distribution, the diversity is given the highest weight to select samples that can represent the overall structure of the new distribution. Outliers are given the second highest weight to capture outliers that may represent new attack patterns. Uncertainty is given the lowest weight because the uncertainty measure of the old model under the new distribution is unreliable. Progressive drift: In such slow changes, new malware variants gradually infiltrate, and the classification confidence in the model decision boundary region is the weakest. The core strategy is to accurately optimize the decision boundary. Therefore, uncertainty indicators are given dominant weights, and annotation resources are concentrated on samples with low model prediction confidence. This improves the model's ability to distinguish progressive changes with optimal efficiency. Low diversity weights are set because the distribution changes are continuous and do not require large-scale exploration. Abnormal sample weights are set to zero to avoid wasting precious annotation budget on potentially noisy data. Cyclic drift: When cyclic drift is detected, it means that a historical concept is reproduced. The strategy goal is to quickly recall historical knowledge and adapt to its possible changes. Therefore, uncertainty sampling is given the highest weight, and the model is given priority to identify ambiguous samples in the current reproduction distribution to adjust the existing knowledge. At the same time, a small amount of weight for diversity and outliers is retained to capture new variants that may appear in this drift.

Images