Sample processing method and device based on energy and interpretation sparsity
By introducing a joint gating mechanism of energy fraction and explained sparsity, the decision type of malware samples is determined and dynamically adjusted, which solves the shortcomings of existing technologies in detecting novel malware, improves the identification capability and system flexibility, and reduces false positives and false negatives.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HARBIN ANTIY TECH
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-10
AI Technical Summary
Existing malware detection technologies perform poorly when facing new or variant malware, are susceptible to zero-day and variant attacks, resulting in high false negative and false positive rates, lack the ability to identify out-of-distribution samples and adversarial samples, and are difficult to handle dynamically changing malware samples.
A joint gating mechanism of energy score and explained sparsity is introduced. The decision type of a sample is determined by calculating the energy score and explained sparsity of the sample, and corresponding processing is performed according to the decision type, including different processing methods for credible samples, rejected samples and uncertain samples. The model decision threshold is dynamically adjusted to reduce false positives and false negatives.
It significantly reduces false positives and false negatives, improves the ability to identify new malware and adversarial samples, enhances the robustness and flexibility of the system, and can effectively cope with dynamically changing malware samples.
Smart Images
Figure CN121834558A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of machine learning, artificial intelligence, and cybersecurity, and in particular to a sample processing method and apparatus based on energy and interpretation sparsity. Background Technology
[0002] As malware attack methods become increasingly sophisticated, existing malware detection technologies face numerous challenges, especially in their poor performance against new or variant malware. They are susceptible to zero-day attacks and variant attacks, resulting in high rates of false negatives and false positives. Summary of the Invention
[0003] This invention provides a sample processing method and apparatus based on energy and interpretation sparsity. The technical solution is as follows: On the one hand, a sample processing method based on energy and interpretable sparsity is provided, the method comprising: Based on the malware detection model, the input feature data of the sample is obtained; The sample is initially classified using the input feature data, and the energy score and explained sparsity of the sample are calculated. Based on the energy score and explained sparsity of the sample, the decision type of the sample is determined; the decision type includes credible sample, rejected sample, and uncertain sample. The samples are processed according to their decision type; the processing methods for samples with different decision types are as follows: If the sample is a reliable sample, output the initial classification result; If the sample is rejected, it will be routed to an alternative analysis process. If the sample is uncertain, the model decision threshold is conditionally adjusted for the sample based on the sample's energy score and explained sparsity, and the sample is re-evaluated using the adjusted decision threshold.
[0004] On the other hand, a sample processing apparatus based on energy and explained sparsity is provided, the apparatus comprising: The acquisition unit is used to acquire the input feature data of the sample based on the malware detection model; The computational unit is used to perform initial classification of samples using input feature data and to calculate the energy score and explained sparsity of the sample. The decision unit is used to determine the decision type of the sample based on its energy score and explained sparsity; the decision type includes a credible sample, a rejected sample, and an uncertain sample. The processing unit is used to process samples according to their decision type. The processing methods for samples with different decision types are as follows: if the sample is a reliable sample, the initial classification result is output; if the sample is a rejected sample, the rejected sample is routed to the backup analysis process; if the sample is an uncertain sample, the model decision threshold is conditionally adjusted for the sample based on the sample's energy score and explained sparsity, and the sample is re-evaluated using the adjusted decision threshold.
[0005] On the other hand, a computer device is provided, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for executing the computer program stored in the memory to implement the steps of the sample processing method based on energy and interpretation sparsity described above.
[0006] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of the sample processing method based on energy and interpretation sparsity described above.
[0007] On the other hand, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the sample processing method based on energy and interpretation sparsity described above.
[0008] The technical solution provided by this invention can bring at least the following beneficial effects: After initial classification of samples using input feature data, the energy score and explained sparsity of each sample are calculated to determine its decision type. Different processing methods are applied to samples belonging to different decision types. If a sample is a reliable sample, the initial classification result is directly output. If a sample is a rejected sample, it is routed to a backup analysis process. If a sample is uncertain, the model's decision threshold is conditionally adjusted based on its energy score and explained sparsity, and the sample is re-evaluated using the adjusted threshold. Therefore, this invention, by introducing a rejection mechanism to actively reject uncertain samples, avoids misclassification and analysis, thereby significantly reducing false positives and false negatives. Attached Figure Description
[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0010] Figure 1 This is a flowchart of a sample processing method based on energy and interpretation sparsity provided in an embodiment of the present invention; Figure 2 This is a flowchart of a method for converting a software binary file into an encoded image according to an embodiment of the present invention; Figure 3 This is a flowchart of an adaptive line width determination method provided in an embodiment of the present invention; Figure 4 This is a structural diagram of a sample processing device based on energy and interpretation sparsity provided in an embodiment of the present invention; Figure 5 This is a hardware architecture diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation
[0011] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0012] Traditional malware detection methods rely on feature-based static analysis, typically depending on predefined rules or known malware signatures. Analysis using existing malware detection methods reveals at least the following problems: Poor detection capability against unknown malware: Traditional methods rely on known signature databases, which often perform poorly against new or variant malware. They are also susceptible to zero-day attacks and variant attacks, resulting in high false negative and false positive rates.
[0013] Vulnerability of adversarial examples: Existing malware detection technologies rely on the output confidence of models to determine whether something is malware. However, adversarial examples are samples specifically designed to evade detection, and therefore can easily bypass detection under some traditional models, causing detection systems to fail to identify new attack methods.
[0014] Limitations of Single-Feature Judgment: Existing technologies often use only a single confidence level metric for malware classification. While this method can determine the maliciousness of a sample, it lacks in-depth analysis of the complex threat patterns behind the sample. For multimodal malware detection, a single model output cannot fully express the complexity and potential threat of malware.
[0015] Weak ability to identify out-of-distribution (OOD) samples: Existing technologies perform poorly when dealing with out-of-distribution samples. Due to biases in the training data, models often struggle to accurately handle samples that do not conform to the training data distribution. This makes malware detection systems vulnerable to new types of malware.
[0016] Lack of robustness and flexibility: Traditional malware detection techniques struggle to handle dynamically changing malware samples. Malware constantly evolves and changes, and existing systems often cannot adapt to rapidly evolving attack methods. Especially in online detection scenarios, models cannot effectively cope with the continuous changes in malware samples and new attack methods.
[0017] Based on this, the inventive concept of the present invention is to introduce a joint gating mechanism of energy fraction and explained dilution to accurately determine whether a sample is an out-of-distribution sample or an adversarial sample. At the same time, through a rejection mechanism, these unknown samples are actively rejected to avoid misjudgment. In addition, by dynamically adjusting the threshold, samples that do not meet the rejection conditions are marked and re-evaluated.
[0018] Please refer to Figure 1 This invention provides a sample processing method based on energy and interpreted sparsity, the method comprising: Step 100: Based on the malware detection model, obtain the input feature data of the sample; Step 102: Perform initial classification of the sample using the input feature data, and calculate the energy score and explained sparsity of the sample; Step 104: Determine the decision type of the sample based on its energy score and explained sparsity; the decision type includes credible sample, rejected sample, and uncertain sample. Step 106: Process the samples according to their decision type; the processing methods for samples with different decision types are as follows: If the sample is a reliable sample, output the initial classification result; If the sample is rejected, it will be routed to an alternative analysis process. If the sample is uncertain, the model decision threshold is conditionally adjusted for the sample based on the sample's energy score and explained sparsity, and the sample is re-evaluated using the adjusted decision threshold.
[0019] In this embodiment of the invention, after initial classification of samples using input feature data, the energy score and explained sparsity of the samples are calculated to determine which decision type the sample belongs to. Different processing methods are applied to samples belonging to different decision types. If the sample is a reliable sample, the initial classification result is directly output. If the sample is a rejected sample, it is routed to a backup analysis process. If the sample is an uncertain sample, the model's decision threshold is conditionally adjusted based on the sample's energy score and explained sparsity, and the sample is re-evaluated using the adjusted decision threshold. Therefore, this invention, by introducing a rejection mechanism to actively reject uncertain samples, avoids misclassification and analysis, thereby significantly reducing false positives and false negatives.
[0020] The following description Figure 1 The execution method for each step is shown.
[0021] First, for step 100, based on the malware detection model, the input feature data of the sample is obtained.
[0022] In this embodiment of the invention, the malware detection model is used to identify whether sample software is malware. It receives input feature data of the sample, uses the input feature data to perform detection, and obtains output detection results.
[0023] Considering that the byte sequences contained in a software binary file contain the essential characteristics of the software, if these essential characteristics can be identified, it is possible to more accurately identify whether the software is malicious or legitimate. Therefore, in one embodiment of the present invention, the input feature data can be obtained by converting the software's binary file into an encoded image.
[0024] For one implementation method, please refer to Figure 2 The software's binary file can be converted into an encoded image using the following method: Step 200: Determine the mapping relationship between binary bytes and image pixels, and determine the encoding method of structural information.
[0025] In this embodiment of the invention, the mapping relationship between binary bytes and image pixels can include single-channel mapping and multi-channel mapping. Single-channel mapping encodes one byte into one pixel; multi-channel mapping encodes multiple bytes (three bytes for RGB channels and four bytes for RGBA channels) into one pixel.
[0026] In one embodiment of the present invention, the structural information includes: segment information and / or high-entropy regions.
[0027] In this embodiment of the invention, the structure of a binary file typically includes multiple distinct parts, such as code segments and data segments, each referred to as a segment. The location and content of segments within the binary file are crucial for malware identification. In this embodiment of the invention, by preserving segment information and optimizing it through an attention mechanism, the malware detection system can better understand the structural characteristics of the binary file.
[0028] In one implementation, segment information includes segment boundaries. Segment boundaries are physical separation points between different segments in a binary file, used to distinguish different parts of the binary file. In another implementation, segment information also includes segment types. Segment types can include: .text segment, .data segment, .bss segment, .rodata segment, .stack segment, and .heap segment.
[0029] High-entropy regions typically contain complex or random content and are often associated with malicious behavior. In this embodiment of the invention, high-entropy regions of the binary file are determined by calculating the byte probability distribution of each window region in the binary file one by one. The attention mechanism is optimized by extracting entropy features to accurately identify key parts of malicious files.
[0030] It is understandable that the encoding methods for structural information include the encoding of segmental information and the encoding methods for high-entropy regions.
[0031] Specifically, the segment information is encoded in at least one of the following ways: each segment occupies a different area in the encoded image; different segments are distinguished by pixel intervals or line intervals; and different segment types are encoded using different colors.
[0032] Encoding high-entropy regions can be achieved by setting the pixel positions of the encoded image corresponding to the byte range of the high-entropy region in the binary file as specific feature markers. These feature markers include specific colors and / or specific shapes. This allows the malware detection model to focus more on these high-entropy regions during training, improving the accuracy of malware detection.
[0033] Step 202: Determine the adaptive line width of the encoded image based on the mapping relationship and encoding method.
[0034] When encoding binary files into images, different binary files have different sizes. If a fixed line width is used for encoding all of them, structural information will be lost, and the structural fidelity of the binary file content cannot be guaranteed. Therefore, in this embodiment of the invention, it is necessary to determine an encoded image with an adaptive line width that is compatible with the binary file, so that the encoded image can adaptably carry all the content of the binary file without the appearance of a large number of blank areas, which would cause distortion of the binary file's own structure.
[0035] For one implementation method, please refer to Figure 3 The adaptive line width can be determined in the following way: Step 300: Determine the total number of pixels in the encoded image based on the mapping relationship, the encoding method, and the total number of bytes in the binary file; Step 302: Determine the initial row width based on the total number of pixels and the target aspect ratio; Step 304: Adjust the initial line width based on the preset line width range so that the adjusted adaptive line width is within the preset line width range.
[0036] Preferably, the preset line width ranges from 32 to 1024 bytes.
[0037] Assuming the target aspect ratio is W / H and the total number of pixels is N, the initial line width can be determined by W×H≥N. To avoid extreme imbalance or excessively large blank areas, the line width can be adjusted to the preset line width range using the following formula: Adaptive line width = max(min(initial line width, maximum line width endpoint), minimum line width endpoint) This ensures that the adaptive line width is within the preset line width range, and that the width and height of the image are as balanced as possible.
[0038] In this embodiment of the invention, by dynamically selecting an adaptive line width, the amount of data in each line is ensured to be appropriate, so as not to lose the relationship between segments. Using different line widths for different binary files ensures that each binary file can adapt to image generation while preserving the structural information of the binary file.
[0039] Step 204: Encode the byte sequence in the binary file into the image to obtain the encoded image.
[0040] In this embodiment of the invention, after obtaining the encoded image, visual features can be extracted from the encoded image using an image processing module. The image processing module can be a convolutional neural network (CNN) or a visual transformer (ViT). Since the encoded image retains structural information, the image processing module will focus on this structural information during the extraction of visual features. In one embodiment, the visual features are used as input feature data for a malware detection model.
[0041] Then, for step 102, the sample is initially classified using the input feature data, and the energy score and explained sparsity of the sample are calculated.
[0042] Energy scores are used to measure a model's confidence in an input sample, distinguishing between samples of known and unknown categories. In malware detection, energy scores are calculated by determining the degree of match between an input sample and a known category. A high energy score indicates that the sample belongs to a known category, while a low energy score suggests that the sample may belong to a new category or be an OOD (Out-of-Depth) sample.
[0043] In one implementation, the energy score of a sample can be calculated as follows: in, Where K is the energy fraction; T is the temperature parameter; and K is the number of output categories. f i (x) This is the unnormalized logical value of the i-th category output by the malware detection model for sample x.
[0044] For example, if the model outputs the categories of family A, family B, and benign for a sample, then K=3, and the model's logits output is [8.5, 2.1, -1.0]. Therefore, the energy score can be calculated as -8.5, indicating that the energy score of this sample is relatively low.
[0045] Interpretive sparsity measures the concentration or sparsity of information in a model's interpretation. When dealing with samples with high uncertainty, the generated interpretations tend to be sparse, meaning that the model cannot focus on certain specific features, or that there are multiple unrelated feature regions in the interpretation process.
[0046] In one embodiment of the present invention, the explanatory sparsity is calculated as follows: a feature importance vector for the initial classification is generated; the explanatory sparsity of the samples is calculated using the feature importance vector.
[0047] For the input sample x and the model's prediction, a feature importance vector A(x) = [a1, a2, ..., a3] can be generated using a post-hoc interpretation method. M ], where M is the number of feature dimensions, a i (i=1,2,…,M) represents the contribution of the i-th feature to the current prediction.
[0048] In this embodiment of the invention, A(x) can be regarded as a distribution to calculate the explained sparsity: In one implementation, the interpreted sparsity of a sample can be calculated as follows: in, To explain sparsity, These are the feature importance values sorted in ascending order of absolute value.
[0049] A higher explanatory sparsity value indicates that the model's decisions rely on a small number of high-weight features, and the explanatory distribution is more concentrated; a lower value indicates that the model's explanations are scattered and the decision basis is unstable.
[0050] Next, for step 104, the decision type of the sample is determined based on the energy score and explained sparsity of the sample; the decision type includes credible sample, rejected sample, and uncertain sample.
[0051] After calculating the energy score and explained sparsity of the sample, the decision type of the sample can be determined in at least three ways: Method 1: Using uncertainty scores to determine Method 2: Using a joint gating function to determine Method 3: Using uncertainty scores and joint gating functions to determine... Regardless of the method used, a preset threshold must be considered for judgment. This preset threshold includes a confidence threshold and a rejection threshold, which are used to distinguish between confident samples, uncertain samples, and rejected samples, respectively.
[0052] The three methods will be explained below.
[0053] Regarding method one: By calculating the uncertainty score of a sample, the statistical significance of the sample's risk intensity can be characterized.
[0054] In one embodiment of the present invention, the uncertainty score can be calculated using the following formula: in, For uncertain fractions, The normalized energy fraction. The normalized sparsity is the explanatory sparsity. These are the weighting coefficients.
[0055] In this embodiment of the invention, uncertainty assessment is accomplished by combining energy score and explained sparsity. The energy score is used to determine the model's confidence in the sample, while the explained sparsity is used to reveal whether the model can clearly explain the features of the sample.
[0056] When determining the decision type of a sample, the uncertainty score can be directly compared with a preset threshold, and the decision type can be determined based on the comparison result. Specifically, if the uncertainty score is greater than the confidence threshold, the sample is determined to be a confidence sample; if the uncertainty score is less than the rejection threshold, the sample is determined to be a rejection sample; if the uncertainty score is greater than the rejection threshold but less than the confidence threshold, the sample is determined to be an uncertain sample. It can be understood that the rejection threshold is less than the confidence threshold.
[0057] Regarding method two: In one embodiment of the present invention, a joint gating function can be constructed based on energy score and explanatory sparsity to form a clear sample decision boundary. Determining the decision type of the sample may include: calculating the function output value using the joint gating function based on the sample's energy score and explanatory sparsity; and determining the decision type of the sample by comparing the function output value with a preset threshold; the preset threshold includes a confidence threshold and a rejection threshold.
[0058] In one implementation, the joint gating function can be: in, For the function output value, For the gating weight parameters, It is a Sigmoid or equivalent monotonic mapping function.
[0059] After obtaining the function output value, the function output value can be compared with the preset threshold as follows to obtain the judgment result: when At that time, the sample is determined to be a reliable sample; when At that time, the sample was identified as a rejected sample; when When this happens, the sample is determined to be an uncertain sample.
[0060] in, As a confidence threshold, This is the rejection threshold.
[0061] Regarding method three: In this third approach, the uncertainty score, as a continuous numerical indicator, is mainly used to continuously characterize the risk level of a sample, for sample risk ranking, statistical analysis, or threshold reference. It may not be directly used for the final rejection or routing decision. The joint gating function is used to form a discrete sample decision boundary based on this, so as to determine the credibility, rejection, or uncertainty state of a sample. The two functions complement each other but do not overlap.
[0062] In one implementation, the uncertainty score is used to adjust the confidence threshold and the rejection threshold, and the function output value of the joint gating function is compared with the adjusted confidence threshold and rejection threshold to obtain a more accurate judgment result.
[0063] Finally, for step 106, the samples are processed accordingly based on their decision type.
[0064] After determining the decision type of the sample, the processing methods for samples of different decision types are as follows: If the sample is a reliable sample, output the initial classification result; If the sample is rejected, it will be routed to an alternative analysis process. If the sample is uncertain, the model decision threshold is conditionally adjusted based on the sample's energy score and explained sparsity, and the sample is then re-evaluated using the adjusted decision threshold. This conditional adjustment of the model decision threshold only applies to the re-evaluation of the current sample and does not affect the decision logic for other samples.
[0065] For trusted samples, the initial classification result can be output directly. This initial classification result is the result obtained after classification based on the malware detection model.
[0066] In this embodiment of the invention, rejected samples can be directly rejected and routed to a backup analysis process, which includes in-depth analysis steps such as sandbox analysis, dynamic behavior analysis, or manual review.
[0067] Uncertain samples indicate that they fall into a gray area of the system; they cannot be released as quickly as trusted samples, yet they are not suspicious enough to warrant in-depth analysis using backup procedures. However, directly using the model's fixed standards for judgment would be very risky.
[0068] For uncertain samples, considering their internal heterogeneity, some uncertain samples might be slightly ambiguous known types, while others might be unknown threats disguised. Although joint gating functions can identify problematic uncertain samples, they cannot distinguish the nature of the problem. For example, suppose the rejection threshold is 0.7, and two samples have a function output value of 0.65. Although both are below the rejection threshold, the first sample has a higher energy score and lower explained sparsity, while the second sample has a lower energy score and higher explained sparsity. Therefore, the first sample is more likely to be an unknown novel malware, while the second sample is more likely to be a variant of known malware. Thus, a fixed threshold cannot differentiate between them.
[0069] Based on this, in this embodiment of the invention, for uncertain samples, a dynamic threshold adjustment mechanism for sample conditionalization is introduced to avoid misjudgment caused by using a uniform fixed threshold. Specifically: In one embodiment of the present invention, the model decision threshold can be adjusted in the following manner: Step 1060: Determine the pattern of the sample in the energy-sparseness two-dimensional space based on the energy fraction and interpreted sparsity of the sample; Step 1062: Calculate the threshold adjustment amount for this sample based on the preset mode-adjustment mapping rule; Step 1064: Adjust the model decision threshold using the threshold adjustment amount.
[0070] In this embodiment of the invention, the sample patterns in the energy-sparseness two-dimensional space include the following three types: Mode 1: High energy score, low explained sparsity Mode 2: Low energy fraction, high explanatory sparsity Mode 3, Medium Energy Fraction, Medium Explained Sparsity When determining the sample pattern, it can be based on two preset energy thresholds and two preset sparsity levels. Specifically: When the energy score of a sample is lower than the first energy threshold, it is determined to be a low energy score. When the energy score of a sample is higher than the first energy threshold but lower than the second energy threshold, it is determined to be a medium energy score. When the energy score of a sample is higher than the second energy threshold, it is determined to be a high energy score. Similarly: When the explained sparsity of a sample is lower than the first sparsity threshold, it is determined to be explained sparsity. When the explained sparsity of a sample is higher than the first sparsity threshold but lower than the second sparsity threshold, it is determined to be medium explained sparsity. When the explanatory sparsity of a sample is higher than the second sparsity threshold, it is determined to be high explanatory sparsity.
[0071] When the high energy score and low explained sparsity of Mode 1 indicate that the sample significantly deviates from the known distribution, but the model's focus is reasonable, and the sample may be a new type of sample. Based on this, the adjustment direction for this mode could be to significantly increase the threshold, with the adjustment magnitude being the first adjustment amount. When the low energy score and high explained sparsity of mode two indicate that the sample seems familiar, but the model decision is relatively fragile. The sample may be an adversarial sample. Based on this, the adjustment direction in this mode can be to moderately increase the threshold, and the adjustment magnitude is the second adjustment amount. When interpreting sparsity in the medium energy fraction of mode 3, it indicates that it is a boundary sample and the adjustment direction needs to be determined according to the specific value. In this mode, the threshold can be fine-tuned, and the adjustment range is the third adjustment amount.
[0072] Among them, the first adjustment amount is greater than the second adjustment amount, and the second adjustment amount is greater than the third adjustment amount.
[0073] In one implementation, the decision threshold is a rejection threshold, which can be calculated using the following formula: in, This is the adjusted rejection threshold. The original rejection threshold, This serves as a reference threshold for energy fraction and the explanation of sparsity. It is a monotonic nonlinear function. This is for adjusting the coefficient.
[0074] It should be noted that the adjusted rejection threshold only applies to the re-evaluation of the current sample and does not affect the global judgment logic of other samples.
[0075] By dynamically adjusting the rejection threshold using the above method, the uncertain sample is then re-evaluated. If the sample remains uncertain based on the adjusted rejection threshold, it is labeled and output. By assigning it the label of uncertain sample, the sample can be evaluated again after subsequent model retraining. If the sample is rejected based on the adjusted rejection threshold, it is routed to the backup analysis process.
[0076] Please refer to Figure 4 This invention provides a sample processing device based on energy and explained sparsity, the device comprising: The acquisition unit 400 is used to acquire input feature data of samples based on a malware detection model; The computing unit 402 is used to perform initial classification of samples using input feature data and to calculate the energy score and explained sparsity of the sample. The decision unit 404 is used to determine the decision type of the sample based on the energy score and explained sparsity of the sample; the decision type includes a credible sample, a rejected sample, and an uncertain sample. Processing unit 406 is used to process samples according to their decision type. The processing methods for samples with different decision types are as follows: if the sample is a reliable sample, the initial classification result is output; if the sample is a rejected sample, the rejected sample is routed to the backup analysis process; if the sample is an uncertain sample, the model decision threshold is conditionally adjusted for the sample based on the sample's energy score and explained sparsity, and the sample is re-evaluated using the adjusted decision threshold.
[0077] In one embodiment of the present invention, the energy fraction is calculated using the following formula: in, Where K is the energy fraction; T is the temperature parameter; and K is the number of output categories. f i (x) This is the unnormalized logical value of the i-th category output by the malware detection model for sample x.
[0078] In one embodiment of the present invention, the explanatory sparsity is calculated as follows: a feature importance vector for the initial classification is generated; the explanatory sparsity of the samples is calculated using the feature importance vector.
[0079] In one embodiment of the present invention, the adjustment method of the model decision threshold is as follows: based on the energy score and explained sparsity of the sample, the pattern of the sample in the energy-sparseness two-dimensional space is determined; based on the preset pattern-adjustment amount mapping rule, the threshold adjustment amount for the sample is calculated; and the threshold adjustment amount is used to adjust the model decision threshold.
[0080] In one embodiment of the present invention, the modes include: high energy fraction with low explained sparsity, low energy fraction with high explained sparsity, and medium energy fraction with medium explained sparsity.
[0081] In one embodiment of the present invention, the decision threshold is a rejection threshold, and the adjusted rejection threshold is calculated by the following formula: in, This is the adjusted rejection threshold. The original rejection threshold, This serves as a reference threshold for energy fraction and the explanation of sparsity. It is a monotonic nonlinear function. This is for adjusting the coefficient.
[0082] It should be noted that the energy- and interpreted sparsity-based sample processing device provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the energy- and interpreted sparsity-based sample processing device and the energy- and interpreted sparsity-based sample processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.
[0083] Embodiments of this application also provide a computer device, please refer to... Figure 5 The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by the processor to implement the energy- and interpretation sparsity-based sample processing method provided in the above-described method embodiments.
[0084] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the sample processing method based on energy and interpretation sparsity provided in the above-described method embodiments.
[0085] Embodiments of this application also provide a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium and executes the computer program, causing the computer device to perform any of the sample processing methods based on energy and interpretation sparsity described in the above embodiments.
[0086] For ease of description, the above systems or devices are described separately as various modules or units based on their functions. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components.
[0087] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.
[0088] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0089] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A sample processing method based on energy and interpreted sparsity, characterized in that, The method includes: Based on the malware detection model, the input feature data of the sample is obtained; The sample is initially classified using the input feature data, and the energy score and explained sparsity of the sample are calculated. Based on the energy score and explained sparsity of the sample, the decision type of the sample is determined; the decision type includes credible sample, rejected sample, and uncertain sample. The samples are processed according to their decision type; the processing methods for samples with different decision types are as follows: If the sample is a reliable sample, output the initial classification result; If the sample is rejected, it will be routed to an alternative analysis process. If the sample is uncertain, the model decision threshold is conditionally adjusted for the sample based on the sample's energy score and explained sparsity, and the sample is re-evaluated using the adjusted decision threshold.
2. The method according to claim 1, characterized in that, The energy fraction is calculated using the following formula: in, Where K is the energy fraction; T is the temperature parameter; and K is the number of output categories. f i (x) This represents the unnormalized logical value of the i-th category output by the malware detection model for sample x. And / or, The method for calculating the explanatory sparsity is as follows: generate a feature importance vector for the initial classification; and use the feature importance vector to calculate the explanatory sparsity of the samples.
3. The method according to claim 1, characterized in that, The determination of the decision type of the sample based on its energy score and explained sparsity includes: Based on the energy fraction and explained sparsity of the sample, the function output value is calculated using a joint gating function; The decision type of the sample is determined by comparing the function output value with a preset threshold; the preset threshold includes a confidence threshold and a rejection threshold.
4. The method according to any one of claims 1-3, characterized in that, The adjustment method for the model decision threshold is as follows: Based on the energy fraction and interpreted sparsity of the sample, determine the pattern of the sample in the energy-sparseness two-dimensional space; Based on the preset pattern-adjustment mapping rule, calculate the threshold adjustment amount for this sample; The decision threshold of the model is adjusted using the threshold adjustment amount.
5. The method according to claim 4, characterized in that, The modes include: high energy fraction with low explained sparsity, low energy fraction with high explained sparsity, and medium energy fraction with medium explained sparsity.
6. The method according to claim 4, characterized in that, This decision threshold is the rejection threshold, and the adjusted rejection threshold is calculated using the following formula: in, This is the adjusted rejection threshold. The original rejection threshold, This serves as a reference threshold for energy fraction and the explanation of sparsity. It is a monotonic nonlinear function. This is for adjusting the coefficient.
7. A sample processing device based on energy and interpretation sparsity, characterized in that, The device includes: The acquisition unit is used to acquire the input feature data of the sample based on the malware detection model; The computational unit is used to perform initial classification of samples using input feature data and to calculate the energy score and explained sparsity of the sample. The decision unit is used to determine the decision type of the sample based on its energy score and explained sparsity; the decision type includes a credible sample, a rejected sample, and an uncertain sample. The processing unit is used to process samples according to their decision type. The processing methods for samples with different decision types are as follows: if the sample is a reliable sample, the initial classification result is output; if the sample is a rejected sample, the rejected sample is routed to the backup analysis process; if the sample is an uncertain sample, the model decision threshold is conditionally adjusted for the sample based on the sample's energy score and explained sparsity, and the sample is re-evaluated using the adjusted decision threshold.
8. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the steps of the method according to any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of claims 1-6.
10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-6.