Streaming model updating method and device based on time rolling evaluation
By introducing streaming data processing and time-rolling evaluation mechanisms, inference messages are generated and model performance indicators are calculated to update the model. This solves the problem of unstable updates in existing malware detection models and enables the model to continuously self-repair and quickly adapt to new attack patterns.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-03
AI Technical Summary
Existing malware detection methods rely on batch training, which cannot effectively cope with new malware or attack patterns that emerge in the changing network environment, resulting in unstable model updates.
A streaming data processing mechanism and a time-rolling evaluation mechanism are introduced. Inference messages are generated through online inference, model performance indicators are calculated, and the model is updated when the performance does not meet the requirements. Incremental training is carried out using risk samples, and hot update technology is used to ensure uninterrupted online service.
It enables continuous self-repair and evolution of the model, allowing it to quickly adapt to emerging malware or changing attack patterns, ensuring the stability and adaptability of model updates.
Smart Images
Figure CN121786487A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence, machine learning, and cybersecurity, and in particular to a streaming incremental training method and apparatus based on time-rolling evaluation. Background Technology
[0002] Existing malware detection methods typically rely on batch training, using historical data to train models and then performing static analysis based on this data. However, as the network environment continues to change, this static training approach often fails to guarantee that malware detection models can cope with emerging malware or changing attack patterns. Summary of the Invention
[0003] This invention provides a method and apparatus for updating streaming models based on time-rolling evaluation. The technical solution is as follows: On the one hand, a streaming model update method based on time-rolling evaluation is provided, the method comprising: The current version of the malware detection model is used to perform online inference to generate inference results. Based on the inference results, an inference message is generated according to a specified message format. The specified message format includes at least: input feature data, inference result field, timestamp, and version number. Based on the inference messages within the time window, calculate the performance metrics of the malware detection model, and trigger a model update decision when the performance metrics are found to be unsatisfactory. Risk samples are obtained based on inference messages. The current version of the malware detection model is updated using the risk samples, and the updated malware detection model is then used for subsequent online inference.
[0004] On the other hand, a streaming model update apparatus based on time-rolling evaluation is provided, the apparatus comprising: The generation unit is used to perform online inference using the current version of the malware detection model to generate inference results, and to generate inference messages according to a specified message format based on the inference results; the specified message format includes at least: input feature data, inference result fields, timestamp, and version number; The computing unit is used to calculate the performance metrics of the malware detection model based on inference messages within a time window, and to trigger a model update decision when the performance metrics are detected to be unsatisfactory. The update unit is used to obtain risk samples based on inference messages, and use the risk samples to update the current version of the malware detection model so that the updated malware detection model can be used for subsequent online inference.
[0005] On the other hand, a computer device is provided, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for executing the computer program stored in the memory to implement the steps of the streaming model update method based on time-rolling evaluation described above.
[0006] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of the streaming model update method based on time-rolling evaluation described above.
[0007] On the other hand, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the streaming model update method based on time-rolling evaluation described above.
[0008] The technical solution provided by this invention can bring at least the following beneficial effects: A streaming data processing mechanism is introduced to generate inference messages according to a specified message format based on the inference results during the online inference process of the malware detection model. A rolling time evaluation mechanism is also introduced to calculate model performance using inference messages within a time window. If the model performance is found to be unsatisfactory, model adjustments are made, and the risk samples used in the model adjustment process are also obtained based on the inference messages. When replacing older versions of the malware detection model, hot-update technology is used for loading and replacement, ensuring that model updates are achieved without interrupting the online inference service. This invention, by introducing streaming data processing and rolling time evaluation mechanisms, enables the model to continuously self-repair and evolve, quickly adapting to newly emerging malware or changing attack patterns. Attached Figure Description
[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0010] Figure 1 This is a flowchart of a streaming model update method based on time-rolling evaluation provided by an embodiment of the present invention; Figure 2 This is a structural diagram of a streaming model update device based on time-rolling evaluation provided in an embodiment of the present invention; Figure 3 This is a hardware architecture diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation
[0011] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0012] Please refer to Figure 1 This invention provides a streaming model update method based on time-rolling evaluation, the method comprising: Step 100: Use the trained malware detection model to perform online inference to generate inference results, and generate an inference message according to the specified message format based on the inference results; the specified message format includes at least: input feature data, inference result field, timestamp and version number; Step 102: Calculate the performance metrics of the malware detection model based on the inference messages within the time window, and trigger a model update decision when the performance metrics are found to be unsatisfactory. Step 104: Obtain risk samples based on inference messages, update the current version of the malware detection model using the risk samples, and then use the updated malware detection model for subsequent online inference.
[0013] In this embodiment of the invention, a streaming data processing mechanism is introduced to generate inference messages according to a specified message format based on the inference results during the online inference process of the malware detection model. A time-rolling evaluation mechanism is also introduced to calculate model performance using inference messages within a time window. If the model performance is determined to be unsatisfactory, model adjustments are made. The risk samples used in the model adjustment process are also obtained based on the inference messages. When replacing an older version of the malware detection model, hot-update technology is used for loading and replacement, ensuring that model updates are achieved without interrupting the online inference service. This invention, by introducing streaming data processing and a time-rolling evaluation mechanism, enables the model to continuously self-repair and evolve, quickly adapting to newly emerging malware or changing attack patterns.
[0014] The following description Figure 1 The execution method of each step is shown.
[0015] First, for step 100, online inference is performed using the current version of the malware detection model to generate inference results, and inference messages are generated according to the specified message format based on the inference results.
[0016] In this embodiment of the invention, after the malware detection model is trained, it needs to be updated periodically and irregularly to ensure that the model can cope with newly emerging malware or changing attack patterns. To ensure that the samples used for model update training are more consistent with newly emerging malware or attack patterns, this embodiment of the invention introduces a streaming data processing mechanism. Specifically, based on the inference results generated online from the current version of the malware detection model, inference messages are generated according to a specified message format.
[0017] Online inference of the malware detection model is the actual application process of the model, which identifies input feature data and outputs inference results. These inference results include at least a classification result and a confidence level for the software. The classification result can be binary, ternary, or fine-grained. When it is binary classification, the result can include both malware and non-malware. To improve the interpretability of the inference results, they may also include at least one of the following: family tags, structured explanatory text, and energy-type uncertainty.
[0018] In this embodiment of the invention, generating a reasoning message according to a specified message format based on the reasoning result includes: Based on the reasoning results, determine whether the samples used for online reasoning in this instance are risk samples for the model; If so, the risk sample is routed to the backup analysis process to re-acquire the empirical inference result of the risk sample using the backup analysis process, and an inference message is generated according to the specified message format based on the verified inference result. Otherwise, a corresponding inference message is generated for the sample according to the specified message format.
[0019] In this embodiment of the invention, the risk samples of the model are samples of unreliable predictions exhibited by the model during the inference phase. Processing such samples can provide a basis for calculating the performance indicators of the malware detection model and provide training data for the incremental training of the model.
[0020] In this embodiment of the invention, the risk samples are at least of the following two types: The first type: risk samples are uncertain samples; The second type: risky samples are high-confidence error samples.
[0021] The methods for determining these two types of risk samples are explained below.
[0022] For the first type of risk sample, determining whether the sample used for online inference is a risk sample of the model based on the inference results can specifically include: determining whether the prediction confidence is less than a confidence threshold based on the prediction confidence in the inference results; if so, the sample is determined to be a risk sample.
[0023] For uncertain samples, identification can be achieved using the confidence score output by the malware detection model. For uncertain samples with low confidence scores, there are two main scenarios: either the model accurately identifies the sample, or the model misidentifies it. In either case, it's highly likely that some features of the sample are new to the model, and the model cannot effectively recognize these new features. Therefore, it's necessary to use uncertain samples for subsequent incremental training.
[0024] For the second type of risk sample, determining whether the sample used for online inference is a risk sample of the model based on the inference results can specifically include: determining, based on the inference results and external feedback, that the sample is a high-confidence error sample with a prediction confidence of not less than the confidence threshold and that the sample was inferred incorrectly, and thus identifying the sample as a risk sample.
[0025] High-confidence error samples are not directly obtained during the inference process, but rather are missed cases discovered afterward. Therefore, this type of sample requires external feedback for identification. The model's misidentification of this sample likely indicates that it possesses hidden features that the model failed to recognize. Therefore, high-confidence error samples need to be used for subsequent incremental training.
[0026] In this embodiment of the invention, after identifying a risk sample, a backup analysis process can be set up to route the risk sample to the backup analysis process for further analysis and processing, so as to obtain a more accurate inference result for the risk sample. The backup analysis process can be sandbox analysis, dynamic behavior analysis, etc.
[0027] In existing streaming model update methods, there is a lack of traceable causal link between model performance evaluation, sample selection, and model update. Especially in the presence of label latency, feedback noise, and frequent hot updates, traditional streaming model update methods cannot guarantee the comparability of evaluation results across different time windows, leading to unstable model update decisions.
[0028] In this embodiment of the invention, after the reasoning result is reacquired, a reasoning message needs to be generated according to a specified message format, which includes at least: input feature data, reasoning result field, timestamp, and version number.
[0029] Generated inference messages for: in, For input feature data, For the inference result fields output by the model, For timestamps, This is the version number.
[0030] To ensure that the inference message can be used for subsequent model updates quickly, accurately, and with rich information, the specified message format may preferably include a verified inference result field, as well as a version number including the model version number and the decision threshold version number.
[0031] The generated inference message is: in, For verified inference result fields, For model version number, This is the version number of the decision threshold.
[0032] Pre-setting a specified message format ensures that inference messages have a unified message format, which at least has the following benefits: 1. Data consistency: The same message format can be used during both time-based rolling evaluation and incremental training, which can reduce system complexity and improve the compatibility of different processing components. 2. Complete information: The specified message format should include at least the input feature data, inference result field, timestamp and version number, to facilitate subsequent rolling evaluation, triggering model adjustment and incremental training; 3. Version Management: The version number in the message enables the system to track data changes and model iterations, facilitating A / B testing, rollback, and traceability; 4. Traceability: Samples can be tracked throughout the entire system lifecycle using version numbers and timestamps.
[0033] In one embodiment of the present invention, the generated inference message can be placed in a message queue for consumption during the time-rolling evaluation phase.
[0034] Then, explanations are given for step 102, "Calculate the performance index of the malware detection model based on the inference messages within the time window, and trigger the model adjustment decision when the performance index does not meet the performance requirements" and step 104, "Obtain risk samples based on inference messages, use the risk samples to update the current version of the malware detection model, and use the updated malware detection model for subsequent online inference".
[0035] In this embodiment of the invention, the time window can be a fixed time window, or the time window can be a time window that is dynamically adjusted based on system resources or model performance.
[0036] Performance metrics for malware detection models can include classification accuracy and false positive rate. When performance metrics deteriorate, automatic model updates are required. The degree of performance degradation is determined by the difference between the calculated performance metric and the target performance metric.
[0037] In this embodiment of the invention, the performance metrics used for time-rolling evaluation are calculated based on verified inference results.
[0038] Specifically, the inference message includes the inference results output by the model in real time, as well as empirically verified inference results formed after obtaining external feedback, alternative analysis process results, or manual review results.
[0039] When the true label of a sample is not available within the current time window, the inference message corresponding to the sample may be temporarily excluded from performance index calculation, or it may be included in the performance evaluation of the corresponding time window in a subsequent time window when the inference result is verified to be available.
[0040] Through the above methods, the present invention can ensure the consistency and comparability of performance index calculations during time-rolling evaluation in practical application scenarios with label delay and feedback noise, thereby improving the stability of model update decisions.
[0041] In this embodiment of the invention, when performing rolling evaluation over time, the inference messages are first filtered according to the version number in the inference message, and only inference messages with the same model version number and decision threshold version number are selected as valid evaluation samples within the current time window.
[0042] By calculating performance metrics for inference messages with the same version number, this embodiment of the invention can avoid cross-version data mixing caused by model hot updates, threshold adjustments, or parameter calibration, thereby ensuring the comparability of performance evaluation results and the accuracy of update decisions within different time windows.
[0043] In one embodiment of the present invention, the model update decision can be a hierarchical update method performed according to different conditions. Specifically: Online calibration is triggered when the performance metric deteriorates below the first-level threshold. When the performance metric deteriorates to the first-level threshold but is below the second-level threshold, a decision threshold reassessment is triggered. Mini-batch incremental training is triggered when the performance metric deteriorates to or exceeds the second-level threshold.
[0044] The following sections will explain each of these three model update methods.
[0045] First method: Online calibration Online calibration primarily addresses the issue of inaccurate confidence scores in model outputs. The calibration goal is to align the model's confidence scores with the actual accuracy, without altering the model parameters. Specific calibration methods include using logistic regression to fit the model's output probabilities to the true labels, or adjusting the temperature parameters of the softmax function using a neural network.
[0046] It should be noted that during the online calibration process, in addition to risky samples, correct samples can also be randomly selected to ensure the balance of the updated model.
[0047] Since online calibration is a post-processing method and can be updated quickly, it will not affect the actual use of the model and can achieve online updates without downtime.
[0048] The second method: Decision threshold reassessment Decision threshold re-estimation redetermines the confidence threshold based on a relatively reliable confidence level and the latest business objectives or data distribution. The decision threshold re-estimation process updates the business decision rules; after redetermining the confidence threshold, the model parameters remain unchanged, thus enabling online, non-disruptive model updates.
[0049] Specifically, the precision-recall curve or ROC curve can be redrawn using risk samples, and the optimal confidence threshold can be determined based on business needs.
[0050] The third method: incremental training with small batches Mini-batch incremental training requires the model to learn new knowledge or correct systematic biases, and the model parameters will change after mini-batch incremental training. The mini-batch training process can utilize risk samples and randomly sampled normal samples to incrementally train the model.
[0051] To ensure that the model can be updated online without downtime even after small-batch incremental training, in one embodiment of the present invention, hot update technology can be used to load the incrementally trained malware detection model into the online inference service to replace the old version of the malware detection model.
[0052] Furthermore, if an anomaly occurs after the new model is loaded into the online inference service, the system can roll back to the previous stable version to ensure service continuity.
[0053] To further explain the incremental training process of this embodiment of the invention, the training method of the malware detection model of this embodiment of the invention will be explained below.
[0054] In one embodiment of the present invention, the training method for the malware detection model can be: The software's binary file is converted into an encoded image, visual features are extracted from the encoded image, input feature data for the sample is determined based on the visual features, and the software's actual recognition results are used as output to train the malware detection model.
[0055] Since the byte sequence of a binary file contains the essential characteristics of the software, by converting these essential characteristics into an encoded image, visual features are extracted from the encoded image, and then the input feature data is determined based on the visual features. During the model training process, the input feature data of the model contains these essential characteristics, which enables the model to learn these essential characteristics in the software binary file, and thus more accurately identify whether the software is malicious or normal.
[0056] In one embodiment of the present invention, converting the binary file of the software into an encoded image specifically includes: A1: Determine the mapping relationship between binary bytes and image pixels, and determine the encoding method of structural information.
[0057] In this embodiment of the invention, the mapping relationship between binary bytes and image pixels can include single-channel mapping and multi-channel mapping. Single-channel mapping encodes one byte into one pixel; multi-channel mapping encodes multiple bytes (three bytes for RGB channels and four bytes for RGBA channels) into one pixel.
[0058] In one embodiment of the present invention, the structural information includes: segment information and / or high-entropy regions.
[0059] In this embodiment of the invention, the structure of a binary file typically includes multiple distinct parts, such as code segments and data segments, each referred to as a segment. The location and content of segments within the binary file are crucial for malware identification. In this embodiment of the invention, by preserving segment information and optimizing it through an attention mechanism, the malware detection system can better understand the structural characteristics of the binary file.
[0060] In one implementation, segment information includes segment boundaries. Segment boundaries are physical separation points between different segments in a binary file, used to distinguish different parts of the binary file. In another implementation, segment information also includes segment types. Segment types can include: .text segment, .data segment, .bss segment, .rodata segment, .stack segment, and .heap segment.
[0061] High-entropy regions typically contain complex or random content and are often associated with malicious behavior. In this embodiment of the invention, high-entropy regions of the binary file are determined by calculating the byte probability distribution of each window region in the binary file one by one. The attention mechanism is optimized by extracting entropy features to accurately identify key parts of malicious files.
[0062] It is understandable that the encoding methods for structural information include the encoding of segmental information and the encoding methods for high-entropy regions.
[0063] Specifically, the segment information is encoded in at least one of the following ways: each segment occupies a different area in the encoded image; different segments are distinguished by pixel intervals or line intervals; and different segment types are encoded using different colors.
[0064] Encoding high-entropy regions can be achieved by setting the pixel positions of the encoded image corresponding to the byte range of the high-entropy region in the binary file as specific feature markers. These feature markers include specific colors and / or specific shapes. This allows the malware detection model to focus more on these high-entropy regions during training, improving the accuracy of malware detection.
[0065] A2: Encode the byte sequence in the binary file into the image according to the mapping relationship and encoding method to obtain the encoded image; It should be noted that, in order to ensure the structural fidelity of the binary file content, in one embodiment of the present invention, the line width of the encoded image corresponding to different binary files is adaptively obtained based on the size and structure of the binary file. The line width of the encoded image is the minimum value that can carry the binary file, and the line width is within a preset line width range. Furthermore, the size of the encoded image satisfies a preset aspect ratio. In this way, the loss of important information caused by a fixed line width can be avoided.
[0066] In this embodiment of the invention, when determining the input feature data of a sample based on visual features, the visual features can be directly used as the input feature data of the sample, or the text features of the security context can be fused with the visual features, and the fused features can be used as the input feature data.
[0067] The above describes the training method for the malware detection model. The incremental training process of this embodiment will be further explained below.
[0068] In this embodiment of the invention, in order to improve the ability of the malware detection model to cope with new attack patterns, not only risky samples can be added, but also new security contexts can be added to perform incremental training on the model.
[0069] One implementation involves incrementally training the model using both risky samples and randomly sampled normal samples. Specifically, this can include: The incremental training is performed using the security context to preserve features of normal samples.
[0070] The security context includes at least one or more of the following: threat intelligence summary, recent detection results, family characteristics associated with malware, and historical information associated with malware. The threat intelligence summary may include known characteristics, attack groups, industry trends, etc. The recent detection results may include alerts, correlation data, sandbox analysis, etc. Family characteristics may include behavior, code, structure, etc. The historical information may include infection, handling, and propagation information, etc.
[0071] The security context of this invention is used to guide the detection decisions of the model, enabling it to better adapt to different malware samples and new attack methods.
[0072] In this embodiment of the invention, feature preservation processing of normal samples using a security context may include: determining the degree of association between each input feature data in the normal sample and the security context, and determining the weight score of each input feature data; retaining the input feature data with a weight score greater than a weight threshold to obtain the preserved features of the normal sample. The degree of association can be calculated using similarity.
[0073] Furthermore, to ensure that key features are not lost during feature preservation, key features can be filtered out from normal samples before feature preservation processing is performed on normal samples. After obtaining the preserved features of the normal sample, the key features are added to the preserved features. In one implementation, the key feature is the structural information of a binary file.
[0074] Please refer to Figure 2 This invention provides a streaming model update device based on time-rolling evaluation, the device comprising: The generation unit 200 is used to perform online inference using the current version of the malware detection model to generate inference results, and to generate inference messages according to a specified message format based on the inference results; the specified message format includes at least: input feature data, inference result fields, timestamps and version numbers; The computing unit 202 is used to calculate the performance index of the malware detection model based on the inference messages within the time window, and to trigger a model update decision when the performance index is detected to be unsatisfactory. The update unit 204 is used to obtain risk samples based on inference messages, and use the risk samples to update the current version of the malware detection model so as to use the updated malware detection model for subsequent online inference.
[0075] In one embodiment of the present invention, when the generation unit performs the step of generating an inference message according to the specified message format based on the inference result, it specifically includes: determining whether the sample for online inference in this instance is a risk sample of the model based on the inference result; if so, routing the risk sample to a backup analysis process to re-obtain the empirical inference result of the risk sample using the backup analysis process, and generating an inference message according to the specified message format based on the verified inference result; otherwise, generating a corresponding inference message for the sample according to the specified message format.
[0076] In one embodiment of the present invention, when the generation unit performs the step of determining whether the sample for online inference is a risk sample of the model based on the inference result, it specifically includes: determining whether the prediction confidence is less than a confidence threshold based on the prediction confidence in the inference result; if so, the sample is determined to be a risk sample.
[0077] In one embodiment of the present invention, when the generation unit performs the step of determining whether the sample for online inference is a risk sample of the model based on the inference result, it specifically includes: determining, based on the inference result and external feedback, that the sample is a high-confidence error sample with a prediction confidence of not less than a confidence threshold and that the sample was inferred incorrectly, and thus determining the sample as a risk sample.
[0078] In one embodiment of the present invention, the time window is a fixed time window, or a time window that is dynamically adjusted based on system resources or model performance.
[0079] In one embodiment of the present invention, when the computing unit executes the decision to trigger a model update when the performance metric is detected to be unsatisfactory, it specifically includes: triggering online calibration when the degree of degradation of the performance metric is lower than a first-level threshold; triggering decision threshold reassessment when the degree of degradation of the performance metric reaches the first-level threshold but is lower than a second-level threshold; and triggering mini-batch incremental training when the degree of degradation of the performance metric reaches or exceeds the second-level threshold.
[0080] It should be noted that the streaming model update device based on time-rolling evaluation provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the streaming model update device based on time-rolling evaluation provided in the above embodiments and the streaming model update method embodiments based on time-rolling evaluation belong to the same concept. The specific implementation process is detailed in the method embodiments and will not be repeated here.
[0081] Embodiments of this application also provide a computer device, please refer to... Figure 3The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by the processor to implement the streaming model update method based on time-rolling evaluation provided in the above-described method embodiments.
[0082] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the streaming model update method based on time-rolling evaluation provided in the above-described method embodiments.
[0083] Embodiments of this application also provide a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium and executes the computer program, causing the computer device to perform the streaming model update method based on time-rolling evaluation as described in any of the above embodiments.
[0084] For ease of description, the above systems or devices are described separately as various modules or units based on their functions. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components.
[0085] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.
[0086] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0087] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A streaming model update method based on time-rolling evaluation, characterized in that, The method includes: The current version of the malware detection model is used to perform online inference to generate inference results. Based on the inference results, an inference message is generated according to a specified message format. The specified message format includes at least: input feature data, inference result field, timestamp, and version number. Based on the inference messages within the time window, calculate the performance metrics of the malware detection model, and trigger a model update decision when the performance metrics are found to be unsatisfactory. Risk samples are obtained based on inference messages. The current version of the malware detection model is updated using the risk samples, and the updated malware detection model is then used for subsequent online inference.
2. The method according to claim 1, characterized in that, The step of generating a reasoning message according to a specified message format based on the reasoning result includes: Based on the reasoning results, determine whether the samples used for online reasoning in this instance are risk samples for the model; If so, the risk sample is routed to the backup analysis process to re-acquire the empirical inference result of the risk sample using the backup analysis process, and an inference message is generated according to the specified message format based on the verified inference result. Otherwise, a corresponding inference message is generated for the sample according to the specified message format.
3. The method according to claim 2, characterized in that, The step of determining whether the sample used for online inference in this instance is a risky sample for the model, based on the inference results, includes: Based on the prediction confidence in the inference results, determine whether the prediction confidence is less than the confidence threshold; if so, determine that the sample is a risky sample.
4. The method according to claim 2, characterized in that, The step of determining whether the sample used for online inference in this instance is a risky sample for the model, based on the inference results, includes: Based on the reasoning results and external feedback, the sample was determined to be a high-confidence error sample with a prediction confidence of not less than the confidence threshold and a reasoning error that was made incorrect. The sample was then identified as a risk sample.
5. The method according to claim 1, characterized in that, The time window can be a fixed time window, or a time window that is dynamically adjusted based on system resources or model performance.
6. The method according to any one of claims 1-5, characterized in that, The step of triggering a model update decision when performance metrics are detected as not meeting performance requirements includes: Online calibration is triggered when the performance metric deteriorates below the first-level threshold. When the performance metric deteriorates to the first-level threshold but is below the second-level threshold, a decision threshold reassessment is triggered. Mini-batch incremental training is triggered when the performance metric deteriorates to or exceeds the second-level threshold.
7. A streaming model update device based on time-rolling evaluation, characterized in that, The device includes: The generation unit is used to perform online inference using the current version of the malware detection model to generate inference results, and to generate inference messages according to a specified message format based on the inference results; the specified message format includes at least: input feature data, inference result fields, timestamp, and version number; The computing unit is used to calculate the performance metrics of the malware detection model based on inference messages within a time window, and to trigger a model update decision when the performance metrics are detected to be unsatisfactory. The update unit is used to obtain risk samples based on inference messages, and use the risk samples to update the current version of the malware detection model so that the updated malware detection model can be used for subsequent online inference.
8. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the steps of the method according to any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of claims 1-6.
10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-6.