Streaming learning method and system based on multi-granularity adaptive window
Through the multi-grained adaptive window adjustment model training window and the model weight is adjusted according to the data distribution offset, the problem of the streaming learning system's accuracy decrease under dynamically changing data is solved, and higher accuracy and stability are achieved.
Patent Information
- Application Number
- CN202510416857.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-03
- Publication Date
- 2025-07-25
AI Technical Summary
Existing streaming learning systems are difficult to effectively cope with the dynamic changes in real-time data flow, resulting in a decrease in model accuracy, especially when data distribution changes, it is difficult to maintain high accuracy quickly and for a long time.
The multi-grained adaptive window method is adopted to set the number of adaptive windows, attenuate the data in the window according to the distribution offset of the labeled data, and adjust the model weight based on the offset distance of the training data distribution when the labeled data flows in, and realize comprehensive prediction.
Improve the accuracy of the streaming learning system without increasing hardware resources, solves the problem of model accuracy decline caused by changes in data distribution, and effectively avoids catastrophic forgetting.
Smart Images
Figure CN120373365A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of computer software, and particularly relates to a streaming learning method and system based on a multi-granularity adaptive window. Background Art
[0002] Nowadays, data mining technology is increasingly applied in various fields to extract knowledge from real-time data generated by Internet of Things sensors, web clicks, and other sources. For example, users can detect network attacks through network traffic and connection types; power plants can supply power according to real-time user consumption; economists can predict economic development trends based on financial markets. The continuously arriving data is crucial for developing robust and accurate artificial intelligence-driven systems.
[0003] Streaming learning has been widely applied to training and prediction scenarios of real-time data. After being deployed, it predicts real-time data streams and updates the model online according to feedback. Compared with traditional batch learning, it does not need to wait for the complete collection of the entire data set, starts the learning process immediately, gradually accumulates knowledge, and continuously optimizes the model. Therefore, it has a faster response ability, lower memory occupancy, and sensitive update ability.
[0004] Existing research has pointed out that streaming data has the significant characteristic of dynamic change, which means that the distribution and classification rules of data may change over time. This requires the data processing system to be able to flexibly adjust the streaming learning model to adapt to these changes. Specifically, due to the randomness of data generation, the instability of data quality, and the trend evolution of data distribution, streaming data shows the characteristics of dynamic change (non-independent and identically distributed). Specifically, slight fluctuations in data distribution may lead to fluctuations in model accuracy, thus reducing the robustness of the model to a certain extent; sudden changes in data distribution may lead to a serious decline in model accuracy; the continuous change of data will lead to the inevitable problem of catastrophic forgetting, bringing the cost of re-learning and a decline in accuracy.
[0005] Streaming data shows dynamic changes over time, especially around the changes in data distribution, such as class imbalance in real-time data, which poses a major challenge to the accuracy of streaming learning. Existing work mainly focuses on accuracy optimization from three aspects: drift detection, data replay, and constrained update. Among them, drift detection work focuses on real-time accuracy, but its processing has a lag; the methods of data replay and constrained update are limited by the generalization performance of the model itself in a complex environment of multi-data distributions and are difficult to be effective for a long time.
[0006] In summary, in the big data era, information is continuously being acquired and processed, and streaming learning is widely applied and concerned. The existing methods for optimizing the accuracy of streaming machine learning systems are difficult to quickly and effectively solve the problem of the accuracy decline of streaming learning when facing dynamically changing data. There is still a lack of optimization methods for the characteristics of real-time data stream changes in streaming learning, as well as training integration methods for multiple models. Summary of the Invention
[0007] Aiming at the problems existing in the prior art, the purpose of the present invention is to provide a streaming learning method and system based on a multi-granularity adaptive window, which can dynamically and adaptively adjust the model training window to match the characteristics of real-time data distribution and achieve accuracy optimization.
[0008] To achieve the above object, the technical solution of the present invention includes the following contents.
[0009] A streaming learning method based on a multi-granularity adaptive window, the method includes:
[0010] Set the number of adaptive windows according to the number of models in the multi-granularity model;
[0011] When labeled data flows into the adaptive window, decay the existing data in the adaptive window according to the distribution offset of the labeled data relative to the data in the adaptive window to generate training data for the corresponding model. When the adaptive window reaches the specified size, train the model based on the training data and save the distribution of the training data in the adaptive window;
[0012] When unlabeled data flows into the adaptive window, obtain the weights of each model in the multi-granularity model based on the offset distance between the unlabeled data and the distribution of the training data in each adaptive window, and combine the prediction results of each trained model on the unlabeled data to obtain the comprehensive prediction result of the unlabeled data.
[0013] Further, the process of calculating the distribution offset of the labeled data relative to the data in the adaptive window includes:
[0014] Calculate the covariance matrix ∑ on a preheating data set, perform eigenvalue decomposition on the covariance matrix ∑, and select the largest d eigenvalues to form the principal component matrix P d ;
[0015] Based on the principal component matrix P d , respectively obtain the principal component information of the labeled data and the principal component information of the data in the adaptive window;
[0016] According to the principal component information of the labeled data and the principal component information of the data in the adaptive window, obtain the distribution offset of the labeled data relative to the data in the adaptive window.
[0017] Further, the existing data in the adaptive window is attenuated according to the distribution offset of the labeled data relative to the data in the adaptive window to generate training data for the corresponding model, including:
[0018] Step 3.1: When time t = 1, store the labeled data in the window; where 1 ≤ t < T, and T represents the survival time of the adaptive window;
[0019] Step 3.2: When time t ≠ 1, store the labeled data in the adaptive window, and attenuate the data in the adaptive window based on the distribution offset of the labeled data relative to the data in the adaptive window;
[0020] Step 3.3: Re - execute Step 3.2 until the adaptive window reaches the specified size or t = T.
[0021] Further, attenuating the data in the adaptive window based on the distribution offset of the labeled data relative to the data in the adaptive window includes:
[0022] Obtain the distribution offset distance di between the data at time t and the data at each time t' in the adaptive window to construct an offset distance set D; where t > t';
[0023] Sort each element in the offset distance set D and calculate the global offset disorder degree β in the adaptive window;
[0024] Obtain the survival duration t of the data at time t' in the adaptive window i ;
[0025] Based on the distribution offset distance d i 、the survival duration t i and the global offset disorder degree β, attenuate the data at each time t' in the adaptive window.
[0026] Further, saving the training data distribution in the adaptive window includes:
[0027] Set a timing or threshold - based trigger condition;
[0028] According to the trigger condition, save the corresponding model and the training data distribution in the adaptive window.
[0029] A streaming learning system based on a multi - granularity adaptive window, the system includes a data offset calculation module, an adaptive window maintenance module, and a multi - granularity model integration prediction module deployed on a computing node, and an adaptive window control module and a model knowledge saving module deployed on a management node; where
[0030] An adaptive window control module, which is used to set the number of adaptive windows according to the number of models in the multi-granularity model;
[0031] A data offset calculation module, which is used to calculate the distribution offset of the labeled data relative to the data within the adaptive window; calculate the offset distance between the unlabeled data and the training data distribution within each adaptive window;
[0032] An adaptive window maintenance module, which is used to decay the existing data within the adaptive window according to the distribution offset of the labeled data relative to the data within the adaptive window when the labeled data flows into the adaptive window, so as to generate the training data for the corresponding model, and when the adaptive window reaches the specified size, train the model based on the training data;
[0033] A model knowledge preservation module, which is used to preserve the training data distribution within the adaptive window;
[0034] A multi-granularity model integrated prediction module, which is used to obtain the weights of each model in the multi-granularity model based on the offset distance between the unlabeled data and the training data distribution within each adaptive window when the unlabeled data flows into the adaptive window, and combine the prediction results of each trained model for the unlabeled data to obtain the comprehensive prediction result of the unlabeled data.
[0035] Further, the adaptive window control module is further used to:
[0036] Preset a threshold;
[0037] Set a timing or a trigger condition based on the threshold;
[0038] Based on the trigger condition, judge whether the adaptive window reaches the specified size.
[0039] An electronic device, the electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the streaming learning method based on multi-granularity adaptive window described in any one of the above is implemented.
[0040] A computer-readable storage medium, characterized in that computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by a processor, the streaming learning method based on multi-granularity adaptive window described in any one of the above is implemented.
[0041] A computer program product, characterized in that when the computer program product runs on a computer device, the computer device is caused to execute the streaming learning method based on multi-granularity adaptive window described in any one of the above.
[0042] Compared with the prior art, the present invention has at least the following beneficial effects.
[0043] 1. The present invention can increase the upper limit of the accuracy rate of a streaming machine learning system without increasing the total amount of available software and hardware resources.
[0044] 2. When the data distribution changes, the present invention can improve the overall accuracy rate of the streaming learning system through the method of multi-granularity model integrated prediction.
[0045] 3. The present invention can effectively solve the problem of catastrophic forgetting by saving the stable model and its data distribution through the model knowledge saving method. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 is a deployment diagram of a streaming learning system based on a multi-granularity adaptive window.
[0047] Figure 2 is a flowchart of a streaming learning method based on a multi-granularity adaptive window. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0048] The present invention will be further described in detail below with reference to the accompanying drawings. The examples given are only used to explain the present invention and are not intended to limit the scope of the present invention.
[0049] As Figure 1 shown, the streaming learning system based on a multi-granularity adaptive window of the present invention, for example, includes a computing node for executing streaming machine learning tasks and a management node for model and window control, and includes a data offset calculation module, an adaptive window maintenance module, a multi-granularity model integrated prediction module, a window control module, and a model knowledge saving module. The deployment methods, main responsibilities, interaction modes, and details of each module are as follows.
[0050] 1. Data offset calculation module.
[0051] The data offset calculation module is deployed on the computing node, accesses real-time prediction and training data, and combines with the historical data distribution and the model training data distribution according to the distribution of the real-time data, and outputs the relative offset situation and severity of the current streaming data distribution. Among them, the specific steps of the data offset calculation module are as follows.
[0052] 1. First, calculate the covariance matrix Σ on a warm-up data set, perform eigen-decomposition, and select the largest d eigenvalues to form the component matrix P d .
[0053] 2. When the data x t at time t flows in, it will be projected by the matrix P d and output the principal component x d .
[0054] 3. Interact with the adaptive window maintenance module to obtain the principal component distribution information of each micro-batch data existing in the adaptive window.
[0055] 4. Interact with the multi-granularity model integrated prediction module to obtain the training data distribution information of the currently deployed model.
[0056] 5. Calculate the deviation situation between the training data and each micro-batch data in the adaptive window.
[0057] 6. Calculate the relative deviation situation between the prediction data and the principal components of the training data trained by the deployed multi-granularity model.
[0058] 7. Output the deviation situation to the adaptive window maintenance module and the multi-granularity model integrated prediction module respectively.
[0059] II. Adaptive window maintenance module.
[0060] The adaptive window maintenance module is deployed on the computing node, receives the real-time training data and its deviation situation output by the data deviation calculation module, attenuates the micro-batch data maintained in the window, outputs data for model training when reaching the specified size or duration, and performs knowledge preservation. Among them, the specific steps of the adaptive window maintenance module are as follows.
[0061] 1. After the real-time training data flows in, check whether the window has reached the specified size or duration. If it has reached, start the model training update process;
[0062] 2. Receive the input of the deviation degree information from the data deviation calculation module, and maintain the deviation distance d between the real-time training data and the data of different batches i in the window i ∈D;
[0063] 3. Sort each element in the deviation distance set D, and calculate the global deviation disorder degree β in the window;
[0064] 4. Attenuate the data of different batches i in the window in turn, considering the relative deviation distance d i , the survival duration t of the window memory i and the current deviation disorder degree β;
[0065] 5. Repeat the above steps 2, 3 and 4 until the window reaches the specified size or duration.
[0066] III. Multi-granularity model integrated prediction module.
[0067] The multi-granularity model integrated prediction module is deployed on the computing node, receives the real-time prediction data and its deviation situation output by the data deviation calculation module, selects appropriate models or knowledge for integrated prediction and outputs the final prediction result. Among them, the specific steps of the multi-granularity model integrated prediction module are as follows.
[0068] 1. Receive the input of the offset degree information between the predicted data of the data offset calculation module and the deployed model.
[0069] 2. After the real-time prediction data flows in, use the deployed streaming learning model to predict it respectively.
[0070] 3. Adopt the Gaussian kernel metric method based on the offset degree of the multi-granularity model as the weight of the model prediction result.
[0071] 4. Output the comprehensive prediction result according to the model prediction result and its weight.
[0072] IV. Window control module.
[0073] The window control module is deployed on the management node, calculates the cluster performance and preset thresholds, and controls the streaming machine learning systems on each computing node to start executing streaming machine learning tasks according to different window sizes. Among them, the specific steps of the window control module are as follows.
[0074] 1. Test the cluster performance information and thresholds, and calculate the adaptive window size and the number of windows.
[0075] 2. Control each computing node to start the adaptive window maintenance module and the corresponding number of model trainings.
[0076] V. Model knowledge preservation module.
[0077] The model knowledge preservation module is deployed on the management node, receives the input of the adaptive window maintenance module, and controls the models on the computing nodes to save knowledge and window data according to the offset trend. Among them, the specific steps of the model knowledge preservation module are as follows.
[0078] 1. Set a timing or threshold-based triggering rule.
[0079] 2. According to the trigger condition, save the corresponding model and the data distribution information within the adaptive window.
[0080] As Figure 2 shown, the present invention also discloses a streaming learning method based on a multi-granularity adaptive window, including the following steps 1) to 9).
[0081] 1) Start the data offset calculation module, the adaptive window maintenance module and the multi-granularity model integrated prediction module on the computing node, and start the window control module and the model knowledge preservation module on the management node.
[0082] 2) The offset calculation module determines the relative offset situation and severity of the current streaming data distribution.
[0083] The offset calculation module first reduces the dimension of the real-time data by using the principal component analysis method. To track the change in the data distribution after dimension reduction, the Euclidean distance dt between the current batch and another batch is calculated, which is the relative offset distance.
[0084] 3) The adaptive window maintenance module attenuates the data saved in the window according to the current streaming data offset situation, thereby reducing the amount of data in the window, while saving the historical data that is closer to the current distribution.
[0085] 3.1) After the real-time data flows in, check whether the window has reached the specified size or duration. If so, start the model training and update process.
[0086] 3.2) After the real-time data flows in, calculate the offset distance di∈D between it and the data of different batches i maintained in the window.
[0087] 3.3) Sort each element in the offset distance set D and calculate the global offset disorder β in the window.
[0088] 3.4) Attenuate the data of different batches i in the window in turn. When attenuating, consider the relative offset distance di, the survival duration ti in the window, and the current offset disorder β.
[0089] 3.5) Repeat steps 3.2), 3.3) and step 3.4) until the window reaches the specified size or duration.
[0090] 4) The multi-granularity model integration prediction module integrates the prediction results of the multi-granularity models trained based on the adaptive window according to the relative offset situation of the current data and outputs the comprehensive prediction result.
[0091] 5) The window control module notifies the streaming machine learning systems on each computing node to start executing the streaming machine learning tasks according to different window sizes.
[0092] 6) The model knowledge preservation module saves the distribution information of the data within a certain duration, calculates the change in the data distribution offset degree within a certain duration, and performs statistical metrics such as sorting and averaging on it, so as to define the data distribution offset trend within the current duration; when the data offset trend is relatively stable, save the relatively stable long-time granularity model; when the data offset trend is relatively changeable, save the relatively sensitive short-time granularity model at the same time.
[0093] 7) After the streaming machine learning task starts to be deployed and executed, the window control module controls each computing node to open adaptive windows of different sizes according to the cluster performance and preset thresholds to support the training of different models; at the same time, the model knowledge preservation module selects a more stable streaming learning model for preservation according to the offset trend.
[0094] 8) The computing node starts the offset calculation module to calculate the offset of real-time data. The offset situation is transmitted to the adaptive window maintenance module in real time. The adaptive window maintenance module attenuates the historical data in the window according to the offset situation. When the window reaches the specified size or duration, it starts the model training and update of the corresponding granularity.
[0095] 9) The multi-granularity model integration prediction module performs integrated prediction based on the offset distance between the data to be predicted at the current moment and the data in the adaptive window corresponding to the model, and outputs the comprehensive prediction result.
[0096] When the real-time data flows in, in the multi-granularity model integration prediction module, the offset distance between the data at the current moment and the data in the window corresponding to the model that is providing the prediction is calculated, and the output results of the multi-granularity models are integrated through a Gaussian kernel metric method and the comprehensive prediction is output.
[0097] Furthermore, in addition to the fixed full-batch window model, a multi-granularity model based on the adaptive window is also maintained. Multiple models provide prediction functions simultaneously and are trained and updated according to the corresponding windows.
[0098] Next, the streaming learning method based on multi-granularity adaptive window proposed by the present invention is applied to streaming machine learning systems such as Flink ML and River. Taking a representative streaming machine learning task as an example, the input data is of numerical type and the micro-batch inflow interval is 1 minute. The specific steps of the present invention are as follows:
[0099] 1. Start the data offset calculation module, the adaptive window maintenance module, and the multi-granularity model integration prediction module on the computing node, and start the window control module and the model knowledge saving module on the management node;
[0100] 2. The window control module sets the number of windows to 2 according to the cluster performance and the preset threshold, and the maximum survival duration of the window is 5 minutes.
[0101] 3. The window control module notifies each computing node to start 2 streaming machine learning models, one of which uses the adaptive window maintenance module. The machine learning system initialization is completed and starts to execute the streaming machine learning task;
[0102] 4. For real-time training data, since the current adaptive window is empty and the survival time is less than 5 minutes, all the data flows in. When the real-time data at T = 2 flows in, calculate the deviation degree between it and the data at T = 1 in the window, and attenuate the data at T = 1 in the window; when the real-time data at T = 3 flows in, calculate the deviation degree between it and the data at T = 1, 2 in the window and the window deviation trend, and attenuate the data at T = 1, 2 in the window; when the real-time data at T = 4 flows in, calculate the deviation degree between it and the data at T = 1, 2, 3 in the window and the window deviation trend, and attenuate the data at T = 1, 2, 3 in the window; when the real-time data at T = 5 flows in, calculate the deviation degree between it and the data at T = 1, 2, 3, 4 in the window and the window deviation trend, and attenuate the data at T = 1, 2, 3, 4 in the window. Since the window survival duration has been reached, trigger the training and update of the model.
[0103] 5. For real-time prediction data, calculate the deviation distance between it and the distribution of the training data of the deployed model, and calculate the model prediction weight based on the deviation distance and the Gaussian kernel metric method. The two deployed models respectively output prediction results, and output the comprehensive prediction result according to the prediction weight.
[0104] 6. When the window reaches the specified survival duration, transmit the deviation trend information in the window to the management node. The management node judges whether the data in the current window is stable according to the threshold. If the data is stable, trigger the model knowledge saving module to save the model and its corresponding training data distribution information.
[0105] Although specific embodiments of the present invention are disclosed for illustrative purposes, the purpose is to help understand the content of the present invention and implement it accordingly. Those skilled in the art can understand that: without departing from the spirit and scope of the present invention and the appended claims, various substitutions, changes and modifications are possible. Therefore, the present invention should not be limited to the content disclosed in the best embodiments, and the scope of protection required by the present invention is subject to the scope defined by the claims.
Claims
1. A streaming learning method based on a multi-granularity adaptive window, characterized in that, The method includes: Setting an adaptive window number according to the number of models in the multi-granularity model; When labeled data flows into the adaptive window, decaying the existing data in the adaptive window according to the distribution offset of the labeled data relative to the data in the adaptive window to generate training data for the corresponding model, and when the adaptive window reaches a specified size, training the model based on the training data and saving the distribution of the training data in the adaptive window; When unlabeled data flows into the adaptive window, obtaining the weights of each model in the multi-granularity model based on the offset distance between the unlabeled data and the distribution of the training data in each adaptive window, and combining the prediction results of each trained model on the unlabeled data to obtain the comprehensive prediction result of the unlabeled data.
2. The method according to claim 1, characterized in that, The process of calculating the distribution offset of the labeled data relative to the data in the adaptive window includes: Calculate the covariance matrix Σ on a preheating dataset, perform eigen decomposition on the covariance matrix Σ, and select the largest d eigenvalues to form the principal component matrix P d ; Based on the principal component matrix P d , respectively obtain the principal component information of the labeled data and the principal component information of the data within the adaptive window; Obtaining the distribution offset of the labeled data relative to the data in the adaptive window according to the principal component information of the labeled data and the principal component information of the data in the adaptive window.
3. The method according to claim 1, wherein Decaying the existing data in the adaptive window according to the distribution offset of the labeled data relative to the data in the adaptive window to generate training data for the corresponding model, including: Step 3.1: When t = 1, storing the labeled data in the window; where 1 ≤ t < T, and T represents the survival time of the adaptive window; Step 3.2: When t ≠ 1, storing the labeled data in the adaptive window and decaying the data in the adaptive window based on the distribution offset of the labeled data relative to the data in the adaptive window; Step 3.3: Re-executing Step 3.2 until the adaptive window reaches the specified size or t = T.
4. The method according to claim 3, characterized in that Decaying the data in the adaptive window based on the distribution offset of the labeled data relative to the data in the adaptive window includes: Obtain the distribution offset distance d of the data at time t relative to the data at each time t′ within the adaptive window i , so as to construct an offset distance set D; where t > t′; Sorting each element in the offset distance set D and calculating the global offset disorder degree β in the adaptive window; Obtain the data at time t′ and the survival duration t in the adaptive window memory i ; Based on the distribution offset distance d i , the survival duration t i and the global offset disorder β, the data at each time t' within the adaptive window is attenuated.
5. The method according to claim 1, characterized in that, Saving the distribution of the training data in the adaptive window includes: Setting a timing or threshold-based trigger condition; According to the trigger condition, saving the corresponding model and the distribution of the training data in the adaptive window.
6. A streaming learning system based on a multi-granularity adaptive window, characterized in that, The system includes a data offset calculation module, an adaptive window maintenance module, and a multi-granularity model integrated prediction module deployed on a computing node, and an adaptive window control module and a model knowledge saving module deployed on a management node; where The adaptive window control module is used to set the number of adaptive windows according to the number of models in the multi-granularity model; The data offset calculation module is used to calculate the distribution offset of the labeled data relative to the data in the adaptive window; calculating the offset distance between the unlabeled data and the distribution of the training data in each adaptive window; The adaptive window maintenance module is used to, when labeled data flows into the adaptive window, decay the existing data in the adaptive window according to the distribution offset of the labeled data relative to the data in the adaptive window to generate training data for the corresponding model, and when the adaptive window reaches a specified size, training the model based on the training data; A model knowledge preservation module, configured to preserve the training data distribution within the adaptive window; A multi-granularity model integration prediction module, configured to, in the case where unlabeled data flows into the adaptive window, obtain the weights of the models in the multi-granularity model based on the offset distance between the unlabeled data and the training data distribution within each adaptive window, and combine the prediction results of each trained model for the unlabeled data to obtain a comprehensive prediction result for the unlabeled data.
7. The system according to claim 6, wherein The adaptive window control module is further configured to: Preset a threshold; Set a timing or a trigger condition based on the threshold; Based on the trigger condition, determine whether the adaptive window reaches a specified size.
8. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the method for streaming learning based on a multi-granularity adaptive window according to any one of claims 1-5 is implemented.
9. A computer-readable storage medium, characterized in that, Computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by a processor, the method for streaming learning based on a multi-granularity adaptive window according to any one of claims 1-5 is implemented.
10. A computer program product, characterized in that, When the computer program product runs on a computer device, the computer device is caused to execute the method for streaming learning based on a multi-granularity adaptive window according to any one of claims 1-5.