Sample stream data concept drift processing method and system based on deep neural network
By constructing a convolutional neural network model based on deep neural networks, and combining ARF algorithm and background tree detection concept offset, the model is updated using adaptive learning strategies, the problem of concept drift of power sample flow data is solved, and the real-time adaptability and prediction accuracy of the model are improved.
Patent Information
- Application Number
- CN202411557599.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-04
- Publication Date
- 2025-04-08
AI Technical Summary
Due to factors such as seasonal changes and technological development, power sample flow data leads to changes in data distribution and characteristics, and concept drift occurs, affecting the model prediction effect.
A deep neural network-based method is adopted to obtain sample flow data, preprocess, standardize and feature extraction, build a convolutional neural network model, and use ARF algorithm and background tree to detect concept offsets, and use adaptive learning strategies to update the model, adjust the network in real time to deal with data changes.
It effectively overcomes the problem of concept drift of power sample flow data, improves the real-time adaptability and prediction accuracy of the model, and ensures the stability of model performance in a dynamic environment.
Smart Images
Figure CN119782695A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of mining power sample stream data, and in particular to a method and system for processing concept drift of sample stream data based on a deep neural network. Background Art
[0002] In today's information age, with the popularity of big data and real-time applications, the processing of sample stream data has become an important and complex issue. Sample stream data usually appears in a continuous and constantly generated form, such as sensor data, social media streams, log information, etc., so its distribution and characteristics may change over time. Streaming data includes stable data streams and dynamic data streams. Dynamic data streams have strong real-time characteristics, and data are not independently distributed, which makes data distribution easy to change and concept drift occurs.
[0003] Concept drift describes the unforeseen changes in the underlying distribution of streaming data over time. If concept drift occurs, the induced patterns of past data may be irrelevant to new data, resulting in poor model prediction results. Concept drift refers to the statistical properties of the target variable that the model attempts to predict changing over time in an unforeseen way.
[0004] Concept drift of streaming data refers to the phenomenon that the data distribution or the concepts that generate the data change over time when processing continuously generated data streams. In short, the nature, distribution, or relationship of data may change over time, thus affecting the performance of the model. Concept drift is one of the common challenges in real-time applications and online learning scenarios.
[0005] Concept drift may be caused by changes in the external environment, instability in the data generation process, adjustments in business needs, and other reasons. Dealing with concept drift in streaming data is an important research direction in the field of machine learning, involving technologies such as online learning, adaptive learning, and real-time monitoring. Effectively dealing with concept drift is the key to maintaining stable model performance in a dynamic environment.
[0006] In the network training process using power sample stream data, due to factors such as seasonal changes and the development of science and technology industries, the power sample stream data in some areas will undergo significant changes, resulting in concept drift in the power sample stream data. Summary of the invention
[0007] The purpose of the embodiments of the present invention is to provide a method and system for processing concept drift of sample stream data based on a deep neural network, which method and system can improve and overcome the problem of concept drift in power sample stream data.
[0008] In order to achieve the above object, an embodiment of the present invention provides a method for processing concept drift of sample stream data based on a deep neural network, comprising:
[0009] Get sample stream data;
[0010] Performing preprocessing operations, standardization operations, and feature extraction operations on the sample stream data in sequence;
[0011] Build a convolutional neural network model;
[0012] Using the sample stream data to train the neural network model, and determining whether concept shift occurs in the current sample stream data;
[0013] In the case where it is determined that a concept shift occurs in the sample stream data, an adaptive learning strategy is adopted to update the convolutional neural network model;
[0014] Calculate the accuracy, precision and recall of the convolutional neural network model respectively;
[0015] The effectiveness of the adaptive learning strategy is determined based on the accuracy, precision and recall.
[0016] Optionally, the sample stream data is sequentially subjected to a preprocessing operation, a standardization operation, and a feature extraction operation, including:
[0017] Determining missing values in the sample stream data;
[0018] Determine whether the length of the missing value is greater than or equal to a preset first length threshold;
[0019] If it is determined that the length of the missing value is greater than or equal to the first length threshold, deleting the missing value;
[0020] When it is determined that the length of the missing value is less than the first length threshold, fill the missing value using a linear interpolation method or a polynomial interpolation method;
[0021] Determine outliers in the sample stream data using at least one of a standard deviation method, a percentile method, and a model-based method;
[0022] The outliers are corrected using at least one of a standard deviation method, a percentile method, and a model-based method.
[0023] Optionally, the sample stream data is sequentially subjected to a preprocessing operation, a standardization operation, and a feature extraction operation, including:
[0024] Sorting the sample stream data in chronological order;
[0025] Performing a data format conversion operation on the sorted sample stream data;
[0026] The sample stream data after the data format conversion operation is subjected to data smoothing and data denoising operations.
[0027] Optionally, the convolutional neural network model includes:
[0028] A first convolution layer, used to perform a first convolution operation on the input sample to obtain a first feature;
[0029] A first maximum pooling layer, used for performing a first maximum pooling operation on the first feature to obtain a second feature;
[0030] A second convolution layer, used for performing a second convolution operation on the second feature to obtain a third feature;
[0031] A second maximum pooling layer, used for performing a second maximum pooling operation on the third feature to obtain a fourth feature;
[0032] A one-dimensional convolution layer, used for performing a one-dimensional convolution operation on the fourth feature to obtain a fifth feature;
[0033] The fully connected layer is used to perform a classification operation according to the fifth feature to obtain a model output.
[0034] Optionally, the first convolution layer includes 8 convolution kernels of size 128*128 connected in sequence, the second convolution layer includes 8 convolution kernels of size 64*64 connected in sequence and 24 convolution kernels of size 48*48, and the one-dimensional convolution layer includes 24 convolution kernels of size 16*16 connected in sequence.
[0035] Optionally, determining whether a concept shift occurs in the current sample stream data includes:
[0036] Using the basic tree of the ARF algorithm to perform a preliminary detection on the input sample stream data to determine whether the sample stream data may have a concept shift;
[0037] In the case that the sample stream data may have a concept shift, a background tree is constructed to determine whether the sample stream data has definitely had a concept shift.
[0038] Optionally, the convolutional neural network model is updated using an adaptive learning strategy, including:
[0039] The background tree is used to replace the classifier of the fully connected layer.
[0040] Optionally, respectively calculating the accuracy, precision and recall of the convolutional neural network model, including:
[0041] The accuracy is calculated according to formula (1):
[0042]
[0043] Wherein, acc is the accuracy rate, TN represents the number of results that are actually negative and predicted to be negative, TP represents the number of results that are actually positive and predicted to be positive, FN represents the number of results that are actually positive but predicted to be negative, and FP represents the number of results that are actually negative but predicted to be positive;
[0044] The accuracy is calculated according to formula (2):
[0045]
[0046] Wherein, precision is the accuracy rate;
[0047] The recall rate is calculated according to formula (3):
[0048]
[0049] Wherein, recall is the recall rate.
[0050] On the other hand, the present invention also provides a sample stream data concept drift processing system based on deep neural network, the system includes a processor, and the processor is configured to execute any of the methods described above.
[0051] Through the above technical solution, the embodiment of the present invention provides a method and system for processing concept drift of sample stream data based on deep neural network. The method and system process the sample stream data and judge in real time whether concept drift occurs in the sample stream data during network training. Finally, in the case of concept drift, an adaptive strategy is adopted to adjust the network model, thereby overcoming the defect of concept drift of sample stream data.
[0052] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:
[0054] Figure 1 is a flowchart of a method for processing sample stream data concept drift based on a deep neural network according to an embodiment of the present invention;
[0055] Figure 2 is a structural block diagram of a convolutional neural network based on a deep neural network according to an embodiment of the present invention;
[0056] Figure 3 It is a schematic diagram of a convolutional neural network based on a deep neural network according to an embodiment of the present invention. DETAILED DESCRIPTION
[0057] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.
[0058] like Figure 1 FIG. 1 is a flowchart of a method for processing sample stream data concept drift based on a deep neural network according to an embodiment of the present invention. Figure 1 In the method, the method may include the following steps:
[0059] In step S10, sample stream data is obtained;
[0060] In step S11, the sample stream data is sequentially subjected to preprocessing, standardization and feature extraction operations;
[0061] In step S12, a convolutional neural network model is constructed;
[0062] In step S13, the neural network model is trained using the sample stream data to determine whether the current sample stream data has a concept shift;
[0063] In step S14, when it is determined that the sample stream data has a concept shift, an adaptive learning strategy is used to update the convolutional neural network model;
[0064] In step S15, the accuracy, precision and recall of the convolutional neural network model are calculated respectively;
[0065] In step S16, the effectiveness of the adaptive learning strategy is determined based on the accuracy, precision and recall.
[0066] In this Figure 1 In the method shown, step 10 is used to obtain sample stream data. The sample stream data may be, for example, power load data of a certain region, power dispatch data, etc. The sample stream data may be obtained by directly accessing a power system server.
[0067] Step S11 can be used to perform preprocessing operations, standardization operations, and feature extraction operations on the sample stream data in sequence. The specific method of the preprocessing operation can be a variety of forms known to those skilled in the art. In one example of the present invention, the preprocessing operation can be to process missing values and abnormal values of the sample stream data. Further, for the missing values in the sample stream, a first length threshold can be first preset, and then it is determined whether the length of the missing value is greater than or equal to the first length threshold. In the case where it is determined that the length of the missing value is greater than or equal to the first length threshold, it means that the data where the missing value is located is difficult to fill, so the missing value and the corresponding data can be directly deleted. On the contrary, in the case where it is determined that the length of the missing value is less than the first length threshold, it means that the missing value can be filled by the point values on both sides of the association, so the linear difference or polynomial difference method can be used to fill the missing value.
[0068] In this example, for outliers in the sample stream data, the standard deviation method, percentile method, and model-based method may be used to identify and process them. The model-based method may be to identify the outliers by pre-training a network model for identifying outliers, and then to correct the outliers by pre-training a network model for correcting outliers.
[0069] The specific method of the standardization operation can also be in various forms known to those skilled in the art. In one example of the present invention, the standardization operation can be to first sort the sample stream data in chronological order, then perform a data format conversion operation on the sorted sample stream data, and finally perform a data smoothing operation and a data denoising operation on the sample stream data after the data format conversion operation. The data smoothing operation and the data denoising operation can be, for example, to smooth the sample stream data using a sliding window, an exponentially weighted moving average, or the like, so as to reduce the influence of noise.
[0070] Step S12 can be used to construct a convolutional neural network model. The specific structure of the convolutional neural network model can be in various forms known to those skilled in the art. In one example of the present invention, Figure 2 and Figure 3As shown, the convolutional neural network may include a first convolutional layer 01, a first maximum pooling layer 02, a second convolutional layer 03, a second maximum pooling layer 04, a one-dimensional convolutional layer 05, and a fully connected layer 06. Among them, the first convolutional layer 01 can be used to perform a first convolution operation on the input sample (sample stream data) to obtain a first feature. The first maximum pooling layer 02 can be used to perform a first maximum pooling operation on the first feature to obtain a second feature. The second convolutional layer 03 can be used to perform a second convolution operation on the second feature to obtain a third feature. The second maximum pooling layer 04 can be used to perform a second maximum pooling operation on the third feature to obtain a fourth feature. The one-dimensional convolutional layer 05 can be used to perform a one-dimensional convolution operation on the fourth feature to obtain a fifth feature. The fully connected layer 06 can be used to perform a classification operation according to the fifth feature to obtain a model output.
[0071] In this Figure 3 In the example, the first convolution layer 01 may include 8 sequentially connected convolution kernels of size 128*128, the second convolution layer 03 may include 8 sequentially connected convolution kernels of size 64*64 and 24 sequentially connected convolution kernels of size 48*48, and the one-dimensional convolution layer 05 may include sequentially connected 24 sequentially connected convolution kernels of size 16*16. In the fully connected layer 06, the tanh function or the softmax function may be used as the activation function. Specifically, the tanh function may be expressed by the following formula (1) and formula (2):
[0072]
[0073] Where z is the output of the neuron, n is the number of input variables, and w j is the weight of the jth input variable, b is the bias, and σ(z) is the output of the activation function.
[0074] In addition, considering that most of the sample stream data of electricity are time stream sequences, in order to highlight the important information of some time periods in the time stream sequence during training, the fully connected layer 06 can further include an attention mechanism. Specifically, the attention mechanism can be determined by the following formulas (3) to (6):
[0075] e t,k-1 =f(h t ,s k-1 ),t∈[1,T], (3)
[0076] e tk-1 =softmax(e t,k-1 ), (4)
[0077] Among them, h t is the hidden state at time t, s k-1is the corresponding (k-1th) output vector, T is the total number of layers, e tk-1 Represents the probability distribution of the input vector.
[0078]
[0079] Among them, l t is the cumulative value of the probability distribution, s t represents the output vector at time t, is the function combination of the accumulated value at time t and the output vector at time t-1, g is the combination representation of the network function, s t Represents the output vector at time t.
[0080] Step S13 can be used to determine whether the current sample stream data has a concept shift. The specific method for determining the concept shift can be a variety of forms known to those skilled in the art. In one example of the present invention, the specific method for determining the concept shift can be to use the base tree of the ARF algorithm to perform a preliminary detection on the input sample stream data to determine whether the sample stream data may have a concept shift. In the case where the sample stream data may have a concept shift, a background tree is constructed to determine whether the sample stream data is determined to have a concept shift. Specifically, the background tree and the base tree can be expressed using the following formulas (7) and (8),
[0081]
[0082] Among them, λ * represents the target parameter, N is the total number of data, is the number of samples correctly classified, λ is the regularization parameter of the model, and h i To represent the i-th hypothesis, is the number of samples misclassified, b is the target variable to be calculated, n is the total number of samples, Acc E is the accuracy of the model.
[0083] When it is determined that the sample stream data has a concept shift, an adaptive learning strategy can be used to update the convolutional neural network model. Specifically, in this embodiment, the background tree can be used to replace the classifier of the fully connected layer.
[0084] Step S15 can be used to calculate the accuracy, precision and recall of the convolutional neural network model. The specific calculation method of the accuracy, precision and recall can be various forms known to those skilled in the art. In one example of the present invention, the accuracy, precision and recall can be calculated using the following formulas (9) to (11):
[0085]
[0086] Among them, acc is the accuracy rate, TN represents the number of results that are actually negative and predicted to be negative, TP represents the number of results that are actually positive and predicted to be positive, FN represents the number of results that are actually positive but predicted to be negative, and FP represents the number of results that are actually negative but predicted to be positive;
[0087]
[0088] Among them, precision is the accuracy;
[0089]
[0090] Among them, recall is the recall rate.
[0091] On the other hand, the present invention also provides a sample stream data concept drift processing system based on deep neural network, the system includes a processor, and the processor is configured to execute any of the methods described above.
[0092] Through the above technical solution, the embodiment of the present invention provides a method and system for processing concept drift of sample stream data based on deep neural network. The method and system process the sample stream data and judge in real time whether concept drift occurs in the sample stream data during network training. Finally, in the case of concept drift, an adaptive strategy is adopted to adjust the network model, thereby overcoming the defect of concept drift of sample stream data.
[0093] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0094] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0095] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0097] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0098] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0099] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0100] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0101] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.
Claims
1. A method for processing concept drift of sample stream data based on deep neural network, characterized in that: The method comprises: Get sample stream data; Performing preprocessing operations, standardization operations, and feature extraction operations on the sample stream data in sequence; Build a convolutional neural network model; Using the sample stream data to train the neural network model, and determining whether concept shift occurs in the current sample stream data; In the case where it is determined that a concept shift occurs in the sample stream data, an adaptive learning strategy is adopted to update the convolutional neural network model; Calculate the accuracy, precision and recall of the convolutional neural network model respectively; The effectiveness of the adaptive learning strategy is determined based on the accuracy, precision and recall.
2. The method according to claim 1, characterized in that The sample stream data is sequentially subjected to preprocessing operations, standardization operations, and feature extraction operations, including: Determining missing values in the sample stream data; Determine whether the length of the missing value is greater than or equal to a preset first length threshold; If it is determined that the length of the missing value is greater than or equal to the first length threshold, deleting the missing value; When it is determined that the length of the missing value is less than the first length threshold, fill the missing value using a linear interpolation method or a polynomial interpolation method; Determine outliers in the sample stream data using at least one of a standard deviation method, a percentile method, and a model-based method; The outliers are corrected using at least one of a standard deviation method, a percentile method, and a model-based method.
3. The method according to claim 1, characterized in that The sample stream data is sequentially subjected to preprocessing operations, standardization operations, and feature extraction operations, including: Sorting the sample stream data in chronological order; Performing a data format conversion operation on the sorted sample stream data; The sample stream data after the data format conversion operation is subjected to data smoothing and data denoising operations.
4. The method according to claim 1, characterized in that: The convolutional neural network model includes: A first convolution layer, used to perform a first convolution operation on the input sample to obtain a first feature; A first maximum pooling layer, used for performing a first maximum pooling operation on the first feature to obtain a second feature; A second convolution layer, used for performing a second convolution operation on the second feature to obtain a third feature; A second maximum pooling layer, used for performing a second maximum pooling operation on the third feature to obtain a fourth feature; A one-dimensional convolution layer, used for performing a one-dimensional convolution operation on the fourth feature to obtain a fifth feature; The fully connected layer is used to perform a classification operation according to the fifth feature to obtain a model output.
5. The method according to claim 4, characterized in that The first convolution layer includes 8 convolution kernels of size 128*128 connected in sequence, the second convolution layer includes 8 convolution kernels of size 64*64 connected in sequence and 24 convolution kernels of size 48*48, and the one-dimensional convolution layer includes 24 convolution kernels of size 16*16 connected in sequence.
6. The method according to claim 5, characterized in that Determine whether the current sample stream data has concept drift, including: Using the basic tree of the ARF algorithm to perform a preliminary detection on the input sample stream data to determine whether the sample stream data may have a concept shift; In the case that the sample stream data may have a concept shift, a background tree is constructed to determine whether the sample stream data has definitely had a concept shift.
7. The method according to claim 6, characterized in that Adopting an adaptive learning strategy to update the convolutional neural network model includes: The background tree is used to replace the classifier of the fully connected layer.
8. The method according to claim 1, characterized in that The accuracy, precision and recall of the convolutional neural network model are calculated respectively, including: The accuracy is calculated according to formula (1): Wherein, acc is the accuracy rate, TN represents the number of results that are actually negative and predicted to be negative, TP represents the number of results that are actually positive and predicted to be positive, FN represents the number of results that are actually positive but predicted to be negative, and FP represents the number of results that are actually negative but predicted to be positive; The accuracy is calculated according to formula (2): Wherein, precision is the accuracy rate; The recall rate is calculated according to formula (3): Wherein, recall is the recall rate.
9. A sample stream data concept drift processing system based on deep neural network, characterized in that: The system comprises a processor configured to execute the method according to any one of claims 1 to 8.