20 results about "Concept drifting" patented technology

The invention discloses a pet personalized physical rehabilitation method and system based on multi-modal data fusion. The method comprises the following steps: establishing a digital file of a target pet; according to the method, multi-modal data such as behavior characteristics and physiological indexes are fused, and the space-time correlation is mined by using a multi-head self-attention mechanism, so that the limitation of a single data source can be overcome, the pain level and the joint limitation degree of the pet can be accurately identified, and an objective basis is provided for scheme formulation; based on medical history and rehabilitation sensitivity base lines in a pet digital file, in combination with real-time environment and expression recognition, rehabilitation strength, frequency and aromatic therapy formula are dynamically adjusted, accurate rehabilitation of'one pet and one strategy 'is achieved, and secondary damage or stress to pets caused by a standardized process is avoided; and in combination with expert-algorithm closed-loop verification and concept drift detection, the system can automatically adapt to new cases and environment changes, and the accuracy of rehabilitation effect evaluation and the scientificity of scheme recommendation are continuously improved.

The invention discloses a new energy power prediction model concept drift detection method and system, and belongs to the technical field of new energy power prediction and data flow mining, and the method comprises the steps: collecting high-availability basic operation data of a new energy station in real time, and carrying out the time alignment processing; a preset three-channel collaborative drift detection module is adopted for detection; the three-channel collaborative drift detection module comprises three monitoring channels which run in parallel, and alarm signals corresponding to the channels are output through prediction residual behavior monitoring, actual power distribution drift detection and physical consistency deviation detection. And calculating the confidence coefficient of each channel, carrying out weighted average on the confidence coefficients of the channels to obtain a total drift score, and judging whether the new energy power prediction model generates concept drift according to the total drift score and the corresponding duration. The method has high availability and strong robustness, can realize high-precision and low-false-alarm detection of concept drift, and provides technical support for optimization of a power prediction model.

This invention discloses an anomaly detection method for industrial IoT time-series data based on concept drift identification, belonging to the field of time-series anomaly detection technology. The method includes: performing multi-scale concept drift detection on industrial IoT time-series data to obtain concept drift detection results; updating an adaptive ensemble model based on the concept drift detection results to obtain an optimal anomaly detection model; and using the optimal anomaly detection model to perform anomaly detection on the industrial IoT time-series data to obtain anomaly detection results. This invention, through an innovative multi-scale window analysis mechanism, can identify data distribution changes earlier, effectively shortening the concept drift detection delay; and by combining model pool management and dynamic ensemble learning methods, it optimizes computational efficiency while ensuring detection accuracy, achieving a smooth transition between old and new data distributions.

The application relates to a network traffic concept drift detection method based on a Count-Min sketch data structure and belongs to the network traffic analysis field. The application records the multidimensional statistical information of network traffic through a CM sketch data structure, starts from the multidimensional probability distribution of network flow, monitors the multidimensional Hellinger distance change condition every certain period, performs network traffic concept drift detection, and detects the type of network traffic concept drift based on the Euclidean distance. The application records the multidimensional statistical information of network traffic through a CM sketch data structure, saves the storage space, each dimension is relatively independent, can be processed in parallel, and saves the detection time; starts from the multidimensional probability distribution of network flow, monitors the multidimensional Hellinger distance change condition, performs network traffic concept drift detection, reduces the concept drift false detection rate and the missed detection rate, makes the detection result more accurate; can correctly identify the network traffic concept drift type, discovers new applications and distributed drift applications, and has important significance in network intrusion detection and the like.

The invention belongs to the technical field of wind power plant SCADA (supervisory control and data acquisition) data detection, and particularly relates to a wind power SCADA data online self-adaptive abnormal value detection method considering concept drift, which comprises the following steps: S100, collecting actual wind power in real time through an SCADA system; carrying out pretreatment and rationality screening; s200, calculating the wind power of the unit through a power model, obtaining a prior wind power sequence, and constructing a residual sequence; introducing an input wind speed as a scaling factor to obtain a scaling residual sequence; processing the scaling residual error sequence, and marking an abnormal value according to a preset threshold value; s300, dynamically monitoring the scaling residual error sequence by adopting an exponentially weighted moving average method; triggering a concept drift candidate event when the EWMA value exceeds a control limit UCL; carrying out difference test on the current residual error distribution and the historical reference distribution by adopting KS test, and if the difference exceeds a preset threshold value, confirming that concept drift occurs; and S400, after judging that the concept drift occurs, updating the parameters of the power model.

The application discloses a cold and hot data recognition method and system based on streaming learning, and belongs to the field of computer storage. The system regards cold and hot recognition as a decision problem, extracts multi-dimensional features including data flow, control flow and system information through a feature extraction module to construct a feature vector, realizes online hotness evaluation and real cold and hot label generation through an online label module, adopts a streaming learning algorithm in a cold and hot recognition module, predicts the cold and hot state of a data block in the future in real time according to the feature vector, and regularly updates a model to cope with concept drift. In addition, the system adopts a dynamic adjustment mechanism of cold and hot perception threshold, can adaptively guide data migration, realizes online judgment and labeling of data cold and hot, and provides efficient and adaptive cold and hot data recognition services for user applications.

This invention discloses a business process anomaly detection method based on concept drift discovery, comprising the following steps: 1) collecting data to form an event log; 2) extracting process information using control flow features from the event log, encoding events, and constructing a process feature dataset; 3) building a prediction model for the next event in the business process based on a GRU model, and training the prediction model using the process feature dataset as input data; 4) calculating the anomaly score s of the business process attributes through probability distribution; 5) performing concept drift detection on the anomaly detection results using a concept drift discovery module; and 6) using an incremental learning method to incorporate the drift case set as new knowledge using an event prediction model update module. This method mines process models from event logs without requiring manual judgment to find concept drift cases, enabling more accurate detection of whether anomalies occur in business process instances and locating and determining whether concept drift has occurred.

The application relates to the technical field of network intrusion detection, in particular to an automatic intrusion detection system for a dynamic network environment, which has the technical scheme that in the autonomous decision module of Gaussian probability, a contrast loss function taking normal traffic as the center is designed, so that the model can efficiently distinguish the behavior patterns of normal traffic and abnormal traffic; in the automatic continuous learning framework, a double memory bank is designed to adapt to the concept drift scene in the dynamic network, wherein the stable memory bank is used for storing old knowledge and preventing the catastrophic forgetting of the model, and the high-confidence pseudo label generated in the autonomous decision module of Gaussian probability is used to update the adaptive memory bank, so that the real-time updating and fine-tuning of the autonomous decision module of Gaussian probability are realized; in the continuous learning process, the system does not need to rely on manual labeling, can effectively capture the constantly evolving patterns in the dynamic network scene, significantly enhances the applicability of the intrusion detection system to the concept drift, and realizes automatic intrusion detection.

Facilitating intelligent concept drift mitigation in advanced communication networks is provided herein. A method includes utilizing, by a system comprising a processor, a first model that facilitates management of resources within a communications network. A reliability level of the first model is determined to satisfy a defined reliability level. The method also includes based on a first determination that the reliability level of the first model no longer satisfies the defined reliability level, replacing, by the system, the first model with a second model that temporarily facilitates management of the resources within the communications network. Further, the method includes, based on a second determination that a third model satisfies the defined reliability level, deploying, by the system, the third model within the communications network. The deploying can include incrementally transitioning facilitation of the management of resources from the second model to the third model.

According to the drift-oriented self-evolution encrypted traffic classification method provided by the invention, self-adaptive classification for coping with concept drift in a development world environment is realized, and the life cycle of a classifier is prolonged; the method specifically comprises the following steps: 1) continuously screening out silver samples of which the Softmax confidence coefficient is higher than a threshold value from a prediction result of a trained classifier based on an extended Laida criterion, and providing reliable data support without manual annotation for subsequent model fine tuning; and 2) monitoring the confidence coefficient of Softmax in real time in a classifier prediction process by adopting a method based on window multi-threshold cumulative measurement, accumulating a drift score when the confidence coefficient is lower than a plurality of preset thresholds, and judging that concept drift occurs and triggering fine tuning if a certain category or the whole model reaches a drift score upper limit in a specified time window. And 3) performing unfreezing layer parameter fine tuning on the model by using a silver sample for the detected concept drift category through category-sensitive layered fine tuning, and prolonging the life cycle of the classifier.

The invention provides a concept drift and unknown category-oriented network intrusion detection incremental learning method, which relates to the technical field of network intrusion detection systems and comprises the steps of collecting original network traffic; carrying out data preprocessing, and extracting a spatiotemporal feature vector from the standardized time sequence feature sequence through a spatiotemporal feature extraction network; processing through a multi-classifier module, and outputting a known class recognition result, unknown traffic and a confidence coefficient sequence of all classifiers; unknown traffic is clustered to obtain a new classifier, whether drifting exists or not is judged based on the spatial-temporal feature vectors and the confidence sequence of the classifier, and local parameter fine tuning is conducted on the spatial-temporal feature extraction network and the multi-classifier module when drifting exists. According to the method, the problems of occurrence of unknown attacks in intrusion detection, continuous change of flow distribution and stability of long-term online operation of a model are solved, and expression learning, classification decision making and updating mechanisms are covered under a unified framework at the same time.

The invention provides a smart power grid load prediction system based on edge computing, and relates to the field of edge computing, and the method comprises the steps: enabling an edge node to locally generate and report an edge portrait data packet containing a data feature vector and an equipment capability vector; the cloud scheduling center performs data concept drift and performance bottleneck detection according to the portrait data packet to trigger a model updating event; then, the dispatching center executes a two-stage matching decision: a candidate model conforming to a load scene is screened out from a model library based on a data feature vector, and then hardware resource constraint screening is performed based on an equipment capability vector, so that an optimal model considering both precision and performance is selected; and finally, the system generates a deployment data packet containing a model difference packet, and the edge node only needs to execute difference updating, so that accurate and efficient model adaptation to heterogeneous equipment and a dynamic scene is realized.

The application provides a knowledge expansion-based data framing retrieval method and system, which comprises the following steps: retrieving according to selected keywords to obtain a retrieval result; completing the retrieval when the number of cycles reaches a set value, otherwise continuing to execute; when performing for the first time, the selected keywords are user input keywords; mining and analyzing the retrieval result to extract keywords; analyzing the relationship between the extracted keywords and the selected keywords and the relationship between the extracted keywords, and dividing the extracted keywords into multiple groups according to different relationship types; constructing different new query statements according to different groups, verifying the effectiveness of the keyword relationship, and putting the keywords verified as effective into a keyword library; taking the keywords in the keyword library as the selected keywords, increasing the number of cycles by 1, and re-executing the retrieval. The application has the advantages that it can retrieve and return information as comprehensively and accurately as possible, and solves the problem of retrieval concept drift.

The invention relates to the technical field of artificial intelligence and machine learning, discloses a clustering-assisted teacher-student model semi-supervised streaming adaptive learning method, and aims to effectively classify a small amount of labeled data and a large amount of unlabeled data in a non-stationary data stream scene. The concept drift phenomenon generally existing in the data stream can be adapted; efficient self-adaptive learning and accurate classification of non-stationary data streams are realized through cooperative work of four core modules of integrated concept drift management, dynamic teacher-student learning, clustering enhancement of pseudo tags and self-adaptive memory and playback; through multi-dimensional dynamic monitoring and architecture-parameter collaborative optimization, an efficient and explainable solution is provided for online learning in a complex dynamic environment.

The invention relates to a data stream concept drift coping method and device based on domain adaptation. The method comprises the following steps: acquiring data in a source domain and a target domain; preprocessing the acquired data to ensure data quality and availability; different weights are given to different training samples of the preprocessed data in an instance weighting mode; mapping data in the source domain and the target domain to a shared low-dimensional subspace, and performing subspace alignment; and inputting the data subjected to instance weighting and subspace alignment into a classifier, and classifying the data by using the classifier. The invention provides an effective solution for concept drift in the data stream, model performance can be improved in different fields and tasks, cost and resources are saved, research and innovation in the transfer learning field are promoted, and important technical support is provided for continuously adapting to application scenes of changing environments.

The application discloses a data labeling rule updating method based on incremental clustering and concept drift detection, comprising the following steps: at least two labelers are configured to collaboratively label samples, and a collaborative labeling hypergraph is constructed by taking the association of sample nodes, labeler nodes and rule nodes in each labeling operation as a three-element hyperedge; incremental clustering is performed on subsequent samples to form new sample clusters; two-way projection is performed on the collaborative labeling hypergraph to obtain a sample-rule projection graph and a labeler-rule projection graph; a concept drift criterion is constructed according to a rule distribution gap; a labeler drift criterion is constructed according to the median of a consistency index of labeler nodes; drift events are divided according to the two-component trigger states; and the current effective labeling rule set is retained, added or refined according to review conclusions, and labelers with low group consistency are calibrated. The application separately visualizes and accurately disposes concept drift and labeler drift in geometry, and improves the adaptive ability of data labeling rules.

The embodiment of the invention provides a flow detection method and device, electronic equipment and a storage medium, and relates to the technical field of flow detection, and the method comprises the steps: obtaining the time sequence data of the bandwidth flow of a target domain name to be detected, extracting a target detection segment from the time sequence data, converting the target detection segment into a first time sequence image, and outputting the first time sequence image; then at least one second time sequence image representing a normal traffic mode is obtained, the first time sequence image and the second time sequence image are jointly input into a multi-mode large model for comparative analysis, a corresponding detection result is output, and if the detection result represents that the bandwidth traffic of the target domain name is not abnormal, the first time sequence image and / or the target detection section are / is input into the multi-mode large model for comparative analysis; the data is added into a normal flow mode data set for generating a second time sequence image, so that the limitation of single-mode analysis is effectively overcome; and meanwhile, the problem that the detection accuracy is reduced due to concept drift is effectively solved, and continuous and reliable anomaly detection in a dynamic network environment is realized.

The application discloses a dynamic pruning flow data frequent item set mining method, comprising the following steps: constructing an unbounded data stream and dividing the unbounded data stream into micro batches, establishing a sliding window model and a global candidate domain, initializing a prefix tree, a global state and a historical micro batch queue, and setting related parameters; a client uploads after standardizing preprocessing and local differential privacy disturbance of original records; a server performs local aggregation, differentially updates the global state with constant time complexity based on the sliding window model; the source of frequency fluctuation is discriminated from privacy noise or concept drift through noise interpretable interval discrimination, and according to the discrimination result, an incremental insertion or dynamic physical pruning operation is performed on the prefix tree; and finally, a frequent item set set in a current window is screened and output.

This invention discloses a dynamic ensemble classification method for imbalanced drift industrial data streams, comprising: cold-starting a base classifier by acquiring the label of an instance in the first data block of the data stream and adding it to a dynamic ensemble framework for classifying continuously arriving new instances in the data stream; determining whether a label request is needed for new instances based on a hybrid label request strategy, and placing the instance in a buffer if a label request occurs; randomly sampling newly arriving data blocks to create a new base classifier to replace the old base classifier with low weights; and selecting the best-performing base classifier in the ensemble framework based on dynamic sub-ensemble to participate in the prediction of newly arriving instances. This invention fully considers the relationship between class imbalance and concept drift, integrating class imbalance into various components of the framework, selecting more valuable samples while saving label requests, and improving the classification accuracy of the framework in imbalanced drift industrial data stream scenarios.

This technology provides the ability to automatically set detection thresholds using a common method for operational data acquired from various devices, thereby suppressing unnecessary retraining and enabling the detection of concept drift. [Solution] The control unit of the detection device inputs a set of test data acquired by the first model to calculate a set of first output values, calculates the first mean and first standard deviation of the set of first output values, inputs a predetermined number of acquired operational data to the first or second model to calculate a second mean by averaging a predetermined number of second output values, calculates the second standard deviation of the second mean from the first standard deviation and a predetermined number, sets a drift threshold from the first mean and the second standard deviation, and detects concept drift based on the fact that the calculated second mean is greater than or equal to the set drift threshold.