45 results about "Concept drifting" patented technology

The invention relates to a cross-domain intelligent reasoning method based on knowledge graph enhancement, and the method achieves the precise reasoning in a complex scene through the construction of a hierarchical knowledge expression framework and a dynamic optimization mechanism. A multi-source heterogeneous data fusion technology is adopted, subject fine-grained knowledge units are generated through multi-modal feature extraction, and a three-dimensional knowledge graph structure comprising a core common concept layer, a subject feature ontology layer and a dynamic semantic mapping layer is established; based on a path exploration algorithm driven by reinforcement learning, cross-domain implicit association is mined while subject independence is reserved, and controllability and interpretability of the reasoning process are achieved in combination with an attention fusion mechanism of a large language model. According to the method, the limitation of traditional unified ontology modeling is broken through, the problems of concept drift and path deviation existing in reasoning in the cross fields of medicine-finance, engineering-law and the like are effectively solved, and the accuracy and knowledge traceability of complex decision tasks are remarkably improved.

The invention relates to the technical field of fault detection, in particular to an industrial fault detection method and system based on dynamic drift perception and diffusion enhancement. The method comprises the following steps: constructing an unsupervised fault detection DDA-DE model, and processing an industrial data flow by utilizing dynamic drift awareness DDA to establish a statistical distribution baseline and a drift threshold; determining an initial model parameter and an anomaly threshold by using diffusion enhanced anomaly detection DE; robustness enhancement of concept drift is carried out on the unsupervised fault detection model based on a diffusion strategy; according to the Mahalanobis distance real-time drift sensing algorithm based on industrial enhancement, the concept drift phenomenon can be detected more efficiently, and collaborative detection of data drift and abnormal events is achieved through the parallel design of the drift sensing algorithm and the fault detection classifier.

The invention discloses a PDU load distribution system based on reinforcement learning, relates to the technical field of data center resource management, and realizes efficient scheduling of PDU load, server migration and start-stop through combination of a multi-dimensional execution cost model and a digital twin environment. Energy consumption and performance can be balanced in different load scenes by using offline pre-training and hierarchical control, and continuous optimization is performed through online learning under concept drift detection; in addition, after the safety cost and the fault-tolerant overhead are overlaid, key services and sensitive data can be protected preferentially, safety risks and downtime losses are effectively reduced, and finally multi-dimensional collaborative efficient, safe and extensible data center management is achieved. Meanwhile, resource impact and performance jitter caused by large-scale operation are further avoided through batch migration and staged starting and stopping, flexible response can be achieved under multi-dimensional risks, and it is ensured that scheduling robustness and sustainability are kept under heterogeneous loads.

A method may include: (1) receiving reference data comprising input texts and corresponding labels; (2) training a covariate drift detector comprising a syntactic drift detector and a semantic draft detector with the reference data; (3) training a concept drift detector comprising a plurality of classifiers with the reference data; (4) receiving production data comprising a plurality of instances; (5) determining that the production data has drifted; (6) calculating similarity scores between each instance of the production data and the reference data; (7) detecting concept drift by generating a predictive distribution using the plurality of classifiers and calculating an entropy of the predictive distribution; (8) identifying final drifted instances from the covariate drifted instances and the concept drifted instances; and (9) receiving updated labels for the final drifted instances.

Facilitating intelligent concept drift mitigation in advanced communication networks is provided herein. A method includes utilizing, by a system comprising a processor, a first model that facilitates management of resources within a communications network. A reliability level of the first model is determined to satisfy a defined reliability level. The method also includes based on a first determination that the reliability level of the first model no longer satisfies the defined reliability level, replacing, by the system, the first model with a second model that temporarily facilitates management of the resources within the communications network. Further, the method includes, based on a second determination that a third model satisfies the defined reliability level, deploying, by the system, the third model within the communications network. The deploying can include incrementally transitioning facilitation of the management of resources from the second model to the third model.

The invention relates to the field of state monitoring and fault prediction of offshore wind turbine generators, in particular to an offshore wind turbine generator fault prediction method and system based on data driving. Comprising the following steps: S1, synchronously acquiring equipment state data of a wind turbine generator state monitoring system and environment state data of a marine environment monitoring system; s2, extracting a preliminary engineering feature vector; s3, generating a dynamic feature signature after the normal influence of the environmental factors is eliminated; s4, calculating a comprehensive concept drift index; s5, comparing the concept drift index with a preset adaptive trigger threshold value; s6, when the concept drift index is greater than an adaptive trigger threshold, generating a trigger signal for model updating, and performing unsupervised adaptive updating on the model parameters of the fault prediction model; and when the concept drift index is not greater than the adaptive trigger threshold, maintaining the model parameters of the fault prediction model unchanged. According to the method, the prediction false alarm rate caused by the sudden change of the environment is remarkably reduced, and the recognition capability of early weak faults is improved.

The invention discloses a federal continuous learning-based network intrusion detection method and system. The method comprises the following steps of initializing a global model and training related parameters; broadcasting the global model and training related parameters together; performing data preprocessing on the collected data, updating a local model, and adding old knowledge into training of a new model through knowledge distillation to obtain new model parameters; after the local model is trained, the adaptability of the model to the complex and changeable network environment is improved, and if concept drift exists, the above steps are executed again; the training related parameters are uploaded; and receiving training related parameters and updating the global model by using federated weighted average to obtain an updated global model. According to the method and the device, the intrusion detection model can be trained under the conditions of multi-party cooperation and no leakage of privacy data, and meanwhile, the accurate recognition capability of the model for novel security threats is improved, and the adaptability of the model to a complex and changeable network environment is improved.

The invention discloses a pet personalized physical rehabilitation method and system based on multi-modal data fusion. The method comprises the following steps: establishing a digital file of a target pet; according to the method, multi-modal data such as behavior characteristics and physiological indexes are fused, and the space-time correlation is mined by using a multi-head self-attention mechanism, so that the limitation of a single data source can be overcome, the pain level and the joint limitation degree of the pet can be accurately identified, and an objective basis is provided for scheme formulation; based on medical history and rehabilitation sensitivity base lines in a pet digital file, in combination with real-time environment and expression recognition, rehabilitation strength, frequency and aromatic therapy formula are dynamically adjusted, accurate rehabilitation of'one pet and one strategy 'is achieved, and secondary damage or stress to pets caused by a standardized process is avoided; and in combination with expert-algorithm closed-loop verification and concept drift detection, the system can automatically adapt to new cases and environment changes, and the accuracy of rehabilitation effect evaluation and the scientificity of scheme recommendation are continuously improved.

The invention discloses a network intrusion detection method based on adaptive ensemble learning and concept drift detection, and the method can improve the detection accuracy and adaptability of an intrusion detection system under the conditions of unknown attacks and data concept drift. The precision of an existing model is remarkably reduced, and manual intervention is needed for recovery. The invention aims to provide a self-adaptive detection framework, so that the system can automatically identify and quickly adjust the failure of the model, autonomously complete the learning of new attacks and the updating of the model, and avoid frequent manual retraining. According to the method, the unknown attack detection capability is improved, and by integrating a plurality of heterogeneous classifiers and dynamically optimizing the combination of the heterogeneous classifiers, the method has stronger detection capability on never seen attack behaviors. Different models identify anomalies from different angles, the coverage rate of unknown attacks is improved, and the defect that a traditional single model misses detection of unknown threats is overcome.

A non-real time control unit (Non-RT RIC) and a near-real time control unit (Near-RT RIC) are hierarchized, a learning and inference unit (11, 12, 13, 16,17, 18), that controls the radio access network based on a result of inference performed by applying newest data to a learning model generated based on data collected from an O-RAN base station device 10, is arranged in the near-real time control unit, and a retraining control unit (14, 15, 19, 20), that detects concept drift based on a history of the data collected and causes the learning and inference unit to retrain the learning model when the concept drift is detected, is arranged in the non-real time control unit.

The invention discloses an Flink and large model collaborative data identification task completion method, device and equipment and a medium, and relates to the technical field of large data processing, and the method comprises the steps: carrying out the preprocessing operation of multi-modal data in a real-time data flow based on Flink, and distributing the preprocessed data to a preset time window according to a preset business demand; loading a preset large model sub-graph from a model registration center, and performing semantic recognition on the preprocessed data based on the preset large model sub-graph; carrying out confidence coefficient verification on the first recognition result, and carrying out semantic recognition on the preprocessed data by utilizing a preset lightweight model after the verification is not passed, so as to fuse the second recognition result and the first recognition result to obtain a target recognition result; and carrying out concept drift detection on the target model through the Flink and the target identification result, and updating the target model based on the Flink after judging that the concept drift occurs so as to execute a data identification task.

The application discloses an edge service anomaly detection method based on concept drift, comprising the following steps: S1, calculating the reliability score of a data stream to obtain a data stream sequence; S2, sampling the data stream sequence through a reservoir sampling method based on random pairing, and storing the sampling result in a sample library; S3, calculating the distribution change index of historical data in the sample library and current data in a sliding window; and S4, performing concept drift anomaly detection according to the distribution change index. The method effectively utilizes the sliding window, improves the traditional reservoir sampling algorithm, introduces the difference degree, realizes the extraction of representative historical sample library features from historical data under the constraint of storage cost, introduces the concept of random pairing to realize the extraction of uniform historical sample library features from historical data, and realizes more accurate concept drift anomaly detection by calculating the minimum cost required for the probability distribution conversion between current reliability data streams and historical reliability data streams in a service environment.

The present invention relates to a concept drift detection method and system based on adaptive data driving, which belongs to the field of machine learning technology. The method comprises the following steps: Step 1: pre-processing water purification plant data and setting prototype neural network parameters; Step 2: adjusting data distribution: extracting data distribution features from historical data and generating an initial data set in combination with resampling probability; Step 3: data variation, introducing three variation factors, generating a re-emergence data set; Step 4: for the re-emergence data set, obtaining meta-features; Step 5: dividing the re-emergence data set into a support set and a query set and inputting them into the prototype neural network for pre-training to obtain a trained prototype neural network; Step 6: inputting the pre-processed real-time water purification plant data into the trained prototype neural network for concept drift detection. Comprehensively considering periodic and non-periodic concept drift scenarios, the drift detection and response capabilities in complex environments are improved.

Systems and methods are provided for an accelerator system that includes a baseline (production) accelerator, optimizing accelerator, and control hardware accelerator, and an operation of alternatingly switching the production / optimizing accelerators between production and optimizing. With two production / optimizing accelerators, at any given point in time, one accelerator adapts while another accelerator processes data. Once the second accelerator starts doing a better job (e.g., has adapted to data drift), the accelerators change their modes, and the trainable accelerator becomes the “optimized” one. The accelerators do this non-stop, thus maintaining redundancy, providing expected quality of service (QoS) and adapting to data / concept drift.

The invention discloses a new energy power prediction model concept drift detection method and system, and belongs to the technical field of new energy power prediction and data flow mining, and the method comprises the steps: collecting high-availability basic operation data of a new energy station in real time, and carrying out the time alignment processing; a preset three-channel collaborative drift detection module is adopted for detection; the three-channel collaborative drift detection module comprises three monitoring channels which run in parallel, and alarm signals corresponding to the channels are output through prediction residual behavior monitoring, actual power distribution drift detection and physical consistency deviation detection. And calculating the confidence coefficient of each channel, carrying out weighted average on the confidence coefficients of the channels to obtain a total drift score, and judging whether the new energy power prediction model generates concept drift according to the total drift score and the corresponding duration. The method has high availability and strong robustness, can realize high-precision and low-false-alarm detection of concept drift, and provides technical support for optimization of a power prediction model.

This invention discloses an anomaly detection method for industrial IoT time-series data based on concept drift identification, belonging to the field of time-series anomaly detection technology. The method includes: performing multi-scale concept drift detection on industrial IoT time-series data to obtain concept drift detection results; updating an adaptive ensemble model based on the concept drift detection results to obtain an optimal anomaly detection model; and using the optimal anomaly detection model to perform anomaly detection on the industrial IoT time-series data to obtain anomaly detection results. This invention, through an innovative multi-scale window analysis mechanism, can identify data distribution changes earlier, effectively shortening the concept drift detection delay; and by combining model pool management and dynamic ensemble learning methods, it optimizes computational efficiency while ensuring detection accuracy, achieving a smooth transition between old and new data distributions.

The application relates to a network traffic concept drift detection method based on a Count-Min sketch data structure and belongs to the network traffic analysis field. The application records the multidimensional statistical information of network traffic through a CM sketch data structure, starts from the multidimensional probability distribution of network flow, monitors the multidimensional Hellinger distance change condition every certain period, performs network traffic concept drift detection, and detects the type of network traffic concept drift based on the Euclidean distance. The application records the multidimensional statistical information of network traffic through a CM sketch data structure, saves the storage space, each dimension is relatively independent, can be processed in parallel, and saves the detection time; starts from the multidimensional probability distribution of network flow, monitors the multidimensional Hellinger distance change condition, performs network traffic concept drift detection, reduces the concept drift false detection rate and the missed detection rate, makes the detection result more accurate; can correctly identify the network traffic concept drift type, discovers new applications and distributed drift applications, and has important significance in network intrusion detection and the like.

The invention discloses a malicious traffic identification method and device for a concept drift scene, and belongs to the technical field of network security. In order to solve the problem that unknown malicious traffic is difficult to identify in a dynamic network environment, an encoder and a label encoder are mainly adopted to collaboratively model potential representation, jointly optimize reconstruction loss, KL divergence loss and classification loss, and combine a class perception mechanism to construct a concept drift detector, so that classification of test samples and drift sample identification are realized. According to the method, the feature representation consistency and aggregation can be improved, outlier sample interference is suppressed, potential variation attack traffic is accurately detected, and the recognition precision and security of the system in a complex environment are improved.

The invention belongs to the technical field of wind power plant SCADA (supervisory control and data acquisition) data detection, and particularly relates to a wind power SCADA data online self-adaptive abnormal value detection method considering concept drift, which comprises the following steps: S100, collecting actual wind power in real time through an SCADA system; carrying out pretreatment and rationality screening; s200, calculating the wind power of the unit through a power model, obtaining a prior wind power sequence, and constructing a residual sequence; introducing an input wind speed as a scaling factor to obtain a scaling residual sequence; processing the scaling residual error sequence, and marking an abnormal value according to a preset threshold value; s300, dynamically monitoring the scaling residual error sequence by adopting an exponentially weighted moving average method; triggering a concept drift candidate event when the EWMA value exceeds a control limit UCL; carrying out difference test on the current residual error distribution and the historical reference distribution by adopting KS test, and if the difference exceeds a preset threshold value, confirming that concept drift occurs; and S400, after judging that the concept drift occurs, updating the parameters of the power model.

The application discloses a cold and hot data recognition method and system based on streaming learning, and belongs to the field of computer storage. The system regards cold and hot recognition as a decision problem, extracts multi-dimensional features including data flow, control flow and system information through a feature extraction module to construct a feature vector, realizes online hotness evaluation and real cold and hot label generation through an online label module, adopts a streaming learning algorithm in a cold and hot recognition module, predicts the cold and hot state of a data block in the future in real time according to the feature vector, and regularly updates a model to cope with concept drift. In addition, the system adopts a dynamic adjustment mechanism of cold and hot perception threshold, can adaptively guide data migration, realizes online judgment and labeling of data cold and hot, and provides efficient and adaptive cold and hot data recognition services for user applications.

This invention discloses a business process anomaly detection method based on concept drift discovery, comprising the following steps: 1) collecting data to form an event log; 2) extracting process information using control flow features from the event log, encoding events, and constructing a process feature dataset; 3) building a prediction model for the next event in the business process based on a GRU model, and training the prediction model using the process feature dataset as input data; 4) calculating the anomaly score s of the business process attributes through probability distribution; 5) performing concept drift detection on the anomaly detection results using a concept drift discovery module; and 6) using an incremental learning method to incorporate the drift case set as new knowledge using an event prediction model update module. This method mines process models from event logs without requiring manual judgment to find concept drift cases, enabling more accurate detection of whether anomalies occur in business process instances and locating and determining whether concept drift has occurred.

The application relates to the technical field of network intrusion detection, in particular to an automatic intrusion detection system for a dynamic network environment, which has the technical scheme that in the autonomous decision module of Gaussian probability, a contrast loss function taking normal traffic as the center is designed, so that the model can efficiently distinguish the behavior patterns of normal traffic and abnormal traffic; in the automatic continuous learning framework, a double memory bank is designed to adapt to the concept drift scene in the dynamic network, wherein the stable memory bank is used for storing old knowledge and preventing the catastrophic forgetting of the model, and the high-confidence pseudo label generated in the autonomous decision module of Gaussian probability is used to update the adaptive memory bank, so that the real-time updating and fine-tuning of the autonomous decision module of Gaussian probability are realized; in the continuous learning process, the system does not need to rely on manual labeling, can effectively capture the constantly evolving patterns in the dynamic network scene, significantly enhances the applicability of the intrusion detection system to the concept drift, and realizes automatic intrusion detection.

Facilitating intelligent concept drift mitigation in advanced communication networks is provided herein. A method includes utilizing, by a system comprising a processor, a first model that facilitates management of resources within a communications network. A reliability level of the first model is determined to satisfy a defined reliability level. The method also includes based on a first determination that the reliability level of the first model no longer satisfies the defined reliability level, replacing, by the system, the first model with a second model that temporarily facilitates management of the resources within the communications network. Further, the method includes, based on a second determination that a third model satisfies the defined reliability level, deploying, by the system, the third model within the communications network. The deploying can include incrementally transitioning facilitation of the management of resources from the second model to the third model.

The application discloses a PDU load distribution system based on reinforcement learning, relates to the technical field of data center resource management, and realizes efficient scheduling of PDU load, server migration and start-stop through the combination of a multi-dimensional execution cost model and a digital twin environment, can balance energy consumption and performance in different load scenarios by using offline pre-training and hierarchical control, and continuously optimizes through online learning under concept drift detection; in addition, after superimposing security cost and fault tolerance overhead, key business and sensitive data can be preferentially protected, security risks and downtime losses can be effectively reduced, and finally, efficient, safe and scalable data center management in multidimensional coordination is achieved. Through batch migration and phased start-stop, resource impact and performance jitter caused by large-scale operations are further avoided, flexible responses can be made under multi-dimensional risks, and the robustness and sustainability of scheduling under heterogeneous loads can be ensured.

The invention discloses a model processing method, which is used for improving the performance of a model obtained by training in processing new data when concept drift occurs in the model. In the method, when it is detected that concept drift occurs in a first model in use, input data obtained after the concept drift occurs in the first model is adopted to train a second model used for replacing the first model, and the first model still continues to be used for a period of time. Moreover, when the second model is trained, the degree of concept drift of the first model can be continuously detected, and the training hyper-parameters of the second model are dynamically adjusted based on the degree of concept drift of the first model, so that the second model can more quickly and effectively adapt to the distribution of new data, and the accuracy of the data distribution is improved. Therefore, the performance of the second model obtained by training in processing new data is improved.

The present application provides a model adaptive training method, device, equipment, medium and program product. By obtaining various operation data of the original model during actual operation and detecting the first concept drift value of the original model during actual operation according to the operation data; then, according to the value model and the first concept drift value, respectively allocate each operation data to a labeled data set and / or an unlabeled data set. When the amount of data in the labeled data set is greater than or equal to a preset threshold, use the adaptive training model to adaptively train the original model according to the labeled data set and the unlabeled data set to determine the trained new model. It solves the technical problem of how to enable the AI model to perform adaptive training with as little human intervention as possible. It achieves the technical effects of reducing the amount of manual annotation required by developers when updating the model and improving the performance stability of the model by integrating multiple models.

According to the drift-oriented self-evolution encrypted traffic classification method provided by the invention, self-adaptive classification for coping with concept drift in a development world environment is realized, and the life cycle of a classifier is prolonged; the method specifically comprises the following steps: 1) continuously screening out silver samples of which the Softmax confidence coefficient is higher than a threshold value from a prediction result of a trained classifier based on an extended Laida criterion, and providing reliable data support without manual annotation for subsequent model fine tuning; and 2) monitoring the confidence coefficient of Softmax in real time in a classifier prediction process by adopting a method based on window multi-threshold cumulative measurement, accumulating a drift score when the confidence coefficient is lower than a plurality of preset thresholds, and judging that concept drift occurs and triggering fine tuning if a certain category or the whole model reaches a drift score upper limit in a specified time window. And 3) performing unfreezing layer parameter fine tuning on the model by using a silver sample for the detected concept drift category through category-sensitive layered fine tuning, and prolonging the life cycle of the classifier.

The invention provides a concept drift and unknown category-oriented network intrusion detection incremental learning method, which relates to the technical field of network intrusion detection systems and comprises the steps of collecting original network traffic; carrying out data preprocessing, and extracting a spatiotemporal feature vector from the standardized time sequence feature sequence through a spatiotemporal feature extraction network; processing through a multi-classifier module, and outputting a known class recognition result, unknown traffic and a confidence coefficient sequence of all classifiers; unknown traffic is clustered to obtain a new classifier, whether drifting exists or not is judged based on the spatial-temporal feature vectors and the confidence sequence of the classifier, and local parameter fine tuning is conducted on the spatial-temporal feature extraction network and the multi-classifier module when drifting exists. According to the method, the problems of occurrence of unknown attacks in intrusion detection, continuous change of flow distribution and stability of long-term online operation of a model are solved, and expression learning, classification decision making and updating mechanisms are covered under a unified framework at the same time.

A non-real time control unit (Non-RT RIC), and a near-real time control unit (Near-RT RIC) are hierarchized, and AI / ML learning function (11, 12, 16) that generates a learning model based on data collected from an O-RAN base station device 10, and a retraining function (14, 15, 19, 20) that detects concept drift based on the data collected and causes the learning model to be retrained, are arranged in the non-real time control unit. Also, AI / ML inference function (13, 17, 18) that controls a radio access network based on a result of inference by applying the newest data to the learning model and sends inference performance data to the retraining function in the near-real time control unit.

The invention provides a smart power grid load prediction system based on edge computing, and relates to the field of edge computing, and the method comprises the steps: enabling an edge node to locally generate and report an edge portrait data packet containing a data feature vector and an equipment capability vector; the cloud scheduling center performs data concept drift and performance bottleneck detection according to the portrait data packet to trigger a model updating event; then, the dispatching center executes a two-stage matching decision: a candidate model conforming to a load scene is screened out from a model library based on a data feature vector, and then hardware resource constraint screening is performed based on an equipment capability vector, so that an optimal model considering both precision and performance is selected; and finally, the system generates a deployment data packet containing a model difference packet, and the edge node only needs to execute difference updating, so that accurate and efficient model adaptation to heterogeneous equipment and a dynamic scene is realized.