Concept drift detection method and system based on weighted sampling and electronic equipment

A concept drift and detection method technology, applied in the direction of instruments, character and pattern recognition, calculation models, etc., can solve the problem that the concept drift of AI models cannot be effectively solved, so as to reduce the time and space complexity of calculation and avoid unnecessary overhead , Improve the effect of timeliness

Active Publication Date: 2021-06-25
SHANGHAI JIAO TONG UNIV
View PDF7 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0015] In view of the shortcomings of the prior art described above, the purpose of the present invention is to provide a concept drift detection method, system and electronic equipment based on weighted sampling, which is used to solve the technical problem that the prior art cannot effectively solve the concept drift of the AI ​​model

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Concept drift detection method and system based on weighted sampling and electronic equipment
  • Concept drift detection method and system based on weighted sampling and electronic equipment
  • Concept drift detection method and system based on weighted sampling and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0059] like figure 1 As shown, this embodiment provides a concept drift detection method based on weighted sampling, and the concept drift detection method based on weighted sampling includes:

[0060] Step S110, train an offline model based on the historical data, and use the offline model to perform model inference on the online data, and output the model inference result.

[0061] Step S120, receiving online real-time data, calculating the concept drift value based on the online real-time data and the historical data, and judging whether the concept drift value is greater than the drift threshold, if so, confirming that there is concept drift in the online real-time data, otherwise confirming the online real-time data There is no concept drift;

[0062] Step S130, when there is concept drift in the online real-time data, updating the offline model and the training data for training the offline model based on the online real-time data and historical data;

[0063] Step S14...

Embodiment 2

[0119] Such as Figure 4 As shown, this embodiment provides a concept drift detection system 10 based on weighted sampling, which includes: a model training module 110 , an inference service module 120 and a concept drift module 130 .

[0120] In this embodiment, the model training module 110 trains an offline model based on historical data.

[0121] In this embodiment, the model training module 110 includes a data pool, training data, model training, and a model warehouse.

[0122] Among them, the data pool is used to store historical data (training data of the initial model). Training data: selected from the data pool, used for offline training of the initial model, and update of the model after drift is detected. Model training: Use the training data set to train the model offline, and the obtained model result will be used as the initial model, and the model result will also be pushed to the model warehouse. Model warehouse: store offline trained models and updated mode...

Embodiment 3

[0148] Such as Figure 8 As shown, this embodiment also provides an electronic device 10, which is, but not limited to, a smart phone, a tablet, a smart wearable device, a personal desktop computer, a notebook computer, a server, a server cluster, and the like.

[0149] The electronic device 10 includes a memory 102 for storing a computer program; a processor 101 for running the computer program to implement the steps of the concept drift detection method with weighted sampling as described in Embodiment 1.

[0150] The memory 102 is connected to the processor 101 through the system bus and completes mutual communication, the memory 102 is used to store computer programs, and the processor 101 is used to run the computer programs, so that the electronic device 10 executes the weighted sampling Concept drift detection methods. The method for detecting concept drift with weighted sampling has been described in Embodiment 1, and will not be repeated here.

[0151] It should be ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a concept drift detection method and system based on weighted sampling and electronic equipment, and the method comprises the steps: training an offline model based on historical data, carrying out the model reasoning of online data through employing the offline model, and outputting a model reasoning result; receiving the online real-time data, calculating a concept drift value based on the online real-time data and the historical data, judging whether the concept drift value is greater than a drift threshold, if so, confirming that the online real-time data has concept drift, and if not, confirming that the online real-time data does not have concept drift; when the concept drift exists in the online real-time data, updating the offline model and training data for training the offline model based on the online real-time data and the historical data; and performing model reasoning on the online data based on the updated offline model, and outputting a model reasoning result. According to the method, the drift degree of the current model can be effectively detected, the drift degree serves as a basis for model retraining / updating, and the concept drift problem of the AI model is intelligently solved.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, in particular to the technical field of AI model design, and specifically relates to a concept drift detection method, system and electronic equipment based on weighted sampling. Background technique [0002] In the actual AI model application, especially in the model application of time-space sequence data, the samples are usually not obtained all at once, but gradually obtained over time, and the information reflected by the samples may also change over time. [0003] Over time, the sample distribution tends to change and the performance of the AI ​​model gradually degrades. To this end, we need to regularly train and update the model, and monitor the performance of the model in real time, which creates a huge workload for our algorithm engineers and operation and maintenance personnel. [0004] The phenomenon that the distribution of samples or the information carried ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/62G06N20/00
CPCG06N20/00G06F18/24G06F18/214
Inventor 林锋施晓华卢宏涛
Owner SHANGHAI JIAO TONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap