Data screening method and data screening device

A technology of data screening and data collection, applied in the field of big data processing, can solve problems such as affecting industry analysis and judgment, incomplete removal of interfering data, affecting data processing and analysis, etc., to achieve the effect of being conducive to accuracy and thoroughness

Inactive Publication Date: 2019-09-06
MIAOZHEN INFORMATION TECH CO LTD
View PDF0 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In the collected data, there are often some interference data due to measurement deviation or statistical deviation. These interference data will affect the processing and analysis of the data, thus affecting the analysis and judgment of the industry.
[0003] At this stage, common methods for removing interference data in large data sets, such as building data m

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data screening method and data screening device
  • Data screening method and data screening device
  • Data screening method and data screening device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0061] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only It is a part of the embodiments of this application, not all of them. The components of the embodiments of the application generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the application. Based on the embodiments of the present application, every other embodiment obtained by those skilled in the art withou...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data screening method and a data screening device. The data screening method comprises the steps of drawing a scatter diagram based on obtained discrete data in a to-be-screened data set; based on the distribution characteristics of each discrete point in the scatter diagram, determining a fitting curve connecting the plurality of discrete points and the fitting degree ofthe fitting curve; if the fitting degree of the fitting curve is smaller than a preset threshold value, constructing a probability distribution model based on the distance from each discrete point tothe fitting curve; determining a confidence interval range of the distance based on the average value and the standard error of the probability distribution model and the obtained saliency level value; and determining discrete data corresponding to the discrete points, of which the distance values with the fitting curve are located outside the confidence interval range, in all the discrete points,screening the determined discrete data from a to-be-screened data set, and determining the data set with the discrete data screened as a target data set. Thus, outliers can be removed with the fitting curve as the reference, and the outliers removal accuracy and thoroughness are improved.

Description

technical field [0001] The present application relates to the technical field of big data processing, in particular to a data screening method and a data screening device. Background technique [0002] With the rapid development of Internet technology, big data technology has penetrated into many businesses in many industries. By collecting a large amount of business-related business data, a large amount of business data is processed and analyzed, and then the industry corresponding to the business data is analyzed. In the collected data, there are often some interference data due to measurement deviation or statistical deviation. These interference data will affect the processing and analysis process of the data, thereby affecting the analysis and judgment of the industry. [0003] At this stage, common methods for removing interference data in large data sets, such as building data models, etc., are all based on predicted values ​​or pre-set standards to remove interferenc...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/18
CPCG06F17/18
Inventor 刘强
Owner MIAOZHEN INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap