Intelligent data detection method and device based on improved SOM algorithm
A technology of intelligent detection and algorithm, applied in the direction of structured data retrieval, digital data information retrieval, relational database, etc., can solve the problems of heavy tasks, relying on manual detection, unable to guarantee the accuracy of manual error detection, etc., to improve data quality, Improve the effect of clustering and reduce the effect of nonlinear effects
Active Publication Date: 2020-10-13
TSINGHUA UNIV +2
View PDF4 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
The task of basic data detection is heavy, but the current system's self-detection ability is insufficient, mainly relying on manual detection
Manual error detection will not only consume a lot of manpower and time, but also cannot guarantee the accuracy of manual error detection
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0026] Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.
[0027] The data intelligent detection method and device based on the improved SOM algorithm proposed according to the embodiments of the present invention will be described below with reference to the accompanying drawings.
[0028] Firstly, a data intelligent detection method based on an improved SOM algorithm proposed according to an embodiment of the present invention will be described with reference to the accompanying drawings.
[0029] figure 1 It is a flowchart of a data intelligent detection method based on an improved S...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a data intelligent detection method and device based on an improved SOM algorithm, and the method comprises the steps: obtaining a sample set, carrying out the decomposition ofthe sample set according to dimensions, carrying out the one-dimensional isolated point detection based on density dimension by dimension, carrying out the preliminary screening of outliers of a multi-dimensional sample set according to dimensions, and removing the outliers; clustering the sample set based on a self-organizing feature mapping algorithm, and removing abnormal data points; improving the mapping algorithm based on the self-organizing characteristics through a kernel function method, clustering the sample set through the improved mapping algorithm based on the self-organizing characteristics, and removing abnormal data points; removing the abnormal data points in the sample set according to expert experience, and completing the intelligent detection of the data. According tothe method, abnormal data can be eliminated by utilizing density-based one-dimensional isolated point detection, the data quality is improved, the influence of nonlinearity of sample data can be reduced by introducing a kernel function into weight updating of the self-organizing mapping algorithm, and the clustering effect of the SOM algorithm is improved.
Description
technical field [0001] The invention relates to the technical field of big data intelligent detection, in particular to a data intelligent detection method and device based on an improved SOM algorithm. Background technique [0002] Install various sensors and other data acquisition devices on the equipment to collect equipment operation data and monitor equipment operation status. Due to factors such as complex system and harsh environment, the data collected by the system has the characteristics of strong nonlinearity, high noise, and extreme instability. Therefore, abnormal data detection is one of the important steps in data preprocessing. [0003] Abnormal data is also called outlier. Abnormal data may be erroneous data caused by equipment failure, human measurement error, etc., or it may be a meaningful event corresponding to the actual situation. Incorrect data has adverse effects on system operation. If the wrong data is not found and eliminated in time, it may c...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/215G06F16/28G06N3/08G06N3/04
CPCG06F16/215G06F16/285G06N3/088G06N3/045
Inventor 胡伟郭秋婷黄建平陈浩盛银波
Owner TSINGHUA UNIV