Dimension reduction and correlation analysis method suitable for large-scale data

A large-scale data and association analysis technology, applied in the fields of computer science and image processing, can solve the problems of insufficient utilization of speed and memory efficiency, and achieve the effect of improving computing speed and memory utilization, and using memory efficiently
CN112149045AInactive Publication Date: 2020-12-29JIANGSU UNIV
0 Cites 2 Cited by

Patent Information

Authority / Receiving Office
CN Β· China
Current Assignee / Owner
JIANGSU UNIV
Publication Date
2020-12-29
Estimated Expiration
Not applicable Β· inactive patent

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a dimension reduction and correlation analysis method suitable for large-scale data, and the method comprises the steps: projecting high-dimensional data to a Fourier domain, and converting a solving feature vector problem of linear correlation analysis into a meaningful Fourier domain basis; because the Fourier domain basis is predefined and the characteristic value distribution of the data is ordered, accelerating training by inputting the training samples in batches until the required Fourier basis is stable and ordered; determining the number of Fourier bases and aprojection matrix, and multiplying the projection matrix by the high-dimensional data set to obtain a low-dimensional data set so as to facilitate rapid processing of data. According to the data dimension reduction method, on the basis of fast Fourier transform and correlation analysis, noise and redundant information in a high-dimension data set can be removed, unnecessary operation processes indata processing are reduced, and the running speed and the memory use efficiency in data dimension reduction calculation are improved.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention belongs to the field of computer science and image processing technology, in particular to a dimensionality reduction and association analysis method suitable for large-scale data. Background technique

[0002] Traditional data processing methods have been unable to effectively analyze massive data. At the same time, with the continuous increase of data dimensions generated by big data processing and cloud computing, in many fields of research and applications, it is usually necessary to observe data containing multiple variables, collect a large amount of data, and then analyze and find patterns. Multivariate large data sets will undoubtedly provide rich information for research and application, but also increase the workload of data collection to a certain extent.

[0003] Canonical Correlation Analysis (CCA) is one of the most commonly used algorithms for mining data association relationships. It is also a dimensionality reduction tec...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More