Construction method for big-data acceleration structure

A technology for accelerating structures and construction methods, applied in electrical digital data processing, special data processing applications, instruments, etc., to solve problems such as increased limitations, unsuitable algorithms, and inability to perform well, to speed up processing and data loading. Speed, simplicity of the build process, overcoming limitations and the effect of platform limitations

Active Publication Date: 2018-01-19
UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF5 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The defect of this method is very obvious, that is, its improvement for a specific platform cannot have better performance on other platforms, and especially under the complex platform of police affairs, its correlation analysis algorithm models are diverse, and some algorithms are not very good. Suitable for this type of parallelization operation, making the limitations of this method increase

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Construction method for big-data acceleration structure
  • Construction method for big-data acceleration structure
  • Construction method for big-data acceleration structure

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0040] Such as figure 1The construction method of the large data acceleration structure of the present invention shown includes:

[0041] A. Data preprocessing: Data cleaning, data integration and data conversion are performed on the original data to form a data set that conforms to the operation process.

[0042] Data cleaning described therein includes deletion of raw data, filling of missing values ​​and smoothing / filtering of noisy data.

[0043] In deleting original data and filling in missing values, the missing data is judged by the attribute weight coverage p of the original data, and the attribute weight coverage p is:

[0044]

[0045] Where A is the data attribute, ε is the attribute weight, indicating the importance of the attribute, and ε 1 +ε 2 +...+ε k =1, α indicates whether the attribute value is missing, α∈{0,1}, set the threshold ω, the value range of the threshold ω is (0,1), if p≥ω, fill in the missing value for the missing data, otherwise, then de...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a construction method for a big-data acceleration structure. The method comprises the following steps that: A: preprocessing data, and forming a dataset which conforms to an operation process; B: carrying out clustering processing, calculating similarity of records in categories, and enabling the most similar records in a group to be minimum on an aspect of space distanceaccording to the grouping result of a clustering algorithm; C: establishing a mapping relationship among a transaction attribute, transaction attribute weight and a transaction record according to three-level indexes, and circulating the process until all data finish being mapped; and D: carrying out initial compression an index structure, a transaction attribute weight index and the transaction attribute, determining the range of continuously recorded shared attribute weight values, traversing an inverted index mapping structure, and compressing the continuous records under the shared attribute weight values through a process compression algorithm. By use of the method, the acceleration structure of big data correlation analysis can be quickly established, and model processing speed and data loading speed can be obviously quickened.

Description

technical field [0001] The invention relates to a data acceleration processing method, in particular to a method for building a large data acceleration structure. Background technique [0002] Big data technology has become the most effective and common technology for processing massive data. Police big data, as one of the most representative scenarios in big data processing scenarios, has attracted more and more attention. In the process of mass data analysis, the processing speed and processing performance of the general big data platform is one of the urgent problems to be solved. In big data analysis scenarios, especially in police big data, the most common analysis method is correlation analysis. Comprehensive correlation analysis of relevant factors involved in the analysis object can effectively improve the accuracy of police analysis. The commonly used big data correlation analysis The algorithm model is an association analysis algorithm model. Association analysis...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 段贵多罗光春田玲秦科
Owner UNIV OF ELECTRONICS SCI & TECH OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap