GPU based data classification method of conditional random field model

Abstract

The present invention provides a GPU based data classification method of a conditional random field model, including the following steps: a CPU reads a training data set and generates a feature set according to matching template definition rules; the CPU initializes a correlation matrix and a vector to first calculate the initial probabilities of all features corresponding to starting nodes of an observation sequence; the CPU determines whether to perform data segmentation processing; after transmitting parameters, the GPU automatically selects a proper parallel computing method according to processing data scale and adopts the parallel computing method to calculate the maximum probability value of each observation node corresponding to each feature. After calculation, with the results being returned, the CPU determines a completion situation of data processing and final output. This method in the invention is characterized by universal use, high training speed, effective processing of large-scale sequence data and self-adaptation capability.

Description

technical field [0001] The invention relates to a data classification method, in particular to a data classification method based on a GPU-based parallel conditional random field model. Background technique [0002] Conditional Random Field (CRF) is one of the important discriminative models in machine learning. It was first used in natural language processing and gradually introduced into data mining, bioinformatics, image intelligent recognition and search engines, etc. in popular fields. It solves the label bias problem of other traditional discriminative models (such as the maximum entropy Markov model, referred to as MEMMS), and has a more accurate training prediction effect. [0003] However, with the widespread application of classification, serial CRF faces some problems. In practical applications, the number of training sets is growing rapidly, the data analysis sequence is getting longer and longer (for example, a DNA contains thousands of amino acid pairs), and ...

AI Technical Summary

Problems solved by technology

This method obtains a linear speedup ratio within a certain range, but compared to the consumed computing resources, the cost performance is not high; on the other hand, the image processor (Graphic Process Units, referred to as GPU) is With the characteristics of many units, strong floating-point computing ability and low cost and power consumption, it has gradually made a difference in the field of general-purpose parallel computing, but GPU also has its own limitations and special processing: a single processor has low performance and requires a large number of computing units Improve performance in parallel at the same time; memory IO consumption has hierarchical overhead characteristics; parallel read shared memory efficiency issues, etc.

Images