Big data feature extraction method and device

Abstract

Embodiments of the invention provide a big data feature extraction method and device. The device comprises a data structured module and a representative learning module, wherein the data structured module is used for pre-processing original big data and networking the pre-processed original big data to obtain a relationship network with nodes and an edges; and the representative learning module is used for obtaining high-dimensional vectors of the relationship network by adoption of an embedded mapping-based representative learning algorithm so as to obtain features of the original big data. The device provided by the embodiments of the invention can effectively extract the feature information in the big data without artificial participation; the feature information is uniformly expressed in a form of high-dimensional vectors, so that the features can provide a uniform effective processing method for a plurality of application services; and by adoption of the embedded mapping-based representative learning algorithm, structure information is retained in the high-dimensional vectors, so that more correct application services can be subsequently provided for the users.

Description

technical field [0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a method and device for feature extraction of big data. Background technique [0002] The insurance industry is undergoing tremendous changes due to technological advancement, and the wide application of big data has changed the way insurance companies provide services. Existing insurance industry websites and software usually collect massive amounts of data and contain a lot of useful information, including users' personal information and consumption habits. Only by making full use of big data in the insurance industry can we adapt to the requirements of the big data era in many aspects such as risk pricing, product design, marketing strategy, customer service, and risk management and control. [0003] Currently in the insurance industry, a database system is usually used to store and manage insurance data. In database systems, data is usually s...

AI Technical Summary

Problems solved by technology

However, this system needs to rely on experience to extract features and collect labeled data, making it impossible to implement effectively

[0007] It can be seen that the existing intelligent processing system of big data in the insurance industry has at least the following disadvantages: 1) The existing insurance industry data technology lacks the analysis of unstructured data, and a large amount of effective information is lost, which affects the analysis results of the insurance industry business; 2) Existing insurance industry recommendation systems, insurance purchaser classification systems, and insurance fraud detection systems rely too much on human-powered feature extraction, which has low accuracy, poor calculation efficiency, slow response to user requests, and affects user experience; 3) Different insurance services usually use different data processing and feature extraction methods, resulting in a large amount of redundant data processing, and the characteristics of data units of different services are not compatible

Images