Large-scale data visualization dimension reduction method based on graph neural network

Abstract

The invention provides a visualized dimension reduction method based on a graph neural network, and relates to the technical field of deep learning and large-scale data processing. The problems that amodel cannot be subjected to large-scale data training, a non-parametric visualization dimensionality reduction model cannot process visualization of unknown data points and a visualization result ofthe parametric visualization dimensionality reduction model is poor in the prior art are solved. After an obtained high-dimensional data set is divided and preprocessed, a heterogeneous graph is constructed, a GNN graph neural network model is established, a loss function is confirmed, then training is carried out, testing is carried out after training is completed, a loss function carries out visual dimensionality reduction on high-dimensional large-scale data, innovative training is carried out by adopting the thought of subgraph negative sampling, the training cost of the model is reduced,the dimensionality of the data can be reduced, but a considerable part of high-dimensional data information is kept; therefore, subsequent data analysis and processing become more meaningful and easier.

Description

technical field [0001] The present invention relates to the technical field of deep learning and large-scale data processing, and more specifically, to a large-scale data visualization dimensionality reduction method based on graph neural network. Background technique [0002] With the advent of the cloud era, big data (Big data) has also attracted more and more attention. Since big data analysis requires a framework like MapReduce to distribute work to tens, hundreds or even thousands of computers, it will It will take too much time and money, so in the era of big data, it is an important task to understand and explore the collected data. The traditional method of drawing data in 2-dimensional (2D) or 3-dimensional (3D) space Make the data scattered in the data mining pipeline, directly visualize the data distribution, and participate in the exploration and analysis in an interactive way, which plays a vital role in the large-scale data analysis and exploration, ensuring th...

AI Technical Summary

Problems solved by technology

[0005] In order to solve the problems in the existing dimensionality reduction technology that the model cannot be trained on large-scale data, the non-parametric visualization dimensionality reduction model cannot handle the visualization of unknown data points, and the visualization results of the parametric visualization dimensionality reduction model are poor, the present invention proposes a A visual dimensionality reduction method based on graph neural network, which realizes efficient data dimensionality reduction while ensuring the original data structure of high-dimensional data, which is convenient for subsequent data analysis and processing operations

Images