SQL interactive analysis method and system based on big data platform

Abstract

The invention discloses an SQL (Structured Query Language) interactive analysis method and system based on a big data platform, which solve the defects in the prior art, and the method comprises the following steps: step 1, the system configures a traditional relational database data source by the system; step 2, the system selects and associates the traditional relational database data source configured in the step 1 with the target data entity type, creates a big data entity matched with the target data entity type in a big data environment, and stores the created big data entity information; 3, the system reads the traditional relational database data source and the big data entity information, configures a synchronous data task from the traditional relational database data source to the big data entity in the big data environment, and executes the task; 4, the system configures big data entity information SQL execution parameters, and then an SQL execution engine matched with the SQL execution parameters is selected by related personnel to perform SQL development; and step 5, SQL analysis is performed by related personnel to obtain an analysis result.

Description

Technical field [0001] The invention relates to the technical field of big data processing, in particular to a SQL interactive analysis method and system based on a big data platform. Background technique [0002] SQL is a data analysis language that is easy to learn and has great flexibility and powerful functions. It needs to be based on the underlying database system and query execution engine. With the gradual increase in data volume, traditional relational databases such as MYSQL, ORACLE, etc. have been unable to meet the needs of query and analysis performance. [0003] The big data cluster environment has good performance for processing large-scale data. SQL also has many applications in the big data environment, such as HIVE, IMPALA and other SQL engines gradually appearing. These SQL engines rely on big data clusters to far exceed the performance of traditional relational databases in analyzing large-scale data. [0004] When the SQL analysis and development of the traditi...

AI Technical Summary

Problems solved by technology

[0004] When the SQL analysis and development of the traditional relational database is transferred to the SQL analysis and development of the big data environment, the following problems will arise: first, the data needs to be migrated to the big data environment, the original data is still stored in the relational database, and SQL developers are not familiar with the migration operation

Most of the SQL execution engines in big data environments do not have a good client environment, which will cause difficulties in development

Images