Rule-based Big data offline batch processing performance capacity scanning method and device

Abstract

The embodiment of the invention provides a rule-based big data offline batch processing performance capacity scanning method and device, relates to the technical field of big data and can also be used for the financial field, and the method comprises the following steps: splitting an HQL script program code according to a preset interval symbol to obtain at least one HQL statement; performing script analysis on the HQL statements in sequence, judging whether the HQL statements subjected to script analysis conform to a preset performance hidden danger grammar rule or not, and if yes, outputting the corresponding HQL statements, HQL script program codes and performance hidden danger grammar to a set summary file. According to the invention, hidden danger checking can be effectively, accurately and conveniently carried out on the performance capacity of the HQL script.

Description

technical field [0001] This application relates to the field of big data technology, and can also be used in the financial field, and specifically relates to a rule-based off-line batch processing performance capacity scanning method and device for big data. Background technique [0002] The enterprise's big data service cloud platform provides data access, storage, computing, security management, resource management and other services for various professional big data analysis applications. Increasingly rich, while the platform operation and maintenance system continues to expand, the performance of the specific models and logic of each application scenario under the new technology architecture has gradually become one of the focuses of the product development process. This also puts forward higher requirements for the performance and capacity testing methods and tools of the big data platform under the "multi-application, multi-tenant" framework system. [0003] The inven...

AI Technical Summary

Problems solved by technology

[0003] The inventor found that in the prior art, when the number of offline batch processing business systems connected to the big data service cloud reaches a certain number (for example, hundreds), in the process of R&D and iterative optimization of each project based on the big data service cloud, Each developer needs to use hive sql (hereinafter referred to as HQL) scripts to perform related operations such as business logic processing on the basic data or subject dat

Images