Data transfer method, system, and storage medium between remote client and hive table

By introducing data relay commands between remote clients and Hive tables, and utilizing the collaborative execution of data transmission by the Hive server and Yarn cluster, the complex operation and server stability issues in existing technologies are resolved, achieving an efficient and simplified data transmission process.

CN117156007BActive Publication Date: 2026-07-10DUXIAOMAN TECH (BEIJING) CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
DUXIAOMAN TECH (BEIJING) CO LTD
Filing Date
2023-08-29
Publication Date
2026-07-10

AI Technical Summary

Technical Problem

In existing technologies, data transmission between remote clients and Hive tables involves cumbersome steps, relies on multiple component modules, leads to complex and redundant development work, affects efficiency, and may affect server stability when transmitting large amounts of data.

Method used

By introducing data transfer instructions between remote clients and Hive tables, and leveraging the collaborative execution of data transfer by the Hive server and Yarn cluster, the system temporarily stores the data to be transferred in a distributed file system, simplifying user operations and avoiding direct data storage on the server.

Benefits of technology

This allows users to complete data transfer by triggering only one data transfer command, reducing operational costs, improving efficiency, ensuring server stability, and preventing data deletion or overwriting on the server.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117156007B_ABST
    Figure CN117156007B_ABST
Patent Text Reader

Abstract

The embodiment of the present disclosure discloses a data transmission method and system between a remote client and a Hive table and a storage medium. The method comprises: in response to a data transmission instruction triggered by a user, the remote client modifies the data transmission instruction into a data relay transmission instruction and sends it to a Hive server, wherein the data relay transmission instruction comprises a relay storage path under a distributed file system, and the relay storage path is used to temporarily store the to-be-transmitted data in the process of data transmission between the remote client and the Hive table; the Hive server analyzes and compiles the data relay transmission instruction, generates a data transmission execution plan and sends it to a Yarn cluster; based on the relay storage path and the data transmission execution plan, the remote client and the Yarn cluster cooperatively execute the data transmission operation to complete the transmission of the to-be-transmitted data between the remote client and the Hive table. The technical scheme of the present disclosure enables the user to trigger the instruction only once, thereby realizing the data transmission between the remote client and the Hive table.
Need to check novelty before this filing date? Find Prior Art