Unlock instant, AI-driven research and patent intelligence for your innovation.

Data processing method and device, computing equipment and storage medium

A technology of data processing and data volume, applied in the field of big data, can solve the problem that the rationality of partition mode needs to be improved, and achieve the effect of improving data processing efficiency and rationality

Active Publication Date: 2019-09-03
TENCENT TECH (SHENZHEN) CO LTD
View PDF2 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the rationality of the current partitioning method needs to be improved

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and device, computing equipment and storage medium
  • Data processing method and device, computing equipment and storage medium
  • Data processing method and device, computing equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some, not all, embodiments of the application. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

[0035] Figure 1A A schematic diagram showing application scenarios according to some embodiments of the present application. Such as Figure 1A As shown, application scenario 100 may include computing device 110 and data processing system 120 . Here, computing device 110 may communicate with data processing system 120 through network 130 . Wherein, the computing device 110 may include a task scheduling application 111 . The task scheduling application 111 may acquire data processing requests. ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data processing method and device, computing equipment and a storage medium. The data processing method comprises: acquiring a data processing request; generating an execution plan corresponding to the data processing request, the execution plan comprising a tree structure, and each node being used for describing one data operation; for each node in the execution plan, generating an elastic distributed data set related to the data operation described by the node; and when it is determined that the repartitioning operation is performed on the elastic distributed data set corresponding to each node in the execution plan, executing the repartitioning operation according to the dependency relationship between the elastic distributed data sets corresponding to each node.

Description

technical field [0001] The present application relates to the technical field of big data, in particular to data processing methods, devices, computing equipment and storage media. Background technique [0002] With the development of the Internet, various big data platforms are widely used. Big data platforms can usually divide big data into data blocks, and then process each data block in a distributed manner. At present, big data platforms usually use fixed partitions to divide big data into blocks. The fixed partition method can divide big data according to the default setting of the system or the number of partitions specified by the user. However, the rationality of the current partitioning method needs to be improved. Contents of the invention [0003] This application proposes a data processing solution, which can improve partition rationality, thereby improving data processing efficiency. [0004] According to one aspect of the present application, a data proc...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/27
CPCG06F16/278Y02D10/00
Inventor 朱锋张韶全
Owner TENCENT TECH (SHENZHEN) CO LTD