Data processing method, device and system in a distributed system
A technology of a data processing device and a distributed system, which is applied in the direction of electrical digital data processing, input/output process of data processing, transmission system, etc., and can solve the problems such as the reduction of data processing efficiency of the distributed system.
Active Publication Date: 2018-10-19
HUAWEI TECH CO LTD
View PDF5 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0008] Embodiments of the present invention provide a data processing method, device, and system in a distributed system, which are used to solve the problem of data processing in a distributed system caused by computing nodes needing to wait for the completion of tasks based on the original partition view before performing data processing based on the new partition view. The problem of reduced efficiency
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0111] The management node in the distributed system is responsible for managing and maintaining the information of the storage nodes of the data, generating a mapping table between the data identifier and the data storage location, and sending the mapping table to each computing node, and the computing node saves the mapping table in the On the local storage, the mapping table is also called the partition view on the computing node. The computing node determines the storage node of the data based on the saved partition view, and then initiates a data service (such as read operation, write operation) request to the corresponding storage node.
[0112] Embodiments of the present invention provide a data processing method, device, and system in a distributed system. The technical solution of the present invention will be described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and th...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a method and a device for processing data in a distributed system, and a system which are used to solve the problem that the data processing efficiency of the distributed system is low because that computing nodes can perform data processing based on new partitioned views until tasks based on original partitioned views are completed. The method for processing the data in the distributed system includes that: when a first computing node uses first updating data to perform an updating operation on first data according to a first partitioned view, a second computing node receives a second partitioned view; the second computing node uses second updating data to perform the updating operation on the first data according to the second partitioned view; wherein at least two storing nodes which can store the first data in a first moment and are indicated by the first partitioned view and at least two storing nodes which can store the first data in a second moment and are indicated by the second partitioned view, are not completely identical.
Description
technical field [0001] The invention relates to the field of data storage, in particular to a data processing method, device and system in a distributed system. Background technique [0002] A distributed system refers to a system that distributes computing resources and storage resources to different nodes and connects the nodes through the network. A distributed system can complete a task by managing and controlling multiple nodes, but it appears as an independent and complete system externally, and users cannot perceive the internal execution details of distributed system tasks. [0003] figure 1 It is a schematic diagram of a distributed system, in which the client is used to receive user requirements (such as write requests and read requests), the computing nodes are responsible for distributing routes, and the storage nodes are responsible for single disk management and actual storage. In addition, the system also includes management nodes , responsible for managing ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/08G06F3/06
CPCG06F3/067H04L67/1097
Inventor 薛贞文
Owner HUAWEI TECH CO LTD