Data processing method and device, data system, computing node, electronic equipment and computer readable storage medium

A data storage and data processing technology, applied in the field of data processing, can solve problems such as hot spots cannot be eliminated

Pending Publication Date: 2021-09-21
ALIBABA GRP HLDG LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Embodiments of the present application provide a data processing method and device, a data system, a computing node, electronic equipment, and a computer-readable storage medium, so as to solve the defect that hot spots cannot be eliminated in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and device, data system, computing node, electronic equipment and computer readable storage medium
  • Data processing method and device, data system, computing node, electronic equipment and computer readable storage medium
  • Data processing method and device, data system, computing node, electronic equipment and computer readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0044] The solutions provided in the embodiments of the present application can be applied to any data processing system with data processing capabilities. Figure 1b It is a schematic diagram of the application scenario of the data processing method provided in the embodiment of the present application, Figure 1b The shown scene is only one example of the scene to which the technical solution of the present application can be applied.

[0045]With the development of distributed database technology, more and more cloud server providers provide tenants with distributed database services. The problem is that the amount of data read and write operations in the database is different. In other words, as a distributed database distributed on multiple nodes, especially a large-scale database needs to use more nodes to store its multiple fragments. Usually, when the management node of the distributed database receives the data that needs to be stored, it calculates the shard to whi...

Embodiment 2

[0055] figure 2 It is a flow chart of an embodiment of the data processing method provided in this application. The execution subject of the method may be various terminals or server devices with data processing capabilities, or may be devices or chips integrated on these devices. Such as figure 2 As shown, the data processing method includes the following steps:

[0056] S201. Receive a data writing request.

[0057]In this embodiment of the application, the tenant of the distributed cloud database may send a data write request to a node of the distributed cloud database, such as the control node RC. The data write request may include first data information indicating data to be written. In other words, the data writing request may be a data writing request caused by a tenant of the distributed database operating on the database. For example, when the cloud database of this application is the sales record database of the e-commerce website, the data writing request may ...

Embodiment 3

[0067] image 3 It is a flow chart of another embodiment of the data processing method provided in this application. The method may be executed by various terminals or server devices with data processing capabilities, or devices or chips integrated on these devices. Such as image 3 As shown, the data processing method includes the following steps:

[0068] S301. Obtain operation status data of at least one data storage node according to a preset index.

[0069] In the embodiment of the present application, in order to accurately determine whether the node of the data storage partition to which the data to be written belongs is a hotspot, the running status of each data storage node may be acquired in real time or periodically. In particular, indicators capable of measuring the load of nodes can be specified in advance, and corresponding operating data can be obtained according to these indicators. Of course, in the embodiment of the present application, it is also possible...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data processing method and device, a data system, a computing node, electronic equipment and a computer readable storage medium. The method comprises the following steps: receiving a data writing request; determining first partition identification information of a first data storage partition to which the to-be-written data belongs according to the first data information; determining at least one piece of second partition identification information of a second data storage partition in which the to-be-written data is written according to a write routing table and the first partition identification information; and writing the to-be-written data into the second data storage partition identified by the second partition identification information according to the second partition identification information. According to the embodiment of the invention, the write routing table reflecting the running state of the processing node can be utilized to adjust the allocation of the fragments calculated based on the data involved in the data write-in request, so that adaptive adjustment of the fragments is realized, hot spots are eliminated, and the influence of node performance reduction on user experience is avoided.

Description

technical field [0001] The present application relates to the technical field of data processing, and in particular to a data processing method and device for data storage, a data system, a computing node, electronic equipment, and a computer-readable storage medium. Background technique [0002] With the development of data processing technology, the demand for data storage is also increasing. For this reason, a distributed storage technology has emerged, which can store a large amount of data through storage servers based on multiple nodes connected by physical connections or Internet connections. Distributed storage has high scalability and flexibility because it disperses data storage to multiple nodes. For the scenario where data is stored in a database, it is necessary to additionally consider the relationship between the data stored in each database and the database. Therefore, a distributed database technology based on distributed storage has emerged, which divides ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/22G06F16/27
CPCG06F16/22G06F16/27
Inventor 梁卜元酆晓杰
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap