Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data writing method and device for distributed file system, equipment and medium

A distributed file and data writing technology, applied in the direction of electronic digital data processing, special data processing applications, digital data information retrieval, etc., can solve the problems of long data writing time, low data writing efficiency, etc. The effect of improving input efficiency and shortening the writing time

Pending Publication Date: 2021-08-03
平安消费金融有限公司
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The technical problem to be solved by the present invention is that the data writing method of the current distributed file system takes a long time to write data, and the data writing efficiency is low

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data writing method and device for distributed file system, equipment and medium
  • Data writing method and device for distributed file system, equipment and medium
  • Data writing method and device for distributed file system, equipment and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0032] see figure 1 , figure 1 It is a schematic flowchart of a method for writing data in a distributed file system disclosed in an embodiment of the present invention. Such as figure 1 As shown, the data writing method of the distributed file system may include the following operations:

[0033] 101. Detect whether a write request for target data sent by a client of the distributed file system is received;

[0034] In the above step 101, the distributed file system usually consists of a client (client), a namenode and several datanodes. Wherein, client (client) may be used to initiate a write request of target data. Optionally, the client (client) can also split the file to be written into multiple data blocks, and then write the multiple data blocks into the distributed file system one by one. For example, the size of the file to be written is 300M, if the default size of the data block in datenode is 128M, the client (client) can split the file into three data blocks,...

Embodiment 2

[0080] see figure 2 , figure 2 It is a schematic structural diagram of a data writing device for a distributed file system disclosed in an embodiment of the present invention. Such as figure 2 As shown, the data writing device of the distributed file system may include:

[0081] A detection module 201, configured to detect whether a write request for target data sent by a client of the distributed file system is received;

[0082] A determination module 202, configured to determine a master data storage node and at least one slave data storage node from multiple data storage nodes of the distributed file system when the detection module detects the write request;

[0083] A writing module 203, configured to write the target data into the primary data storage node;

[0084] return module 204, configured to write the target data to the distributed file system after the writing module 203 finishes writing the target data in the primary data storage node A confirmation mes...

Embodiment 3

[0088] see image 3 , image 3 It is a schematic structural diagram of a computer device disclosed in an embodiment of the present invention. Such as image 3 As shown, the computer equipment may include:

[0089] A memory 301 storing executable program codes;

[0090] A processor 302 connected to the memory 301;

[0091] The processor 302 invokes the executable program code stored in the memory 301 to execute the steps in the data writing method of the distributed file system disclosed in the first embodiment of the present invention.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data writing method of a distributed file system. The data writing method comprises the following steps: detecting whether a writing request for target data sent by a client of the distributed file system is received or not; when the write-in request is detected, determining a master data storage node and at least one slave data storage node from a plurality of data storage nodes of the distributed file system; writing the target data into the main data storage node; after writing of the target data in the main data storage node is completed, returning confirmation information used for representing that the target data is successfully written into the distributed file system to the client; and writing the target data into each slave data storage node based on a preset data synchronization background thread. Therefore, the writing time of the target data can be shortened, and the writing efficiency of the target data is improved. The invention also relates to the technical field of block chains.

Description

technical field [0001] The present invention relates to the technical field of cluster storage, in particular to a data writing method, device, computer equipment and storage medium of a distributed file system. Background technique [0002] Distributed File System (HDFS) is a system suitable for storing large files on inexpensive machines. In order to ensure data security, distributed file systems usually store data in a data redundant manner, that is, there are usually multiple copies of files stored in distributed file systems, and each copy is stored in a different on the node. In this way, when writing data to the distributed file system, it is necessary to wait for multiple copies of the written data to be successfully written to the distributed file system before it can be confirmed that the written data is successfully written. The number of copies of data in the file system is three. When writing a piece of data to the distributed file system, it is necessary to w...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/178G06F16/182
CPCG06F16/178G06F16/182Y02D10/00
Inventor 赵宏军
Owner 平安消费金融有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com