A data processing method and device based on a distributed file system

A technology for distributed files and data processing. It is used in electrical digital data processing, special data processing applications, instruments, etc. It can solve problems such as inability to fully utilize iSCSI performance, and achieve the effect of improving processing performance.

Inactive Publication Date: 2017-07-28
TOYOU FEIJI ELECTRONICS
View PDF2 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] When a traditional block device processes block data, when multiple initiators (clients) store data in a distributed file system through iSCSI (internet Small Computer System Interface), multiple initiators can only connect to one iSCSI. Instead of connecting to multiple

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data processing method and device based on a distributed file system
  • A data processing method and device based on a distributed file system
  • A data processing method and device based on a distributed file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0045] It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0046] The present invention provides a method for processing data based on a distributed file system, referring to figure 1 and Figure 7 , in the first embodiment, the method for processing data based on the distributed file system includes:

[0047] Step S10, connect with at least one client;

[0048] In this embodiment, by establishing multiple processes on the basis of iSCSI, each process can correspond to one or more initators (clients), and improve the throughput of system storage through multiple threads. Assigned to multiple threads, this is equivalent to handing over the processing of a block device to multiple tasks for processing, and each task processes a part of it, so that processing at the same time speeds up the speed.

[0049] Step S20, start the epoll network model, when detecting that there is t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data processing method based on a distributed file system. The method comprises the steps of performing connection with at least one initiator; starting an epoll network model, and when detecting transmitted network data, processing the data in response to corresponding threads, wherein multiple threads correspond to one initiator; starting the corresponding threads to receive a request message sent by the initiator and packaging the request message into SCSI command words; processing the received request message and storing metadata corresponding to the processed request message into a data memory pool. The invention also provides a data processing device based on a distributed file system. The method and the device combine multiple threads with iSCSIs, so that each thread is corresponding to one or more initators and the processing properties of a distributed file system block device is improved.

Description

technical field [0001] The invention relates to the field of distributed system storage, in particular to a method and device for processing data based on a distributed file system. Background technique [0002] When a traditional block device processes block data, when multiple initiators (clients) store data in a distributed file system through iSCSI (internet Small Computer System Interface), multiple initiators can only connect to one iSCSI. Instead of connecting to multiple iSCSI devices, iSISI can transmit data to each other, but externally, the initator can only have one connection. Since multiple initators cannot connect to multiple iSCSI devices at the same time, there is a technical bottleneck. The performance of iSCSI cannot be fully utilized. Contents of the invention [0003] The main purpose of the present invention is to provide a data processing method and device based on a distributed file system, aiming at solving the single-threaded data transmission mo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/185
Inventor 李进张勤
Owner TOYOU FEIJI ELECTRONICS
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap