A data processing method, a distributed file system and a data server

A data server and distributed file technology, applied in the field of big data analysis, can solve the problem that the distributed file system cannot provide data processing.

Inactive Publication Date: 2017-09-15
SHENZHEN IPANEL TECH LTD
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In view of this, the present invention provides a data processing method, a distributed file system and a data server to solve the problem in the prior art that the distributed file system cannot provide its internal data processing conditions

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data processing method, a distributed file system and a data server
  • A data processing method, a distributed file system and a data server
  • A data processing method, a distributed file system and a data server

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0059] A flow chart of a data processing method provided by the present invention is as follows figure 1 As shown, the method is applied to a distributed file system, including:

[0060] Step S11: receiving the data uploaded by the client;

[0061] In this embodiment, the distributed file system corresponds to multiple clients, and each client collects user behavior data in a "buried point" manner. When the data collected by the client's buried point meets any of the following two conditions, it will be uploaded. The two conditions are: ① User behavior data meets the preset number, such as 20, and the preset number can be set according to the actual situation; ② Satisfies the preset time interval, such as 1 minute. Similarly, the preset time interval can also be Set it according to the actual situation.

[0062]The distributed file system receives the data uploaded by the client.

[0063] Step S12: counting the total number of received data of all the data uploaded by the ...

Embodiment 2

[0073] On the basis of the first embodiment, this embodiment specifically introduces the specific processes of step S12 and step S14 in the first embodiment. In this embodiment, it should first be explained that the distributed file system includes multiple data servers.

[0074] In the distributed file system, receiving the data uploaded by the client and filtering unqualified data in the data uploaded by the client are all completed by each data server in the system. Similarly, multiple clients corresponding to the distributed file system correspond to respective data servers.

[0075] For step S12, another flow chart of a data processing method provided by the present invention is as follows figure 2 As shown, the specific steps of counting the total number of received data of all the data uploaded by the client are shown, including:

[0076] Step S21: Obtain the number of received data pieces received by each data server; wherein, the number of received data pieces is c...

Embodiment 3

[0089] On the basis of Embodiment 1, another flow chart of a data processing method provided by the present invention is as follows Figure 4 As shown, after step S14, it may also include:

[0090] Step S15: Displaying the total number of received data and the total number of filtered data in real time.

[0091] The data processing method provided in this embodiment displays the total number of received data and the total number of filtered data in real time, which is convenient for visual viewing. When displaying, a single digital display method can be used, or a combination of numbers and graphs can be used, and the historical curve of a historical time period (such as 1 hour, etc.) is retained in the graph.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data processing method applied to a distributed file system. The method comprises the steps of receiving data uploaded by a client; counting the total number of pieces of received data for reception of all the data uploaded by the client; filtering off unqualified data in the data uploaded by the client; counting the total number of pieces of filtered data with all unqualified data being filtered off. Thus, the distributed file system performs statistical processing on the number of pieces of all received data uploaded by a client and the number of pieces of all filtered-off unqualified data, so that technicians can judge the operation condition of the distributed file system according to the statistical data processing condition.

Description

technical field [0001] The invention relates to the field of big data analysis, in particular to a data processing method, a distributed file system and a data server. Background technique [0002] Big data analysis means that enterprises capture massive, diverse and real data in various ways, and then analyze and process these data. The distributed file system is the basis of big data analysis, which contains multiple data servers to provide the required data for big data analysis. The distributed file system corresponds to multiple clients. Each client uses the page buried point technology to collect user behavior data, and then reports the collected data to the distributed file system. The distributed file system filters unqualified data and retains qualified ones. data. [0003] In the prior art, the distributed file system cannot provide its internal data processing conditions, so it cannot provide a basis for technicians to judge the operation of the distributed file...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/182
Inventor 徐佳宏赖国琴陈恋
Owner SHENZHEN IPANEL TECH LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap