Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Unstructured data processing method and unstructured data processing system

A technology of unstructured data and processing methods, applied in the field of data processing, can solve the problems affecting distributed processing efficiency, waste of storage space, etc., and achieve the effects of improving distributed processing efficiency, saving storage space, and simple storage structure

Pending Publication Date: 2019-08-09
BOE TECH GRP CO LTD
View PDF6 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In view of this, the present invention provides an unstructured data processing method and an unstructured data processing system, which are used to solve the problem of storing a large amount of small unstructured data in the existing distributed file system, resulting in waste of storage space and affecting The problem of distributed processing efficiency

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Unstructured data processing method and unstructured data processing system
  • Unstructured data processing method and unstructured data processing system
  • Unstructured data processing method and unstructured data processing system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0043] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the following will clearly and completely describe the technical solutions of the embodiments of the present invention in conjunction with the drawings of the embodiments of the present invention. Apparently, the described embodiments are some, not all, embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention belong to the protection scope of the present invention.

[0044] In order to solve the problem of storing a large number of small files in the existing distributed file system, which causes waste of storage space and affects the efficiency of distributed processing, please refer to figure 1 , figure 1 It is a schematic flowchart of an unstructured data processing method according to an embodiment of the present invention, and the unstructured dat...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an unstructured data processing method and an unstructured data processing system. The unstructured data processing method comprises the following steps: acquiring unstructureddata; performing serialization processing on the unstructured data to obtain serialized data; connecting the serialized data with the index information of the unstructured data to obtain target data;and storing the plurality of target data into a target structured data file, wherein the target structured data file is used for a distributed file system. According to the method and the device, theplurality of unstructured data are subjected to serialization processing and are combined and stored into one structured data file for the distributed file system, and compared with the mode of storing a plurality of small unstructured data in the distributed file system, the required storage space can be effectively saved.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to an unstructured data processing method and an unstructured data processing system. Background technique [0002] Distributed file system (DFS) can effectively solve the storage and management problems of massive data: expand a certain file system fixed at a certain location to any number of locations / multiple file systems, and many nodes form a file system network . Each node can be distributed in different locations, and the communication and data transmission between nodes can be carried out through the network. When people use a distributed file system, they don't need to care about which node the data is stored on or from which node it is obtained from. They only need to manage and store the data in the file system like a local file system. [0003] However, in the face of increasingly large and massive files, the distributed file system also encounters some problem...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/182
CPCG06F16/182
Inventor 樊林
Owner BOE TECH GRP CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com