Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Mass-unstructured data distributed type processing structure for description information

A technology of unstructured data and distributed processing, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve problems such as powerlessness, achieve the effects of reducing access pressure, saving computing resources, and satisfying system availability

Active Publication Date: 2014-12-17
JINAN GRANDLAND DATA TECH
View PDF2 Cites 15 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The traditional centralized storage and processing architecture has been unable to cope with such a massive data scale, and the existing distributed system cannot simply solve this problem

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Mass-unstructured data distributed type processing structure for description information
  • Mass-unstructured data distributed type processing structure for description information
  • Mass-unstructured data distributed type processing structure for description information

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0030] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0031] figure 1 , a distributed processing architecture for massive unstructured data with descriptive information. It includes the following modules:

[0032] Module 1. Data acquisition module. Contains several collection terminals, responsible for collecting unstructured data and sending them to the data buffer queue. If the terminal device supports it, the basic description information identification or labeling work can be completed here. If the device is relatively lightweight, this work can be transferred to the next module for completion.

[0033] Module 2. Data buffering and preprocessing module. It mainly includes a high-availability distributed buffer queue, which is used to temporarily store the data sent by the d...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a mass-unstructured data distributed type processing structure for description information. The mass-unstructured data distributed type processing structure comprises a data collecting module, a data buffering and pre-processing module, a data separating and filing storage module, a stream processing module, a distributed type data storage module, a distributed type service processing module and distributed type massage-oriented middleware. The data collecting module is used for collecting and sending unstructured data to a data buffering queue. The data buffering and pre-processing module is used for temporally storing the data sent by the data collecting module and selectively repairing or carrying out secondary processing on the data. The data separating and filing storage module is used for acquiring data from a prior module distributed type queue, selectively separating the unstructured data from the description information, and the separated data are forwarded or stored in a successor module. The stream processing module is used for monitoring, comparing, calculating and processing new access data. The distributed type data storage module is used for storing the unstructured data and the description information. The distributed type service processing module comprises a service processor, a data access unit and a data buffering assembly. The distributed type massage-oriented middleware is used fore receiving front-end requests for a service processor to carry out selective executing or returning the background processing result to a front end.

Description

technical field [0001] The invention relates to a distributed system architecture, which belongs to the field of computer application software development, in particular to a distributed processing architecture for massive unstructured data with description information. Background technique [0002] In recent years, with the improvement of data collection and sensing technology, people can collect more and more data and extract valuable information from it for analysis and decision-making. Generally, the collected data can be divided into three categories: one category is structured data that can be logically expressed through a two-dimensional table structure; the second category is semi-structured data that has a certain structure but the structure itself changes greatly; the last category is Classes are unstructured data such as electronic documents, pictures, videos, etc. Combining the third type of data with the first two types becomes unstructured data with descriptiv...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/337
Inventor 林立伟崔星灿董振
Owner JINAN GRANDLAND DATA TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com