Distributed unstructured database association query method based on big data

An unstructured, associative query technology, applied in the field of data processing, can solve the problems that relational databases are difficult to solve for massive data storage, affect write performance, and have serious query performance bottlenecks, and achieve the effect of solving real-time storage problems.

Pending Publication Date: 2020-11-13
银盛支付服务股份有限公司
View PDF0 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, with the rapid growth of business, the bottlenecks of relational databases have gradually become prominent, such as: (1) the data volume has increased sharply, and the query performance bottleneck is serious; (2) the query bottleneck of traditional relational databases affects the writing performance in the transaction process, and urgently needs Consider reading and writing separation; (3) Traditional relational databases are difficult to solve the problem of massive data storage

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed unstructured database association query method based on big data
  • Distributed unstructured database association query method based on big data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0037] The idea, specific structure and technical effects of the present invention will be clearly and completely described below in conjunction with the embodiments and accompanying drawings, so as to fully understand the purpose, features and effects of the present invention. Apparently, the described embodiments are only some of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative efforts belong to The protection scope of the present invention. In addition, all the connection / connection relationships involved in the patent do not simply refer to the direct connection of components, but mean that a better connection structure can be formed by adding or reducing connection accessories according to specific implementat...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distributed unstructured database association query method based on big data, and relates to the technical field of data processing. The method comprises the following stepsof: S1, generating data tables, and designing data table instances associated with multiple tables by adopting mapping in combination with service requirements so as to obtain a plurality of data tables; s2, monitoring a data table; s3, data synchronization; s4, ETL processing and data aggregation: cleaning and preprocessing the data subjected to data synchronization in the step S3 through a kafkacluster, and storing the aggregated data into an unstructured database in a landing manner; s5, selection of a fragmentation algorithm: when the data reaches the Tb-level data volume, designing the fragmentation algorithm in combination with a query scene to realize distributed mass storage of the data; the method has the advantages that read-write separation of the structured database and the unstructured database can be achieved, and the mass data storage problem is solved.

Description

technical field [0001] The present invention relates to the technical field of data processing, and more specifically, the present invention relates to a large data-based distributed unstructured database associated query method. Background technique [0002] A relational database is a database based on a relational model, which uses mathematical concepts and methods such as set algebra to process data in the database. Various entities in the real world and various links between entities are represented by relational models. The standard data query language SQL is a language based on relational databases. This language executes the retrieval and operation of data in relational databases. The relational model consists of three parts: relational data structure, relational operation set, and relational integrity constraints. [0003] Traditional relational databases involve associated queries between tables, which can be implemented through views. However, with the rapid gro...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/25G06F16/215G06F16/27
CPCG06F16/215G06F16/254G06F16/27
Inventor 刘跃红曹浩曾国江郭倩
Owner 银盛支付服务股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap