Heterogeneous data real-time search method in big data environment

A heterogeneous data and big data technology, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve problems such as single application scenarios, unfavorable data retrieval expansion, and inflexible result set storage mode, etc., to achieve The effect of increased accuracy

Inactive Publication Date: 2017-05-03
CHINA CHANGFENG SCI TECH IND GROUPCORP
View PDF0 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The implementation of retrieval technology in the traditional mode has relatively single application scenarios and strict requirements on the data source environment. The storage mode of the result set (or index library) of data retrieval is not flexible enough, which is not conducive to solving the retrieval problems brought about by the continuous growth of data. inflation problem

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Heterogeneous data real-time search method in big data environment
  • Heterogeneous data real-time search method in big data environment
  • Heterogeneous data real-time search method in big data environment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0011] figure 1 It is the overall architecture diagram of the present invention. Based on the comprehensive analysis of the current mature technical framework, the present invention adopts a scalable technical framework, which can provide reserved space for future data growth.

[0012] figure 2 Executing a schematic diagram for the indexing service cluster, the specific technical implementation mainly includes the following steps:

[0013] Step 1: Build a massive data index cloud service to balance the index storage load.

[0014] figure 2 The detailed execution process is generally described as follows:

[0015] A arrow indicates the start of a search request

[0016] B means to search for each shard according to the command of the main console.

[0017] C means to get the records retrieved respectively

[0018] D aggregates the initial result set of each fragment.

[0019] E sorts the initial result set, and returns qualified records according to the preset conditi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a heterogeneous data real-time search method in a big data environment. By establishing a mass data index cloud service, balance of index storage loads is realized; by optimizing a heterogeneous data content analytical algorithm, analysis is more accurate; by considering a weight of each peak namely a word in applying textrank, voting and iterative operating in a next step are performed to obtain characteristic words of an article; and before performing textrank keyword extraction, a weight of each word in a document in a data set is calculated, and then the weight value of the word is taken as textrank for inputting each corresponding word, and calculation of a next step is executed.

Description

technical field [0001] The invention relates to a real-time retrieval method for heterogeneous data in a big data environment. The main application fields include safe cities, smart transportation, smart cities and other fields. The method is not limited to specific application scenarios and has a wide range of applications. Background technique [0002] With the increasing popularity of information technology applications, information systems are showing an upward trend year by year, so the data generated by these information systems will also become more and more extensive. Especially with the birth of emerging platforms such as Safe City and Smart City, higher requirements are put forward for data integration and rapid data response. The implementation of retrieval technology in the traditional mode has relatively single application scenarios and strict requirements on the data source environment. The storage mode of the result set (or index library) of data retrieval is ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/2471G06F16/2453
Inventor 陈瑞蓝飞翔张宏左浩雷蒋志鸿
Owner CHINA CHANGFENG SCI TECH IND GROUPCORP
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap