ElasticSearch-based data indexing method and device, computer equipment and storage medium

A technology of data indexing and indexing, which is applied in the field of data processing, and can solve problems such as difficult to meet the needs of indexing and low indexing efficiency

Pending Publication Date: 2020-10-20
CHINA PING AN PROPERTY INSURANCE CO LTD
View PDF0 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The existing technology uses the distributed search engine ElasticSearch to perform distributed search, and then merges the search result data to obtain the final data. This indexing scheme will have very low indexing efficiency before facing massive data of more than 100 terabytes , it is difficult to meet the needs of indexing; there is an urgent need for a solution that can improve the efficiency of data indexing for massive data of more than 100 terabytes

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • ElasticSearch-based data indexing method and device, computer equipment and storage medium
  • ElasticSearch-based data indexing method and device, computer equipment and storage medium
  • ElasticSearch-based data indexing method and device, computer equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the application; the terms used herein in the description of the application are only to describe specific embodiments The purpose is not to limit the present application; the terms "comprising" and "having" and any variations thereof in the description and claims of the present application and the description of the above drawings are intended to cover non-exclusive inclusion. The terms "first", "second" and the like in the specification and claims of the present application or the above drawings are used to distinguish different objects, rather than to describe a specific order.

[0042] Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The occurrenc...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an ElasticSearch-based data indexing method and a device, computer equipment and a storage medium. The method comprises the steps: setting a main fragment based on a distributed search engine ElasticSearch, and writing data into the main fragment; aiming at the data of each main fragment, respectively creating an index task, importing the corresponding index fragment into the main fragment according to the type of the index task, and creating an index; if a data query request instruction is detected, performing index query on the index fragments through each node serverto obtain the index data volume of each node server; combining the index data volume of each node server to obtain a total index data volume; and if it is detected that the total index data volume exceeds a preset numerical value, indexing in a preset mode. The invention also relates to the blockchain technology. Index data is stored in the blockchain. According to the method, the data indexing efficiency is improved for mass data of hundreds of T or above.

Description

technical field [0001] The present application relates to the technical field of data processing, in particular to an ElasticSearch-based data indexing method, device, computer equipment and storage medium. Background technique [0002] In the current age of big data, there are many data warehouses used to store massive data, and the distributed search engine ElasticSearch (search engine, ES) is one of them. The distributed search engine ElasticSearch is an open source, distributed, Restful search server based on Lucene, which is generally used in cloud computing. It can easily enable large amounts of data to be searched, analyzed and explored. Making full use of the horizontal scalability of the distributed search engine ElasticSearch can make data more valuable in the production environment. [0003] For the indexing scheme of the distributed search engine ElasticSearch for massive data of more than 100 terabytes, due to the huge amount of data, if there is no correspond...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/2458G06F16/27
CPCG06F16/2228G06F16/2471G06F16/27
Inventor 王康任
Owner CHINA PING AN PROPERTY INSURANCE CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap