Unlock instant, AI-driven research and patent intelligence for your innovation.

A data update method

A data update and data technology, which is applied in the field of big data processing, can solve the problems of unknown update, inability to know, and inability to perform incremental data processing, and achieve the effect of reducing waste of resources and reducing the amount of data processing.

Active Publication Date: 2019-11-01
杭州费尔斯通科技有限公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In a big data processing system, data updates are generally performed through batch operations. Without full data comparison, it is impossible to know whether the data is actually updated during the data update process.
The actual update of data is unknown, resulting in the inability to perform incremental data processing in the whole process of data processing, resulting in a huge waste of resources in the entire computing system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data update method
  • A data update method
  • A data update method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0018] The present invention will be described in further detail below in conjunction with the accompanying drawings and specific embodiments.

[0019] like figure 1 As shown, the present invention proposes a data update method, which combines the synchronization, update, and comparison of the Elasticsearch data internal version number _version and the self-defined business data version number to determine whether the data is actually updated. This method can be implemented repeatedly and remains effective. The method specifically includes the following steps:

[0020] The first step is to add a business data version number field. In order to determine whether the data is actually updated, add a business data version number field to the data record, assuming the field name is out_version, and initialize out_version, for example, it can be directly set to 1.

[0021] The second step is to synchronize the business data version number and internal version number _version. The...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data updating method. The data updating method comprises the four steps of adding a service data version number, synchronizing the service data version number and an internalversion number, executing actual service data updating operation, checking and marking actual data updating. Adding a service data version number is an implementation basis of the method; Synchronizing service data version numbers and internal version numbers are key steps of the method, and a comparison standard is kept uniform. Checking and marking actual data updating, determining whether thedata is actually updated or not by comparing the version number of the service data with the internal version number, and optimizing the subsequent data processing performance according to the actualdata updating condition. In a traditional Elasticsearch-based big data processing system, a large amount of time and performance waste is caused due to the fact that whether data is actually updated or not cannot be known and a large amount of full data reading, writing and calculating conditions exist. According to the method, the defects are effectively overcome, and system resources and data processing time are saved.

Description

technical field [0001] The invention relates to the technical field of big data processing, in particular to an Elasticsearch-based data update method. Background technique [0002] Elasticsearch is an open source search server. It provides a distributed multi-user capable full-text search engine based on a RESTful web interface. Elasticsearch, developed in Java and released as open source under the terms of the Apache license, is currently a popular enterprise-grade search engine. [0003] Elasticsearch is currently a very popular system component of enterprise big data solutions, which has the functions of data storage and search engine. In the technical solution using Elasticsearch as data storage, data update is a common data operation. Elasticsearch provides the following features for data update: elasticsearch records an internal version number (_version) for each data record. When a record is created for the first time, elasticsearch sets _version=1 for it; every t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/21G06F16/23
Inventor 金霞
Owner 杭州费尔斯通科技有限公司