NiFi-based two-level cache data processing method and system

A data processing system and secondary cache technology, applied in the field of data management, can solve problems such as affecting the quality of business data, unable to capture data, unable to capture data, etc.

Active Publication Date: 2019-01-11
LINEWELL SOFTWARE
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] (1) When the existing technology caches the database query results, the deleted data in the database cannot be captured, resulting in the retention of the discarded data. When the cache query is matched, the outdated cache is matched, which affects the quality of business data
[000

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • NiFi-based two-level cache data processing method and system
  • NiFi-based two-level cache data processing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0040] In order to make the object, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with the examples. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0041] The present invention provides a practical and convenient method to solve the defects of NiFi's own cache components, so that when the cache components are used, the invalid cache can be removed in time without affecting the use of other business components.

[0042] Below in conjunction with accompanying drawing, the present invention is described in further detail;

[0043] Such as figure 1 As shown, the NiFi-based secondary cache data processing method provided by the embodiment of the present invention includes:

[0044] S101: Start the secondary cache server, and monitor the data signals of "put Secondary Cach...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the technical field of data management, and discloses a NiFi-based two-level cache data processing method and system. The method comprises the steps of starting two-level cache server to monitor data signals; receiving a user operation request and converting it into a signal for sending to a server; sending the request buffer data to the secondary buffer; sending a SecondaryCache signal to a secondary buffer server; storing the cached data in SecondaryCache. The secondary cache client service sends the applySecondaryCache signal to the secondary cache server; The cached data of the SecondaryCache are copied into a new cache object and the new cache object pointer is pointed to the cache used for the query. The invention adds a two-level cache for temporarily storing cache data, and has atomicity to the operation of the original cache, so as to achieve the purpose of safely deleting the invalid cache.

Description

technical field [0001] The invention belongs to the technical field of data management, and in particular relates to a NiFi-based secondary cache data processing method and system. Background technique [0002] At present, the existing technologies commonly used in the industry are as follows: [0003] With the componentization of big data ETL, many ETL functions have been implemented to allow users to connect and configure the ETL process in the interface by dragging and dropping. NiFi is such a powerful ETL engine, and it has realized the requirements of many ETL components, which greatly reduces the threshold for users to perform ETL processing on data. As a caching component that is often used in the NiFi engine, it cannot meet most of the needs in the face of actual business scenarios. This is because when caching database query results, the deleted data in the database cannot be captured, resulting in discarded data. When performing cache query matching, outdated cac...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/2455G06F12/0897G06F9/54
CPCG06F9/544G06F12/0897
Inventor 吴晓梅郭廷镕陈耿坤
Owner LINEWELL SOFTWARE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap