Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

Data storage processing method, apparatus, computing device, and computer storage medium

A computer storage and data storage technology, applied in the Internet field, can solve problems such as memory explosion, low data storage efficiency, and poor flexibility, and achieve the effects of reducing memory processing pressure, improving efficiency and flexibility, and optimizing data storage and processing methods

Inactive Publication Date: 2018-12-25
BEIJING QIHOO TECH CO LTD
View PDF3 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In some stream processing frameworks, list data (such as ListState in Flink) is persistently stored in the database as a whole, so even if only a single element in a list data needs to be operated, it needs to be obtained from the database The data of the entire list data, therefore, this processing method has the problems of low data storage efficiency and poor flexibility
In addition, if the list data has a very large data volume, it may also cause a memory explosion

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data storage processing method, apparatus, computing device, and computer storage medium
  • Data storage processing method, apparatus, computing device, and computer storage medium
  • Data storage processing method, apparatus, computing device, and computer storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0022] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0023] figure 1 A schematic flow diagram showing a data storage and processing method according to an embodiment of the present invention, such as figure 1 As shown, the method includes the following steps:

[0024] Step S100, acquiring list data.

[0025] This method is suitable for stream processing frameworks such as Flink. Taking the Flink stream processing framework as an example, according to its fault tolerance mechanism,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data storage processing method, a device, a computing device and a computer storage medium. The method comprises obtaining list data; converting the list data into converteddata in the form of at least two data key-value pairs, wherein at least two data key-value pairs have index information on the converted data in the form; converted data in the form of at least two data key-value pairs being stored in a database. According to the technical proposal provided by the invention, the list data can be conveniently converted into converted data in the form of at least two data key-value pairs and stored, the data storage efficiency and flexibility are effectively improved, the memory processing pressure is reduced, and the data storage processing mode is optimized.

Description

technical field [0001] The present invention relates to the technical field of the Internet, in particular to a data storage and processing method, device, computing equipment and computer storage medium. Background technique [0002] With the continuous development of Internet technology, data processing technology has also developed rapidly, and many streaming processing frameworks such as Storm, Spark, and Flink have emerged. In these stream processing frameworks, most of the state information is managed and stored persistently. When a machine, node, etc. fails for some reason, the stored state information can be used for state restoration. In some stream processing frameworks, list data (such as ListState in Flink) is persistently stored in the database as a whole, so even if only a single element in a list data needs to be operated, it needs to be obtained from the database The data of the entire list data, therefore, this processing method has the problems of low data...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30
Inventor 张馨予左欢
Owner BEIJING QIHOO TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com