Data aggregation method, device and equipment and computer readable storage medium

A technology of data aggregation and computer program, applied in the field of data aggregation, which can solve problems such as data loss

Inactive Publication Date: 2020-11-03
INSPUR SUZHOU INTELLIGENT TECH CO LTD
View PDF0 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In view of this, the purpose of this application is to provide a data aggregation method, a data aggregation device, a data aggregation device, and a computer-readable storage medium, which solve the problem of data loss that may be caused by related technologies during data aggregation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data aggregation method, device and equipment and computer readable storage medium
  • Data aggregation method, device and equipment and computer readable storage medium
  • Data aggregation method, device and equipment and computer readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0045] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments It is only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

[0046] In one possible implementation, please refer to figure 1 , figure 1 It is a flow chart of a data aggregation method provided by the embodiment of this application. The method includes:

[0047] S101: Obtain initial data, and write the initial data into a source storage pool.

[0048] It should be noted that all or pa...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data aggregation method, device and equipment and a computer readable storage medium, and the method comprises the steps: obtaining initial data, and writing the initial datainto a source storage pool; reading a plurality of pieces of to-be-aggregated data from the source storage pool; aggregating the to-be-aggregated data to obtain aggregated data; writing the aggregated data into a target storage pool, and updating metadata corresponding to each piece of to-be-aggregated data; after the initial data is obtained, the initial data is not immediately aggregated, but is firstly written in and read out when aggregation is needed. If a cache failure fault occurs when the to-be-aggregated data is read out and waits for aggregation processing, or the cache failure fault occurs in the aggregation processing process, the to-be-aggregated data is written into the source storage pool, so that the problem of data loss does not occur.

Description

technical field [0001] The present invention relates to the technical field of data aggregation, in particular to a data aggregation method, a data aggregation device, a data aggregation device, and a computer-readable storage medium. Background technique [0002] In the era of big data, there are more and more massive data, and there are more and more applications for reading and writing small files with a size of a few K or more than a dozen K. In order to reduce the pressure of reading and writing data on the disk, the speed of using the disk to read and write large files is often significantly higher than the principle of small files. Related technologies do not directly write small files (such as a few K or more than a dozen K) Write to the cache first, wait in the cache and merge it into a large file (for example, the size is M level) before performing the disk operation, so as to improve the read and write speed of subsequent file reads and writes, and reduce the pres...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/2455
CPCG06F16/2219G06F16/24552
Inventor 孙业宽孟祥瑞
Owner INSPUR SUZHOU INTELLIGENT TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap