Data migration method and device, equipment and storage medium

A data and technology to be migrated, applied in the field of big data, to achieve the effect of increasing the migration rate and reducing the impact

Pending Publication Date: 2022-04-29
AGRICULTURAL BANK OF CHINA
View PDF0 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, in the era of big data, the access frequency of data often changes dramatically in a short period of time. When the system is in the peak period of transaction, a large number of data migration between layers will put a huge pressure on the system.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data migration method and device, equipment and storage medium
  • Data migration method and device, equipment and storage medium
  • Data migration method and device, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0050] figure 1It is a flow chart of a data migration method provided by Embodiment 1 of the present invention. This embodiment is applicable to data migration of data stored in two storage areas based on data access conditions. This method can be implemented by the data migration device, which can be implemented in software and / or hardware, such as figure 1 As shown, the method specifically includes the following steps:

[0051] S110. Obtain access information and the current data migration rate between the first storage area and the second storage area, where the access information includes: current system access information and historical data access information.

[0052] Among them, the access information includes: current system access information and historical data access information. The current system access information is used to indicate the real-time access information of the unstructured data service system requested by external parties at the current access time...

Embodiment 2

[0068] Figure 2A It is a flow chart of a data migration method in Embodiment 2 of the present invention. This embodiment is optimized based on the above embodiments. In this embodiment, the adaptive control model includes: target neural network control model and target adaptive Correction model; correspondingly, the current system access information and the current data migration rate are input into the adaptive control model to obtain the migration control factor, including: the current system access information is input into the target neural network control model to obtain the output factor, and the target neural network control model passes the historical The initial neural network control model is obtained by iteratively training the system access information sample set; the output factor is compressed through the activation function; the current data migration rate is input into the target adaptive correction model to obtain the adaptive correction factor, and the target...

Embodiment 3

[0111] image 3 It is a schematic structural diagram of a data migration device provided by Embodiment 3 of the present invention. This embodiment is applicable to the situation where data stored in two storage areas is migrated based on data access conditions. The device can be implemented in software and / or hardware, and the device can be integrated into any device that provides the function of data migration in, such as image 3 As shown, the data migration apparatus specifically includes: an acquisition module 310 , a migration control factor determination module 320 , a data information determination module 330 to be migrated, and a data migration module 340 .

[0112] Wherein, the acquiring module 310 is configured to acquire access information and the current data migration rate between the first storage area and the second storage area, the access information includes: current system access information and historical data access information;

[0113] A migration cont...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data migration method and device, equipment and a storage medium. The method comprises the steps that access information and the current data migration rate between a first storage area and a second storage area are obtained, and the access information comprises current system access information and historical data access information; inputting the current system access information and the current data migration rate into an adaptive control model to obtain a migration control factor; inputting the historical data access information into a migration data analysis model to obtain to-be-migrated data information; threshold parameters of the migration data analysis model are determined by migration control factors; and based on the to-be-migrated data information, executing a data migration operation between the first storage area and the second storage area. According to the technical scheme, data migration can be adaptively controlled according to the data access information and the current data migration rate, the migration rate is reduced in the peak access period, the migration rate is improved in the low peak access period, and therefore the influence of data migration on system services is reduced.

Description

technical field [0001] The embodiments of the present invention relate to the technical field of big data, and in particular, to a data migration method, device, equipment, and storage medium. Background technique [0002] In the era of big data, traditional data centers are shifting to cloud computing, big data, and artificial intelligence, and more and more applications and enterprises are using object storage services. When unstructured data grows massively, object storage data does not separate hot and cold data, which is not conducive to access to hot data. Among them, hot data refers to data with high access frequency, and cold data refers to data with low access frequency. [0003] In response to the above problems, the prior art proposes a data layered storage system, including a data monitoring module, an automatic layering module, and a data migration module, which establishes sample data for each data layer, and analyzes the data access hit rate of each layer, W...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06G06N3/08
CPCG06F3/0647G06F3/0604G06F3/0644G06N3/08
Inventor 宋成王凯刘言杰张磊孙蕾
Owner AGRICULTURAL BANK OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap