Hierarchical storage method and device, electronic equipment, medium and program product

Abstract

The invention discloses a hierarchical storage method and device, electronic equipment, a medium and a program product.The method comprises the steps that at least one parameter associated with a to-be-analyzed file is obtained, and the at least one parameter comprises a self-defined first parameter and / or a second parameter of the to-be-analyzed file; based on the at least one parameter, a data state corresponding to the to-be-analyzed file is determined, and the data state comprises cold data or hot data; and migrating the to-be-analyzed file to the cold storage pool or the hot storage pool according to the data state. Through the method and the device, migration conditions of different to-be-analyzed files can be flexibly controlled, it can be determined that the to-be-analyzed files need to be migrated through the data state, the to-be-analyzed files are migrated in time, and the data storage efficiency is improved.

Description

Technical Field

[0001] The present application relates to, but is not limited to, the field of computer storage technology, and in particular to a hierarchical storage method, device, electronic device, medium, and program product. Background Art

[0002] With the rapid development of the Internet, the speed of big data generation has exploded. For the surging data, upper-layer applications have put forward higher storage and access performance requirements. In order to meet the above requirements, data tiered storage technology can be used to store frequently accessed hot data in high-performance devices to ensure performance, and store infrequently accessed cold data in low-cost large-capacity storage devices to save costs.

[0003] Migration strategy is the key to data tiered storage technology, which can determine the storage and access performance of data in the storage system. In related technologies, tiered storage determines the popularity of files based on global and historical access conditions, lacks real-time file access monitoring and analysis, and thus leads to untimely file migration, especially for files that are frequently accessed suddenly. Summary of the invention

[0004] In view of this, the present application provides a hierarchical storage method, device, electronic device, medium and program product, which can flexibly control the migration conditions of different files to be analyzed, and timely migrate the files to be analyzed, thereby improving the efficiency of data storage.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, the present application provides a hierarchical storage method, comprising: obtaining at least one parameter associated with a file to be analyzed, wherein the at least one parameter includes a customized first parameter and / or a second parameter of the file to be analyzed; based on the at least one parameter, determining a data state corresponding to the file to be analyzed, wherein the data state includes cold data or hot data; and migrating the file to be analyzed to a cold storage pool or a hot storage pool according to the data state.

[0007] In some embodiments, before migrating the file to be analyzed to a cold storage pool or a hot storage pool according to the data status, the hierarchical storage method also includes: obtaining the migration status and usage count corresponding to the file to be analyzed through preset fields; and determining, based on the migration status and usage count, that the file to be analyzed is not in other migration tasks and is not being used.

[0008] In some embodiments, based on at least one parameter, the data state corresponding to the file to be analyzed is determined, including: determining the heat value corresponding to the file to be analyzed in the hot storage pool according to the at least one parameter; when the heat value exceeds a preset heat threshold, determining that the data state corresponding to the file to be analyzed is cold data; according to the data state, migrating the file to be analyzed to the cold storage pool or the hot storage pool, including: when the data state corresponding to the file to be analyzed is cold data, migrating the file to be analyzed from the hot storage pool to the cold storage pool.

[0009] In some embodiments, the first parameter includes the first time required for the file to be analyzed to be reduced from hot data to cold data and the score corresponding to the file to be analyzed, and the second parameter includes the hot time, current time, file size and baseline file size of the file to be analyzed; according to at least one parameter, the hot value corresponding to the file to be analyzed in the hot storage pool is determined, including: determining a first ratio based on the hot time, current time and first time; determining a second ratio based on the baseline file size and the file size corresponding to the file to be analyzed; determining the hot value corresponding to the file to be analyzed based on the first ratio, the second ratio and the score corresponding to the file to be analyzed.

[0010] In some embodiments, based on the first ratio, the second ratio and the score corresponding to the file to be analyzed, the heat value corresponding to the file to be analyzed is determined, including: multiplying the first ratio and the first weight corresponding to the first ratio to obtain a first product; multiplying the second ratio and the second weight corresponding to the second ratio to obtain a second product; summing the first product and the second product, and multiplying the result of the summation operation by the heat value corresponding to the file to be analyzed to obtain the heat value corresponding to the file to be analyzed.

[0011] In some embodiments, based on at least one parameter, determining the data status corresponding to the file to be analyzed, including: determining the number of accesses corresponding to the file to be analyzed in the cold storage pool according to the at least one parameter; when the number of accesses exceeds a preset access number threshold, determining that the data status corresponding to the file to be analyzed is hot data; and migrating the file to be analyzed to the cold storage pool or the hot storage pool according to the data status, including: when the data status corresponding to the file to be analyzed is hot data, migrating the file to be analyzed from the cold storage pool to the hot storage pool.

[0012] In some embodiments, the first parameter includes a preset time; determining the number of accesses corresponding to the file to be analyzed in the cold storage pool based on at least one parameter includes: obtaining a storage queue of a hot time corresponding to the file to be analyzed; and determining the number of accesses through the storage queue and the preset time.

[0013] In some embodiments, the first parameter includes a score corresponding to the file to be analyzed; the hierarchical storage method further includes: when the score corresponding to the file to be analyzed in the cold storage pool is a preset score, the file to be analyzed is not migrated.

[0014] In a second aspect, the present application provides a hierarchical storage device, comprising: a first acquisition module, used to acquire at least one parameter associated with a file to be analyzed, wherein the at least one parameter includes a customized first parameter, and / or a second parameter of the file to be analyzed; a first determination module, used to determine a data state corresponding to the file to be analyzed based on at least one parameter, wherein the data state includes cold data or hot data; a migration module, used to migrate the file to be analyzed to a cold storage pool or a hot storage pool according to the data state.

[0015] In some embodiments, the hierarchical storage device also includes: a second acquisition module, which is used to obtain the migration status and usage count corresponding to the file to be analyzed through a preset field before migrating the file to be analyzed to a cold storage pool or a hot storage pool according to the data status; and a second determination module, which is used to determine that the file to be analyzed is not in other migration tasks and is not being used based on the migration status and usage count.

[0016] In some embodiments, the first determination module includes: a first determination unit, used to determine the heat value corresponding to the file to be analyzed in the hot storage pool according to at least one parameter; a second determination unit, used to determine that the data state corresponding to the file to be analyzed is cold data when the heat value exceeds a preset heat threshold; the migration module is also used to: when the data state corresponding to the file to be analyzed is cold data, migrate the file to be analyzed from the hot storage pool to the cold storage pool.

[0017] In some embodiments, the first parameter includes the first time required for the file to be analyzed to be reduced from hot data to cold data and the score corresponding to the file to be analyzed, and the second parameter includes the hot time, current time, file size and benchmark file size of the file to be analyzed; the first determination unit includes: a first determination subunit, used to determine a first ratio based on the hot time, current time and first time; a second determination subunit, used to determine a second ratio based on the benchmark file size and the file size corresponding to the file to be analyzed; a third determination subunit, used to determine the hot value corresponding to the file to be analyzed based on the first ratio, the second ratio and the score corresponding to the file to be analyzed.

[0018] In some embodiments, the third determination subunit is used to perform the following steps: multiply the first ratio and the first weight corresponding to the first ratio to obtain a first product; multiply the second ratio and the second weight corresponding to the second ratio to obtain a second product; sum the first product and the second product, and multiply the result of the summation operation by the heat value corresponding to the file to be analyzed to obtain the heat value corresponding to the file to be analyzed.

[0019] In some embodiments, the first determination module further includes: a third determination unit, used to determine the number of accesses corresponding to the file to be analyzed in the cold storage pool based on at least one parameter; a fourth determination unit, used to determine that the data state corresponding to the file to be analyzed is hot data when the number of accesses exceeds a preset access number threshold; the migration module is also used to: when the data state corresponding to the file to be analyzed is hot data, migrate the file to be analyzed from the cold storage pool to the hot storage pool.

[0020] In some embodiments, the first parameter includes a preset time; the third determination unit is used to perform the following steps: obtain a storage queue of a heat time corresponding to the file to be analyzed; and determine the number of accesses through the storage queue and the preset time.

[0021] In some embodiments, the first parameter includes a score corresponding to the file to be analyzed; the hierarchical storage device further includes: a third determination module, configured to not migrate the file to be analyzed if the score corresponding to the file to be analyzed in the cold storage pool is a preset score.

[0022] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and when the processor executes the computer program, some or all of the steps in the above method are implemented.

[0023] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements some or all of the steps in the above method when the computer program is executed by a processor.

[0024] In a fifth aspect, the present application provides a computer program product, including a computer program or instructions, which implement some or all of the steps in the above method when executed by a processor.

[0025] In a sixth aspect, the present application provides a computer program, comprising a computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes some or all of the steps for implementing the above method.

[0026] In the present application, by obtaining at least one parameter associated with the file to be analyzed, that is, a customized first parameter, and / or a second parameter of the file to be analyzed, based on at least one parameter, the data state corresponding to the file to be analyzed is determined, wherein the data state includes cold data or hot data, and the file to be analyzed is migrated to a cold storage pool or a hot storage pool according to the data state. In this way, the migration conditions of different files to be analyzed can be flexibly controlled through at least one parameter in the above scheme, and it can be determined through the data state that the file to be analyzed needs to be migrated, so that the file to be analyzed is migrated in time, thereby improving the efficiency of data storage.

[0027] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0029] Figure 1 is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0030] Figure 2 is a structural schematic diagram of a hierarchical storage device provided in an embodiment of the present application;

[0031] Figure 3 is a structural diagram of a hierarchical storage system provided in an embodiment of the present application;

[0032] Figure 4 This is a schematic diagram of an implementation process of the hierarchical storage method provided in an embodiment of the present application;

[0033] Figure 5 It is a structural diagram of a background migration task model provided in an embodiment of the present application;

[0034] Figure 6 This is a schematic diagram of a processing process of a processing coroutine provided in an embodiment of the present application;

[0035] Figure 7 is a schematic diagram of a processing process of the first task provided in an embodiment of the present application;

[0036] Figure 8 is a schematic diagram of a processing process of the second task provided in an embodiment of the present application;

[0037] Fig. 9 It is a flowchart of concurrency control provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The following is a description of the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. In the following description, reference is made to the drawings that form a part of the present application and show the specific aspects of the embodiments of the present application or the specific aspects of the embodiments of the present application in an illustrative manner. It should be understood that the embodiments of the present application can be used in other aspects and may include structural or logical changes not depicted in the drawings. Therefore, the following detailed description should not be understood in a restrictive sense, and the scope of the present application is defined by the appended claims. For example, it should be understood that the disclosure of the described method can be equally applicable to the corresponding device or system for performing the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units such as functional units to perform the one or more method steps described (for example, one unit performs one or more steps, or multiple units, each of which performs one or more of the multiple steps), even if such one or more units are not explicitly described or illustrated in the drawings. On the other hand, for example, if a specific device is described based on one or more units such as functional units, the corresponding method may include a step to perform the functionality of the one or more units (e.g., a step to perform the functionality of the one or more units, or multiple steps, each of which performs the functionality of one or more of the multiple units), even if such one or more steps are not explicitly described or illustrated in the drawings. Further, it should be understood that, unless otherwise explicitly stated, the features of the various exemplary embodiments and / or aspects described herein may be combined with each other.

[0039] Below, the terms involved in this application are described and explained.

[0040] 1. Metadata

[0041] Metadata, also known as intermediate data or relay data, is data used to describe data. It mainly contains attribute information about the data and can be used to support a variety of functions, such as indicating storage location, historical data, resource search, and file records.

[0042] In some embodiments, metadata can be viewed as an electronic catalog whose purpose is to assist in data retrieval by describing and cataloging the content or characteristics of the data.

[0043] 2. Distributed file system (DFS).

[0044] DFS means that the physical storage resources managed by the file system are not necessarily directly connected to the local node, but are connected to the node through a computer network, or are a complete hierarchical file system formed by combining several different logical disk partitions or volumes. DFS provides a logical tree-like file system structure for resources distributed anywhere on the network, making it easier for users to access shared files distributed on the network.

[0045] 3. Cold and hot tiered storage.

[0046] Tiered storage is a storage management strategy that aims to optimize storage efficiency and performance by storing different types of data on different storage devices.

[0047] Hot storage can be used to store frequently used data in faster storage media such as high-speed disks, solid state drives (SSDs) or random access memory (RAM) for quick access. In contrast, cold storage can be used to store data that is less frequently accessed in low-cost storage media (such as tapes) and generally does not require frequent access. Therefore, hot storage is mainly used for caching and real-time data processing, while cold storage is used for long-term archiving and backup.

[0048] In some embodiments, the solution for hot and cold tiered storage may include multiple aspects such as data placement strategy, data hot and cold setting strategy, data migration strategy, and cluster background task design.

[0049] 4. Volumes of distributed file systems.

[0050] Volumes usually refer to logical storage units that are shared and managed on multiple nodes or servers. Volumes can span multiple physical nodes to form a unified file storage system, allowing distributed users and applications to share files on the network.

[0051] For hierarchical storage, a hybrid storage method is provided in the related technology. This solution is a hot and cold hierarchical file storage solution developed for the Google File System (GFS). SSD is used to store hot storage and hard disk drive (HDD) is used to store cold storage. In the placement strategy, it is planned to save all files in hot storage at the beginning, and set a scheduled task to migrate the cooled data to cold storage.

[0052] The related technology has the following disadvantages:

[0053] 1. The solution only includes the process of converting hot storage to cold storage, and does not consider the process of converting cold storage to hot storage due to the increase in user access frequency. In this process, files will be migrated from cold storage to hot storage. This cold-to-hot migration mechanism is not reflected in the solution.

[0054] 2. The solution does not describe the metadata operations during the migration process in detail, nor does it consider the data consistency issues during the hot migration process. How to ensure that user operations on files are not lost during the migration process is not reflected in the solution.

[0055] 3. The data placement strategy is too simple, without considering the different default storage media for different files. At the same time, when judging whether files of different sizes and types are transformed into cold data, there should also be a distinction, and time should not be used as a single condition to judge.

[0056] Based on the above problems, the present application proposes a hierarchical storage method, by obtaining at least one parameter associated with the file to be analyzed, namely: a customized first parameter, and / or a second parameter of the file to be analyzed, and based on at least one parameter, determining the data state corresponding to the file to be analyzed, wherein the data state includes cold data or hot data, and according to the data state, migrating the file to be analyzed to a cold storage pool or a hot storage pool. In this way, the migration conditions of different files to be analyzed can be flexibly controlled through at least one parameter in the above scheme, and it can be determined through the data state that the file to be analyzed needs to be migrated, so that the file to be analyzed is migrated in time, thereby improving the efficiency of data storage.

[0057] In some embodiments, the method described in the embodiments of the present application can be applied to the hot and cold hierarchical storage scenarios of distributed file systems. In some embodiments, the method described in the embodiments of the present application can be implemented based on metadata services. In some embodiments, the method described in the embodiments of the present application can also be applied to many application systems such as hierarchical storage systems for cloud file storage, flexible file-level migration tools, and customizable and scalable file storage consoles. In some embodiments, the method described in the embodiments of the present application can also be applied to distributed file storage systems, file migration systems, and recycle bins.

[0058] The following describes an exemplary application of the electronic device provided in the embodiment of the present application. The electronic device provided in the embodiment of the present application can be a laptop, a tablet computer, a desktop computer, a mobile device (e.g., a mobile phone, a wearable smart watch, a dedicated messaging device), an electric car, an electric bicycle, and other rechargeable devices, but is not limited thereto. Alternatively, the electronic device can also be implemented as a server.

[0059] In some embodiments, the server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and big data and artificial intelligence platforms, but is not limited thereto. In some embodiments, the server and the electronic device may be directly or indirectly connected via wired or wireless communication, and this is not specifically limited in the embodiments of the present application.

[0060] See also Figure 1 , Figure 1 is a structural schematic diagram of an electronic device provided in an embodiment of the present application, Figure 1 The electronic device 100 shown includes: at least one processor 110, a memory 150, at least one network interface 120 and a user interface 130. The various components in the electronic device 100 are coupled together via a bus system 140. It is understood that the bus system 140 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 140 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 140 is not described in detail. Figure 1 Various buses are labeled as bus system 140 .

[0061] The processor 110 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0062] The user interface 130 includes one or more output devices 131 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 130 also includes one or more input devices 132, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0063] The memory 150 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 150 may optionally include one or more storage devices that are physically remote from the processor 110.

[0064] The memory 150 includes a volatile memory or a nonvolatile memory, and may also include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 150 described in the embodiments of the present application is intended to include any suitable type of memory.

[0065] In some embodiments, the memory 150 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0066] The operating system 151 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic businesses and process hardware-based tasks.

[0067] The network communication module 152 is used to reach other computing devices via one or more (wired or wireless) network interfaces 120. Exemplary network interfaces 120 include: Bluetooth, WiFi, and universal serial bus (USB).

[0068] The presentation module 153 is used to enable presentation of information (eg, a user interface for operating peripheral devices and displaying content and information) via one or more output devices 131 (eg, a display screen, a speaker, etc.) associated with the user interface 130 .

[0069] The input processing module 154 is configured to detect one or more user inputs or interactions from one of the one or more input devices 132 and to translate the detected inputs or interactions.

[0070] In some embodiments, the hierarchical storage method provided in the embodiments of the present application can be implemented in software and stored in the memory 150. Figure 2 , Figure 2 15 is a structural diagram of a hierarchical storage device provided in an embodiment of the present application, which may be software in the form of a program and a plug-in, etc. The hierarchical storage device 155 includes the following software modules: a first acquisition module 1551, a first determination module 1552, and a migration module 1553. These modules are logical, and therefore can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0071] In other embodiments, the hierarchical storage device 155 provided in the embodiment of the present application may be implemented in hardware. As an example, the hierarchical storage device 155 provided in the embodiment of the present application may be a processor in the form of a hardware decoding processor, which is programmed to execute the hierarchical storage method provided in the embodiment of the present application. For example, the processor in the form of a hardware decoding processor may be one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field-programmable gate array (FPGA) or other electronic components.

[0072] The hierarchical storage method provided in the embodiment of the present application will be described below in conjunction with exemplary applications and implementations of the electronic device provided in the embodiment of the present application.

[0073] In some embodiments, the hierarchical storage method provided in the embodiments of the present application may be applied to a hierarchical storage system, and the hierarchical storage system may be used to implement a hierarchical storage process.

[0074] See also Figure 3 , Figure 3 300 is a structural diagram of a hierarchical storage system provided in an embodiment of the present application. The hierarchical storage system 300 may include: a hot and cold hierarchical server (referred to as a server) 301, a client 302, a server volume virtual file system (VFS) interface 303, a client volume VFS interface 304, a data service 305, and a metadata service 306.

[0075] In some embodiments, the client 302 may be a file system in user space (FUSE) client, which may be deployed in an electronic device on the user side. Both the hot and cold grading server 301 and the client 302 use the VFS interface as a window for interacting with the data service 305 and the metadata service 306. The mapping between the volume and the VFS may be one-to-one or one-to-many.

[0076] In some embodiments, VFS can be a functional client of a volume, and a volume can be a logical entity of VFS. Each volume can correspond to one or more VFS interfaces on the server side, and each volume can create one or more VFS interfaces in any client, which is not specifically limited in the embodiments of the present application.

[0077] Here, the VFS interface is used to implement communication between the distributed file system and the storage pool.

[0078] In some embodiments, the client volume VFS interface 304 can be deployed in the electronic device on the user side. The hot and cold classification server 301, the server volume VFS interface 303, the data service 305 and the metadata service 306 can be deployed in a server or a server cluster, and this embodiment of the application does not specifically limit this.

[0079] In some embodiments, the client 302 may create a client volume VFS interface 304 when executing a mount command (mount) for mounting and using a volume.

[0080] Here, the mount command can be understood as the process of connecting a storage device (such as a hard disk, optical disk, etc.) to a distributed file system, so that files and directories on the device can be accessed through the distributed file system. The mount command can be input by a user or automatically executed by a tool, and the embodiments of the present application do not specifically limit this.

[0081] In some embodiments, the node where the client 302 is located may also perform a thermal update and migration of files from the cold storage pool to the hot storage pool due to the thermal update of the cold data.

[0082] In some embodiments, the client 302 can initiate a volume creation request to the hot and cold grading server 301. The hot and cold grading server 301 can create a server volume VFS interface 303 for the volume based on the request, and store the mapping relationship between the volume name (volume name, volName) and the server volume VFS interface 303 in the form of a dictionary in the software grading service (e.g., the central object server) of the hot and cold grading server, and save the volume information in the cache and the database.

[0083] In some embodiments, creating a volume may be understood as creating a distributed file system.

[0084] In some embodiments, after the server volume VFS interface 303 is successfully created, the hot and cold grading server 301 can register a background task using the VFS interface method, which can periodically perform heat checks on files in the volume and migrate the resulting files from the hot storage pool to the cold storage pool.

[0085] In some embodiments, data storage can use two storage units, such as SSD and HDD, and each storage unit is abstracted as a storage pool, which can include a cold storage pool and a hot storage pool. The hot storage pool can include one or more SSDs, and the cold storage pool can include one or more HDDs. In some embodiments, the cold and hot classification service end 301 can be provided with an interface for manually entering storage pool information, so that the user can set an identification for the information of each storage pool and save it in the database.

[0086] In some embodiments, the VFS object may store available identifiers of the cold storage pool and the hot storage pool, such as an identity (ID).

[0087] In some embodiments, after the volume is created, the client 302 can initiate a volume information query request to the server at any time. The volume information may include: the volume capacity limit, flow limiting information, bandwidth information, the number of blocks into which the files in the volume are split, etc. The embodiments of the present application do not make specific limitations on this.

[0088] In some embodiments, the data service 305 may be responsible for the storage and management of data. It may implement data storage, access, and updating through technologies such as a distributed file system or a distributed database to ensure high availability, reliability, and high performance of the data.

[0089] In some embodiments, the metadata service 306 may be responsible for the access and management of metadata, and by providing access to metadata, it helps users and management systems to efficiently locate, manage, and operate data.

[0090] The following describes the hierarchical storage method provided in the embodiment of the present application in combination with the above-mentioned hierarchical storage system.

[0091] Figure 4 Schematic diagram of an implementation process of the hierarchical storage method provided in the embodiment of the present application. Figure 4 As shown, the hierarchical storage method of the embodiment of the present application may include steps S410 to S430.

[0092] In step S410, at least one parameter associated with the file to be analyzed is obtained.

[0093] The at least one parameter includes a customized first parameter and / or a second parameter of the file to be analyzed.

[0094] Here, the file to be analyzed can be understood as a file included in the volume that needs to be checked for heat to determine whether it needs to be migrated. The first parameter can be an optional parameter added for the user when creating a volume, which can be a parameter related to the file to be analyzed and can determine the heat value of the file to be analyzed, for example, the first time required for the file to be analyzed to be reduced from hot data to cold data (recorded as HotTime, abbreviated as HT) and the score corresponding to the file to be analyzed (recorded as TypeScore, abbreviated as t). TypeScore can be the type score corresponding to the file to be analyzed, which can be determined according to the file type corresponding to the file to be analyzed. The second parameter can be an inherent attribute parameter corresponding to the file to be analyzed, such as the hot time, current time, file size and benchmark file size of the file to be analyzed.

[0095] It can be understood that in order to determine the migration strategy corresponding to the file to be analyzed so as to store it in a hierarchical manner, it is necessary to determine the data state currently corresponding to the file to be analyzed, and the data state is determined based on at least one parameter associated with the file to be analyzed. Therefore, it is necessary to obtain at least one parameter associated with the file to be analyzed, namely: the first parameter, and / or the second parameter.

[0096] In some embodiments, the first parameter can be obtained through the hot and cold grading service end 301, and after the hot and cold grading service end 301 obtains the first parameter, the first parameter is stored in the volume information library of the volume where the file to be analyzed is located, and the first parameter is written into the attributes HotTime and TypeScore of the VFS object in the software grading service.

[0097] In some embodiments, for TypeScore in step S410, multiple file type groups can be set in JavaScript object notation (json) format, and a score can be set for each file type group. Each file type group can include at least one file to be analyzed. The file to be analyzed can be the score of the file type group to which it belongs, or can be determined based on the score of the file type group to which it belongs. The embodiments of the present application do not make specific limitations on this.

[0098] In some embodiments, the file type can be determined based on the file category, such as text, video, audio, etc.; it can also be determined based on the file storage format, such as: docx, txt, json, etc., which is not specifically limited in the embodiments of the present application. In one example, different types of files can be distinguished by file extension or other methods.

[0099] In step S420, based on at least one parameter, a data state corresponding to the file to be analyzed is determined.

[0100] The data status includes cold data or hot data.

[0101] Here, the data state can be understood as the current state of the file to be analyzed. Usually, files in the hot storage pool are hot data, and files in the cold storage pool are cold data. However, as the popularity of files is updated, hot data may become cold data, and cold data may become hot data. Therefore, it is necessary to determine the data state corresponding to the file to be analyzed in order to perform data migration.

[0102] It can be understood that since at least one parameter is associated with the file to be analyzed, the heat value or access count corresponding to the file to be analyzed can be determined based on the at least one parameter, and then based on the heat value or access count, the data state corresponding to the file to be analyzed can be determined.

[0103] In some embodiments, the data status (also referred to as hot or cold status) in step S420 may be recorded in the status field of the file attribute (Attr) saved by the metadata service.

[0104] In step S430, the file to be analyzed is migrated to a cold storage pool or a hot storage pool according to the data status.

[0105] It can be understood that after determining the data status corresponding to the file to be analyzed, if the data status is cold data and the data to be analyzed is currently in a hot storage pool, it means that the file to be analyzed needs to be migrated, and the file to be analyzed is migrated from the current hot storage pool to the cold storage pool; if the data status is hot data and the data to be analyzed is currently in a cold storage pool, it means that the file to be analyzed needs to be migrated, and the file to be analyzed is migrated from the current cold storage pool to the hot storage pool.

[0106] In the embodiment of the present application, through the above steps S410 to S430, at least one parameter can flexibly control the migration conditions of different files to be analyzed, and it can be determined through the data status that the files to be analyzed need to be migrated, so that the files to be analyzed are migrated in time, thereby improving the efficiency of data storage.

[0107] In some embodiments, before the above step S430, the hierarchical storage method further includes: obtaining the migration status and usage count corresponding to the file to be analyzed through preset fields; and determining that the file to be analyzed is not in other migration tasks and is not being used according to the migration status and usage count.

[0108] Here, the preset fields may be a migration status field (e.g., isTrans) and a usage count field (e.g., using). isTrans and using may be stored in the control attributes of the file attributes in the metadata service. The value of isTrans corresponding to the file to be analyzed may be yes (or true) and no (or false). Yes (or true) indicates that the file to be analyzed is in other migration tasks; no (or false) indicates that the file to be analyzed is not in other migration tasks. The value of using corresponding to the file to be analyzed may be an integer. If its value is 0, it indicates that the file to be analyzed is not being used; if its value is an integer greater than 0, it indicates that the file to be analyzed is being used. The preset number of times may be a preset number of times, such as 0 times, and the embodiments of the present application do not make specific limitations thereto. The migration task may be a task of migrating the file to be analyzed from the hot storage pool to the cold storage pool, or a task of migrating the file to be analyzed from the cold storage pool to the hot storage pool, or a task of migrating the file to be analyzed from hot storage pool A to hot storage pool B, or a task of migrating the file to be analyzed to other storage locations. The embodiments of the present application do not make specific limitations thereto.

[0109] In some embodiments, since the metadata is a key-value pair of <inode (node), Attr (attribute)>, and isTrans and using are part of Attr, therefore, the transaction of the metadata storage engine can be used to implement the synchronization control of these two fields.

[0110] It can be understood that the migration status corresponding to the file to be analyzed can be determined through the migration status field, and the usage times corresponding to the file to be analyzed can be determined through the usage count field. When the migration status is not in other migration tasks and the usage times are the preset number of times, it indicates that the file to be analyzed is neither being used by the user nor in other migration tasks. At this time, the file to be analyzed can be migrated; when the migration status is in other migration tasks or the usage times are not the preset number of times, it indicates that the file to be analyzed is in other migration tasks or is being used by the user. At this time, the file to be analyzed cannot be migrated. In this way, through the migration principle in the above manner, the smooth progress of the migration can be ensured and the execution of other migration tasks or the user's usage will not be interrupted.

[0111] In some embodiments, the above step S420 may include: step a, determining the heat value corresponding to the file to be analyzed in the hot storage pool according to at least one parameter; step b, determining that the data state corresponding to the file to be analyzed is cold data when the heat value exceeds the preset heat threshold; the above step S430 may include: when the data state corresponding to the file to be analyzed is cold data, migrating the file to be analyzed from the hot storage pool to the cold storage pool.

[0112] Here, the preset heat threshold may be a preset heat value, such as 1, or other values, which is not specifically limited in the embodiments of the present application.

[0113] It can be understood that after obtaining at least one parameter associated with the file to be analyzed, that is, the first parameter and / or the second parameter, the heat value corresponding to the file to be analyzed in the hot storage pool can be determined according to the first parameter and / or the second parameter through a weighted algorithm, a weighted sum algorithm, a weighted average algorithm or other algorithms. After determining the heat value, the size relationship between the heat value and the preset heat threshold is compared. When the heat value exceeds the preset heat threshold, it means that the heat of the file to be analyzed has expired, and the data state corresponding to the file to be analyzed has changed from hot data to cold data. At this time, the file to be analyzed needs to be migrated, and the file to be analyzed is migrated from the hot storage pool to the cold storage pool. In this way, by the above method, the file to be analyzed is migrated from the hot storage pool to the cold storage pool, which can avoid the file with expired heat occupying the memory space of the hot storage pool and realize efficient storage of data.

[0114] In some embodiments, after the above step a, the method may further include: when the heat value does not exceed a preset heat threshold, determining that the data state corresponding to the file to be analyzed is hot data.

[0115] It can be understood that after determining the heat value, the heat value is compared with the preset heat threshold. If the heat value does not exceed the preset heat threshold, it means that the data state corresponding to the file to be analyzed is still hot data, and the heat of the file to be analyzed has not expired. At this time, the file to be analyzed does not need to be migrated. In this way, by performing a heat check on the file to be analyzed in the above manner, the file to be analyzed whose heat has not expired can be retained in the hot storage pool, which is convenient for users to access and use later.

[0116] In some embodiments, the first parameter includes the first time required for the file to be analyzed to be reduced from hot data to cold data and the score corresponding to the file to be analyzed, and the second parameter includes the hot time, current time, file size and benchmark file size of the file to be analyzed; the above step a may include: step a1, determining a first ratio based on the hot time, current time and first time; step a2, determining a second ratio based on the benchmark file size and the file size corresponding to the file to be analyzed; step a3, determining the hot value corresponding to the file to be analyzed based on the first ratio, the second ratio and the score corresponding to the file to be analyzed.

[0117] Here, the heat time can be saved in the form of a storage queue, such as: HTime fixed-length queue. HTime can be a first-in-first-out queue with a fixed length. Whenever there is an operation on the file to be analyzed, such as creation, opening, modification, and deletion, the operation time will be saved at the end of HTime. At the same time, since HTime is a fixed-length queue, historical data that exceeds the queue length will be abandoned. HTime can be stored in the file attributes of the metadata service (such as Attr). The value of the heat time can be the latest value in the HTime fixed-length queue. The benchmark file size can be understood as the size of the benchmark file corresponding to the file to be analyzed, for example, 10 megabytes (M), which can be customized by the user or pre-set by the operation and maintenance personnel. The embodiment of the present application does not specifically limit this. The score corresponding to the file to be analyzed determines the impact of the file type on the difficulty of triggering the migration.

[0118] It can be understood that based on the heat time of the file to be analyzed (recorded as LT), the current time (recorded as NT) and the first time (HT) required for the file to be analyzed to be reduced from hot data to cold data, the first ratio can be determined through corresponding mathematical operations. Then, based on the benchmark file size (recorded as MS) and the file size corresponding to the file to be analyzed (recorded as S), the benchmark file size is divided by the file size (t) corresponding to the file to be analyzed, and the second ratio can be obtained. Finally, based on the first ratio, the second ratio and the score corresponding to the file to be analyzed, the heat value corresponding to the file to be analyzed can be determined by weighted summation or other calculation methods. In this way, the heat value is determined from three perspectives of time, file size and file type, and the heat value is used as a condition for judging whether the file to be analyzed needs to be migrated from the hot storage pool to the cold storage pool. Compared with the method of judging from a single perspective of time, it has stronger customization and file differentiation, allowing users to flexibly control the migration conditions for different files to be analyzed, and the above process is implemented in the form of an algorithm, which enhances the scalability of the mechanism.

[0119] In one example, if MS is 10M, a to-be-analyzed file larger than 10M is unlikely to trigger migration, and a to-be-analyzed file smaller than 10M is more likely to trigger migration.

[0120] In some embodiments, for step a1, the current time is subtracted from the heat time to obtain a first difference, and the first difference is placed in the first time to obtain a first ratio. In this way, determining the first ratio in the above manner is simple and efficient.

[0121] In some embodiments, the above step a3 may include: multiplying the first ratio and the first weight corresponding to the first ratio to obtain a first product; multiplying the second ratio and the second weight corresponding to the second ratio to obtain a second product; summing the first product and the second product, and multiplying the result of the summation operation by the heat value corresponding to the file to be analyzed to obtain the heat value corresponding to the file to be analyzed.

[0122] Here, the weight corresponding to the first ratio (denoted as W1) and the weight corresponding to the second ratio (denoted as W2) can be set by the user or pre-set, and the embodiment of the present application does not make any specific limitation on this.

[0123] In some embodiments, the sum of W1 and W2 is 1. The larger W1 is, the greater the influence of heat time on heat value; the larger W2 is, the greater the influence of file size on heat value.

[0124] It can be understood that after determining the first ratio, the second ratio, and the score corresponding to the file to be analyzed, the first weight corresponding to the first ratio and the second weight corresponding to the second ratio are obtained, and then the first ratio and the first weight corresponding to the first ratio are multiplied to obtain the first product, and the second ratio and the second weight corresponding to the second ratio are multiplied to obtain the second product. Then the first product and the second product are added, and the result of the addition is multiplied by the heat value corresponding to the file to be analyzed to obtain the heat value corresponding to the file to be analyzed. In this way, by setting the first weight and the second weight, the migration conditions for different files to be analyzed can be flexibly controlled, and the above method of determining the heat value is simple, fast, and efficient.

[0125] In some embodiments, the influence of the heat time on the heat value can be adjusted by adjusting the factor λ1, and the influence of the file size on the heat value can be adjusted by adjusting the factor λ2, where λ1 and λ2 are constants. In some embodiments, λ2 can prevent division by zero.

[0126] In some embodiments, the heat value in step a3 above can be determined by the following formula (1):

[0127]

[0128] Among them, S c Indicates the heat value.

[0129] In some embodiments, the first parameter includes the first time required for the file to be analyzed to be reduced from hot data to cold data and the score corresponding to the file to be analyzed; the above step a may include: dividing the current time by the first time to obtain a third ratio; obtaining a third weight corresponding to the third ratio; multiplying the third ratio, the third weight and the score corresponding to the file to be analyzed respectively to obtain a heat value corresponding to the file to be analyzed.

[0130] Here, the third weight can be set by the user or pre-set, and can be the same as the first weight or different, and the embodiment of the present application does not specifically limit this. In this way, by determining the heat value corresponding to the file to be analyzed through the first parameter and the third weight, the migration conditions for different files to be analyzed can be flexibly controlled to improve the user experience.

[0131] In some embodiments, the second parameter includes the heat time, current time, file size and benchmark file size of the file to be analyzed; the above step a may include: determining a fourth ratio based on the current time and the first time; determining a fifth ratio based on the benchmark file size and the file size corresponding to the file to be analyzed; determining the heat value corresponding to the file to be analyzed based on the fourth ratio, the fourth weight corresponding to the fourth ratio, the fifth ratio and the weight corresponding to the fifth ratio.

[0132] Here, the fourth weight and the fifth weight may be set by the user or may be pre-set, and the embodiment of the present application does not specifically limit this.

[0133] It can be understood that based on the current time and the first time, dividing the two can obtain a fourth ratio; based on the benchmark file size and the file size corresponding to the file to be analyzed, dividing the two can obtain a fifth ratio. Then, the fourth ratio is multiplied by the fourth weight corresponding to the fourth ratio to obtain a first value, and the fifth ratio is multiplied by the weight corresponding to the fifth ratio to obtain a second value. Finally, the first value and the second value are added together to obtain the heat value corresponding to the file to be analyzed. In this way, determining the heat value through the two dimensions of time and file size has stronger customization and file differentiation than the method of judging from a single perspective of time, and can flexibly control the migration conditions for different files to be analyzed.

[0134] In some embodiments, the first parameter includes a score corresponding to the file to be analyzed; the hierarchical storage method may further include: when the score corresponding to the file to be analyzed in the cold storage pool is a preset score, the file to be analyzed is not migrated.

[0135] Here, the preset score can be pre-set, such as 0 points, 0.5 points, etc., or can be determined by other methods. The embodiment of the present application does not make specific limitations on this.

[0136] It can be understood that since the score corresponding to the file to be analyzed can determine the impact of the file type on the difficulty of triggering migration, therefore, when the score corresponding to the file to be analyzed is the preset score, it means that the file to be analyzed will always be cold data, and it will not change from cold data to hot data. Therefore, by default, this type of cold data will not trigger migration, that is, when the score corresponding to the file to be analyzed in the cold storage pool is the preset score, the file to be analyzed will not be migrated. In this way, through the above method, it can be ensured that cold data that meets the conditions will not be migrated, thereby improving the efficiency of data hierarchical storage.

[0137] In some embodiments, the default initial file storage strategy may be: when the score corresponding to the file to be analyzed is a preset score, the file to be analyzed is stored in a cold storage pool; when the score corresponding to the file to be analyzed exceeds the preset score, the file to be analyzed is stored in a hot storage pool.

[0138] In one example, the current scenario is video surveillance. In this scenario, some compressed files are backups of past surveillance videos. These files are generally not frequently accessed. In this case, the scores of these RAR, ZIP, TAR, and GZ (compressed file format) types of files to be analyzed can be set to preset scores, and then the above types of files to be analyzed can be stored in a cold storage pool.

[0139] In some embodiments, the above step S420 may include: step c, determining the number of accesses corresponding to the file to be analyzed in the cold storage pool according to at least one parameter; step d, determining that the data state corresponding to the file to be analyzed is hot data when the number of accesses exceeds a preset access number threshold; the above step S430 may include: when the data state corresponding to the file to be analyzed is hot data, migrating the file to be analyzed from the cold storage pool to the hot storage pool.

[0140] Here, the preset access count threshold (referred to as MaxCount) can be set by the user or determined by other methods, and the present application embodiment does not specifically limit this. In one example, the user can set the preset access count threshold when mounting the client.

[0141] It can be understood that after obtaining at least one parameter associated with the file to be analyzed, the number of accesses (denoted as Count) corresponding to the file to be analyzed in the cold storage pool can be determined by the first parameter and / or the second parameter according to a statistical method or other methods. After determining the number of accesses, the size relationship between the number of accesses and the preset number of accesses threshold is compared. When the number of accesses exceeds the preset number of accesses heat threshold, it indicates that the file to be analyzed has undergone a heat update, and after the heat update, the data state corresponding to the file to be analyzed changes from cold data to hot data. At this time, the file to be analyzed needs to be migrated, and the file to be analyzed is migrated from the cold storage pool to the hot storage pool. In this way, through the above method, the file to be analyzed with a heat update can be migrated from the cold storage pool to the hot storage pool in a timely manner, which is convenient for user access and can achieve efficient storage of data.

[0142] In some embodiments, the first parameter may include a score corresponding to the file to be analyzed. The step of migrating the file to be analyzed from the cold storage pool to the hot storage pool included in step S430 may be performed when the score corresponding to the file to be analyzed in the cold storage pool exceeds a preset score.

[0143] In some embodiments, the second parameter may be the number of accesses corresponding to the file to be analyzed in the cold storage pool. For example, the data status corresponding to the file to be analyzed in the cold storage pool may be directly determined by the second parameter and a preset access number threshold.

[0144] In some embodiments, the first parameter includes a preset time; the above step c may include: obtaining a storage queue of a heat time corresponding to the file to be analyzed; and determining the number of accesses through the storage queue and the preset time.

[0145] Here, the preset time (referred to as CountTime) can be set by the user or determined by other methods, and the embodiment of the present application does not specifically limit this. In one example, the user can set the preset time when mounting the client.

[0146] It can be understood that the storage queue of the hot time corresponding to the file to be analyzed, such as HTime, is obtained, and the number of times in the storage queue that are included in the preset time is determined through the storage queue and the preset time, and this number is the number of accesses. In this way, it can be ensured that each access to the file to be analyzed corresponding to the access number occurs within the preset time, and then, if the access number exceeds the preset access number threshold, it can be determined that the data state corresponding to the file to be analyzed is hot data, so as to trigger the migration of the file to be analyzed from the cold storage pool to the hot storage pool.

[0147] In some embodiments, the logic of triggering the migration of the file to be analyzed from the cold storage pool to the hot storage pool when the number of accesses corresponding to the file to be analyzed exceeds the preset access number threshold within the preset time can be encapsulated as a checking method, such as: checkCountInCountTime method, which can check whether the first Count values ​​in HTime satisfy the occurrence within CountTime, so as to determine whether the number of accesses corresponding to the file to be analyzed in the cold storage pool (denoted as Count) satisfies the occurrence within CountTime, and then determine whether it can trigger the migration of the file to be analyzed from the cold storage pool to the hot storage pool.

[0148] In some embodiments, the first parameter may be a preset time, and the second parameter may be the number of accesses corresponding to the file to be analyzed in the cold storage pool. For example, the target number of accesses corresponding to the file to be analyzed within the preset time may be directly determined by the first parameter and the second parameter, so that the data status corresponding to the file to be analyzed in the cold storage pool may be determined subsequently by the target number of accesses and the preset number of accesses threshold.

[0149] In some embodiments, the migration task corresponding to migrating the to-be-analyzed file to the cold storage pool or the hot storage pool in the above step S430 may be executed by a background migration task model. The background migration task model is introduced below.

[0150] Through the solution described in the above embodiment, compared with the hybrid storage method in the related art, the present application has the following advantages:

[0151] 1. Clear migration mechanism: Through the detailed description of the migration process and metadata modification, it not only provides the process of migrating from the hot storage pool to the cold storage pool, but also provides the process of migrating from the cold storage pool to the hot storage pool. However, the hybrid storage method does not provide the mechanism and process of migrating from the cold storage pool to the hot storage pool.

[0152] 2. Multi-angle migration judgment: Through the heat value calculation algorithm, the migration conditions of the files to be analyzed are comprehensively judged from three angles: time, file size and file type. Compared with the method of judging from the single angle of time, it has stronger customization and file differentiation, allowing users to flexibly control the migration conditions of different files to be analyzed by setting weights. At the same time, the above judgment process is realized in the form of an algorithm, which also enhances the scalability of the mechanism. The hybrid storage method only considers the migration conditions from the single angle of time.

[0153] 3. Added file attributes: By adding the two file attributes isTrans and using in the metadata, the migration of the files to be analyzed can be achieved without suspending user services, and there is no need for additional operations such as diversion and external copies. The hybrid storage method diverts the file business during the migration process by means of an external database.

[0154] 4. Reliable migration synchronization mechanism: Relying on the transactional features provided by the metadata service, multiple clients can be concurrently controlled during the migration process and metadata multi-field query and modification synchronization can be achieved to ensure data consistency before and after the migration. However, in the hybrid storage method, how to merge the database copy data with the migrated data is not involved.

[0155] Figure 5 Schematic diagram of a background migration task model provided by an embodiment of the present application. Figure 5 As shown, the background migration task model may include: a reactor (Reactor) coroutine 501, a handler (Handler) coroutine 502, a handler coroutine 503, a handler coroutine 504, a handler coroutine 505, a channel A, a channel B and a channel C.

[0156] The user request can be used to request the background migration task model to perform the migration task. The Reactor coroutine 501 can respond to the user request, traverse the files included in the volume, determine the files to be analyzed that need to be migrated, and encapsulate the original storage pool ID of the files to be analyzed before migration and the file attributes queried from the metadata service into an event (e.g., event A), and put event A into channel A. The background processing coroutine group can include multiple processing coroutines (e.g., Figure 5 The processing coroutines 502 to 505 in the channel will compete for the processing right of the event in the channel, and the processing coroutine that grabs the event will process the event. When receiving the event, the processing coroutine can determine whether the migration task is executable. If the file to be analyzed in the migration task is being used or in other migration events, the event is abandoned. Otherwise, if the file to be analyzed in the migration task is not in other migration tasks and is not being used, the event can be executed.

[0157] In some embodiments, the periodic heat check of the server and the heat update of the client will lead to migration tasks, and the migration tasks can be implemented in the mode of background tasks.

[0158] In some embodiments, if there is only one migration task in the background, then only one buffer channel can be opened (such as Figure 5 Channel A in the data center), the capacity of channel A can be set larger. Channel A can be used to store the original storage pool ID of the file to be analyzed before migration and the events encapsulated by the file attributes queried from the metadata service. It can store one or more events, which is not limited in the embodiments of the present application.

[0159] In some embodiments, if there are multiple migration tasks in the background, multiple buffer channels can be opened (such as Figure 5In the example, channel A, channel B, and channel C are included in the event data, each channel can be for a type of migration task. For example, if event B and event C are events encapsulated for different migration tasks, event B can be placed in channel B, and event C can be placed in channel C.

[0160] In some embodiments, the background migration task model is extensible. When multiple different types of events occur subsequently, the processing coroutines 502 to 505 can select events from multiple different channels and use different methods to process the events.

[0161] In some embodiments, the background migration task model may be deployed inside a node of a hot and cold tiered server, or may not belong to one or more clients, and this embodiment of the present application does not specifically limit this.

[0162] It should be noted that: Figure 5 The modules included in the illustrated structure are only for illustrative purposes and are not intended to be limiting.

[0163] In some embodiments, the processing coroutine in the above-mentioned background migration task model can execute the migration task. The process of the processing coroutine processing the migration task is introduced below.

[0164] Figure 6 Schematic diagram of a processing process of a processing coroutine provided in an embodiment of the present application. Figure 6 As shown, the processing process may include steps S6001 to S6019.

[0165] In step S6001, start.

[0166] In step S6002, wait for an event to be processed to appear in the channel.

[0167] Here, the processing coroutine will continue to wait until a pending event appears in the channel. The pending event can be a migration task.

[0168] In step S6003, the tasks in the event to be processed are read.

[0169] Here, after receiving the task in the pending event in the channel, the processing coroutine reads the task and unpacks the task into two parts: file attributes and the original storage pool.

[0170] In step S6004, a first transaction is started.

[0171] Here, a transaction can be understood as a program execution unit.

[0172] It can be understood that the processing coroutine starts the first transaction of the metadata service, and the first transaction is used to execute subsequent steps S6005 to S6007.

[0173] In step S6005, the file attributes are read.

[0174] Here, the processing coroutine queries the metadata of the to-be-analyzed file corresponding to the task in the first transaction to obtain the file attributes.

[0175] In step S6006, it is determined that isTrans is false and using=0.

[0176] If so, execute step S6007; if not, execute step S6002.

[0177] Here, the processing coroutine determines whether isTrans is false and using is 0 according to the file attributes, that is, whether the file to be analyzed is not in other migration tasks and is not being used.

[0178] In step S6007, isTrans is set to true.

[0179] Here, if the file to be analyzed is not in other migration tasks and is not being used, the isTrans in the file attribute is updated and is set to true, indicating that the file to be analyzed is in a migration task, which can prevent the file to be analyzed from being accessed by other transactions. If the file to be analyzed is in other migration tasks or is being used, the migration task is not executed, and the waiting state is re-entered after the migration task is submitted, that is, the execution returns to step S6002.

[0180] In step S6008, the first transaction is committed.

[0181] In step S6009, it is determined whether the first transaction conflicts with other transactions.

[0182] If so, execute step S6004; if not, execute step S6010.

[0183] Here, if the first transaction conflicts with other transactions, the process returns to step S6004 ; if the first transaction does not conflict with other transactions, the process proceeds to step S6010 .

[0184] In step S6010, it is determined that the file to be analyzed has not been modified.

[0185] Here, it can be determined that the file to be analyzed has not been modified by flushing (Flush) VFS to determine whether the data modification on the file to be analyzed is submitted. If the data modification on the file to be analyzed is not submitted in VFS, it can be determined that the file to be analyzed has not been modified.

[0186] In step S6011, a first interface is created.

[0187] Here, the first interface may be an object storage (ObjectStorage) interface, which may be an interface between a processing coroutine and a storage pool.

[0188] In step S6012, the data of the file to be analyzed is read.

[0189] Here, the processing coroutine reads the data of the file to be analyzed in the original storage pool through the first interface.

[0190] In step S6013, the data is stored in the target storage pool.

[0191] Here, the target storage pool may be a storage pool to which the to-be-analyzed file is to be migrated. The processing coroutine stores the data of the to-be-analyzed file into the target storage pool.

[0192] In step S6014, it is determined whether the data is consistent.

[0193] Here, after the data is flushed into the target storage pool, the processing coroutine can check the data consistency of the files to be analyzed in the original storage pool and the target storage pool through a data checking tool or other methods to determine whether the data is consistent.

[0194] If so, execute step S6015; if not, execute step S6013.

[0195] If the data is consistent, execute step S6015; if the data is inconsistent, return and re-execute step S6013.

[0196] In step S6015, the second transaction is started.

[0197] Here, the processing coroutine starts the second transaction of the metadata service, and the second transaction is used to execute the subsequent step S6016.

[0198] In step S6016, isTrans is set to false.

[0199] Here, isTrans in the file attribute is updated and is set to false, indicating that the file to be analyzed is not in the migration task, and the file to be analyzed can be accessed by other transactions.

[0200] In step S6017, the second transaction is committed.

[0201] In step S6018, it is determined whether the second transaction conflicts with other transactions.

[0202] If so, execute step S6015; if not, execute step S6019.

[0203] Here, if the second transaction conflicts with other transactions, the process returns to step S6015; if the second transaction does not conflict with other transactions, the process executes step S6019.

[0204] In step S6019, the files to be analyzed in the original storage pool are deleted.

[0205] Here, after the second transaction is successfully submitted, the files to be analyzed in the original storage pool are deleted, and the migration task is completed.

[0206] In some embodiments, the migration task may include a task of migrating the to-be-analyzed file from the hot storage pool to the cold storage pool (referred to as the first task). The processing process of the first task is introduced below.

[0207] Figure 7 Schematic diagram of a processing process of the first task provided in the embodiment of the present application. Figure 7 As shown, the processing process can be executed by the server, and specifically can include steps S7001 to S7012.

[0208] In step S7001, start.

[0209] In step S7002, a volume is created.

[0210] Here, the first task is based on volumes, and is started after the volume is created. Therefore, the server needs to create a volume, that is, CreateVolume.

[0211] In step S7003, a server volume VFS interface is created.

[0212] Here, after creating the volume, the server creates a server volume VFS interface, such as: server.cluster.vfs[volName]=vfs.

[0213] In step S7004, a background periodic task is started.

[0214] Here, the server starts a background periodic task, the purpose of which is to migrate the files to be analyzed that have expired from the hot storage pool to the cold storage pool (i.e., the first task). The background periodic task can be a task that periodically checks the heat of the files in the volume. The period can be pre-set, such as every 3 hours, or can be determined according to the specific implementation process. The embodiment of the present application does not make specific limitations on this.

[0215] In step S7005, all metadata of the volume are traversed.

[0216] Here, the background periodic task periodically checks the metadata corresponding to all files to be analyzed in the volume and traverses all metadata of the volume.

[0217] In step S7006, the heat value corresponding to each file to be analyzed is determined.

[0218] Here, the implementation method of step S7006 can refer to the above steps a1 to a3, which will not be repeated here.

[0219] In step S7007, a metadata modification form is created.

[0220] Here, after determining the heat value, it is determined whether the first task exists based on the heat value. If the first task exists, a metadata modification form is created to modify the pool identifier (such as PoolID), data status (Status), heat time (HTime) and other fields of the file attributes of the file to be analyzed corresponding to the first task in the metadata.

[0221] In step S7008, the first event after encapsulation is determined.

[0222] Here, the server encapsulates the original storage pool ID of the file to be analyzed corresponding to the first task before migration and the modified file attributes into a first event, thereby obtaining the encapsulated first event.

[0223] In step S7009, the first event is stored in the channel.

[0224] Here, the server stores the first event in a channel (such as channel A) corresponding to the first event.

[0225] In step S7010, a first task processing mechanism is performed.

[0226] Here, the first event in step S7009 is processed by the first task processing mechanism, and its implementation process can refer to the above Figure 5 and Figure 6 The process shown will not be repeated here.

[0227] In step S7011, wait for the next cycle.

[0228] Here, after the server has traversed all the files to be analyzed in the volume, the processing coroutine processing the first task can fall into sleep. The sleep can call the sleep command (such as Sleep) and other methods that will give up the central processing unit (CPU), and then wait for the next cycle. When the next cycle comes, repeat the above steps S7005 to S7011.

[0229] In step S7012, end.

[0230] In some embodiments, the migration task may include a task of migrating the to-be-analyzed file from the cold storage pool to the hot storage pool (referred to as the second task). The processing process of the second task is introduced below.

[0231] Figure 8 Schematic diagram of a processing process of the second task provided in the embodiment of the present application. Figure 8 As shown, the processing process can be executed by the client, and specifically can include steps S8001 to S8014.

[0232] In step S8001, start.

[0233] In step S8002, an open operation is received.

[0234] Here, the open operation can be an access operation for a user to open a file to be analyzed, or can be an open operation for calling VFS, which is not specifically limited in the embodiments of the present application. In addition to the open operation, the access operation can also be a create operation, a modify operation, etc., which is not specifically limited in the embodiments of the present application. After receiving the above access operation, the HTime and using fields in the file attributes need to be updated.

[0235] In step S8003, the heat is updated.

[0236] Here, the client performs a heat update in response to the opening operation, and the purpose of the heat update is to determine whether the second task exists.

[0237] In step S8004, it is determined whether the score corresponding to the file to be analyzed is a preset score.

[0238] If so, execute step S8014; if not, execute step S8005.

[0239] Here, if the score corresponding to the file to be analyzed is the preset score, it means that the data state of the file to be analyzed is cold data (ColdType) and migration will not be triggered. At this time, the process ends directly; if the score corresponding to the file to be analyzed is not the preset score, execute step S8005.

[0240] In step S8005, it is determined whether status is 0.

[0241] Here, if status is 0, it means that the data status of the file to be analyzed is cold data, and if status is not 0, it means that the data status of the file to be analyzed is hot data. Therefore, when status is 0, there is no need to migrate the file to be analyzed, but since an open operation is received, the time in Htime needs to be updated. After updating Htime, it ends directly. When status is not 0, the client determines whether there is a second task.

[0242] In step S8006, Htime is updated.

[0243] In step S8007, it is determined whether a second task exists.

[0244] Here, the implementation method of step S8007 can refer to the above steps c to d, which will not be repeated here.

[0245] If so, execute step S8008; if not, execute step S8013.

[0246] Here, after executing step S8013, step S8014 is executed.

[0247] In step S8008, HTime is updated.

[0248] Here, in the case where the second task exists, since an open operation is received, the time in Htime also needs to be updated.

[0249] In step S8009, a metadata modification form is created.

[0250] Here, after determining the existence of the second task, it is necessary to create a metadata modification form to modify the pool identifier (eg, PoolID), data status (Status), and heat time (HTime) fields of the file attributes of the file to be analyzed corresponding to the second task in the metadata.

[0251] In step S8010, the encapsulated second event is determined.

[0252] Here, the client encapsulates the original storage pool ID of the file to be analyzed before migration and the modified file attributes corresponding to the second task into a second event, thereby obtaining the encapsulated second event.

[0253] In step S8011, the second event is stored in the channel.

[0254] Here, the client stores the second event in a channel corresponding to the second event (eg, channel B).

[0255] In step S8012, a second task processing mechanism is performed.

[0256] Here, the second event in step S8011 is processed by the second task processing mechanism, and its implementation process can refer to the above Figure 5 and Figure 6 The process shown will not be repeated here.

[0257] In step S8013, Htime is updated.

[0258] Here, in the absence of the second task, since an open operation is received, the time in Htime also needs to be updated.

[0259] In step S8014, end.

[0260] In some embodiments, multiple clients and servers may perform concurrent control on the same file to be analyzed. The process of the concurrent control is described below.

[0261] Fig. 9 This is a flow chart of concurrency control provided by the embodiment of the present application. Fig. 9 As shown, the process may include Fig. 9 (a) to (d) in the text.

[0262] For details, see Fig. 9 (a) in the figure shows the process of querying and modifying metadata before migration, which may include step S9101 and step S9112.

[0263] In step S9101, start.

[0264] In step S9102, a first request is received.

[0265] Here, the first request may be a request for querying the file to be analyzed and modifying metadata before migration.

[0266] In step S9103, a third transaction is created.

[0267] Here, the third transaction is used to execute subsequent steps S9104 to S9108.

[0268] In step S9104, metadata is queried.

[0269] In step S9105, it is determined whether isTrans is true.

[0270] If so, execute step S9111; if not, execute step S9106.

[0271] In step S9106, using is calculated.

[0272] Here, the total number of times the file to be analyzed is used can be calculated, and if it times out, it will be reset.

[0273] In step S9107, it is determined whether using is greater than 0.

[0274] If so, execute step S9108; if not, execute step S9111.

[0275] In step S9108, isTrans is set to true.

[0276] In step S9109, the third transaction is committed.

[0277] In step S9110, determine whether the execution is successful.

[0278] Here, it is determined whether the third transaction is executed successfully.

[0279] If so, execute step S9111; if not, execute step S9103.

[0280] In step S9111, the first response is returned.

[0281] Here, the first response is the response information of the first request, which may be the successful execution of the first request or the failed execution of the first request. The embodiment of the present application does not make any specific limitation on this.

[0282] In step S9112, end.

[0283] See also Fig. 9 (b) in the figure shows the process of modifying metadata after migration, which may include step S9201 and step S9209.

[0284] In step S9201, start.

[0285] In step S9202, a second request is received.

[0286] Here, the second request may be a request for modifying metadata of the file to be analyzed after migration.

[0287] In step S9203, a fourth transaction is created.

[0288] Here, the fourth transaction is used to execute subsequent steps S9204 and S9205.

[0289] In step S9204, metadata is queried.

[0290] In step S9205, isTrans is set to false.

[0291] In step S9206, the fourth transaction is committed.

[0292] In step S9207, determine whether the execution is successful.

[0293] Here, it is determined whether the fourth transaction is executed successfully.

[0294] If so, execute step S9208; if not, execute step S9203.

[0295] In step S9208, a second response is returned.

[0296] Here, the second response is the response information of the second request, which may be the successful execution of the second request or the failed execution of the second request. The embodiment of the present application does not make any specific limitation on this.

[0297] In step S9209, end.

[0298] See also Fig. 9 (c) in the figure shows the process of modifying information before opening and creating, which may include step S9301 and step S9309.

[0299] In step S9301, start.

[0300] In step S9302, a third request is received.

[0301] Here, the third request may be a request to modify information of the file to be analyzed before opening or creating it.

[0302] In step S9303, a fifth transaction is created.

[0303] Here, the fifth transaction is used to execute subsequent steps S9304 and S9305.

[0304] In step S9304, metadata is queried.

[0305] In step S9305, the using of the metadata is updated.

[0306] Here, you can increment (++) using by 1.

[0307] In step S9306, the fifth transaction is committed.

[0308] In step S9307, determine whether the execution is successful.

[0309] Here, it is determined whether the fifth transaction is executed successfully.

[0310] If so, execute step S9308; if not, execute step S9303.

[0311] In step S9308, a third response is returned.

[0312] Here, the third response is the response information of the third request, which may be the successful execution of the third request or the failed execution of the third request. The embodiment of the present application does not make any specific limitation on this.

[0313] In step S9309, end.

[0314] See also Fig. 9 (d) in the figure shows the process of information modification after release, which may include step S9401 and step S9409.

[0315] In step S9401, start.

[0316] In step S9402, the fourth request is received.

[0317] Here, the fourth request may be a request for modifying information of the file to be analyzed after being released.

[0318] In step S9403, a sixth transaction is created.

[0319] Here, the sixth transaction is used to execute subsequent steps S9404 and S9405.

[0320] In step S9404, metadata is queried.

[0321] In step S9405, the using of the metadata is updated.

[0322] Here, you can decrement (--) using by 1.

[0323] In step S9406, the sixth transaction is committed.

[0324] In step S9407, it is determined whether the execution is successful.

[0325] Here, it is determined whether the sixth transaction is executed successfully.

[0326] If so, execute step S9408; if not, execute step S9403.

[0327] In step S9408, a fourth response is returned.

[0328] Here, the fourth response is the response information of the fourth request, which may indicate that the fourth request is executed successfully or fails to be executed. The embodiment of the present application does not make any specific limitation on this.

[0329] In step S9409, end.

[0330] Understandably, for the above Fig. 9(a) to (d), the third to sixth transactions are all transactions. Transactions themselves have the characteristics of ACID (Atomicity, Consistency, Isolation, and Durability). When a transaction is committed, if Attr is inconsistent with the state before the transaction is established, the commit will fail due to a conflict and return an error message. At this time, the server or client will re-create a new transaction, read the new metadata information, and continue to try to modify Attr.

[0331] Here, the Attr read by the transaction recreated due to the conflict failure is different from the previous transaction, that is, all clients' modifications to Attr will be visible to all clients in the next transaction.

[0332] This feature can solve the data consistency problem of the two fields (isTrans and using). At the same time, due to the consistency of transactions, at most only one transaction can be submitted successfully at the same time, thus achieving synchronous control of multiple clients and servers. In addition, the update of the using field can be performed in the hot update operation, which also utilizes the metadata transaction mechanism. In this way, based on the transaction characteristics of the metadata service, it is possible to achieve concurrent control of multiple clients modifying the metadata of the file to be analyzed, as well as a synchronization mechanism for multi-field query and modification of metadata.

[0333] Based on the same inventive concept, the present application embodiment also provides a hierarchical storage device, such as the hierarchical storage device 155 in the above embodiment. Figure 2 As shown, the hierarchical storage device 155 includes: a first acquisition module 1551, used to obtain at least one parameter associated with the file to be analyzed, wherein the at least one parameter includes a customized first parameter and / or a second parameter of the file to be analyzed; a first determination module 1552, used to determine the data state corresponding to the file to be analyzed based on the at least one parameter, wherein the data state includes cold data or hot data; a migration module 1553, used to migrate the file to be analyzed to a cold storage pool or a hot storage pool according to the data state.

[0334] In some embodiments, the hierarchical storage device also includes: a second acquisition module, which is used to obtain the migration status and usage count corresponding to the file to be analyzed through a preset field before migrating the file to be analyzed to a cold storage pool or a hot storage pool according to the data status; and a second determination module, which is used to determine that the file to be analyzed is not in other migration tasks and is not being used based on the migration status and usage count.

[0335] In some embodiments, the first determination module 1552 includes: a first determination unit, used to determine the heat value corresponding to the file to be analyzed in the hot storage pool according to at least one parameter; a second determination unit, used to determine that the data state corresponding to the file to be analyzed is cold data when the heat value exceeds a preset heat threshold; the migration module 1553 is also used to: when the data state corresponding to the file to be analyzed is cold data, migrate the file to be analyzed from the hot storage pool to the cold storage pool.

[0336] In some embodiments, the first parameter includes the first time required for the file to be analyzed to be reduced from hot data to cold data and the score corresponding to the file to be analyzed, and the second parameter includes the hot time, current time, file size and benchmark file size of the file to be analyzed; the first determination unit includes: a first determination subunit, used to determine a first ratio based on the hot time, current time and first time; a second determination subunit, used to determine a second ratio based on the benchmark file size and the file size corresponding to the file to be analyzed; a third determination subunit, used to determine the hot value corresponding to the file to be analyzed based on the first ratio, the second ratio and the score corresponding to the file to be analyzed.

[0337] In some embodiments, the third determination subunit is used to perform the following steps: multiply the first ratio and the first weight corresponding to the first ratio to obtain a first product; multiply the second ratio and the second weight corresponding to the second ratio to obtain a second product; sum the first product and the second product, and multiply the result of the summation operation by the heat value corresponding to the file to be analyzed to obtain the heat value corresponding to the file to be analyzed.

[0338] In some embodiments, the first determination module 1552 further includes: a third determination unit, used to determine the number of accesses corresponding to the file to be analyzed in the cold storage pool based on at least one parameter; a fourth determination unit, used to determine that the data state corresponding to the file to be analyzed is hot data when the number of accesses exceeds a preset access number threshold; the migration module 1553 is further used to: when the data state corresponding to the file to be analyzed is hot data, migrate the file to be analyzed from the cold storage pool to the hot storage pool.

[0339] In some embodiments, the first parameter includes a preset time; the third determination unit is used to perform the following steps: obtain a storage queue of a heat time corresponding to the file to be analyzed; and determine the number of accesses through the storage queue and the preset time.

[0340] In some embodiments, the first parameter includes a score corresponding to the file to be analyzed; the hierarchical storage device further includes: a third determination module, configured to not migrate the file to be analyzed if the score corresponding to the file to be analyzed in the cold storage pool is a preset score.

[0341] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiment of the present application can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0342] It should be noted that in the embodiment of the present application, if the above-mentioned hierarchical storage method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a ROM, a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.

[0343] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0344] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.

[0345] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is run in an electronic device, a processor in the electronic device executes some or all of the steps for implementing the above method.

[0346] The present application embodiment provides a computer program product, including a computer program or an instruction, which implements some or all of the steps in the above method when the computer program or instruction is executed by a processor. The computer program product can be implemented specifically by hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0347] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0348] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial number of each step / process mentioned above does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.

[0349] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0350] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0351] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0352] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0353] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks or optical disks.

[0354] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0355] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A hierarchical storage method, characterized in that: include: Acquire at least one parameter associated with the file to be analyzed, wherein the at least one parameter includes a user-defined first parameter and / or a second parameter of the file to be analyzed; Based on the at least one parameter, determining a data state corresponding to the file to be analyzed, wherein the data state includes cold data or hot data; According to the data status, the to-be-analyzed file is migrated to a cold storage pool or a hot storage pool.

2. The method according to claim 1, characterized in that Before migrating the to-be-analyzed file to the cold storage pool or the hot storage pool according to the data state, the method further includes: Obtaining the migration status and usage count corresponding to the file to be analyzed through the preset fields; According to the migration status and the number of uses, it is determined that the file to be analyzed is not in other migration tasks and is not used.

3. The method according to claim 1, characterized in that The determining, based on the at least one parameter, a data state corresponding to the file to be analyzed includes: Determine, according to the at least one parameter, a heat value corresponding to the to-be-analyzed file in the hot storage pool; When the heat value exceeds a preset heat threshold, determining that the data state corresponding to the file to be analyzed is cold data; The step of migrating the to-be-analyzed file to a cold storage pool or a hot storage pool according to the data state includes: When the data state corresponding to the to-be-analyzed file is cold data, the to-be-analyzed file is migrated from the hot storage pool to the cold storage pool.

4. The method according to claim 3, characterized in that The first parameter includes the first time required for the file to be analyzed to be reduced from hot data to cold data and the score corresponding to the file to be analyzed, and the second parameter includes the hot time, current time, file size and reference file size of the file to be analyzed; The step of determining the heat value corresponding to the to-be-analyzed file in the hot storage pool according to the at least one parameter includes: Determining a first ratio based on the heat time, the current time, and the first time; Determining a second ratio based on the benchmark file size and the file size corresponding to the file to be analyzed; Based on the first ratio, the second ratio, and the score corresponding to the file to be analyzed, a heat value corresponding to the file to be analyzed is determined.

5. The method according to claim 4, characterized in that The determining, based on the first ratio, the second ratio, and the score corresponding to the file to be analyzed, a heat value corresponding to the file to be analyzed, includes: Multiplying the first ratio by a first weight corresponding to the first ratio to obtain a first product; multiplying the second ratio by a second weight corresponding to the second ratio to obtain a second product; A sum operation is performed on the first product and the second product, and a result obtained by the sum operation is multiplied by the heat value corresponding to the file to be analyzed to obtain the heat value corresponding to the file to be analyzed.

6. The method according to claim 1, characterized in that The determining, based on the at least one parameter, a data state corresponding to the file to be analyzed includes: Determine, according to the at least one parameter, the number of accesses corresponding to the file to be analyzed in the cold storage pool; When the access count exceeds a preset access count threshold, determining that the data state corresponding to the file to be analyzed is hot data; The step of migrating the to-be-analyzed file to a cold storage pool or a hot storage pool according to the data state includes: When the data state corresponding to the to-be-analyzed file is hot data, the to-be-analyzed file is migrated from the cold storage pool to the hot storage pool.

7. The method according to claim 6, characterized in that The first parameter includes the preset time; The determining, according to the at least one parameter, the number of accesses corresponding to the file to be analyzed in the cold storage pool includes: Obtaining a storage queue of the heat time corresponding to the file to be analyzed; The access times are determined by the storage queue and the preset time.

8. The method according to claim 1, characterized in that: The first parameter includes a score corresponding to the file to be analyzed; The method further comprises: When the score corresponding to the file to be analyzed in the cold storage pool is the preset score, the file to be analyzed is not migrated.

9. A hierarchical storage device, characterized in that: include: A first acquisition module, used to acquire at least one parameter associated with the file to be analyzed, wherein the at least one parameter includes a user-defined first parameter and / or a second parameter of the file to be analyzed; A first determination module, configured to determine a data state corresponding to the file to be analyzed based on the at least one parameter, wherein the data state includes cold data or hot data; A migration module is used to migrate the to-be-analyzed file to a cold storage pool or a hot storage pool according to the data status.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the steps in the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps in the method according to any one of claims 1 to 8 are implemented.

Images