Heterogeneous storage system management method, electronic equipment and storage medium

Abstract

The invention discloses a management method of a heterogeneous storage system, electronic equipment and a storage medium. The method comprises the following steps: determining an access data type for each storage subsystem; and for each storage subsystem, a storage pool matched with the access data type corresponding to the storage subsystem is created by using the storage resources corresponding to the storage subsystem, and the storage pool is used for storing data corresponding to the access data type. According to the scheme, the reliability and performance of the heterogeneous storage system can be improved.

Description

Technical Field

[0001] The present application relates to the field of storage technology, and in particular to a management method, electronic equipment, and storage medium for a heterogeneous storage system. Background Art

[0002] Existing management and scheduling strategies for heterogeneous storage systems are often limited to the heterogeneity of a single storage layer, failing to fully tap and utilize the optimal performance potential of each storage subsystem. Consequently, heterogeneous storage systems cannot operate in an optimal environment, resulting in information exchange between data having to pass through different storage subsystems, leading to low reading efficiency. Summary of the Invention

[0003] This application at least provides a management method, electronic device, and storage medium for a heterogeneous storage system, which can improve the reliability and performance of the heterogeneous storage system.

[0004] In a first aspect, the present application provides a method for managing a heterogeneous storage system. The heterogeneous storage system includes several storage subsystems, each storage subsystem corresponds to at least one storage device, and is used to virtualize storage resources of the corresponding storage devices. In the heterogeneous storage system, there are at least two storage subsystems corresponding to storage devices of different types. The method includes: determining an access data type for each storage subsystem; for each storage subsystem, using the storage resources corresponding to the storage subsystem, creating a storage pool that matches the access data type corresponding to the storage subsystem, wherein the storage pool is used to store data corresponding to the access data type.

[0005] Among them, each storage subsystem corresponds to a Platform as a Service domain; using the storage resources corresponding to the storage subsystem, a storage pool that matches the access data type corresponding to the storage subsystem is created, including: for each storage subsystem, using the Platform as a Service domain corresponding to the storage subsystem to call the management interface of the storage subsystem to divide at least part of the storage resources corresponding to the storage subsystem into storage pools that match the access data type corresponding to the storage subsystem.

[0006] The method further includes: detecting the business data in each storage subsystem to obtain a detection result; determining the storage subsystem whose detection result shows that the business data has an abnormality as an abnormal storage subsystem, and deleting the abnormal storage subsystem after waiting for a preset time; and / or determining the storage subsystem whose detection result shows that the business data has no abnormality as a normal storage subsystem, in response to receiving a deletion instruction for the first normal storage subsystem, obtaining the remaining storage capacity of the second normal storage subsystem, and in response to the remaining storage capacity meeting a first preset migration condition, migrating the business data in the storage pool corresponding to the first normal storage subsystem to the storage pool of the second normal storage subsystem, and deleting the first normal storage subsystem, wherein the second normal storage subsystem is a normal storage subsystem other than the first normal storage subsystem.

[0007] Wherein, the detection result of the normal storage subsystem is that business data exists and the business data has not expired; the storage subsystem whose detection result is that business data exists abnormally is determined to be an abnormal storage subsystem, including at least one of the following steps: determining the storage subsystem whose detection result is that business data exists and the business data has expired as a first abnormal storage subsystem; determining the storage subsystem whose detection result is that business data does not exist as a second abnormal storage subsystem; after waiting for a preset time, deleting the abnormal storage subsystem, including: in response to the abnormal storage subsystem being the first abnormal storage subsystem, marking several storage pools corresponding to the first abnormal storage subsystem, and after waiting for a preset time, deleting the first abnormal storage subsystem and the marked several storage pools; and / or, in response to the abnormal storage subsystem being the second abnormal storage subsystem, after waiting for a preset time, the second abnormal storage subsystem and its corresponding storage pools are deleted; and / or, the first preset migration condition includes: the remaining storage capacity of the second normal storage subsystem is greater than the migration data capacity of the first normal storage subsystem and the planned usage capacity of the current storage plan, and the total network load of the second normal storage subsystem is less than a preset multiple of the network load of the heterogeneous storage system; and / or, deleting the first normal storage subsystem includes: setting the second normal storage subsystem to be unavailable, marking the storage pools corresponding to the second normal storage subsystem, and after waiting for a preset time, deleting the second normal storage subsystem and the marked storage pools.

[0008] The method further includes: detecting that a storage subsystem satisfies a preset fault condition, and determining the storage subsystem that satisfies the preset fault condition as a faulty storage subsystem; obtaining a first capacity value and a first remaining storage capacity of each third normal storage subsystem in the current cycle, wherein the third normal storage subsystem is a storage subsystem other than the faulty storage subsystem in the heterogeneous storage system; and allocating the capacity value corresponding to the faulty storage subsystem to at least one third normal storage subsystem based on the first capacity value and the first remaining storage capacity.

[0009] Among them, based on the first capacity value and the first remaining storage capacity, the capacity value corresponding to the faulty storage subsystem is allocated to at least one third normal storage subsystem, including: determining a first storage pool and a second storage pool from the storage pools corresponding to each third normal storage subsystem, wherein the first storage pool is a storage pool of the same type as the storage pool corresponding to the faulty storage subsystem, and the second storage pool is a storage pool of a different type from the storage pool corresponding to the faulty storage subsystem; in response to the existence of at least one first storage pool, using at least one first storage pool to take over at least part of the capacity value corresponding to the faulty storage subsystem; and / or, in response to the absence of the first storage pool, using at least one second storage pool to take over all capacity values of the faulty storage subsystem.

[0010] The method further includes: obtaining a second capacity value and a second remaining storage capacity recorded in the previous cycle of the faulty storage subsystem, wherein the second capacity value is obtained through the system average load, input / output average load and disk average total load of the faulty storage subsystem; using at least one first storage pool to take over at least part of the capacity value corresponding to the faulty storage subsystem, including: in response to the first storage pool meeting a preset takeover condition, using the first storage pool to take over all capacity values of the faulty storage subsystem, wherein the preset takeover condition is that the sum of the first capacity values corresponding to all first storage pools and the second capacity value satisfies a first size relationship, and the sum of the first remaining storage capacity corresponding to all first storage pools and the second remaining storage capacity satisfies a second size relationship, the first size relationship is that the first multiple of the sum of the first capacity values of all first storage pools is greater than the second capacity value, and the second size relationship is that the second multiple of the sum of the first remaining storage capacity of all first storage pools is greater than the second remaining storage capacity; in response to the first storage pool not meeting the preset takeover condition, allocating the capacity value of the faulty storage subsystem to each first storage pool for takeover, and using at least one second storage pool to take over the faulty storage subsystem if there is a remaining capacity value after the allocation. The remaining capacity value of the storage subsystem, wherein the capacity value of the faulty storage subsystem is allocated to each first storage pool for takeover, including: selecting a first target storage pool from each first storage pool, and determining a first takeover capacity value of the first target storage pool, wherein the first takeover capacity value is related to at least one of the first capacity value corresponding to the first target storage pool and the first remaining storage capacity; allocating part of the capacity value of the faulty storage subsystem to the first target storage pool for takeover according to the first takeover capacity value; repeating the above steps until all first storage pools are selected or the capacity value allocation of the faulty storage subsystem is completed; wherein, using at least one second storage pool to take over all capacity values of the faulty storage subsystem, or using at least one second storage pool to take over the remaining capacity value of the faulty storage subsystem, includes: selecting a second storage pool as a second target storage pool, and determining a second takeover capacity value of the second target storage pool, wherein the second takeover capacity value is related to at least one of the first capacity value corresponding to the second target storage pool and the first remaining storage capacity; allocating part of the capacity value of the faulty storage subsystem to the second target storage pool for takeover according to the second takeover capacity value; repeating the above steps until the capacity value allocation of the faulty storage subsystem is completed.

[0011] In a second aspect, the present application provides a heterogeneous storage system, comprising: a plurality of storage subsystems; a domain management service, configured to execute the heterogeneous storage system management method of the first aspect to create a storage pool for each storage subsystem; a software service platform, connected to the domain management service, configured to receive basic information of the storage subsystems uploaded by the domain management service; and a storage pool space, comprising a storage pool corresponding to each storage subsystem, the storage pool being configured to store business data sent by the software service platform.

[0012] A third aspect of the present application provides an electronic device, comprising a memory and a processor coupled to each other, wherein the processor is configured to execute program instructions stored in the memory to implement the heterogeneous storage system management method of the first aspect.

[0013] In a fourth aspect, the present application provides a computer-readable storage medium having program instructions stored thereon. When the program instructions are executed by a processor, the heterogeneous storage system management method in the first aspect is implemented.

[0014] The above scheme determines the access data type for each storage subsystem, and based on the access data type, uses the storage device corresponding to the storage subsystem to generate a storage pool that matches the access data type, so as to store data corresponding to the access data type through the storage pool, use the storage pool to shield the differences between storage devices in heterogeneous storage systems, and use the storage pool to store corresponding types of access data to shield the differences in storage types, thereby improving the reliability and performance of heterogeneous storage systems.

[0015] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to illustrate the technical solutions of the present application.

[0017] Figure 1 This is a flow chart of an embodiment of a method for managing a heterogeneous storage system of the present application;

[0018] Figure 2 This is a schematic diagram of the framework of an embodiment of the heterogeneous storage system of the present application;

[0019] Figure 3 This is a flow chart of an embodiment of a deletion management method for a heterogeneous storage system of the present application;

[0020] Figure 4 This is a flowchart of another embodiment of the deletion management method for a heterogeneous storage system of the present application;

[0021] Figure 5 This is a flow chart of an embodiment of a method for managing a heterogeneous storage system according to the present invention;

[0022] Figure 6 This is a flowchart of another embodiment of the fault management of a heterogeneous storage system of the present application;

[0023] Figure 7 This is a schematic diagram of a framework of an embodiment of a management device for a heterogeneous storage system of the present application;

[0024] Figure 8 This is a schematic diagram of the framework of an embodiment of the electronic device of the present application;

[0025] Figure 9 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0026] The following describes the embodiments of the present application in detail with reference to the accompanying drawings.

[0027] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0028] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0029] The heterogeneous storage system in this application includes several storage subsystems, each storage subsystem corresponds to at least one storage device, and is used to virtualize the storage resources of the corresponding storage device. In the heterogeneous storage system, there are at least two storage subsystems corresponding to storage devices of different types.

[0030] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of a method for managing a heterogeneous storage system of the present application. Specifically, the method may include the following steps:

[0031] Step S110: Determine the access data type for each storage subsystem.

[0032] The heterogeneous storage system in this application can connect to new storage subsystems as the storage scale increases and technology develops, and allocate a Platform as a Service domain, namely, a PaaS (Platform as a Service) domain, to the newly connected storage subsystem. The Platform as a Service domain is used to manage the corresponding storage subsystem.

[0033] Specifically, see Figure 2 , Figure 2This is a schematic diagram of the framework of an embodiment of a heterogeneous storage system 200 of the present application. The heterogeneous storage system 200 includes several storage subsystems 210, a domain management service 220, a software service platform 230, and a storage pool space 240. The domain management service 220 is used to execute a management method for a heterogeneous storage system to create a storage pool 241 for each storage subsystem 210. The domain management service 220 includes several Platform as a Service domains 221, and each storage subsystem 210 corresponds to a Platform as a Service domain 221. The software service platform 230 (i.e., SaaS platform, Software as a Service) is connected to the domain management service 220 and is used to receive basic information of the storage subsystem 210 uploaded by the domain management service 220. The storage pool space 240 includes a storage pool 241 corresponding to each storage subsystem 210, and the storage pool 241 is used to store business data sent by the software service platform 230.

[0034] In the architecture of the heterogeneous storage system 200 of the present application, the domain management service 220 can be used as a central controller, responsible for coordinating and managing the resources of each storage subsystem 210. By collecting basic information about the storage subsystems 210 (such as available capacity, total network bandwidth, supported storage types, etc.), the domain management service 220 implements global control of the storage subsystems 210 and performs resource allocation and scheduling based on business needs.

[0035] In some embodiments, a heterogeneous storage system 200 may add a new storage subsystem b to a homogeneous storage system. Storage subsystem b may have different storage media from storage subsystem a, or may have some or all of the same storage media, resulting in a heterogeneous storage system. A homogeneous storage system may have storage subsystem a composed of devices with the same storage media. Different storage subsystems 210 may be configured with different storage types, such as object storage, block storage, and file storage, resulting in heterogeneous storage types.

[0036] In other embodiments, the heterogeneous storage system 200 includes at least two storage subsystems 210 , and the types of storage devices corresponding to the storage subsystems 210 are not completely the same.

[0037] In some embodiments, the type of access data can be determined based on the storage device of storage subsystem 210. For example, if storage subsystem 210 is composed of a hard disk drive, due to its large capacity, low cost, medium read and write speed (compared to SSDs), and susceptibility to physical damage, it can store a large amount of access data such as files, operating systems, applications, videos, audio, documents, etc. For another example, if storage subsystem 210 is composed of an optical disk, due to its non-volatile storage, long life (if properly stored), fast read speed, but slow write speed, and limited capacity, it can store access data such as software installation packages and backup data.

[0038] It is understandable that the storage device of the storage subsystem 210 may also be a magnetic tape, a memory card or other device, and the type of accessed data is determined according to the characteristics of the storage device, which is not specifically limited here.

[0039] In other embodiments, the type of data accessible may be determined based on the type of data accessible by the heterogeneous storage system 200 and the characteristics of the storage devices in the storage subsystem 210. For example, a storage subsystem 210 may consist of a hard drive and an optical disk, and the type of data accessible by the heterogeneous storage system 200 is operating system and application data. This type of data is applicable to the hard drive portion of the storage devices in the storage subsystem 210. Therefore, only the hard drive portion of the storage subsystem 210 is subsequently converted into a storage pool of the corresponding type, while the optical disk portion is not processed and is not used for data storage.

[0040] Step S120: For each storage subsystem, use the storage resources corresponding to the storage subsystem to create a storage pool that matches the access data type corresponding to the storage subsystem.

[0041] The storage pool 241 is used to store data corresponding to the access data type.

[0042] In some embodiments, for each storage subsystem 210, after determining the access data type of the storage subsystem 210, the platform as a service domain 221 corresponding to the storage subsystem 210 can be used to call the management interface of the storage subsystem 210 to divide at least part of the storage resources corresponding to the storage subsystem 210 into a storage pool 241 that matches the access data type corresponding to the storage subsystem 210.

[0043] Specifically, each storage subsystem 210 is independently managed by a Platform as a Service (PaS) domain 221. Different PaS domains 221 have different data access capabilities, allowing different types of storage pool resources to be created based on the type of data being accessed. In this embodiment, by uniformly encapsulating and managing the PaS domains 221, the differences between storage subsystems 210 are shielded, enabling management of heterogeneous storage systems 200. The PaS domains 221 recognize the type of data being accessed and adaptively create corresponding storage pools 241, enabling management of heterogeneous storage types.

[0044] Furthermore, when a new storage subsystem 210 is added to the heterogeneous storage system 200, the domain management service 220 allocates a Platform as a Service domain 221 to the newly added storage subsystem 210. At the same time, when a new storage subsystem 210 is added to the heterogeneous storage system 200, the basic information of the new storage subsystem 210 must also be recorded. Therefore, the Platform as a Service domain 221 corresponding to the storage subsystem 210 can be used to obtain the basic information of the storage subsystem 210, where the basic information includes at least one of the following: the available capacity of the storage subsystem 210, the total network bandwidth, the supported storage types, the overall system load, and the IP address and port used by the storage subsystem 210 to provide external services. At the same time, the basic information of the storage subsystem 210 can also be synchronized to the software service platform 230, where the software service platform 230 is used to write business data to the storage pool 241 corresponding to the storage subsystem 210.

[0045] In addition, the domain management service 220 allocates a Platform as a Service (PaaS) domain 221 to the newly added storage subsystem 210. The allocated Platform as a Service (PaaS) domain 221 is an idle and unassigned Platform as a Service (PaaS) domain 221 managed by the domain management service 220. If a storage subsystem 210 is newly added to the heterogeneous storage system 200 and there are no idle and unassigned Platform as a Service (PaaS) domains 221 managed by the domain management service 220, the domain management service 220 will issue a prompt to inform the operating object that the Platform as a Service (PaaS) domain 221 needs to be expanded.

[0046] The addition of new storage subsystems 210 is a key component of the dynamic scalability of heterogeneous storage system 200. These new storage subsystems 210 should have a unified entry point (IP address and port) to facilitate easy access and management by domain management service 220. Furthermore, storage subsystems 210 should provide standardized information interfaces, such as available capacity, total network bandwidth, and supported storage types, to ensure that domain management service 220 can accurately access and effectively manage this information.

[0047] Although the specific implementation within the storage subsystem 210 may vary due to different technical architectures and hardware devices, the domain management service 220 only focuses on the basic information and service interfaces provided by the storage subsystem 210 and does not need to worry about its internal implementation details, thereby achieving management at the heterogeneous storage system 200 level.

[0048] When adding a new storage subsystem 210, security and isolation must also be considered. This includes authenticating and authorizing the new storage subsystem 210, and ensuring network and data isolation between the new storage subsystem 210 and existing storage subsystems 210. By implementing strict security policies and access control mechanisms, the security and data integrity of the storage subsystem 210 can be protected.

[0049] See also Figure 3 , Figure 3 This is a flow chart of an embodiment of a deletion management method for a heterogeneous storage system of the present application. Deleting a storage subsystem 210 that is idle or abnormal is an important step in resource recovery and optimal utilization of the heterogeneous storage system 200. Specifically, the deletion management method includes the following steps:

[0050] Step S310: Detect the business data in each storage subsystem to obtain a detection result.

[0051] Before deleting a storage subsystem 210, it is necessary to check the current status and service data of the storage subsystem 210 to determine whether the storage subsystem 210 is idle or in an abnormal state. Therefore, the PaaS domain 221 can be used to test the service data in the corresponding storage subsystem 210 and obtain test results. Specifically, a timed test method can be used, where the PaaS domain 221 tests the service data in the corresponding storage subsystem 210 at set time intervals.

[0052] Step S320: determining the abnormal storage subsystem based on the detection result that the business data has abnormal conditions, and deleting the abnormal storage subsystem after waiting for a preset time;

[0053] In some embodiments, the detection results are generally divided into two situations: one is that the business data has an abnormal situation; the other is that the business data has a normal situation. Among them, the abnormal situation can be further divided into two situations: one is that the business data exists and the business data has expired; the other is that the business data does not exist. When the detection result is that the business data exists and the business data has expired, the storage subsystem 210 with the detection result of the business data exists and the business data has expired can be determined as the first abnormal storage subsystem, and the storage subsystem 210 is in an abnormal state at this time. When the detection result is that the storage subsystem 210 does not have business data, the storage subsystem 210 with the detection result of the business data does not exist can be determined as the second abnormal storage subsystem, and the storage subsystem 210 is in an idle state at this time.

[0054] In a specific embodiment, when the detection result indicates that business data exists and has expired, in response to the abnormal storage subsystem being the first abnormal storage subsystem, several storage pools corresponding to the first abnormal storage subsystem are marked, and after waiting for a preset time, the first abnormal storage subsystem and the marked storage pools are deleted. When the detection result indicates that business data does not exist, in response to the abnormal storage subsystem being the second abnormal storage subsystem, after waiting for a preset time, the second abnormal storage subsystem and its corresponding storage pools are deleted. The preset time can be 10 minutes, 30 minutes, etc., and can also be set by the operation object or a default time, which is not specifically limited here.

[0055] Step S330: The storage subsystem whose detection result indicates that no abnormality exists in the business data is determined to be a normal storage subsystem.

[0056] In some cases, deleting a storage subsystem 210 may require active intervention and decision-making support from the operator. For example, if business data exists in the storage subsystem 210, the operator must decide whether to delete the storage subsystem 210 and how to perform data migration and business recovery. In this case, the domain management service 220 can provide relevant decision-making support information, such as the usage of the storage subsystem 210 and feasibility analysis of data migration, to help the operator make an informed decision.

[0057] Specifically, if the detection result of the normal storage subsystem is that business data exists and the business data has not expired, then the storage subsystem 210 is considered to be in a normal state. In order to improve the usability of the heterogeneous storage system 200, the normal storage subsystem allows deletion operations, and the normal storage subsystem has business data and the business data has not expired. Therefore, deleting the normal storage subsystem requires migrating and saving the business data stored in it. Specifically, in response to receiving a deletion instruction from the operation object to the first normal storage subsystem, the remaining storage capacity of the second normal storage subsystem is obtained, and in response to the remaining storage capacity meeting the first preset migration condition, the business data in the storage pool corresponding to the first normal storage subsystem is migrated to the storage pool of the second normal storage subsystem, and the first normal storage subsystem is deleted, wherein the second normal storage subsystem is a normal storage subsystem other than the first normal storage subsystem.

[0058] The first preset migration condition includes: the remaining storage capacity of the second normal storage subsystem is greater than the migration data capacity of the first normal storage subsystem, the planned usage capacity of the current storage plan, and the total network load of the second normal storage subsystem is less than a preset multiple of the network load of the heterogeneous storage system. The planned usage capacity of the current storage plan represents the amount of business data to be stored in the current heterogeneous storage system 200. The preset multiple can be 0.5, 0.7, 0.85, etc., and is not specifically limited here.

[0059] Furthermore, in order to delete the first normal storage subsystem, the first normal storage subsystem can be set to unavailable, and several storage pools corresponding to the first normal storage subsystem can be marked. After waiting for a preset time, the first normal storage subsystem and the marked storage pools can be deleted. In addition, the basic information corresponding to the first normal storage subsystem stored in the software service platform 230 can also be deleted together. Specifically, the deletion operation should include steps such as deleting the physical device and logical information of the storage subsystem 210, releasing occupied resources (such as IP addresses, ports, etc.), and updating the storage information in the domain management service 220. By completely deleting the storage subsystem 210 that is no longer needed, valuable resources can be released and the storage environment can be optimized.

[0060] Data migration should adhere to certain rules and standards, such as the available capacity of the storage subsystem, system load, and migration strategy. By selecting the appropriate migration path and strategy, you can ensure efficient and secure data migration. Data migration can be categorized into online and offline scenarios.

[0061] In an online data migration scenario, during the migration process, block-level replication technology is first used. This technology can accurately copy data from the source storage (the storage pool corresponding to the normal storage subsystem to be deleted) to the target storage (the storage pool corresponding to other normal storage subsystems) using storage blocks as the basic unit. By operating the block mapping table at the bottom layer of the storage subsystem 210, the pointer of the source block is redirected to the target block, so that the upper-layer application can hardly perceive the change in data location during the data migration process. At the same time, combined with double-write technology, in the early stage of data copying from the source storage to the target storage, the newly generated data is written to the source storage and the target storage at the same time to ensure data consistency. Once the data migration is completed and verified to be correct, the business is switched to fully rely on the target storage to achieve a seamless transition.

[0062] To ensure the stability of the migration process, it is also necessary to obtain real-time information on the migration progress, data consistency, and performance indicators of the business system. By deploying intelligent detection software in the storage system, key indicators such as the completion rate of the migration task, the checksum information of the migrated data, and the response time and throughput of the business system are collected at regular intervals (such as 5 seconds). If the migration progress is found to be abnormally slow, such as the migration rate is lower than the preset threshold for multiple consecutive time intervals, or an error occurs in the data consistency check, the system will automatically suspend the migration process to troubleshoot and repair the problem, and resume the migration after the problem is resolved, ensuring that the entire migration process is smooth and reliable and the business continues to operate normally.

[0063] For example, a large e-commerce company, with its rapid business growth, found that its existing storage system based on traditional mechanical hard drives could no longer meet the increasing read and write demands. This was especially true during promotional events, when problems like order processing delays and slow product information loading became frequent. To improve system performance, the company decided to introduce solid-state drives (SSDs) to build a heterogeneous storage system. Online data migration technology became crucial.

[0064] In an offline data migration scenario, the migration process must be carefully planned before the offline migration begins. First, the source data (business data in the storage pool corresponding to the normal storage subsystem to be deleted) is thoroughly sorted and categorized according to data type, age, and related projects. Differentiated migration plans are then developed for different data categories. Historical data that is very old and rarely accessed can be migrated during non-business hours or at a low priority. Data involved in recent analysis projects should be prioritized for migration to ensure business continuity.

[0065] During the migration process, a dedicated data migration tool, such as a high-speed storage area network (SAN), is used to connect the source tape library and the heterogeneous storage system 200. Multi-threading technology is used to read data blocks from the tape library in parallel and write them to the corresponding locations on the target storage (the storage pool corresponding to other normal storage subsystems). This fully utilizes the relatively idle system resources during offline migration and significantly improves migration efficiency.

[0066] Furthermore, to mitigate data risks during the migration process, rigorous data validation is performed both before and after the migration. Before the migration, a hash value is calculated for the source data and recorded. After the migration is complete, the hash value of the target data is recalculated and compared against the source data hash value to ensure data integrity. If a hash value inconsistency is detected, a data retransmission mechanism is immediately activated to ensure accurate data migration and provide a solid and reliable data foundation for subsequent business data analysis.

[0067] For example, a research institute had accumulated a massive amount of experimental data, stored in an outdated tape library. As data analysis requirements became increasingly complex, they needed to migrate this data to a heterogeneous storage system based on a distributed storage architecture to improve data processing efficiency. Because the data volume reached hundreds of TB and real-time performance was not a priority, an offline migration strategy became the preferred approach.

[0068] To ensure the traceability and security of the operation of deleting the storage subsystem 210, the entire deletion process can be logged and audited. This includes recording information such as the time of the deletion operation, the name of the operation object, and the reason, as well as auditing whether the deletion operation complies with regulations and standards. Through logging and auditing, potential problems and risks can be discovered and resolved in a timely manner.

[0069] In one implementation scenario, the platform as a service domain 221 detects the storage subsystem 210 according to a set time interval and obtains the detection result. Figure 4 The Platform as a Service domain 221 detects whether the storage subsystem 210 contains any business data. If not, the storage subsystem 210 is determined to be a second abnormal storage subsystem, and the corresponding storage pools are marked. After a preset time, the second abnormal storage subsystem and the marked storage pools are deleted. If the business data exists, it is further determined whether the business data has expired. If the business data has expired, the storage subsystem 210 is determined to be a first abnormal storage subsystem, and the corresponding storage pools are marked. After a preset time, the first abnormal storage subsystem and the marked storage pools are deleted. If not, it is determined whether the operation object actively intervenes in the decision-making. Specifically, whether the operation object actively intervenes in the decision-making is determined by whether a deletion instruction for the first normal storage subsystem is received from the operation object. If a deletion instruction is received, it is further determined whether the remaining storage capacity of the second normal storage subsystem meets the first preset migration condition. If not, the detection process is terminated; if so, the business data in the storage pool corresponding to the first normal storage subsystem is migrated. After the migration is completed, the first normal storage subsystem and its corresponding storage pool are marked, and after waiting for a preset time, the first normal storage subsystem and its corresponding storage pool are deleted.

[0070] See also Figure 5 , Figure 5 This is a flow chart illustrating an embodiment of a method for managing a heterogeneous storage system. Fault management of storage subsystem 210 is crucial for ensuring the reliability and availability of heterogeneous storage system 200. When a storage subsystem 210 failure occurs, a prompt response and effective recovery measures must be implemented to minimize service interruption and data loss. Specifically, the following steps may be included:

[0071] Step S510: Detecting that a storage subsystem meets a preset fault condition, and determining the storage subsystem meeting the preset fault condition as a faulty storage subsystem.

[0072] In some embodiments, the Platform as a Service domain 221 corresponding to the storage subsystem 210 can be used to perform fault detection on the storage subsystem 210, obtain a fault detection result, and determine whether the fault detection result meets a preset fault condition. If so, the storage subsystem 210 is considered a faulty storage subsystem. The fault detection can be set as a scheduled detection.

[0073] In other embodiments, real-time monitoring of the operating status and capacity of the storage subsystem 210 can be used to promptly identify potential faults and initiate alerts. Alert information should include information such as the fault type, impact scope, and severity, enabling management entities to quickly locate the problem and take action. Furthermore, the alert mechanism should be flexible and scalable to adapt to changing requirements in different scenarios.

[0074] Step S520: Acquire the first capability value and the first remaining storage capacity of each third normal storage subsystem in the current cycle.

[0075] The third normal storage subsystem is the storage subsystem 210 in the heterogeneous storage system 200 except the faulty storage subsystem.

[0076] When a storage subsystem 210 fails, another functioning storage subsystem 210 must take over the storage capacity of the failed storage subsystem to ensure business continuity and data integrity. Takeover task allocation should adhere to certain policies and principles, such as prioritizing storage pools of the same type and allocating resources based on capacity and available capacity, to ensure the rationality and efficiency of the takeover process. Furthermore, factors such as system load and network bandwidth should be considered during the takeover process to avoid bottlenecks or problems caused by the takeover operation.

[0077] In some embodiments, the heterogeneous storage system 200 will periodically collect the capacity value and available capacity of each storage subsystem and save them in the software service platform 230. When a storage subsystem 210 fails, in order to take over the capacity value of the failed storage subsystem, the first capacity value and first remaining storage capacity of each third normal storage subsystem in the current cycle can be obtained from the software service platform 230, as well as the second capacity value and second remaining storage capacity recorded in the previous cycle of the failed storage subsystem, as a basis for determining whether the heterogeneous storage system 200 can take over the capacity value of the failed storage subsystem, wherein the second capacity value is obtained through the system average load, input / output average load and average total disk load of the failed storage subsystem. The second capacity value is calculated as follows:

[0078] V=1 / (Sys Load +IO LOAD +Disk Load ) (1).

[0079] Among them, V represents the second capability value, Sys Load Indicates the average system load, IO LOAD Indicates the average input / output load, Disk Load Indicates the average total disk load.

[0080] Step S530: Based on the first capability value and the first remaining storage capacity, the capability value corresponding to the faulty storage subsystem is allocated to at least one third normal storage subsystem.

[0081] In some embodiments, when taking over the capacity of a failed storage subsystem, priority should be given to using storage pools of the same type as the failed storage subsystem within a third, functioning storage subsystem. Therefore, the storage pools of each third, functioning storage subsystem must first be categorized. Specifically, a first storage pool and a second storage pool are determined from the storage pools corresponding to each third, functioning storage subsystem. The first storage pool is of the same type as the storage pool corresponding to the failed storage subsystem, and the second storage pool is of a different type than the storage pool corresponding to the failed storage subsystem.

[0082] Afterwards, a takeover task is assigned to the capacity value of the failed storage subsystem. In response to the existence of at least one first storage pool, at least a portion of the capacity value corresponding to the failed storage subsystem is taken over by the at least one first storage pool.

[0083] Specifically, in response to a first storage pool satisfying a preset takeover condition, all capacity values of the failed storage subsystem are taken over by the first storage pool that satisfies the takeover condition, wherein the preset takeover condition is that the sum of the first capacity values corresponding to all first storage pools and the second capacity value satisfy a first size relationship, and the sum of the first remaining storage capacity corresponding to all first storage pools and the second remaining storage capacity satisfy a second size relationship. Furthermore, the first size relationship may be that a first multiple of the sum of the first capacity values of all first storage pools is greater than the second capacity value, and the second size relationship may be that a second multiple of the sum of the first remaining storage capacity of all first storage pools is greater than the second remaining storage capacity.

[0084] For example, the first size relationship may be that 0.8 times the sum of the first capacity values corresponding to all first storage pools is greater than the second capacity value, and the second size relationship may be that 0.8 times the sum of the first remaining storage capacities corresponding to all first storage pools is greater than the second remaining storage capacity.

[0085] In another specific scenario, in response to each first storage pool failing to meet the preset takeover conditions, the capacity value of the faulty storage subsystem is allocated to each first storage pool for takeover, and if there is a remaining capacity value of the faulty storage subsystem after allocation, at least one second storage pool is used to take over the remaining capacity value of the faulty storage subsystem.

[0086] For example, a first target storage pool is selected from each first storage pool, and a first takeover capacity value of the first target storage pool is determined, wherein the first takeover capacity value is related to at least one of the first capacity value corresponding to the first target storage pool and the first remaining storage capacity. According to the first takeover capacity value, part of the capacity value of the faulty storage subsystem is allocated to the first target storage pool for takeover. Repeat the aforementioned steps until all first storage pools are selected or the capacity value allocation of the faulty storage subsystem is completed. For the remaining capacity value of the faulty storage subsystem, a second storage pool is selected as the second target storage pool, and a second takeover capacity value of the second target storage pool is determined, wherein the second takeover capacity value is related to at least one of the first capacity value corresponding to the second target storage pool and the first remaining storage capacity. According to the second takeover capacity value, part of the capacity value of the faulty storage subsystem is allocated to the second target storage pool for takeover. Repeat the aforementioned steps until the capacity value allocation of the faulty storage subsystem is completed.

[0087] In another specific scenario, in response to the absence of the first storage pool, at least one second storage pool is used to take over all capacity values of the failed storage subsystem. Specifically, a second storage pool is selected as the second target storage pool, and a second takeover capacity value of the second target storage pool is determined, where the second takeover capacity value is related to at least one of the first capacity value and the first remaining storage capacity corresponding to the second target storage pool. Based on the second takeover capacity value, a portion of the capacity value of the failed storage subsystem is allocated to the second target storage pool for takeover. The aforementioned steps are repeated until the capacity value allocation of the failed storage subsystem is complete.

[0088] In some embodiments, the first takeover capability value is related to the first capability value and the first remaining storage capacity corresponding to the first target storage pool, and the second takeover capability value is related to the first capability value and the first remaining storage capacity corresponding to the second target storage pool. Therefore, the first takeover capability value may be the smaller value of the ratio between the first capability value corresponding to the first target storage pool and the sum of the first capability values, and the ratio between the first remaining storage capacity corresponding to the first target storage pool and the sum of the first capacity values, the sum of the first capability values being the sum of the first capability values and the second capability values corresponding to each first target storage pool, and the sum of the first capacity being the sum of the first remaining storage capacity and the second remaining storage capacity corresponding to each first target storage pool. The second takeover capability value may be the smaller value of the product of the second capability value and the capability ratio, and the product of the second remaining storage capacity and the capability ratio, the capability ratio being the ratio between the first capability value and the sum of the second capability values corresponding to the second target storage pool, the sum of the second capability values being the sum of the first capability values and the second capability values corresponding to each second storage pool, the capacity ratio being the ratio between the first remaining storage capacity corresponding to the second target storage pool and the sum of the second capacity, and the sum of the second capacity being the sum of the first remaining storage capacity and the second remaining storage capacity corresponding to each second storage pool.

[0089] Specifically,

[0090]

[0091] in, Represents the first capability value and, represents the first capacity and, Indicates the first takeover capability value, V 1i represents the first capacity value corresponding to the first target storage pool, V a Indicates the second ability value, AC 1i Indicates the first remaining storage capacity corresponding to the first target storage pool, AC a represents the second remaining storage capacity, i=1,…,n.

[0092]

[0093] in, Represents the second capability value and, Represents the second capacity and V′ 2i Indicates the capability ratio, AC′ 2i Indicates the capacity ratio, Indicates the second takeover capability value, V 2i represents the first capacity value corresponding to the second storage pool, and AC2i represents the first remaining storage capacity corresponding to the second storage pool.

[0094] After the takeover is complete, fault recovery and data reconstruction must be performed as quickly as possible to restore normal operation and data integrity of the storage subsystem 210. This includes repairing faulty hardware, recreating data replicas, and restoring service access. Implementing an effective fault recovery and data reconstruction strategy can minimize service interruption and the risk of data loss.

[0095] In addition to the aforementioned fault management measures, regular preventive maintenance is also required to reduce the probability of storage subsystem 210 failures and improve system stability and reliability. This includes regularly checking hardware status, updating software versions, and optimizing storage configurations. Furthermore, optimization suggestions and improvement measures are proposed based on system operating conditions and changes in business needs to continuously improve the performance and efficiency of heterogeneous storage system 200.

[0096] In one implementation scenario, the platform as a service domain 221 can be used to detect in real time whether the corresponding storage subsystem 210 has failed. Figure 6 The Platform as a Service domain 221 performs fault detection on the storage subsystem 210, determines the storage subsystem that meets the preset fault conditions as the faulty storage subsystem, and obtains the second capacity value and the second remaining storage capacity recorded in the previous cycle of the faulty storage subsystem. At the same time, the storage pool corresponding to the faulty storage subsystem is marked as unavailable to avoid storing the newly stored business data in the storage pool corresponding to the faulty storage subsystem, causing damage to the newly stored business data.

[0097] Afterwards, the Platform as a Service domain 221 issues a capacity value reporting task for the third normal storage subsystem to the software service platform 230 to obtain the first capacity value and the first remaining storage capacity of each third normal storage subsystem in the current cycle. Afterwards, it is determined whether the storage pool corresponding to each third normal storage subsystem has a first storage pool. If not, at least one second storage pool is used to take over all capacity values of the faulty storage subsystem; if so, it is further determined whether the first storage pool meets the preset takeover conditions. If the preset takeover conditions are met, all are taken over by the first storage pool, and the capacity values taken over by each first storage pool can refer to Formula 2; if not, a first target storage pool is selected from each first storage pool, and the first takeover capacity value of the first target storage pool is determined. The first takeover capacity value can refer to Formula 2. According to the first takeover capacity value, part of the capacity value of the faulty storage subsystem is allocated to the target storage pool for takeover. The above steps are repeated until all first storage pools are selected or the capacity value allocation of the faulty storage subsystem is completed. Based on the remaining capacity of the failed storage subsystem, a second storage pool is selected as the second target storage pool. A second takeover capacity value is determined for the second target storage pool. For the first takeover capacity value, refer to Formula 3. Based on the second takeover capacity value, a portion of the failed storage subsystem's capacity is allocated to the second target storage pool for takeover. Repeat the above steps until all capacity allocations for the failed storage subsystem are complete.

[0098] The management method of the heterogeneous storage system in this application is of great significance for improving the utilization of storage resources, ensuring business continuity and data integrity. By implementing effective management strategies for adding new storage subsystems, deleting storage subsystems, and managing storage subsystem faults, an efficient, reliable, and scalable heterogeneous storage system can be built to provide strong support for the rapid development of the business. In the future, with the continuous advancement of technology and the continuous changes in business needs, the management method of heterogeneous storage systems will continue to develop and improve to adapt to new challenges and opportunities. For example, with the widespread application of artificial intelligence and big data technologies, it can be considered to introduce intelligent algorithms into the management of heterogeneous storage systems in the future to achieve more intelligent and automated management decisions and optimization adjustments. At the same time, with the rise of emerging technologies such as cloud computing and edge computing, future heterogeneous storage systems also need to consider how to better integrate and innovate with these technologies to provide more efficient and convenient data storage services.

[0099] Compared to existing technologies, the heterogeneous storage system management method of this application can shield differences in storage devices and storage types, utilizing a unified management technology to manage heterogeneous storage systems and ensure the reliability and performance of the storage systems. Furthermore, the heterogeneous storage system management method of this application uses rules based on storage system performance indicators and storage capacity to further improve the availability of heterogeneous systems.

[0100] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0101] See also Figure 7 , Figure 7 The present invention is a schematic diagram of a framework of an embodiment of a management device 700 for a heterogeneous storage system. The management device 700 includes a new addition detection module 710, a deletion detection module 720, and a fault detection module 730. The new addition detection module 710 includes a determination unit 711 and a creation unit 712. The determination unit 711 determines the access data type for each storage subsystem. The creation unit 712 creates a storage pool for each storage subsystem, using the storage resources corresponding to the storage subsystem, that matches the access data type corresponding to the storage subsystem. The storage pool is used to store data corresponding to the access data type.

[0102] In some embodiments, the creation unit 712 executes a platform as a service domain corresponding to each storage subsystem; uses the storage resources corresponding to the storage subsystem to create a storage pool that matches the access data type corresponding to the storage subsystem, including: for each storage subsystem, using the platform as a service domain corresponding to the storage subsystem to call the management interface of the storage subsystem to divide at least part of the storage resources corresponding to the storage subsystem into storage pools that match the access data type corresponding to the storage subsystem.

[0103] In some embodiments, the determination unit 711 executes the use of the platform as a service domain corresponding to the storage subsystem to obtain basic information of the storage subsystem, wherein the basic information includes at least one of the following: the available capacity of the storage subsystem, the total network bandwidth, the supported storage types, the overall system load, the IP address and port for the storage subsystem to provide external services; and / or, synchronizes the basic information of the storage subsystem to the software service platform, wherein the software service platform is used to write business data to the storage pool corresponding to the storage subsystem.

[0104] The deletion detection module 720 includes a deletion detection unit 721, a first deletion unit 722, and a second deletion unit 723. The deletion detection unit 721 performs a detection on the business data in each storage subsystem to obtain a detection result. The first deletion unit 722 determines the storage subsystem whose detection result is that the business data has an abnormality as an abnormal storage subsystem, waits for a preset time, and then deletes the abnormal storage subsystem. The second deletion unit 723 determines the storage subsystem whose detection result is that the business data has no abnormality as a normal storage subsystem, and in response to receiving a deletion instruction for a normal storage subsystem, obtains the remaining storage capacity of other normal storage subsystems except the normal storage subsystem to be deleted, and in response to the remaining storage capacity meeting the first preset migration condition, migrates the business data in the storage pool corresponding to the normal storage subsystem to be deleted to the storage pool of other normal storage subsystems, and deletes the normal storage subsystem to be deleted.

[0105] In some embodiments, the deletion detection unit 721 executes the step of deleting a storage subsystem whose detection result indicates that business data has an abnormality, and determining the abnormal storage subsystem, including at least one of the following steps: determining a storage subsystem whose detection result indicates that business data exists and has expired as a first abnormal storage subsystem; and determining a storage subsystem whose detection result indicates that no business data exists as a second abnormal storage subsystem. The first deletion unit 722 executes the step of deleting the abnormal storage subsystem after waiting for a preset time, including: in response to the abnormal storage subsystem being the first abnormal storage subsystem, marking several storage pools corresponding to the first abnormal storage subsystem, and after waiting for a preset time, deleting the first abnormal storage subsystem and the marked several storage pools; and / or, in response to the abnormal storage subsystem being the second abnormal storage subsystem, deleting the second abnormal storage subsystem and its corresponding several storage pools after waiting for a preset time.

[0106] In some embodiments, the detection result of the second deletion unit 723 executing the normal storage subsystem is that business data exists and the business data has not expired; and / or, the first preset migration condition includes: the remaining storage capacity of other normal storage subsystems is greater than the migration data capacity of the normal storage subsystem to be deleted and the planned usage capacity of the current storage plan, and the total network load of other normal storage subsystems is less than a preset multiple of the network load of the heterogeneous storage system; and / or, deleting the normal storage subsystem to be deleted, including: setting the normal storage subsystem to be deleted as unavailable, marking several storage pools corresponding to the normal storage subsystem to be deleted, and deleting the normal storage subsystem to be deleted and the marked storage pools after waiting for a preset time.

[0107] The fault detection module 730 includes a fault detection unit 731, an acquisition unit 732, and an allocation unit 733. The fault detection unit 731 detects that a storage subsystem meets a preset fault condition and determines the storage subsystem meeting the preset fault condition as a faulty storage subsystem. The acquisition unit 732 acquires the first capacity value and first remaining storage capacity of each third normal storage subsystem in the current cycle, wherein the third normal storage subsystem is a storage subsystem in the heterogeneous storage system other than the faulty storage subsystem. The allocation unit 733 allocates the capacity value corresponding to the faulty storage subsystem to at least one third normal storage subsystem based on the first capacity value and the first remaining storage capacity.

[0108] In some embodiments, the allocation unit 733 performs allocation of the capacity value corresponding to the faulty storage subsystem to at least one third normal storage subsystem based on the first capacity value and the first remaining storage capacity, including: determining a first storage pool and a second storage pool from the storage pools corresponding to each third normal storage subsystem, wherein the first storage pool is a storage pool of the same type as the storage pool corresponding to the faulty storage subsystem, and the second storage pool is a storage pool of a different type from the storage pool corresponding to the faulty storage subsystem; in response to the existence of at least one first storage pool, using at least one first storage pool to take over at least part of the capacity value corresponding to the faulty storage subsystem; and / or, in response to the absence of the first storage pool, using at least one second storage pool to take over all capacity values of the faulty storage subsystem.

[0109] In some embodiments, the acquisition unit 732 acquires a second capacity value and a second remaining storage capacity recorded in the previous cycle of the failed storage subsystem, wherein the second capacity value is obtained based on the system average load, input / output average load, and total disk average load of the failed storage subsystem. The allocation unit 733 utilizes at least one first storage pool to take over at least a portion of the capacity value corresponding to the failed storage subsystem, including: in response to a first storage pool that meets a preset takeover condition, utilizing the first storage pool that meets the takeover condition to take over all capacity values of the failed storage subsystem, wherein the preset takeover condition is that the first capacity value and the second capacity value corresponding to the first storage pool meet a first size relationship, and the first remaining storage capacity and the second remaining storage capacity corresponding to the first storage pool meet a second size relationship; in response to each first storage pool not meeting the preset takeover condition, allocating the capacity value of the failed storage subsystem to each first storage pool for takeover, and if the failed storage subsystem has a remaining capacity value after the allocation, utilizing at least one second storage pool to take over the remaining capacity value of the failed storage subsystem.

[0110] In some embodiments, the allocation unit 733 executes a first size relationship in which a first multiple of the first capacity value of the first storage pool is greater than a second capacity value, and a second size relationship in which a second multiple of the first remaining storage capacity of the first storage pool is greater than the second remaining storage capacity; and / or, allocates the capacity value of the faulty storage subsystem to each first storage pool for takeover, including: selecting a first target storage pool from each first storage pool, and determining a first takeover capacity value of the first target storage pool, wherein the first takeover capacity value is related to at least one of the first capacity value and the first remaining storage capacity corresponding to the first target storage pool; allocating part of the capacity value of the faulty storage subsystem to the target storage pool for takeover according to the first takeover capacity value; repeating the aforementioned Steps, until all first storage pools are selected or the capacity value allocation of the faulty storage subsystem is completed; using at least one second storage pool to take over all capacity values of the faulty storage subsystem, or using at least one second storage pool to take over the remaining capacity value of the faulty storage subsystem, including: selecting a second storage pool as a second target storage pool, and determining a second takeover capacity value of the second target storage pool, wherein the second takeover capacity value is related to at least one of the first capacity value and the first remaining storage capacity corresponding to the second target storage pool; according to the second takeover capacity value, allocating part of the capacity value of the faulty storage subsystem to the second target storage pool for takeover; repeating the above steps until the capacity value allocation of the faulty storage subsystem is completed.

[0111] In some embodiments, the allocation unit 733 executes a first takeover capability value which is the smaller value of the ratio between the first capability value corresponding to the first target storage pool and the sum of the first capability values and the ratio between the first remaining storage capacity corresponding to the first target storage pool and the sum of the first capacity values, the sum of the first capability values is the sum of the first capability value and the second capability value corresponding to each first target storage pool, and the sum of the first capacity is the sum of the first remaining storage capacity and the second remaining storage capacity corresponding to each first target storage pool; a second takeover capability value which is the smaller value of the product of the second capability value and the capability ratio and the product of the second remaining storage capacity and the capability ratio, the capability ratio is the ratio between the first capability value and the sum of the second capability values corresponding to the second target storage pool, the sum of the second capability values is the sum of the first capability value and the second capability values corresponding to each second storage pool, the capacity ratio is the ratio between the first remaining storage capacity corresponding to the second target storage pool and the sum of the second capacity, and the sum of the second capacity is the sum of the first remaining storage capacity and the second remaining storage capacity corresponding to each second storage pool.

[0112] See also Figure 8 , Figure 81 is a schematic diagram of an embodiment of an electronic device 80 of the present application. Electronic device 80 includes a memory 81 and a processor 82 coupled to each other. Processor 82 is configured to execute program instructions stored in memory 81 to implement the steps of any of the aforementioned heterogeneous storage system management method embodiments. In a specific implementation scenario, electronic device 80 may include, but is not limited to, a microcomputer and a server. Furthermore, electronic device 80 may also include mobile devices such as laptops and tablet computers, without limitation herein.

[0113] Specifically, the processor 82 is used to control itself and the memory 81 to implement the steps in any of the above-mentioned heterogeneous storage system management method embodiments. The processor 82 can also be called a CPU (Central Processing Unit). The processor 82 may be an integrated circuit chip with signal processing capabilities. The processor 82 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 82 can be implemented by an integrated circuit chip.

[0114] See also Figure 9 , Figure 9 1 is a schematic diagram of a framework of an embodiment of a computer-readable storage medium 90 of the present application. The computer-readable storage medium 90 stores program instructions 901 that can be executed by a processor, and the program instructions 901 are used to implement the steps of any of the above-mentioned heterogeneous storage system management method embodiments.

[0115] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0116] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced with each other and will not be repeated herein for the sake of brevity.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation methods described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0118] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0119] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

Claims

1. A method for managing a heterogeneous storage system, characterized in that: The heterogeneous storage system includes a plurality of storage subsystems, each of which corresponds to at least one storage device and is used to virtualize storage resources of the corresponding storage device. In the heterogeneous storage system, at least two of the storage subsystems correspond to storage devices of different types. The method comprises: Determining an access data type for each of the storage subsystems; For each of the storage subsystems, a storage pool matching the access data type corresponding to the storage subsystem is created using the storage resources corresponding to the storage subsystem, wherein the storage pool is used to store data corresponding to the access data type.

2. The method according to claim 1, characterized in that Each of the storage subsystems corresponds to a Platform as a Service domain; The step of utilizing storage resources corresponding to the storage subsystem to create a storage pool that matches the access data type corresponding to the storage subsystem includes: For each of the storage subsystems, the platform as a service domain corresponding to the storage subsystem is used to call the management interface of the storage subsystem to divide at least part of the storage resources corresponding to the storage subsystem into storage pools that match the access data type corresponding to the storage subsystem.

3. The method according to claim 1, characterized in that The method further comprises: Detecting the business data in each of the storage subsystems to obtain a detection result; Determine the storage subsystem whose business data has abnormality as a detected result as an abnormal storage subsystem, and delete the abnormal storage subsystem after a preset time; and / or, The storage subsystem whose detection result is that there is no abnormality in the business data is determined to be a normal storage subsystem. In response to receiving a deletion instruction for the first normal storage subsystem, the remaining storage capacity of the second normal storage subsystem is obtained. In response to the remaining storage capacity meeting a first preset migration condition, the business data in the storage pool corresponding to the first normal storage subsystem is migrated to the storage pool of the second normal storage subsystem, and the first normal storage subsystem is deleted, wherein the second normal storage subsystem is a normal storage subsystem other than the first normal storage subsystem.

4. The method according to claim 3, characterized in that The detection result of the normal storage subsystem is that the business data exists and the business data is not expired; The step of determining the abnormal storage subsystem based on the detection result that the business data has an abnormality includes at least one of the following steps: Determine the storage subsystem, for which the detection result shows that the business data exists and the business data is expired, as a first abnormal storage subsystem; Determine the storage subsystem, for which the detection result shows that the business data does not exist, as a second abnormal storage subsystem; After waiting for a preset time, deleting the abnormal storage subsystem includes: In response to the abnormal storage subsystem being the first abnormal storage subsystem, marking the storage pools corresponding to the first abnormal storage subsystem, and after waiting for a preset time, deleting the first abnormal storage subsystem and the marked storage pools; and / or, In response to the abnormal storage subsystem being the second abnormal storage subsystem, after waiting for a preset time, deleting the second abnormal storage subsystem and the corresponding storage pools; And / or, the first preset migration condition includes: the remaining storage capacity of the second normal storage subsystem is greater than the migration data capacity of the first normal storage subsystem and the planned usage capacity of the current storage plan, and the total network load of the second normal storage subsystem is less than a preset multiple of the network load of the heterogeneous storage system; And / or, deleting the first normal storage subsystem includes: The first normal storage subsystem is set to be unavailable, several storage pools corresponding to the first normal storage subsystem are marked, and after waiting for a preset time, the first normal storage subsystem and the marked storage pools are deleted.

5. The method according to claim 1, wherein The method further comprises: detecting that a storage subsystem meets a preset fault condition, and determining the storage subsystem meeting the preset fault condition as a faulty storage subsystem; Obtaining a first capability value and a first remaining storage capacity of each third normal storage subsystem in a current cycle, wherein the third normal storage subsystem is a storage subsystem in the heterogeneous storage system excluding the faulty storage subsystem; Based on the first capacity value and the first remaining storage capacity, the capacity value corresponding to the faulty storage subsystem is allocated to at least one of the third normal storage subsystems.

6. The method according to claim 5, characterized in that The allocating the capacity value corresponding to the faulty storage subsystem to at least one of the third normal storage subsystems based on the first capacity value and the first remaining storage capacity includes: Determining a first storage pool and a second storage pool from the storage pools corresponding to each of the third normal storage subsystems, wherein the first storage pool is a storage pool of the same type as the storage pool corresponding to the faulty storage subsystem, and the second storage pool is a storage pool of a different type than the storage pool corresponding to the faulty storage subsystem; In response to the existence of at least one first storage pool, using the at least one first storage pool to take over at least part of the capacity value corresponding to the failed storage subsystem; and / or, In response to the absence of the first storage pool, at least one second storage pool is used to take over all capacity values of the failed storage subsystem.

7. The method according to claim 6, characterized in that The method further comprises: Obtaining a second capacity value and a second remaining storage capacity recorded in a previous cycle of the faulty storage subsystem, wherein the second capacity value is obtained by using the system average load, the input / output average load, and the average total load of the disks of the faulty storage subsystem; The using the at least one first storage pool to take over at least part of the capacity value corresponding to the failed storage subsystem includes: In response to the first storage pool satisfying a preset takeover condition, the first storage pool is used to take over all capacity values of the failed storage subsystem, wherein the preset takeover condition is that a sum of first capacity values corresponding to all the first storage pools and the second capacity value satisfy a first size relationship, and a sum of first remaining storage capacities corresponding to all the first storage pools and the second remaining storage capacity satisfy a second size relationship, the first size relationship being that a first multiple of the sum of the first capacity values of all the first storage pools is greater than the second capacity value, and the second size relationship being that a second multiple of the sum of the first remaining storage capacities of all the first storage pools is greater than the second remaining storage capacity; In response to the first storage pool not satisfying the preset takeover condition, the capacity value of the faulty storage subsystem is allocated to each of the first storage pools for takeover, and if there is a remaining capacity value of the faulty storage subsystem after allocation, at least one of the second storage pools is used to take over the remaining capacity value of the faulty storage subsystem, wherein the allocating the capacity value of the faulty storage subsystem to each of the first storage pools for takeover comprises: selecting a first target storage pool from each of the first storage pools, and determining a first takeover capacity value of the first target storage pool, wherein the first takeover capacity value is related to at least one of the first capacity value and the first remaining storage capacity corresponding to the first target storage pool; allocating part of the capacity value of the faulty storage subsystem to the first target storage pool for takeover according to the first takeover capacity value; repeating the aforementioned steps until each of the first storage pools is selected or the capacity value allocation of the faulty storage subsystem is completed; Among them, the use of at least one second storage pool to take over all capacity values of the faulty storage subsystem, or the use of at least one second storage pool to take over the remaining capacity value of the faulty storage subsystem, includes: selecting a second storage pool as a second target storage pool, and determining a second takeover capacity value of the second target storage pool, wherein the second takeover capacity value is related to at least one of the first capacity value and the first remaining storage capacity corresponding to the second target storage pool; according to the second takeover capacity value, allocating part of the capacity value of the faulty storage subsystem to the second target storage pool for takeover; repeating the above steps until the capacity value allocation of the faulty storage subsystem is completed.

8. A heterogeneous storage system, characterized in that: include: several storage subsystems; A domain management service, configured to execute the method according to any one of claims 1 to 7 to create a storage pool for each of the storage subsystems; a software service platform connected to the domain management service and configured to receive basic information of the storage subsystem uploaded by the domain management service; The storage pool space includes storage pools corresponding to each of the storage subsystems, and the storage pools are used to store business data sent by the software service platform.

9. An electronic device, characterized in that: The system comprises a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the heterogeneous storage system management method according to any one of claims 1 to 7.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the heterogeneous storage system management method according to any one of claims 1 to 7 is implemented.

Images