Online switching system, method, device and storage medium of file storage system

Abstract

The present invention discloses an online switching system for a file storage system, comprising: a business system, a file storage gateway and a file storage system, wherein the file storage system comprises: an original file storage system and a target file storage system; the business system is used for receiving a file storage system switching instruction, switching an access address of the business system to an address of a file storage gateway based on the file storage system switching instruction, and sending file data to the file storage gateway in real time; the file storage gateway is used for receiving and caching file data sent by the business system in real time, persisting the file data in the original file storage system, and migrating the file data to be migrated stored in the original file storage system to the target file storage system; after the migration of the file data to be migrated is completed, the original file storage system is switched to the target file storage system, and the cached file data obtained after the switching is completed is persisted in the target file storage system, so as to shorten the inaccessible time of the business system as much as possible.

Description

Technical Field

[0001] The present invention relates to the technical field of file data storage, and in particular to an online switching system, method, device and storage medium of a file storage system. Background Art

[0002] With the development of information technology, the unstructured data of enterprises has exploded. The back-end storage of enterprise business systems mostly uses Network Attached Storage (NAS) or Storage Area Network (SAN) to store massive amounts of unstructured data. However, the old original file storage devices have problems such as poor scalability and poor maintainability. Therefore, it is necessary to use advanced target file storage systems on the market, such as unstructured data management systems, to replace the old original file storage systems and take over the continuously growing unstructured data.

[0003] like Figure 1 As shown, the process of replacing the original file storage system with the prior art mainly includes: the first stage: the business system IO is connected to the original file storage system; the second stage: the data of the original file storage system is migrated to the target file storage system through the migration tool, and then the business system IO is connected to the unstructured data management system, the second stage: the original file storage system equipment is removed to complete the process of the target file storage system replacing the original file storage system; the fourth stage: the business system IO is connected to the target file storage system.

[0004] In the process of replacing the original file storage system in the prior art, it is often necessary to interrupt the access to the business system for a long time. The reason is that during the migration process, the incremental data generated by the business system to the original file storage system may be missed in the migration to the unstructured data management system. In order to ensure data consistency, it is necessary to stop the IO access of the business system, compare the original file storage system and the target file storage system to determine the incremental data, and migrate the incremental data to the target file storage system. When the business system generates a large amount of file data, the waiting time for determining the incremental data and the waiting time for migrating the incremental data to the target file storage system are relatively long. It takes at least two hours to scan and compare 10 million data, which results in the business system being inaccessible for a long time during the file data migration period in the process of switching the file storage system. This is unacceptable for business systems that need to ensure high availability. Summary of the invention

[0005] The present invention provides an online switching system, method, device and storage medium for a file storage system to solve the problem in the prior art that a business system is inaccessible during a long period of time during file data migration. The file storage system corresponding to the business system can be switched online by simply suspending access to the business system for a short period of several seconds when the business system IO is connected to a NAS gateway system, thereby shortening the inaccessible time of the business system as much as possible.

[0006] According to one aspect of the present invention, there is provided an online switching system for a file storage system, the online switching system comprising: a business system, a file storage gateway and a file storage system, the file storage system comprising: an original file storage system and a target file storage system; the file storage gateway is connected to the original file storage system and the target file storage system respectively;

[0007] The business system is used to receive a file storage system switching instruction, switch the access address of the business system to the address of the file storage gateway based on the file storage system switching instruction, and send file data to the file storage gateway in real time;

[0008] The file storage gateway is used to receive and cache file data sent by the business system in real time to obtain cached file data; persist the cached file data in the original file storage system, and migrate the file data to be migrated stored in the original file storage system to the target file storage system, wherein the file data to be migrated includes: the cached file data and existing file data; after the migration of the file data to be migrated is completed, the original file storage system is switched to the target file storage system, and the cached file data obtained after the switching is completed is persisted in the target file storage system.

[0009] According to another aspect of the present invention, there is provided an online switching method for a file storage system, which is applied to a file storage gateway in an online switching system of the file storage system, and the method comprises:

[0010] Receiving and caching file data sent in real time by the business system to obtain cached file data; wherein the business system is used to receive a file storage system switching instruction, switch the access address of the business system to the address of the file storage gateway based on the file storage system switching instruction, and send file data to the file storage gateway in real time;

[0011] Persisting the cached file data in the original file storage system, and migrating the file data to be migrated stored in the original file storage system to the target file storage system;

[0012] After the migration of the file data to be migrated is completed, the original file storage system is switched to the target file storage system, and the cached file data obtained after the switching is completed is persisted in the target file storage system.

[0013] According to another aspect of the present invention, an online switching method of a file storage system is applied to a business system in an online switching system of the file storage system, the method comprising:

[0014] Receiving a file storage system switching instruction;

[0015] Switching the access address of the business system to a file storage gateway address based on the file storage system switching instruction;

[0016] Sending file data to the file storage gateway in real time;

[0017] Among them, the file storage gateway is used to receive and cache file data sent by the business system in real time to obtain cached file data; persist the cached file data in the original file storage system, and migrate the file data to be migrated stored in the original file storage system to the target file storage system, and the file data to be migrated includes: the cached file data and the existing file data; after the migration of the file data to be migrated is completed, the original file storage system is switched to the target file storage system, and the cached file data obtained after the switching is completed is persisted in the target file storage system.

[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the online switching method of the file storage system described in any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the online switching method of the file storage system described in any embodiment of the present invention when executed.

[0023] An online switching system for a file storage system provided by an embodiment of the present invention connects a business system with a file storage gateway, takes over the subsequent online switching and file data persistence of the file storage system through the file storage gateway, caches the file data sent by the business system, sends the cached file data to the original file storage system, and migrates the file data stored in the original file storage system to the target file storage system. After the migration is completed, the original file storage system is switched to the target file storage system, and the cached file data obtained after the file storage gateway completes the switching is directly stored in the target file storage system. Therefore, it is only necessary to interrupt the access to the business system during the short period of time when the input and output interfaces of the business system are connected to the file storage gateway, and it is not necessary to interrupt the access to the business system during the long period of data migration, thereby solving the problem of the inaccessibility of the business system during the long period of file storage system switching in the prior art, greatly shortening the inaccessibility time of the business system, and realizing the online switching of the file storage system corresponding to the business system.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 It is a schematic diagram of a switching process of a file storage system in the prior art;

[0027] Figure 2 This is a structural diagram of an online switching system for a file storage system provided according to Embodiment 1 of the present invention;

[0028] Figure 3 is a schematic diagram of an online switching process of a file storage system provided according to Embodiment 1 of the present invention;

[0029] Figure 4 is a structural diagram of an online switching system of a file storage system provided according to Embodiment 2 of the present invention;

[0030] Figure 5 is a flowchart of an online switching method of a file storage system provided according to Embodiment 3 of the present invention;

[0031] Figure 6 is a flowchart of an online switching method of a file storage system provided according to a fourth embodiment of the present invention;

[0032] Figure 7 It is a structural schematic diagram of an electronic device for implementing the online switching method of the file storage system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] Embodiment 1

[0036] Figure 2 The first embodiment of the present invention provides a structural diagram of an online switching system for a file storage system. This embodiment is applicable to the case where an original file storage system connected to a business system is switched to a target file storage system. Figure 2 As shown, the online switching system of the file storage system includes: a business system 10, a file storage gateway 20 and a file storage system 30, wherein the file storage system includes: an original file storage system 31 and a target file storage system 32; the file storage gateway 20 is connected to the original file storage system 31 and the target file storage system 32 respectively;

[0037] The business system 10 is used to receive a file storage system switching instruction, switch the access address of the business system to the address of the file storage gateway based on the file storage system switching instruction, and send file data to the file storage gateway in real time;

[0038] The file storage gateway 20 is used to receive and cache file data sent by the business system in real time to obtain cached file data; persist the cached file data in the original file storage system 31, and migrate the file data to be migrated stored in the original file storage system 31 to the target file storage system 32, and the file data to be migrated includes: the cached file data and the existing file data; after the migration of the file data to be migrated is completed, the original file storage system is switched to the target file storage system, and the cached file data obtained after the switching is completed is persisted in the target file storage system 32.

[0039] The original file storage system 31 refers to the file storage system that the business system is connected to in the initial stage, and the target file storage system 32 refers to the file storage system that the business system is connected to after the file storage system is switched. The file storage system switching instruction is used to instruct the business system 10 to connect to the file storage gateway 20. Since the file storage gateway 20 supports connecting to the original file storage system 31 and the target file storage system 32, the subsequent file storage system switching work can be implemented through the file storage gateway 20. The file storage system switching instruction may include: the file storage gateway address.

[0040] Wherein, the target file storage system includes: unstructured data management system.

[0041] Specifically, Figure 3 As shown in FIG. 1 , the online switching process of the file storage system mainly includes the following three stages.

[0042] In the first stage, the business system 10 is in a state of waiting to switch the file storage system, and the input and output interface IO of the business system 10 is connected to the original file storage system 31 , that is, the access address of the business system 10 is the address of the original file storage system 31 .

[0043] In the second stage, after receiving the file storage system switching instruction, the business system 10 switches the access address of the business system 10 to the address of the file storage gateway 20 according to the address of the file storage gateway included in the file storage system switching instruction, so that the file storage gateway 20 is connected with the business system 10 and receives the file data sent by the business system 10 in real time. The file storage gateway and the file system can be connected through a file protocol, and the file protocol can be, for example, SMB, NFS or FTP.

[0044] The file storage gateway 20 receives the file data sent by the business system 10 in real time, caches the file data to obtain cached file data, and persists the cached file data in the original file storage system 31. To use the target file storage system 32 to replace the original file storage system 31, it is first necessary to migrate all the file data stored in the original file storage system 31, that is, the file data to be migrated, to the target file storage system 32. The file data to be migrated stored in the original file storage system 31 includes: the stock file data stored in the historical time and the cached file data received from the file storage gateway 20. In the process of migrating the data to be migrated in the original file storage system 31 to the target file storage system 32, the file data generated by the business system 10 can be temporarily cached by the file storage gateway 20. Therefore, data migration can be achieved without interrupting the access of the business system, ensuring the consistency of data migration. After all the file data to be migrated stored in the original file storage system 31 are migrated to the target file storage system 32, the file storage gateway 20 switches the file storage system storing the cached file data from the original file storage system 31 to the target file storage system 32, thereby switching the file storage system connected to the business system.

[0045] In the third stage, after the file storage system connected to the business system 10 is switched to the target file storage system, the file storage gateway 20 continues to receive and cache the file data sent by the business system in real time, and directly persists the cached file data obtained after the switch is completed to the target file storage system, thereby realizing the connection between the business system and the target file storage system.

[0046] An online switching system for a file storage system provided by an embodiment of the present invention connects a business system with a file storage gateway, takes over the subsequent online switching and file data persistence of the file storage system through the file storage gateway, caches the file data sent by the business system, sends the cached file data to the original file storage system, and migrates the file data stored in the original file storage system to the target file storage system. After the migration is completed, the original file storage system is switched to the target file storage system, and the cached file data obtained after the file storage gateway completes the switching is directly stored in the target file storage system. Therefore, it is only necessary to interrupt the access to the business system during the short period of time when the input and output interfaces of the business system are connected to the file storage gateway, and it is not necessary to interrupt the access to the business system during the long period of data migration, thereby solving the problem of the inaccessibility of the business system during the long period of file storage system switching in the prior art, greatly shortening the inaccessibility time of the business system, and realizing the online switching of the file storage system corresponding to the business system.

[0047] Embodiment 2

[0048] Figure 4This is a structural diagram of an online switching system of a file storage system provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment further refines the structure of the file storage gateway.

[0049] like Figure 4 As shown, the online switching system includes: a business system 10, a file storage gateway 20 and a file storage system 30, the file storage system 30 includes: an original file storage system 31 and a target file storage system 32; the file storage gateway 20 includes: a cache file subsystem 21 and a migration file subsystem 22; the cache file subsystem 21 is respectively connected to the business system 10 and the original file storage system 31, and the migration file subsystem 22 is respectively connected to the cache file subsystem 21, the original file storage system 31 and the target file storage system 32.

[0050] Specifically, the cache file subsystem 21 is used to receive and cache the file data sent by the business system 10 in real time to obtain the cache file data; and persist the cache file data in the original file storage system 31. The migration file subsystem 22 is used to migrate the file data to be migrated stored in the original file storage system to the target file storage system, and the file data to be migrated includes: cache file data and stock file data. The cache file subsystem 21 is also used to switch the original file storage system to the target file storage system after the migration of the file data to be migrated is completed, and persist the cache file data obtained after the switching is completed in the target file storage system.

[0051] It should be noted that the cache file subsystem 21 and the migration file subsystem 22 of the file storage gateway 20 can be executed in parallel. While the cache file subsystem 21 receives and caches the file data sent by the business system 10 to obtain the cache file data, and persists the cache file data to the original file storage system 31, the migration file subsystem 22 also migrates the data to be migrated stored in the original file storage system to the target file storage system.

[0052] An online switching system for a file storage system provided by an embodiment of the present invention connects a business system with a file storage gateway, and takes over the subsequent online switching and file data persistence of the file storage system through the file storage gateway. Specifically, the file data sent by the business system is cached through the cache file subsystem of the file storage gateway, and the cached file data is sent to the original file storage system; the file data stored in the original file storage system is migrated to the target file storage system through the migration file subsystem of the file storage gateway; after the migration is completed, the original file storage system is switched to the target file storage system through the cache file subsystem of the file storage gateway, and the cache file data obtained after the file storage gateway completes the switching is directly stored in the target file storage system. Therefore, it is only necessary to interrupt the access to the business system during the short time when the input and output interface of the business system is connected to the file storage gateway, and it is not necessary to interrupt the access to the business system during the long time of data migration, thereby solving the problem of the inaccessibility of the business system during the long period of file storage system switching in the prior art, greatly shortening the inaccessibility time of the business system, and realizing the online switching of the file storage system corresponding to the business system.

[0053] Based on the above embodiment, the cache file subsystem 21 includes: a cache module, a persistence module, a control module and a switching module;

[0054] The cache module is used to receive and cache the file data sent in real time by the business system to obtain first cache file data;

[0055] The control module is used to control the persistence module to open or close the file data reading function;

[0056] The persistence module is used to periodically read cache file data from the cache module when the file data reading function is turned on, and persist the cache file data to the original file storage system to obtain incremental file data before the original file storage system is switched to the target file storage system, and persist the cache file data to the target file storage system after the original file storage system is switched to the target file storage system; in the closed state, stop reading cache file data from the cache module, and continue to persist the read cache file data to the original file storage system or the target file storage system;

[0057] The switching module is used to switch the original file storage system to the target file storage system after the migration file subsystem completes the migration of the file data to be migrated stored in the original file storage system.

[0058] The cache module of the cache file subsystem may be a module integrated by one or more caches, and the cache module may cache the file data sent by the business system. The cache file subsystem may support adaptation of the original file storage system or the target file storage system.

[0059] Specifically, the cache module of the cache file subsystem receives the file data sent by the business system in real time, and caches the file data to obtain the cache file data. Since the cache module is limited by the cache space, the persistence module needs to read the cache file data from the cache module at regular intervals when the file data reading function is turned on, and send the cache file data to the file storage system to realize the persistence of the cache file data. The control module is used to control the persistence module to turn on or off the file data reading function.

[0060] Among them, when the file data reading function is turned on, the persistence module persists the cached file data to the original file storage system before the original file storage system is switched to the target file storage system, so that the migration file subsystem migrates the file data to be migrated in the original file storage system to the target file storage system; after the migration file subsystem migrates all the file data to be migrated in the original file storage system to the target file storage system, the switching module can switch the original file storage system to the target file storage system; after the original file storage system is switched to the target file storage system, the persistence module persists the cached file data to the target file storage system.

[0061] When the file data reading function of the persistence module is turned off, the persistence module stops reading cache file data from the cache module, but the cache file data that has been read will continue to be persisted to the original file storage system or the target file storage system.

[0062] Exemplarily, the condition under which the control module controls the persistence module to enable or disable the file data reading function may be determined according to the cache space of the cache module and the migration status of the migration file subsystem migrating the original file storage system to the target file storage system.

[0063] Optionally, the control module is specifically used for:

[0064] Acquire data volume information of unmigrated file data in the original file storage system in real time;

[0065] Determine whether the data volume information meets a preset threshold condition,

[0066] If so, controlling the persistence module to close the file data reading function;

[0067] If not, the persistence module is controlled to keep the file data reading function enabled.

[0068] Among them, all file data stored in the original file storage system are data to be migrated. According to whether the file data to be migrated is migrated to the target file storage system, the file data to be migrated can be divided into migrated file data and unmigrated file data, wherein the unmigrated file data refers to the file data to be migrated that is still stored in the original file storage system. The data volume information of the unmigrated file data may include the total number of unmigrated files and the total data volume of the unmigrated files.

[0069] The switching module can switch the original file storage system to the target file storage system only after the migration file subsystem has migrated all the file data to be migrated in the original file storage system to the target file storage system. In order for the migration file subsystem to migrate all the file data to be migrated in the original file storage system to the target file storage system, the persistence module needs to turn off the file data reading function so that the data in the original file storage system no longer changes. However, when the persistence module turns off the file data reading function, it stops reading cache file data from the cache module, while the business system will continue to generate file data and cache the file data in the cache module of the cache file subsystem. Therefore, when controlling the persistence module to turn off the file data reading function, it is necessary to consider whether the migration file subsystem can migrate all the unmigrated file data in the original file storage system to the target file storage system before the cache module runs out of space.

[0070] Specifically, in order to solve the above problem, a preset threshold condition is set. The preset threshold condition can be determined according to the working parameters of the cache file subsystem and the migration file subsystem, such as the historical maximum throughput of the cache file subsystem or the maximum migration rate of the migration file subsystem; it can also be determined according to an empirical value. In the process of the migration file subsystem migrating the file data to be migrated stored in the original file storage system to the target file storage system, the data volume information of the unmigrated file data in the original file storage system is obtained in real time to determine whether the data volume information meets the preset threshold condition. If the data volume information does not meet the preset threshold condition, it means that the migration file subsystem cannot migrate all the file data to be migrated stored in the original file storage system to the target file storage system before the cache module space is insufficient; the persistence module cannot be controlled to remain closed, and the persistence module needs to be controlled to keep the file data reading function turned on so that the persistence module can continue to read the cache file data from the cache module and persist the cache file data to the original file storage system. Until the data volume information meets the preset threshold condition, the persistence module is controlled to close the file data reading function so that the persistence module stops reading the cache file data from the cache module and only persists the cache file data that has been read to the original file system.

[0071] Optionally, the data volume information includes: the total number of unmigrated files and the total data volume of unmigrated files; the preset threshold condition includes: the total number of unmigrated files is less than the number threshold and the total data volume of unmigrated files is less than the data volume threshold. The preset threshold condition is determined according to the throughput of the cache file subsystem and the migration rate of the migration file subsystem.

[0072] Specifically, in the process of the migration file subsystem migrating the file data to be migrated, the migration progress is directly related to the total number of unmigrated files and the size of each unmigrated file. Therefore, the quantity threshold and the data volume threshold are determined according to the throughput of the cache file subsystem and the migration rate of the migration file subsystem. The preset threshold conditions that the data volume information of the unmigrated file data needs to meet include: the total number of unmigrated files is less than the quantity threshold and the total data volume of the unmigrated files is less than the data volume threshold.

[0073] Based on any of the above embodiments, the cache file subsystem 21 further includes: a first judgment module and a second judgment module;

[0074] The first judgment module is used to judge whether the persistence module of the cache file subsystem persists all cache file data read from the cache module to the original file storage system within a preset time when the file data reading function of the persistence module is turned off;

[0075] The second judgment module is used to judge whether the file migration subsystem has migrated all the file data to be migrated in the original file storage system to the target file storage system within a preset time if the output result of the first judgment module is yes;

[0076] The control module of the cache file subsystem is also used to: if the output result of the first judgment module or the output result of the second judgment module is no, control the persistence module to be reopened, so that the persistence module reads the cache file data from the cache module cache and persists the cache file data to the original file storage system.

[0077] The preset time may be the time when the file data generated by the business system fills up the cache space of the cache module, and thus the preset time may be determined according to the throughput of the cache sub-file system.

[0078] Specifically, after the persistence module turns off the file data reading function, the persistence module no longer reads cache file data from the cache module, but the persistence module still needs to persist the cache file data that has been read to the original file storage system, and the cache module also needs to receive and cache the file data generated by the business system at this time. Therefore, the cache file subsystem determines through the first judgment module whether the persistence module persists all the cache file data read from the cache module to the original file storage system within the preset time. If so, it further determines through the second judgment module whether the migration file subsystem migrates all the file data to be migrated in the original file storage system to the target file storage system within the preset time. If so, the switching module can switch the original file storage system to the target file storage system.

[0079] If the output result of the first judgment module or the output result of the second judgment module is no, that is, within the preset time, the persistence module of the cache file subsystem has not persisted all the cache file data read from the cache module to the original file storage system or the migration file subsystem has not migrated all the file data to be migrated in the original file storage system to the target file storage system, then the cache module will have no free space to cache the file data generated by the business system, and the control module is required to control the persistence module to be reopened so that the persistence module reads the cache file data from the cache module cache and persists the cache file data to the original file storage system to release the cache space of the cache module for caching the file data generated by the business system.

[0080] Based on any of the above embodiments, the cache file subsystem further includes: a notification module;

[0081] The notification module is used to send a notification message to the migration file subsystem after the persistence module persists the cache file data in the original file storage system to obtain the incremental file data, and the notification message includes data information of the incremental file data.

[0082] Specifically, after the persistence module of the cache file subsystem persists the cache file data to the original file storage system to obtain the incremental file data, the notification module of the cache file subsystem sends a notification message to the migration file subsystem, which may include: data information of the incremental file data, such as the storage location of the incremental file data in the original storage file and the incremental file index number, etc.

[0083] Optionally, the migration file subsystem is specifically used for:

[0084] Scanning the stock file data stored in the original file storage system;

[0085] Acquire the incremental file data from the original file storage system according to the notification message;

[0086] The sum of the existing file data and the incremental file data is used as the file data to be migrated;

[0087] Migrate the file data to be migrated to the target file storage system.

[0088] Among them, the existing file data refers to the data directly received from the business system and stored in the original file storage system; the incremental file data refers to the data stored in the cache module in the original file storage system.

[0089] Specifically, the specific process of the migration file subsystem migrating the file data to be migrated stored in the original file storage system to the target file storage system can be: the migration file subsystem scans the historical file data stored in the original file storage system to obtain the existing file data; obtains the incremental file data from the original file storage system according to the file index number and / or storage location contained in the notification message, takes the sum of the existing file data and the incremental file data as the file data to be migrated, and migrates the file data to be migrated to the target file storage system.

[0090] Exemplarily, the method of migrating the file data to be migrated to the target file storage system may be performed in parallel or in series, and the embodiment of the present invention does not impose any limitation on this.

[0091] Embodiment 3

[0092] Figure 5 This is a flowchart of an online switching method for a file storage system provided in Embodiment 3 of the present invention. This embodiment is applicable to the case where an original file storage system connected to a business system is switched to a target file storage system. The method can be executed by a file storage gateway in an online switching system of the file storage system. Figure 5 As shown, the method includes:

[0093] S310, receiving and caching file data sent by the business system in real time to obtain cached file data; wherein the business system is used to receive a file storage system switching instruction, switch the access address of the business system to the address of the file storage gateway based on the file storage system switching instruction, and send file data to the file storage gateway in real time.

[0094] Specifically, when the business system is in a state of waiting to switch the file storage system, the input and output interfaces of the business system are connected to the original file storage system, that is, the access address of the business system is the address of the original file storage system. After receiving the file storage system switching instruction, the business system switches the access address of the business system to the address of the file storage gateway according to the address of the file storage gateway contained in the file storage system switching instruction, so that the file storage gateway is connected to the business system. After the file storage gateway is connected to the business system, it receives the file data sent by the business system in real time, and caches the received file data to obtain cached file data.

[0095] S320, persisting the cached file data in the original file storage system, and migrating the file data to be migrated stored in the original file storage system to the target file storage system; the file data to be migrated includes: cached file data and stock file data.

[0096] The file data to be migrated includes: cached file data, and may also include stock file data directly received and stored from the business system when the business system is in a state of a file storage system to be switched.

[0097] Specifically, the file storage gateway can periodically persist cached file data to the original file storage system, and migrate the to-be-migrated file data stored in the original file storage system to the target file storage system.

[0098] It should be noted that persisting cached file data to the original file storage system can be performed in parallel with migrating the file data to be migrated stored in the original file storage system to the target file storage system. That is, during the process of migrating the data to be migrated from the original file storage system to the target file storage system, the file data generated by the business system can be temporarily cached by the file storage gateway. Therefore, data migration can be achieved without interrupting access to the business system, ensuring the consistency of data migration.

[0099] S330: After the migration of the file data to be migrated is completed, the original file storage system is switched to the target file storage system, and the cached file data obtained after the switching is completed is persisted in the target file storage system.

[0100] Specifically, after all the file data to be migrated stored in the original file storage system are migrated to the target file storage system, the file storage gateway switches the file storage system storing the cached file data from the original file storage system to the target file storage system, and persists the cached file data obtained after the switch to the target file storage system, thereby connecting the business system with the target file storage system.

[0101] The technical solution of the embodiment of the present invention is to switch the file storage gateway in the online switching system of the file storage system, wherein the business system is used to receive the file storage system switching instruction, switch the access address of the business system to the address of the file storage gateway based on the file storage system switching instruction, and send file data to the file storage gateway in real time; persist the cached file data in the original file storage system, and migrate the file data to be migrated stored in the original file storage system to the target file storage system; the file data to be migrated includes: cached file data stock file data; after the migration of the file data to be migrated is completed, the original file storage system is switched to the target file storage system, and the cached file data obtained after the switching is completed is persisted in the target file storage system, which can solve the problem of the inaccessibility of the business system during the long period of file data migration in the prior art, and only needs to suspend the access of the business system for a short period of several seconds when the business system IO is connected to the NAS gateway system, so as to realize the online switching of the file storage system corresponding to the business system, and shorten the inaccessibility time of the business system as much as possible.

[0102] Embodiment 4

[0103] Figure 6 This is a flowchart of an online switching method for a file storage system provided in Embodiment 4 of the present invention. This embodiment is applicable to the case where an original file storage system connected to a business system is switched to a target file storage system. The method can be executed by a business system in an online switching system of a file storage system. Figure 6 As shown, the method includes:

[0104] S410: Receive a file storage system switching instruction.

[0105] Specifically, the business system receives a file storage system switching instruction, which may include an address of a file storage gateway so that the business system can be connected to the file storage gateway.

[0106] S420: Switch the access address of the business system to the file storage gateway address based on the file storage system switching instruction.

[0107] Specifically, according to the address of the file storage gateway included in the file storage system switching instruction, the access address of the business system is switched to the address of the file storage gateway, so that the file storage gateway is connected to the business system.

[0108] S430. Send file data to the file storage gateway in real time, so that the file storage gateway receives and caches the file data sent by the business system in real time to obtain cached file data; persist the cached file data in the original file storage system, and migrate the file data to be migrated stored in the original file storage system to the target file storage system, where the file data to be migrated includes: the existing file data of the cached file data; after the migration of the file data to be migrated is completed, switch the original file storage system to the target file storage system, and persist the cached file data obtained after the switching is completed in the target file storage system.

[0109] Specifically, after the business system is connected to the business system, the file data is sent to the file storage gateway in real time, and the file storage gateway takes over the subsequent online switching of the file storage system and the persistence of file data. Therefore, it is only necessary to interrupt the access to the business system during the short period of time when the input and output interface of the business system is connected to the file storage gateway, and there is no need to interrupt the access to the business system during the long period of data migration.

[0110] The technical solution of the embodiment of the present invention is to receive a file storage system switching instruction through a business system in an online switching system of the file storage system; switch the access address of the business system to a file storage gateway address based on the file storage system switching instruction; send file data to the file storage gateway in real time, so that the file storage gateway takes over the subsequent online switching of the file storage system and the persistence of file data, and only needs to interrupt the access to the business system during the short time when the input and output interfaces of the business system are connected to the file storage gateway, and there is no need to interrupt the access to the business system during the long time of data migration, thereby solving the problem of inaccessibility of the business system during the long period of file storage system switching in the prior art, greatly shortening the inaccessibility time of the business system, and realizing online switching of the file storage system corresponding to the business system.

[0111] Embodiment 5

[0112] Figure 7 A schematic diagram of an electronic device 50 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, gateway devices, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0113] like Figure 7As shown, the electronic device 50 includes at least one processor 51, and a memory connected to the at least one processor 51 in communication, such as a read-only memory (ROM) 52, a random access memory (RAM) 53, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 51 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 52 or the computer program loaded from the storage unit 58 to the random access memory (RAM) 53. In the RAM 53, various programs and data required for the operation of the electronic device 50 can also be stored. The processor 51, the ROM 52, and the RAM 53 are connected to each other via a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0114] A number of components in the electronic device 50 are connected to the I / O interface 55, including: an input unit 56, such as a keyboard, a mouse, etc.; an output unit 57, such as various types of displays, speakers, etc.; a storage unit 58, such as a disk, an optical disk, etc.; and a communication unit 59, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 59 allows the electronic device 50 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0115] The processor 51 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 51 executes the various methods and processes described above, such as an online switching method for a file storage system.

[0116] In some embodiments, the online switching method of the file storage system can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 58. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 50 via the ROM 52 and / or the communication unit 59. When the computer program is loaded into the RAM 53 and executed by the processor 51, one or more steps of the online switching method of the file storage system described above can be performed. Alternatively, in other embodiments, the processor 51 can be configured to execute the online switching method of the file storage system by any other appropriate means (e.g., by means of firmware).

[0117] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0118] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0119] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0120] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0121] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0122] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0123] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0124] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An online switching system for a file storage system, characterized in that: The online switching system includes: a business system, a file storage gateway and a file storage system, wherein the file storage system includes: an original file storage system and a target file storage system; the file storage gateway is connected to the original file storage system and the target file storage system respectively; The business system is used to receive a file storage system switching instruction, switch the access address of the business system to the address of the file storage gateway based on the file storage system switching instruction, and send file data to the file storage gateway in real time; The file storage gateway is used to receive and cache file data sent by the business system in real time to obtain cache file data; persist the cache file data in the original file storage system, and migrate the file data to be migrated stored in the original file storage system to the target file storage system, wherein the file data to be migrated includes: the cache file data and the stock file data; after the migration of the file data to be migrated is completed, the original file storage system is switched to the target file storage system, and the cache file data obtained after the switching is completed is persisted in the target file storage system; The file storage gateway includes: a cache file subsystem and a migration file subsystem; the cache file subsystem is connected to the business system and the original file storage system respectively, and the migration file subsystem is connected to the cache file subsystem, the original file storage system and the target file storage system respectively; The cache file subsystem includes: a cache module, a control module, a persistence module and a switching module; Wherein, the cache module is used to receive and cache the file data sent by the business system in real time to obtain first cache file data; The control module is used to control the persistence module to turn on or off the file data reading function; The persistence module is used to periodically read the cache file data from the cache module when the file data reading function is turned on, and before the original file storage system is switched to the target file storage system, persist the cache file data to the original file storage system to obtain incremental file data, and after the original file storage system is switched to the target file storage system, persist the cache file data to the target file storage system; in the turned-off state, stop reading the cache file data from the cache module, and continue to persist the read cache file data to the original file storage system or the target file storage system; The switching module is used to switch the original file storage system to the target file storage system after the migration file subsystem completes the migration of the to-be-migrated file data stored in the original file storage system.

2. The online switching system according to claim 1, characterized in that: The control module is specifically used for: Acquire data volume information of unmigrated file data in the original file storage system in real time; Determining whether the data volume information meets a preset threshold condition; If so, controlling the persistence module to close the file data reading function; If not, the persistence module is controlled to keep the file data reading function enabled.

3. The online switching system according to claim 2, characterized in that: The data volume information includes: the total number of unmigrated files and the total data volume of unmigrated files; Correspondingly, the preset threshold condition includes: the total number of the unmigrated files is less than the number threshold and the total data volume of the unmigrated files is less than the data volume threshold.

4. The online switching system according to claim 3, characterized in that: The preset threshold condition is determined according to the throughput of the cache file subsystem and the migration rate of the migration file subsystem.

5. The online switching system according to claim 2, characterized in that: The cache file subsystem further includes: a first judgment module and a second judgment module; The first judgment module is used to judge whether the persistence module of the cache file subsystem persists all cache file data read from the cache module to the original file storage system within a preset time when the persistence module is in a state where the file data reading function is turned off; The second judgment module is used to judge whether the migration file subsystem has migrated all the to-be-migrated file data in the original file storage system to the target file storage system within the preset time if the output result of the first judgment module is yes.

6. The online switching system according to claim 5, characterized in that: The control module is also used for: If the output result of the first judgment module or the output result of the second judgment module is no, the persistence module is controlled to be reopened so that the persistence module reads the cache file data from the cache module cache and persists the cache file data to the original file storage system.

7. The online switching system according to claim 1, characterized in that: The cache file subsystem also includes: a notification module; The notification module is used to send a notification message to the migration file subsystem after the persistence module persists the cache file data in the original file storage system to obtain the incremental file data, and the notification message includes data information of the incremental file data.

8. The online switching system according to claim 7, characterized in that: The migration file subsystem is specifically used for: Scanning the stock file data stored in the original file storage system; Acquire the incremental file data from the original file storage system according to the notification message; The sum of the existing file data and the incremental file data is used as the file data to be migrated; Migrate the file data to be migrated to the target file storage system.

9. The online switching system according to any one of claims 1 to 8, characterized in that: The target file storage system includes: an unstructured data management system.

10. An online switching method for a file storage system, characterized in that: A file storage gateway applied to an online switching system of a file storage system according to any one of claims 1 to 9, the method comprising: Receiving and caching file data sent in real time by the business system to obtain cached file data; wherein the business system is used to receive a file storage system switching instruction, switch the access address of the business system to the address of the file storage gateway based on the file storage system switching instruction, and send file data to the file storage gateway in real time; Persisting the cached file data in the original file storage system, and migrating the file data to be migrated stored in the original file storage system to the target file storage system; After the migration of the file data to be migrated is completed, the original file storage system is switched to the target file storage system, and the cached file data obtained after the switching is completed is persisted in the target file storage system; The file storage gateway includes: a cache file subsystem and a migration file subsystem; the cache file subsystem is connected to the business system and the original file storage system respectively, and the migration file subsystem is connected to the cache file subsystem, the original file storage system and the target file storage system respectively; The cache file subsystem includes: a cache module, a control module, a persistence module and a switching module; Wherein, the cache module is used to receive and cache the file data sent by the business system in real time to obtain first cache file data; The control module is used to control the persistence module to turn on or off the file data reading function; The persistence module is used to periodically read the cache file data from the cache module when the file data reading function is turned on, and before the original file storage system is switched to the target file storage system, persist the cache file data to the original file storage system to obtain incremental file data, and after the original file storage system is switched to the target file storage system, persist the cache file data to the target file storage system; in the turned-off state, stop reading the cache file data from the cache module, and continue to persist the read cache file data to the original file storage system or the target file storage system; The switching module is used to switch the original file storage system to the target file storage system after the migration file subsystem completes the migration of the to-be-migrated file data stored in the original file storage system.

11. An online switching method for a file storage system, characterized in that: A method applied to a business system in an online switching system of a file storage system according to any one of claims 1 to 9, the method comprising: Receiving a file storage system switching instruction; Switching the access address of the business system to a file storage gateway address based on the file storage system switching instruction; Sending file data to the file storage gateway in real time; The file storage gateway is used to receive and cache file data sent by the business system in real time to obtain cache file data; persist the cache file data in the original file storage system, and migrate the file data to be migrated stored in the original file storage system to the target file storage system, wherein the file data to be migrated includes: the cache file data and the stock file data; after the migration of the file data to be migrated is completed, the original file storage system is switched to the target file storage system, and the cache file data obtained after the switching is completed is persisted in the target file storage system; The cache file subsystem includes: a cache module, a control module, a persistence module and a switching module; Wherein, the cache module is used to receive and cache the file data sent by the business system in real time to obtain first cache file data; The control module is used to control the persistence module to turn on or off the file data reading function; The persistence module is used to periodically read the cache file data from the cache module when the file data reading function is turned on, and before the original file storage system is switched to the target file storage system, persist the cache file data to the original file storage system to obtain incremental file data, and after the original file storage system is switched to the target file storage system, persist the cache file data to the target file storage system; in the turned-off state, stop reading the cache file data from the cache module, and continue to persist the read cache file data to the original file storage system or the target file storage system; The switching module is used to switch the original file storage system to the target file storage system after the migration of the to-be-migrated file data stored in the original file storage system of the migration file subsystem is completed.

12. A computer device, characterized in that: The computer device is configured with a business system, and the computer device includes: at least one processor; and A memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the online switching method of the file storage system described in any one of claims 10-11.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the online switching method of the file storage system described in any one of claims 10-11 when executed.

Images