Method for file storage clustering and related apparatus

Abstract

The embodiment of the application discloses a file storage clustering method and a related device. The method comprises the following steps: inputting file storage information and routing rules of a cluster on a configuration page; deploying a system instance of the cluster; reading storage configuration of the cluster; initializing storage, so that an access path is consistent with the storage configuration; generating a file routing rule engine based on the storage configuration; and controlling the file routing rule engine to generate a saving path of a saved file, and saving the file according to the saving path. The method can greatly improve the efficiency of multi-file storage.

Description

Technical Field

[0001] This application belongs to the field of information processing technology, and mainly relates to a method and related apparatus for file storage clustering. Background Technology

[0002] Currently, with the increasing demand for file storage, a large number of multi-file storage operations are required during the file storage process.

[0003] In existing technologies, when multiple files are stored, each file is stored individually, which results in low efficiency for multi-file storage. Summary of the Invention

[0004] One objective of this application is to provide a method and related apparatus for file storage clustering, which has the advantage of significantly improving the efficiency of multi-file storage.

[0005] To achieve the above objectives, in a first aspect, embodiments of this application provide a method for file storage clustering, including:

[0006] Enter the cluster's file storage information and routing rules on the configuration page;

[0007] Deploy system instances of the cluster;

[0008] Read the storage configuration of the cluster;

[0009] Initialize the storage to ensure that the access path matches the storage configuration;

[0010] Based on the storage configuration, a file routing rule engine is generated;

[0011] The file routing rule engine is controlled to generate a save path for the saved files, and the files are saved according to the save path.

[0012] It is understandable that by entering the cluster's file storage information and routing rules on the configuration page, deploying the system instance of the cluster, reading the cluster's storage configuration, initializing the storage, ensuring that the access path is consistent with the storage configuration, generating a file routing rule engine based on the storage configuration, controlling the file routing rule engine to generate the save path for saving the files, and saving the files according to the save path, the efficiency of multi-file storage can be improved.

[0013] In one possible example, entering the cluster's file storage information and routing rules on the configuration page includes:

[0014] The file storage information includes storage instance, storage type, and storage directory prefix;

[0015] The routing rules include routing policies, routing directory prefixes, routing algorithms, and corresponding algorithm effectiveness levels.

[0016] It is understood that the file storage information includes storage instance, storage type and storage directory prefix, and the routing rules include routing policy, routing directory prefix, routing algorithm and corresponding algorithm effectiveness level, which can optimize the configuration efficiency of file storage information and routing rules.

[0017] In one possible example, the process of controlling the file routing rule engine to generate a save path for the file and saving the file according to the save path includes the following steps:

[0018] The file routing rule engine is based on cluster routing configuration, which includes routing algorithm, routing directory prefix, cluster mount instance and path;

[0019] The cluster's separate configuration file stores the extended algorithm, file storage instances, and corresponding mount directories.

[0020] It is understood that the file routing rule engine is based on cluster routing configuration, which includes routing algorithm, routing directory prefix, cluster mount instance and path. The cluster separate configuration file stores the extended algorithm, file storage instance and corresponding mount directory, which can improve the efficiency of saving path generation.

[0021] In one possible example, the cluster's individual configuration file storage expansion algorithm, file storage instance, and corresponding mount directory include the following steps:

[0022] The file storage expansion algorithm operates on the prefix directory;

[0023] The file storage expansion algorithms include a default expansion algorithm, a load balancing algorithm, and a time-slicing algorithm.

[0024] It is understood that the file storage expansion algorithm operates on the prefix directory, and the file storage expansion algorithm includes a default expansion algorithm, a load balancing algorithm, and a time sharding algorithm, which improves the file processing efficiency of each cluster.

[0025] In one possible example, the file storage expansion algorithm, including the default expansion algorithm, load balancing algorithm, and time sharding algorithm, includes the following steps:

[0026] The default extension algorithm directly stores the mounted instances;

[0027] The load balancing algorithm performs file load balancing on the instance;

[0028] The time-slicing algorithm stores files according to the saved time nodes.

[0029] It is understood that the default extension algorithm directly stores the mounted instances, the load balancing algorithm performs file load balancing on the instances, and the time sharding algorithm stores files according to the time node of the save, which can improve the storage efficiency of various files.

[0030] In one possible example, the load balancing algorithm performs file load balancing on instances, including the following steps:

[0031] Read the cluster routing configuration;

[0032] Set the route type to cluster-wide load balancing;

[0033] Get the base path of the file;

[0034] Parse filenames and extended directories;

[0035] Combine the extended directory to generate the actual file directory.

[0036] It is understandable that reading the cluster routing configuration, setting the routing type to cluster-wide load balancing, obtaining the file base path, parsing the file name, parsing the extended directory, and combining the extended directory to generate the actual file directory can improve the efficiency of file load balancing.

[0037] In one possible example, the time-slicing algorithm stores files according to the saved time nodes, including the following steps:

[0038] Get file index information and file upload time;

[0039] Calculate the mount directory where the file is stored based on the file index information and the file upload time;

[0040] Based on the mounted directory, the actual storage path for the files is generated.

[0041] It is understandable that obtaining file index information and file upload time, calculating the mount directory where the file is stored based on the file index information and file upload time, and generating the actual storage path of the file based on the mount directory can improve the accuracy of file saving time.

[0042] In a second aspect, an apparatus for file storage clustering includes a module for performing the method provided in the first aspect or any embodiment of the first aspect.

[0043] Thirdly, a file storage clustering device includes a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, the programs including instructions for performing the methods provided in the first aspect or any embodiment of the first aspect.

[0044] Fourthly, a computer-readable storage medium storing a computer program that causes a computer to perform the method provided in the first aspect or any embodiment of the first aspect.

[0045] Implementing the embodiments of this application will have the following beneficial effects:

[0046] Enter the cluster's file storage information and routing rules on the configuration page; deploy the system instance of the cluster; read the cluster's storage configuration; initialize the storage to ensure the access path matches the storage configuration; generate a file routing rule engine based on the storage configuration; control the file routing rule engine to generate save paths for the saved files, and save the files according to the save paths, thereby significantly improving the efficiency of multi-file storage. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0048] Figure 1 This application provides an illustration of a file storage clustering application scenario.

[0049] Figure 2 A schematic diagram of a file system overall architecture provided for an embodiment of this application;

[0050] Figure 3 A flowchart of a file routing rule engine provided in an embodiment of this application;

[0051] Figure 4 This application provides a schematic diagram of a file storage clustering process.

[0052] Figure 5 A schematic diagram of a file storage clustering device provided in an embodiment of this application;

[0053] Figure 6 This is a structural diagram of a file storage clustering device provided in an embodiment of this application. Detailed Implementation

[0054] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0055] The terms “1” and “2”, etc., in this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0056] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0057] Please see Figure 1 , Figure 1 This diagram illustrates an application scenario for file storage clustering, as provided in an embodiment of this application. Figure 1 As shown in the diagram, this application scenario includes user 101, electronic device 102, and server 103. It should be noted that... Figure 1 The number of devices, the form of each device, and the number of users in the system shown are for illustrative purposes only and do not constitute a limitation on the embodiments of this application. A user may use multiple electronic devices.

[0058] User 101 is the user who actually operates electronic device 102 to control it to perform corresponding operations. Electronic device 102 can be... Figure 1The laptop shown can also be a personal computer (PC), all-in-one computer, handheld computer, tablet computer, smartphone, smart TV playback terminal, and portable device. PC-based electronic devices, such as all-in-one computers, can have operating systems including, but not limited to, Linux, Unix, and Windows series systems (such as Windows XP and Windows 7). Mobile electronic devices, such as smartphones, can have operating systems including, but not limited to, Android, iOS (Apple's operating system), and Windows.

[0059] The following describes a file storage clustering method provided in the embodiments of this application. This method can be executed by a file storage clustering device, which can be implemented by software and / or hardware, and is generally integrated into an electronic device or server.

[0060] Please see Figure 2 , Figure 2 This is a schematic diagram of the overall architecture of a file system provided in an embodiment of this application. The file system consists of a file system instance 201 and a file storage cluster 202. The file system instance 201 is the entry point for accessing the file storage cluster, generating file routing rules according to the configuration and calculating the physical paths in the file storage cluster. The file storage cluster 202 is the underlying physical storage of the file system, characterized in that it is composed of multiple independent storage instances mounted together 203. Each instance is integrated through routing rules to provide unified file services to the file system instance 201.

[0061] Please refer to Figure 3 , Figure 3 This is a flowchart illustrating a file routing rule engine provided in an embodiment of this application. The specific process is as follows: First, start 301; second, file access 302; third, file save path calculation 303; fourth, file operation 304; fifth, end 305. For example, the file routing rule engine is based on cluster routing configuration. Each cluster has a separate configuration file storing the extended algorithm, the file storage instance used, and the corresponding mount directory. When accessing a file, the actual file storage path (this is the file routing rule engine) must first be calculated according to the configuration before file operations are performed.

[0062] Please refer to Figure 4 , Figure 4 This is a schematic diagram illustrating a file storage clustering process provided in an embodiment of this application. Taking the application of this method to a file storage clustering process as an example, the file storage clustering device may include a server or electronic device. The method includes the following steps S401-S407, wherein...

[0063] S401: Enter the cluster's file storage information and routing rules on the configuration page.

[0064] In one possible example, step S401 includes steps A1-A2, wherein,

[0065] A1: The file storage information includes storage instance, storage type, and storage directory prefix.

[0066] For example, in file storage (also known as file-level storage or file-based storage), data is stored as individual records within folders. When this data needs to be accessed, the computer needs to know the appropriate search path. The data stored in the files is organized and retrieved based on a limited amount of metadata that tells the computer the exact location of the file; it's like a KUKA directory for data files.

[0067] For example, a storage instance can be implemented using different technologies. It can be an open-source distributed system, a server host, or an accessible disk. As long as it can be accessed through mounting, it can be used by the file system as part of the storage cluster.

[0068] A2: The routing rules include routing policies, routing directory prefixes, routing algorithms, and corresponding algorithm effectiveness levels.

[0069] It should be noted that the cluster includes service switch configuration and cluster routing configuration.

[0070] S402: The system instance where the cluster is deployed.

[0071] S403: Read the storage configuration of the cluster.

[0072] S404: Initialize the storage to make the access path consistent with the storage configuration.

[0073] S405: Based on the storage configuration, generate a file routing rule engine.

[0074] For example, business scenarios are typically complex and intertwined, with code prone to nesting, intricate structures, and unclear logic. This results in high maintenance costs and poor readability and extensibility. A rules engine integrates the set of rules passed to the system to derive conclusions. This can be understood as triggering (inferring) one or more business operations given a set of states (a finite state machine), or executing certain tasks under certain conditions.

[0075] S406: Control the file routing rule engine to generate a save path for the saved file, and save the file according to the save path.

[0076] In one possible example, the file routing rule engine is based on a cluster routing configuration, which includes a routing algorithm, a routing directory prefix, cluster mount instances, and paths.

[0077] For example, multiple storage types within an instance are configured and combined into a unified file storage cluster for system use by a file routing rule engine. Storage routing management settings include routing policies, extended directory prefixes, extended routing algorithms, and the extended routing algorithm's effectiveness level. For instance, the routing policy might be set to extended, the extended directory prefix to / app / 1, the extended routing algorithm to load balancing, and the extended routing algorithm's effectiveness level to the entire cluster. Storage mount management settings include the mount instance ID, instance model, mount directory, instance number, effectiveness method, and effectiveness date. For example, configure two storage mounts. The first mount has an instance ID of `mount01`, an instance type of `mountType01` (where `mountType` refers to the file system type, such as cloud file storage, cloud object storage, network storage, network file system, distributed file system, or custom), a mount directory of ` / app / 1 / 1`, an instance number of `1`, and an immediate activation method (no activation date needed). The second mount has an instance ID of `mount02`, an instance type of `mountType02`, a mount directory of ` / app / 1 / 2`, an instance number of `2`, and an immediate activation method (no activation date needed). After configuring these examples, once the file routing rules are initialized, files will be stored in the first and second instances based on their indexes, corresponding to the physical paths ` / app / 1 / 1` and ` / app / 1 / 2`.

[0078] In one possible example, the cluster's individual configuration file storage expansion algorithm, file storage instance, and corresponding mount directory include steps B11-B12, wherein:

[0079] B11: The file storage expansion algorithm applies to the prefix directory.

[0080] B12: The file storage expansion algorithm includes the default expansion algorithm, the load balancing algorithm, and the time sharding algorithm.

[0081] In one possible example, step B12 includes steps B121-B123, wherein,

[0082] B121: The default extension algorithm directly stores the mounted instances.

[0083] B122: The load balancing algorithm performs file load balancing on the instance.

[0084] For example, load balancing means distributing requests or data across multiple processing units to complete tasks together. Its purpose is to optimize resource utilization, maximize throughput, minimize response time, and avoid single-point overload by scheduling the cluster. Load balancing algorithms are divided into two categories: static load balancing and dynamic load balancing. For instance, the round-robin method in static load balancing distributes requests sequentially to each node, regardless of the actual number of connections or the current system load on each node. Its advantages are simplicity, efficiency, and easy horizontal scaling, with each node achieving a literal balance. Its disadvantage is that it doesn't consider machine performance; according to the "shortest plank in the barrel" theory, cluster performance bottlenecks are more often affected by poorly performing servers. The source address hashing method in static load balancing calculates a value based on the client's IP address using a hash function, modulo this value with the number of server nodes, and the result is the node number to be accessed. Using source address hashing for load balancing, clients with the same IP address will always access the same server when the backend server list remains unchanged. Its advantage is that the same IP always lands on the same node, allowing for manual intervention in client request direction, such as canary releases. Its disadvantages are: if a node fails, clients on that node will become unusable, compromising high availability. When a user becomes a hot user, a huge influx of traffic will flow to that node, leading to an uneven distribution of hot and cold traffic and hindering the effective utilization of the cluster's performance. Therefore, when hot events occur, the source address hashing method is generally switched to a round-robin method.

[0085] For example, the least connections method in dynamic load balancing algorithms dynamically selects the node with the fewest backlogged connections to handle the current request, maximizing the utilization efficiency of the backend service and rationally distributing requests to each server. Its advantage is its dynamic nature and ability to change in real-time based on node status. Its disadvantage is increased complexity, requiring counting each connection loss. The fastest response time method in dynamic load balancing algorithms dynamically adjusts the weight of each node based on the request's response time, allocating more requests to nodes with faster response times and fewer to nodes with slower response times. Its advantages are its dynamic nature, real-time changes, finer-grained control, and high sensitivity. Its disadvantage is higher complexity, requiring calculation of the request's response time each time.

[0086] B123: The time-slicing algorithm stores files according to the saved time nodes.

[0087] In one possible example, step B122 includes steps B1221-B1225, wherein,

[0088] B1221: Read the cluster routing configuration.

[0089] B1222: Set the routing type to load balancing at the entire cluster level.

[0090] B1223: Get the basic path of the file.

[0091] B1224: Parses filenames and extended directories.

[0092] B1225: Combine the extended directory to generate the actual file directory.

[0093] In one possible example, step B123 includes the following steps B1231-B1233, wherein,

[0094] B1231: Get file index information and file upload time.

[0095] For example, a file index refers to storing a file's information across several non-contiguous physical blocks. The system establishes a dedicated data structure for each file: an index table, and stores the block numbers of these blocks in the index table. The entries in the index table are stored contiguously, and each entry is of equal size. For instance, if the index number, length, and pointer fields each occupy 4 bytes, then each entry requires a length of 12 bytes. Therefore, the index table itself can be understood as a fixed-length sequential file. Fixed-length sequential files support random access, so the location of the index entry corresponding to the i-th record can be quickly found. Its advantages include retaining the advantages of linked structures, enabling both sequential and random access, meeting the needs of dynamic file growth, insertion, and deletion, and fully utilizing external storage space.

[0096] B1232: Calculate the mount directory where the file is stored based on the file index information and file upload time.

[0097] For example, mounting refers to connecting the top-level directory of a device file to a directory under the root directory (preferably an empty directory). Accessing this directory is equivalent to accessing the device file. Not any directory under the root directory can be used as a mount point. Because mounting hides files in the original directory, newly added devices, the root directory, and existing system directories should not be used as mount points, as this can cause system abnormalities or even crashes. The best mount point is a newly created empty directory.

[0098] B1233: Based on the mounted directory, generate the actual file storage path.

[0099] Please see Figure 5 , Figure 5This is a schematic diagram of a file storage clustering device provided in an embodiment of this application. Based on the above system architecture, the file storage clustering device 500 can be a server or a module within a server. The device 500 includes at least: an input module 501, a deployment module 502, a reading module 503, a processing module 504, and a control module 505.

[0100] The input module 501 is used to input the cluster's file storage information and routing rules on the configuration page;

[0101] Deployment module 502 is used to deploy system instances of the cluster;

[0102] The read module 503 is used to read the storage configuration of the cluster;

[0103] Processing module 504 is used to initialize the storage, ensuring that the access path matches the storage configuration; and to generate a file routing rule engine based on the storage configuration.

[0104] The control module 505 is used to control the file routing rule engine to generate a save path for the saved file, and to save the file according to the save path.

[0105] In one possible example, the processing module 504 makes the file storage information include storage instance, storage type and storage directory prefix; makes the routing rules include routing policy, routing directory prefix, routing algorithm and corresponding algorithm effectiveness level; and divides the cluster into service switch configuration and cluster routing configuration.

[0106] In one possible example, the file routing rule engine is based on cluster routing configuration, and the processing module 504 makes the cluster routing configuration include routing algorithm, routing directory prefix, cluster mount instance and path; and stores the cluster separate configuration file with extended algorithm, file storage instance and corresponding mount directory.

[0107] In one possible example, the processing module 504 applies the file storage expansion algorithm to the prefix directory, making the file storage expansion algorithm include a default expansion algorithm, a load balancing algorithm, and a time sharding algorithm.

[0108] In one possible example, the processing module 504 causes the default extension algorithm to directly store the mounted instance; causes the load balancing algorithm to perform file load balancing on the instance; and causes the time sharding algorithm to store the file according to the saved time node.

[0109] In one possible example, the processing module 504 reads the cluster routing configuration; sets the routing type to cluster-wide load balancing; obtains the file base path; parses the file name and the extended directory; and combines the extended directory to generate the file's actual directory.

[0110] In one possible example, the processing module 504 obtains file index information and file upload time; calculates the mount directory where the file is stored based on the file index information and file upload time; and generates the actual storage path of the file based on the mount directory.

[0111] Please see Figure 6 , Figure 6 This is a structural diagram of a file storage clustering device provided in an embodiment of this application. Figure 6 As shown, the file storage clustering device 600 includes a processor 601, a memory 602, a communication interface 604, and at least one program 603. The at least one program 603 is stored in the memory 602 and configured to be executed by the processor 601. The at least one program 603 includes instructions for performing the following steps:

[0112] Enter the cluster's file storage information and routing rules on the configuration page;

[0113] Deploy system instances of the cluster;

[0114] Read the storage configuration of the cluster;

[0115] Initialize the storage to ensure that the access path matches the storage configuration;

[0116] Based on the storage configuration, a file routing rule engine is generated;

[0117] The file routing rule engine is controlled to generate a save path for the saved files, and the files are saved according to the save path.

[0118] In one possible example, the at least one program 603 is specifically configured to perform the following instructions:

[0119] The file storage information includes storage instance, storage type, and storage directory prefix;

[0120] The routing rules include routing policies, routing directory prefixes, routing algorithms, and corresponding algorithm effectiveness levels.

[0121] In one possible example, the at least one program 603 is specifically configured to perform the following instructions:

[0122] The file routing rule engine is based on cluster routing configuration, which includes routing algorithm, routing directory prefix, cluster mount instance and path;

[0123] The cluster's separate configuration file stores the extended algorithm, file storage instances, and corresponding mount directories.

[0124] In one possible example, the at least one program 603 is specifically configured to perform the following instructions:

[0125] The file storage expansion algorithm operates on the prefix directory;

[0126] The file storage expansion algorithms include a default expansion algorithm, a load balancing algorithm, and a time-slicing algorithm.

[0127] In one possible example, the at least one program 603 is specifically configured to perform the following instructions:

[0128] The default extension algorithm directly stores the mounted instances;

[0129] The load balancing algorithm performs file load balancing on the instance;

[0130] The time-slicing algorithm stores files according to the saved time nodes.

[0131] In one possible example, the at least one program 603 is specifically configured to perform the following instructions:

[0132] Read the cluster routing configuration;

[0133] Set the route type to cluster-wide load balancing;

[0134] Get the base path of the file;

[0135] Parse filenames and extended directories;

[0136] Combine the extended directory to generate the actual file directory.

[0137] In one possible example, the at least one program 603 is specifically configured to perform the following instructions:

[0138] Get file index information and file upload time;

[0139] Calculate the mount directory where the file is stored based on the file index information and the file upload time;

[0140] Based on the mounted directory, the actual storage path for the files is generated.

[0141] Those skilled in the art will understand that, for ease of explanation, Figure 6Only one memory 602 and processor 601 are shown in the illustration. In a real terminal or server, multiple processors and memories may exist. Memory can also be called storage medium or storage device, etc., and this application embodiment does not limit this.

[0142] It should be understood that in the embodiments of this application, the processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may also be a general-purpose microprocessor, graphics processing unit (GPU), or one or more integrated circuits to execute relevant programs to achieve the functions required by the embodiments of this application.

[0143] Processor 601 can also be an integrated circuit chip with signal processing capabilities. In implementation, each step of this application can be completed by the integrated logic circuitry in the hardware of processor 601 or by instructions in software form. The processor 601 described above can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 602. Processor 601 reads the information in memory 602 and, in conjunction with its hardware, completes the functions required by the units included in the methods, apparatus, and storage media of the embodiments of this application.

[0144] It should also be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). The memory can also be, but is not limited to, Compact Disc Read-Only Memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer. The memory can be independent and connected to the processor via a bus. The memory can also be integrated with the processor. The memory can store programs, and when the programs stored in the memory are executed by the processor, the processor is used to perform the various steps in the above embodiments of this application.

[0145] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) is integrated into the processor. It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0146] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0147] In implementation, each step of the above method can be completed by integrated logic circuits in the processor hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules in the processor. The software modules can reside in mature storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. Those skilled in the art will recognize that the various illustrative logical blocks (ILBs) and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0148] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer-programmed program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a processor, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber) or wireless (e.g., infrared, wireless, microwave, etc.) means, or from one website, computer, server, or data center to a mobile phone processor via a wired means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0149] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for clustering file storage, characterized in that, Includes the following steps: Enter the cluster's file storage information and routing rules on the configuration page; Deploy system instances of the cluster; Read the storage configuration of the cluster; Initialize the storage to ensure that the access path matches the storage configuration; Based on the storage configuration, a file routing rule engine is generated; The file routing rule engine is controlled to generate a save path for the saved files, and the files are saved according to the save path. The file routing rule engine is based on cluster routing configuration, which includes routing algorithm, routing directory prefix, cluster mount instance and path; The cluster's separate configuration file stores the extension algorithm, file storage instances, and corresponding mount directories; The file storage expansion algorithm operates on the prefix directory; The file storage expansion algorithm includes a first preset algorithm, a second preset algorithm, and a third preset algorithm. The first preset algorithm stores the mounted instances, the second preset algorithm performs file load balancing on the instances, and the third preset algorithm stores files according to the time node of the save.

2. The method according to claim 1, characterized in that, The process of entering cluster file storage information and routing rules on the configuration page includes: The file storage information includes storage instance, storage type, and storage directory prefix; The routing rules include routing policies, routing directory prefixes, routing algorithms, and corresponding algorithm effectiveness levels.

3. The method according to claim 1, characterized in that, The preset second algorithm performs file load balancing on the instance, including the following steps: Read the cluster routing configuration; Set the route type to cluster-wide load balancing; Get the base path of the file; Parse filenames and extended directories; Combine the extended directory to generate the actual file directory.

4. The method according to claim 1, characterized in that, The preset third algorithm stores files according to the saved time point, including the following steps: Get file index information and file upload time; Calculate the mount directory where the file is stored based on the file index information and the file upload time; Based on the mounted directory, the actual storage path for the files is generated.

5. A file storage clustering device, characterized in that, Used to perform the method as described in any one of claims 1-4.

6. A file storage clustering device, characterized in that, It includes a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, the programs including instructions for performing the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that causes a computer to execute in order to implement the method as described in any one of claims 1-4.

Images