File storage method and apparatus

Abstract

This disclosure provides a file storage method and apparatus, relating to the field of computer storage technology, and particularly to the fields of distributed storage, cloud storage, and hybrid disk collaborative storage technology. One specific embodiment of the method includes: receiving a file storage request sent by a user, the file storage request including a file to be stored; in response to determining that the length of the file to be stored is not greater than a file partitioning threshold, storing the file to be stored in a first type of storage medium; in response to determining that the length of the file to be stored is greater than the file partitioning threshold, splitting the file to be stored into a first sub-file and a second sub-file, storing the first sub-file in the first type of storage medium, and storing the second sub-file in a second type of storage medium, wherein the length of the first sub-file is less than the length of the second sub-file and less than the file partitioning threshold, and the read / write performance of the first type of storage medium is higher than that of the second type of storage medium.

Description

Technical Field

[0001] This disclosure relates to the field of computer storage technology, and in particular to the fields of distributed storage, cloud storage, and hybrid disk collaborative storage technology. Background Technology

[0002] With the iterative upgrades of hard drive hardware technology, the cost-effectiveness of solid-state drives (SSDs) has significantly improved. Distributed object storage customers are increasingly inclined to configure a small number (e.g., 1-2) of SSDs in servers equipped with large-capacity mechanical hard drives to accelerate metadata access performance, while massive amounts of data are still stored on mechanical hard drives, which have a greater advantage in price and capacity, forming a hybrid disk storage architecture.

[0003] Among the currently available hybrid disk solutions, one approach is to use high-performance disks for block storage and mechanical hard drives for object storage, with the object storage serving as a cold pool for block storage and providing block storage services to the outside world. Another approach is to evaluate the priority of objects and hard drives and store objects of different priorities on the corresponding types of hard drives. This approach is mainly aimed at database objects. Summary of the Invention

[0004] This disclosure provides a file storage method, apparatus, device, storage medium, and program product.

[0005] In a first aspect, embodiments of this disclosure propose a file storage method, comprising: receiving a file storage request sent by a user, the file storage request including a file to be stored; in response to determining that the length of the file to be stored is not greater than a file partitioning threshold, storing the file to be stored in a first type of storage medium; in response to determining that the length of the file to be stored is greater than the file partitioning threshold, splitting the file to be stored into a first sub-file and a second sub-file, storing the first sub-file in the first type of storage medium, storing the second sub-file in a second type of storage medium, wherein the length of the first sub-file is less than the length of the second sub-file and less than the file partitioning threshold, and the read / write performance of the first type of storage medium is higher than that of the second type of storage medium.

[0006] Secondly, embodiments of this disclosure provide a file storage device, comprising: a request receiving module configured to receive a file storage request sent by a user, the file storage request including a file to be stored; a file storage module configured to store the file to be stored in a first type of storage medium in response to determining that the length of the file to be stored is not greater than a file partitioning threshold; and a file splitting module configured to split the file to be stored into a first sub-file and a second sub-file in response to determining that the length of the file to be stored is greater than the file partitioning threshold, store the first sub-file in the first type of storage medium, and store the second sub-file in a second type of storage medium, wherein the length of the first sub-file is less than the length of the second sub-file and less than the file partitioning threshold, and the read / write performance of the first type of storage medium is higher than that of the second type of storage medium.

[0007] Thirdly, embodiments of this disclosure provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0008] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the methods described in the first aspect.

[0009] Fifthly, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0010] The key or essential features of the embodiments disclosed herein are not intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. Wherein: Figure 1 This is a flowchart of an embodiment of the file storage method according to the present disclosure; Figure 2 This is a flowchart of yet another embodiment of the file storage method according to the present disclosure; Figure 3 This is a flowchart illustrating the hierarchical splitting logic of object storage files in a hybrid disk model. Figure 4 This is a schematic diagram of a structure of a file storage device according to an embodiment of the present disclosure; Figure 5 This is a block diagram of an electronic device used to implement the file storage method of the embodiments of this disclosure. Detailed Implementation

[0012] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0013] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0014] Figure 1 A flow 100 of an embodiment of a file storage method according to the present disclosure is shown. The file storage method includes the following steps: Step 101: Receive the file storage request sent by the user.

[0015] In this embodiment, the entity executing the file storage method can receive file storage requests sent by the user.

[0016] File storage requests can include files to be stored. Typically, the type of file to be stored is not restricted; whether it's a log file, video file, archived data file, or other type of file, it is stored in binary format, without distinguishing file type during storage.

[0017] Step 102: Determine whether the length of the file to be stored is greater than the file partition threshold.

[0018] In this embodiment, the execution entity can determine whether the length of the file to be stored is greater than the file partition threshold (small_size). If the length of the file to be stored is not greater than the file partition threshold, step 103 is executed; if the length of the file to be stored is greater than the file partition threshold, step 104 is executed. The file partition threshold is determined based on the hardware conditions of the current storage cluster; different hardware conditions in storage clusters may correspond to different file partition thresholds.

[0019] Step 103: Store the file to be stored in the first type of storage medium.

[0020] In this embodiment, if the length of the file to be stored is not greater than the file partition threshold, it indicates that the file is a small file, and the aforementioned execution entity can store the file to be stored in the first type of storage medium. The first type of storage medium is a solid-state drive (SSD) with higher read / write performance. Utilizing the high QPS (Queries Per Second) and IO (Input / Output) capabilities of the SSD improves the read / write response speed of small files, enhancing the user experience. Simultaneously, storing small files in the SSD fully utilizes its storage capacity, avoiding the resource waste caused by storing only metadata.

[0021] Step 104: Split the file to be stored into a first sub-file and a second sub-file.

[0022] In this embodiment, if the length of the file to be stored is greater than the file partitioning threshold, it indicates that the file is a large file. The aforementioned execution entity can then split the file to be stored into a first sub-file and a second sub-file. The length of the first sub-file is less than the length of the second sub-file and less than the file partitioning threshold, ensuring that the first sub-file can still meet the storage optimization requirements of the solid-state drive.

[0023] Step 105: Store the first sub-file to the first type of storage medium and store the second sub-file to the second type of storage medium.

[0024] In this embodiment, the aforementioned executing entity can store the first sub-file on a first type of storage medium and the second sub-file on a second type of storage medium. The first type of storage medium has higher read / write performance than the second type. Typically, the first type of storage medium is a solid-state drive (SSD) with higher read / write performance. The second type of storage medium is a hard disk drive (HDD) with a greater advantage in price and capacity. Its massive storage capacity meets the storage needs of large files, while the sheer number of HDDs provides high throughput for large files, achieving a complementary advantage between the high I / O performance of the SSD and the high storage capacity of the HDD.

[0025] In some embodiments, to further optimize the space utilization of the solid-state drive and avoid a large number of extremely small files occupying too much storage space, the aforementioned execution entity may, if it is determined that there are multiple target files (i.e., extremely small files) in the first type of storage medium with a length not exceeding a preset length threshold, merge the multiple target files into a merged file; generate an index file for the merged file, the index file recording the offset and length of each target file in the merged file. When reading a target file subsequently, the position of the file in the merged file can be located first through the index file before the reading operation is performed.

[0026] The file storage method provided in this disclosure divides files into large and small files based on a fixed threshold determined by hardware conditions, and stores small files and some sub-files of large files on solid-state drives (SSDs), while storing the main body of large files on mechanical hard drives (HDDs). This method fully utilizes the performance and capacity resources of SSDs and leverages the massive storage advantages of HDDs, effectively solving the problems of resource waste and small file performance bottlenecks in hybrid drive systems.

[0027] Figure 2 A flow 200 of another embodiment of the file storage method according to this disclosure is shown. The file storage method includes the following steps: Step 201: Receive the file storage request sent by the user.

[0028] In this embodiment, the entity executing the file storage method can receive file storage requests sent by the user.

[0029] File storage requests can include the file to be stored and its metadata. Metadata refers to structured data describing the core attributes of the file to be stored, not the file's content data itself. It is mainly used for file management, indexing, searching, and access control. Common examples include file name, file size, creation time, modification time, storage path, access permissions, and data distribution information. Typically, the type of file to be stored is not restricted; whether it is a log file, video file, archived data file, or other type of file, it is stored in binary form, without distinguishing between file types during storage.

[0030] Step 202: Store the metadata of the file to be stored in the first type of storage medium.

[0031] In this embodiment, the aforementioned execution entity can store the metadata of the file to be stored in a first type of storage medium (solid-state drive). The storage location of the metadata is fixed to the solid-state drive. With the high IO performance of the solid-state drive, it is ensured that when users traverse objects in the bucket, modify or view object metadata, they can quickly obtain relevant information, thereby improving operation response efficiency.

[0032] Step 203: Determine whether the length of the file to be stored is greater than the file partition threshold.

[0033] In this embodiment, the execution entity can determine whether the length of the file to be stored is greater than the file partition threshold (small_size). If the length of the file to be stored is not greater than the file partition threshold, step 204 is executed; if the length of the file to be stored is greater than the file partition threshold, step 205 is executed. The file partition threshold is determined based on the hardware conditions of the current storage cluster; different hardware conditions in storage clusters may correspond to different file partition thresholds.

[0034] Step 204: Store the file to be stored in the first type of storage medium.

[0035] In this embodiment, if the length of the file to be stored is not greater than the file partition threshold, it indicates that the file is a small file, and the aforementioned execution entity can store the file to be stored in the first type of storage medium. The first type of storage medium is a solid-state drive (SSD) with higher read / write performance. Utilizing the high QPS IO capability of the SSD improves the read / write response speed of small files, thus enhancing the user experience. Simultaneously, storing small files in the SSD fully utilizes its storage capacity, avoiding the resource waste caused by storing only metadata.

[0036] Step 205: Calculate the remainder of the length of the file to be stored relative to the file splitting baseline threshold.

[0037] In this embodiment, the execution entity can calculate the remainder of the length of the file to be stored divided by the file splitting baseline threshold (align_size). The file splitting baseline threshold, similar to the file partitioning boundary threshold, is determined based on the current storage cluster hardware conditions; different storage clusters with different hardware conditions may correspond to different file splitting baseline thresholds.

[0038] Step 206: Split the file to be stored based on the remainder, taking the part of the file corresponding to the remainder as the first sub-file and the remaining part of the file as the second sub-file.

[0039] In this embodiment, the execution entity can split the file to be stored based on the remainder, using the portion of the file corresponding to the remainder as the first sub-file (small_obj) and the remaining portion as the second sub-file (big_obj). If the length of the file to be stored is modulo 0 when divided by the file splitting baseline threshold, then the data portion of the file is not stored on the solid-state drive; only its metadata is stored.

[0040] Step 207: Store the first sub-file to the first type of storage medium and store the second sub-file to the second type of storage medium.

[0041] In this embodiment, the execution entity stores the first sub-file to a first type of storage medium (solid-state drive), leveraging the high IO performance of the solid-state drive to ensure fast access to the first sub-file; and stores the second sub-file to a second type of storage medium (hard disk drive), utilizing the massive storage capacity and high throughput of the hard disk drive to meet the storage and access requirements of the main body of the large file.

[0042] Step 208: In response to determining that the available space of the first type of storage medium is lower than a preset space threshold, some files in the first type of storage medium are migrated to the second type of storage medium.

[0043] In this embodiment, if the available space of the first type of storage medium is lower than a preset space threshold, the aforementioned execution entity can migrate some files in the first type of storage medium to the second type of storage medium.

[0044] In this embodiment, if the available space of the first type of storage medium (solid-state drive) is lower than the preset space threshold, it means that the solid-state drive is about to be filled. In order to avoid affecting the normal operation of the KV (Key-Value) storage system, the above-mentioned execution entity can migrate some files in the first type of storage medium to the second type of storage medium (hard disk drive).

[0045] In some embodiments, the file migration process may involve the following steps: First, based on the Least Recently Accessed algorithm, the first file to be migrated is selected from the first type of storage medium. This first file is considered "cold data," and the determination is based on its access frequency.

[0046] Then, the first file to be migrated is stored in the second type of storage medium.

[0047] Finally, in response to the confirmation that the first file to be migrated has been successfully stored in the second type of storage medium, the first file to be migrated in the first type of storage medium is deleted, and the corresponding storage space is released to ensure that the solid-state drive has sufficient space to store new small files and metadata.

[0048] In addition, users can set regular expression rules according to their actual needs, such as specifying files with specific suffixes or character strings as low-access-frequency files, and not loading them onto the solid-state drive in the future, thus further optimizing the data migration logic.

[0049] Step 209: In response to determining that there is a second file to be migrated in the second type of storage medium with an access frequency greater than a preset access frequency threshold and a length not greater than a file partitioning threshold, the second file to be migrated is migrated from the second type of storage medium to the first type of storage medium.

[0050] In this embodiment, if a second file to be migrated exists in the second type of storage medium with an access frequency greater than a preset access frequency threshold and a length not greater than a file partitioning threshold, it indicates that the file is a hot data small file. The aforementioned execution entity can migrate the second file to be migrated from the second type of storage medium to the first type of storage medium. The second file to be migrated is then migrated from the mechanical hard drive to the solid-state drive. Through the hot data loading logic, the high-frequency access small files benefit from the high IO performance of the solid-state drive to improve read speed and reduce subsequent access time; while large objects, because the read bandwidth provided by a large number of mechanical hard drives exceeds the bandwidth provided by a small number of solid-state drives, are always retained on the mechanical hard drive and are not hot-loaded and migrated.

[0051] The file storage method provided in this disclosure adds dedicated metadata storage, cold data flushing, and hot data loading mechanisms on the basis of hierarchical file storage. This not only solves the problems of low SSD capacity utilization and small file performance bottlenecks, but also avoids the situation where the SSD is filled up due to too many small files. At the same time, the stability and adaptability of the storage strategy are ensured by setting a fixed threshold, maximizing the storage advantages of hybrid disk models and providing users with high-throughput and low-latency object storage services.

[0052] Figure 3This document presents a flowchart illustrating the hierarchical splitting logic of object storage files on a hybrid disk system. It clearly demonstrates the complete judgment and processing flow for classifying and storing files. The process begins in step 301, writing the object, which involves receiving the file to be stored and initiating the storage processing flow. Then, in step 302, it determines whether the file length is less than the file partitioning threshold (small_size). If the result is yes, step 303 is executed, classifying the file as a small object with a length equal to the original file length, and storing it directly on the first type of storage medium (SSD). If the result is no, step 304 is executed, classifying the file as a large object and splitting it. The remainder of the file length divided by the file partitioning baseline threshold (align_size) is calculated, and the portion of the file corresponding to the remainder is designated as a small object (length = remainder). Step 305 then executes, classifying the remaining portion of the file as a large object (length = original file length - remainder). Finally, the small objects are stored on the SSD, and the large objects are stored on the HDD, ending the process. This flowchart visually presents the hierarchical splitting logic of this disclosure based on file length and fixed thresholds, clarifies the criteria for dividing large and small objects and their storage destinations, and reflects the core optimization idea of ​​data distribution under hybrid disk models.

[0053] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a file storage device, which is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0054] like Figure 4 As shown, the file storage device 400 of this embodiment may include: a request receiving module 401, a file storage module 402, and a file splitting module 403. The request receiving module 401 is configured to receive a file storage request sent by a user, the file storage request including a file to be stored; the file storage module 402 is configured to store the file to be stored in a first type of storage medium in response to determining that the length of the file to be stored is not greater than a file partitioning threshold; the file splitting module 403 is configured to split the file to be stored into a first sub-file and a second sub-file in response to determining that the length of the file to be stored is greater than the file partitioning threshold, storing the first sub-file in the first type of storage medium and the second sub-file in a second type of storage medium, wherein the length of the first sub-file is less than the length of the second sub-file and less than the file partitioning threshold, and the read / write performance of the first type of storage medium is higher than that of the second type of storage medium.

[0055] In this embodiment, the specific processing of the request receiving module 401, the file storage module 402, and the file splitting module 403 in the file storage device 400, and the resulting technical effects, can be found in reference to [reference needed]. Figure 1The relevant descriptions of steps 101-105 in the corresponding embodiments will not be repeated here.

[0056] In some optional implementations of this embodiment, the file splitting module 403 is further configured to: calculate the remainder of the length of the file to be stored divided by the file splitting baseline threshold; split the file to be stored based on the remainder, taking the file portion corresponding to the remainder as the first sub-file and the remaining file portion as the second sub-file.

[0057] In some optional implementations of this embodiment, the file storage device 400 further includes: a first migration module, configured to migrate a portion of the files in the first type of storage medium to a second type of storage medium in response to determining that the available space of the first type of storage medium is lower than a preset space threshold.

[0058] In some optional implementations of this embodiment, the first migration module is further configured to: select a first file to be migrated from a first type of storage medium based on the least recent access algorithm; store the first file to be migrated to a second type of storage medium; and, in response to determining that the first file to be migrated has been successfully stored in the second type of storage medium, delete the first file to be migrated from the first type of storage medium and release the corresponding storage space.

[0059] In some optional implementations of this embodiment, the file storage device 400 further includes: a second migration module, configured to migrate the second file to be migrated from the second type of storage medium to the first type of storage medium in response to determining that there is a second file to be migrated in the second type of storage medium with an access frequency greater than a preset access frequency threshold and a length not greater than a file division boundary threshold.

[0060] In some optional implementations of this embodiment, the file storage device 400 further includes: a file merging module configured to merge multiple target files into a merged file in response to determining that there are multiple target files in the first type of storage medium with a length not greater than a preset length threshold; and an index generation module configured to generate an index file for the merged file, wherein the index file records the offset and length of each target file in the merged file.

[0061] In some optional implementations of this embodiment, the file storage request may further include metadata of the file to be stored; and the file storage device 400 may further include: a metadata storage module configured to store the metadata of the file to be stored in a first type of storage medium.

[0062] The collection, storage, use, processing, transmission, provision, and disclosure of any type of information, such as user personal information, in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0063] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0064] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0065] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0066] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0067] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the file storage method. For example, in some embodiments, the file storage method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the file storage method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the file storage method by any other suitable means (e.g., by means of firmware).

[0068] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0069] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0070] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0071] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0072] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0073] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, distributed system servers, or servers incorporating blockchain technology.

[0074] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution provided in this disclosure can be achieved, and this is not limited herein.

[0075] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A file storage method, comprising: Receive a file storage request sent by a user, the file storage request including a file to be stored; In response to determining that the length of the file to be stored is not greater than the file partitioning threshold, the file to be stored is stored in the first type of storage medium; In response to determining that the length of the file to be stored is greater than the file partitioning threshold, the file to be stored is split into a first sub-file and a second sub-file, the first sub-file is stored in the first type of storage medium, and the second sub-file is stored in the second type of storage medium. The length of the first sub-file is less than the length of the second sub-file and less than the file partitioning threshold. The read / write performance of the first type of storage medium is higher than that of the second type of storage medium.

2. The method according to claim 1, wherein, The step of splitting the file to be stored into a first sub-file and a second sub-file includes: Calculate the remainder of the length of the file to be stored divided by the file splitting baseline threshold; The file to be stored is split based on the remainder, with the portion of the file corresponding to the remainder being used as the first sub-file and the remaining portion being used as the second sub-file.

3. The method according to claim 1, wherein, The method further includes: In response to determining that the available space of the first type of storage medium is lower than a preset space threshold, some files in the first type of storage medium are migrated to the second type of storage medium.

4. The method according to claim 3, wherein, The step of migrating a portion of files from the first type of storage medium to the second type of storage medium includes: Based on the Least Recently Accessed algorithm, the first file to be migrated is selected from the first type of storage medium; Store the first file to be migrated to the second type of storage medium; In response to determining that the first file to be migrated has been successfully stored in the second type of storage medium, the first file to be migrated in the first type of storage medium is deleted, and the corresponding storage space is released.

5. The method according to claim 1, wherein, The method further includes: In response to determining that there is a second file to be migrated in the second type of storage medium with an access frequency greater than a preset access frequency threshold and a length not greater than the file partition threshold, the second file to be migrated is migrated from the second type of storage medium to the first type of storage medium.

6. The method according to claim 1, wherein, The method further includes: In response to determining that there are multiple target files in the first type of storage medium with a length not greater than a preset length threshold, the multiple target files are merged into a merged file; An index file is generated for the merged file, which records the offset and length of each target file in the merged file.

7. The method according to claim 1, wherein, The file storage request also includes metadata of the file to be stored; as well as The method further includes: The metadata of the file to be stored is stored in the first type of storage medium.

8. A file storage device, comprising: The request receiving module is configured to receive file storage requests sent by users, the file storage requests including files to be stored. The file storage module is configured to store the file to be stored in a first type of storage medium in response to determining that the length of the file to be stored is not greater than a file partition threshold. The file splitting module is configured to split the file to be stored into a first sub-file and a second sub-file in response to determining that the length of the file to be stored is greater than the file splitting threshold. The first sub-file is stored in the first type of storage medium, and the second sub-file is stored in the second type of storage medium. The length of the first sub-file is less than the length of the second sub-file and less than the file splitting threshold. The read and write performance of the first type of storage medium is higher than that of the second type of storage medium.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-7.

11. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images