Data query method and device, computer equipment and readable storage medium

Abstract

The invention relates to a data processing method and device, computer equipment, a storage medium and a computer program product. The method is applied to an index server of a file storage system and comprises the steps that a target file uploaded by object storage equipment in an object storage system based on a file uploading interface is received, and the file uploading interface conforms to object storage interface semantics of the object storage system; performing interface semantic conversion on the file uploading interface to obtain a file storage regulation and control interface; the file storage regulation and control interface conforms to the interface semantics of a portable operating system of the file storage system; creating index information for the target file based on the file storage regulation and control interface, storing the target file according to the index information, and hiding the index information in the storage process of the target file; when the index information disclosure condition of the target file is met, the index information is disclosed based on the file storage regulation and control interface. The method can enable the file storage system to be compatible with the data processing function of the object storage system.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data query method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] With the development of computer technology, data storage technology has emerged. Data storage technology can store massive amounts of data. For example, data storage technologies can store data through object storage systems or distributed file systems.

[0003] In traditional technologies, object storage systems or distributed file systems typically store data independently. However, because different users use different types of clients, distributed file systems can only read and write data with clients that support portable operating system interface semantics, and cannot read and write data with clients that support object storage interface semantics. Therefore, traditional distributed file systems suffer from poor compatibility. Summary of the Invention

[0004] Therefore, it is necessary to provide a data processing method, apparatus, computer device, computer-readable storage medium, and computer program product that is compatible with object storage functions to address the above-mentioned technical problems.

[0005] In a first aspect, this application provides a data processing method applied to an index server of a file storage system, the method comprising:

[0006] Receive a target file uploaded by an object storage device in the object storage system via a file upload interface, wherein the file upload interface conforms to the object storage interface semantics of the object storage system.

[0007] The file upload interface is semantically transformed to obtain a file storage control interface; the file storage control interface conforms to the interface semantics of the portable operating system of the file storage system.

[0008] An index information for the target file is created based on the file storage control interface, the target file is stored according to the index information, and the index information is hidden during the storage of the target file.

[0009] When the conditions for disclosing the index information of the target file are met, the index information is disclosed based on the file storage control interface.

[0010] Secondly, this application also provides a data processing apparatus for use as an index server in a file storage system, the apparatus comprising:

[0011] The receiving module is used to receive a target file uploaded by an object storage device in the object storage system based on a file upload interface, wherein the file upload interface conforms to the object storage interface semantics of the object storage system.

[0012] The conversion module is used to perform interface semantic conversion on the file upload interface to obtain a file storage control interface; the file storage control interface conforms to the portable operating system interface semantics of the file storage system.

[0013] A creation module is used to create index information for the target file based on the file storage control interface, store the target file according to the index information, and hide the index information during the storage process of the target file;

[0014] The public module is used to publicly disclose the index information based on the file storage control interface when the conditions for disclosing the index information of the target file are met.

[0015] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0016] Receive a target file uploaded by an object storage device in the object storage system via a file upload interface, wherein the file upload interface conforms to the object storage interface semantics of the object storage system.

[0017] The file upload interface is semantically transformed to obtain a file storage control interface; the file storage control interface conforms to the interface semantics of the portable operating system of the file storage system.

[0018] An index information for the target file is created based on the file storage control interface, the target file is stored according to the index information, and the index information is hidden during the storage of the target file.

[0019] When the conditions for disclosing the index information of the target file are met, the index information is disclosed based on the file storage control interface.

[0020] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0021] Receive a target file uploaded by an object storage device in the object storage system via a file upload interface, wherein the file upload interface conforms to the object storage interface semantics of the object storage system.

[0022] The file upload interface is semantically transformed to obtain a file storage control interface; the file storage control interface conforms to the interface semantics of the portable operating system of the file storage system.

[0023] An index information for the target file is created based on the file storage control interface, the target file is stored according to the index information, and the index information is hidden during the storage of the target file.

[0024] When the conditions for disclosing the index information of the target file are met, the index information is disclosed based on the file storage control interface.

[0025] Fifthly, this application also provides a computer program product. The computer program product includes a computer program, which, when executed by a processor, performs the following steps:

[0026] Receive a target file uploaded by an object storage device in the object storage system via a file upload interface, wherein the file upload interface conforms to the object storage interface semantics of the object storage system.

[0027] The file upload interface is semantically transformed to obtain a file storage control interface; the file storage control interface conforms to the interface semantics of the portable operating system of the file storage system.

[0028] An index information for the target file is created based on the file storage control interface, the target file is stored according to the index information, and the index information is hidden during the storage of the target file.

[0029] When the conditions for disclosing the index information of the target file are met, the index information is disclosed based on the file storage control interface.

[0030] The aforementioned data processing methods, apparatus, computer equipment, storage media, and computer program products include a file storage system with portable operating system interface semantics and an object storage system with object storage interface semantics. The index server of the file storage system receives target files uploaded by object storage devices in the object storage system via a file upload interface. This file upload interface conforms to the object storage interface semantics of the object storage system. Interface semantic conversion is performed on the file upload interface to transform it into a file storage control interface that conforms to the portable operating system interface semantics. This interface conversion enables the file storage system to support the object storage system's interface. Index information for the target file is created based on the file storage control interface. The target file is stored according to the index information, and the index information is hidden during the storage process. When the conditions for revealing the target file's index information are met, the index information is revealed based on the file storage control interface. This allows the file storage system to achieve the functionality of the object storage system—hiding index information during file upload and revealing it after upload—based on the converted interface. This ensures that the file storage system is compatible with the object storage system's functionality. Attached Figure Description

[0031] Figure 1 This is a diagram illustrating the application environment of a data processing method in one embodiment.

[0032] Figure 2 This is a flowchart illustrating a data processing method in one embodiment;

[0033] Figure 3 This is a schematic diagram of the architecture of an object storage system and a file storage system in one embodiment;

[0034] Figure 4 This is a schematic diagram illustrating the index information being set to a hidden state and a public state in another embodiment;

[0035] Figure 5 This is a schematic diagram of the process for generating public index information for a target file in a public index branch, as shown in one embodiment.

[0036] Figure 6 This is a schematic diagram of hidden index branches and public index branches in a preset directory structure in one embodiment;

[0037] Figure 7 This is a schematic diagram of the globally hidden index branch, locally hidden index branch, and publicly visible index branch in a preset directory structure in one embodiment;

[0038] Figure 8 This is a correspondence between interfaces conforming to object storage interface semantics and interfaces conforming to portable operating system interface semantics in one embodiment;

[0039] Figure 9 This is a schematic diagram of a file access process in one embodiment;

[0040] Figure 10 This is a schematic diagram illustrating the process of reading file blocks according to local hidden index information to obtain a merged file in one embodiment.

[0041] Figure 11 This is a schematic diagram of the architecture of a traditional file storage system in one embodiment;

[0042] Figure 12 This is a schematic diagram of the architecture of a file storage system and an object storage system in one embodiment;

[0043] Figure 13 This is a schematic diagram of chunked upload in one embodiment;

[0044] Figure 14 This is a structural block diagram of a data processing device in one embodiment;

[0045] Figure 15 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0046] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0047] Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology and distributed storage file systems to bring together a large number of storage devices of various types in the network (storage devices are also called storage nodes) to work together through application software or application interfaces to provide data storage and business access functions to the outside world.

[0048] The storage system employs the following method: Logical volumes are created. During creation, physical storage space is allocated to each logical volume. This physical storage space may consist of a single storage device or the disks of several storage devices. Clients store data on a logical volume, which means storing the data on the file system. The file system divides the data into many parts, each part being an object. Each object contains not only the data but also additional information such as a data identifier (ID, ID entity). The file system writes each object to the physical storage space of that logical volume and records the storage location information of each object. Therefore, when a client requests data access, the file system can allow the client to access the data based on the storage location information of each object.

[0049] The process by which a storage system allocates physical storage space to a logical volume is as follows: the physical storage space is pre-divided into strips according to the capacity estimate of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of Redundant Array of Independent Disks (RAID). A logical volume can be understood as a strip, thus allocating physical storage space to the logical volume.

[0050] The data processing method provided in this application embodiment can be applied to a distributed cloud storage system, which can be a distributed file system.

[0051] The data processing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the object storage device 102 in the object storage system communicates with the index server 104 of the file storage system via a network. The index server 104 receives the target file uploaded by the object storage device 102 using a file upload interface, which conforms to the object storage interface semantics of the object storage system. The index server 104 performs interface semantic conversion on the file upload interface to obtain a file storage control interface, which conforms to the portable operating system interface semantics of the file storage system. Based on the file storage control interface, the index server 104 creates index information for the target file, stores the target file according to the index information, and hides the index information during the storage process. When the conditions for publicizing the index information of the target file are met, the index server 104 publishes the index information based on the file storage control interface.

[0052] The object storage device 102 can be a terminal or a server. Terminals can be, but are not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server can be a standalone physical server, a server cluster consisting of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0053] The index server 104 can be a standalone physical server, a server cluster consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0054] In one embodiment, such as Figure 2 As shown, a data processing method is provided, which can be applied to... Figure 1 Taking the index server of the Chinese file storage system as an example, the following steps are included:

[0055] Step S202: Receive the target file uploaded by the object storage device in the object storage system based on the file upload interface, wherein the file upload interface conforms to the object storage interface semantics of the object storage system.

[0056] The file storage system is a distributed file system with Portable Operating System Interface (POSIX) semantics. The index server is the server in the file storage system that creates and manages index information. The index can be a metadata server.

[0057] Portable operating system interface semantics defines the interface standards that an operating system should provide to applications; it is a set of interface standards defined for software running on an operating system.

[0058] Portable operating system interface semantics refers to standards or specifications used for transferring data between clients and servers in a file storage system.

[0059] A portable operating system interface (OSI) refers to an interface that conforms to the semantics of a portable operating system interface. Portable OSI interfaces include various interfaces in file storage systems, such as file storage control interfaces, relation establishment interfaces, index marking interfaces, and index access interfaces, but are not limited to these.

[0060] Cloud Object Storage (COS) is a distributed storage service with object storage interface semantics. Users can access the system through the object storage interface to perform operations such as uploading, downloading, and managing files in various formats.

[0061] Object storage interface semantics refers to the data transmission protocol of an object storage system, that is, the standard or specification used to transmit data between clients and servers in an object storage system.

[0062] An object storage interface refers to an interface that conforms to the semantics of an object storage interface. Object storage interfaces include various interfaces in an object storage system, such as file upload interfaces, file access interfaces, file deletion interfaces, upload completion interfaces, and file deletion interfaces, but are not limited to these.

[0063] An object storage interface represents at least one instruction. For example, a file upload interface represents a file upload instruction, and a file access interface represents a file upload instruction.

[0064] This file upload interface is used for interaction between the object storage system and the file storage system. Furthermore, it is used for interaction between the object storage device in the object storage system and the index server in the file storage system.

[0065] The file upload interface conforms to the semantics of the object storage interface, indicating that the target file needs to be uploaded according to the file upload method of the object storage system. The file upload method of the object storage system refers to the visibility of the file upload process; that is, the index information is hidden during the upload process and made public after the upload is completed.

[0066] Specifically, the user identifies the target file that needs to be uploaded from the object storage system to the file storage system. Then, through the object storage device in the object storage system, the user calls a file upload interface that conforms to the object storage interface semantics, and the target file is uploaded to the file storage system via this interface.

[0067] The index server in the file storage system receives the target file uploaded by the object storage device through the file upload interface.

[0068] In one embodiment, the object storage device can be an object storage client or an object storage server. The target file can be a file local to the object storage client or a file stored on the object storage server. The file upload interface can upload the target file local to the object storage client to the file storage system. The file upload interface can also upload the target file stored on the object storage server to the file storage system; that is, in a file migration scenario, the target file stored on the object storage server can be migrated to the file storage system.

[0069] Step S204: Perform interface semantic conversion on the file upload interface to obtain the file storage control interface; the file storage control interface conforms to the interface semantics of the portable operating system of the file storage system.

[0070] The file storage control interface is the interface in the file storage system that corresponds to the file upload interface. The file storage control interface conforms to the interface semantics of a portable operating system inherent in the file storage system.

[0071] The file storage control interface is an interface used to implement the file upload function of an object storage system.

[0072] Specifically, the file upload interface conforms to the object storage interface semantics of the object storage system. Since the index server in the file storage system cannot directly recognize and execute the instructions represented by the file upload interface, the index server performs interface semantic conversion on the file upload interface, converting the file upload interface into a file storage control interface. This file storage control interface conforms to the portable operating system interface semantics of the file storage system, so as to convert it into instructions that the index server can recognize.

[0073] In this embodiment, the file storage system includes an escaping device and an index server. The escaping device receives the target file uploaded by the object storage device based on the file upload interface. The escaping device performs interface semantic conversion on the file upload interface to obtain the file storage control interface. The escaping device feeds back the target file and the file storage control interface to the index server, which then calls the file storage control interface to execute the storage of the target file.

[0074] In this embodiment, the file storage system includes a proxy server, an escaping device, and an index server. The escaping device converts the file upload interface into a file storage control interface and feeds back the target file and the file storage control interface to the proxy server. The proxy server then feeds back the target file to the index server, which in turn calls the file storage control interface to store the target file on the file storage server.

[0075] Furthermore, such as Figure 3As shown, the object storage system includes object storage devices and a proxy server. The object storage device sends the target file to the proxy server. The proxy server calls the file upload interface to upload the target file to the escape device in the file storage system.

[0076] In one embodiment, the file upload interface undergoes interface semantic transformation to obtain a file storage control interface, including:

[0077] Determine the file upload type corresponding to the file upload interface; perform interface semantic conversion on the file upload interface based on the file upload type to obtain the file storage control interface.

[0078] File upload types include global upload and chunked upload. Global upload refers to uploading the target file as a whole. Chunked upload refers to splitting the target file into multiple file chunks and uploading the target file in chunk form.

[0079] The file storage control interface may include an index creation interface and an index public interface. The index creation interface may include a global index creation interface and a local index creation interface.

[0080] In this embodiment, when the file upload type is global upload, the file upload interface is converted into a global index creation interface and an index public interface. When the file upload type is chunked upload, the file upload interface is converted into a global index creation interface, a local index creation interface, and an index public interface. The global index creation interface, the local index creation interface, and the index public interface all conform to the interface semantics of a portable operating system.

[0081] Step S206: Create index information for the target file based on the file storage control interface, store the target file according to the index information, and hide the index information during the storage process of the target file.

[0082] The index information is used to locate the target file in the file storage system. The index information may include a file index identifier and a file storage path. The file storage path refers to the location where the target file is stored in the file storage system.

[0083] Specifically, the index server creates index information for the target file based on the file storage control interface. The index server stores the target file according to the index information and hides the index information during the storage process.

[0084] Step S208: When the conditions for publicizing the index information of the target file are met, the index information is publicized based on the file storage control interface.

[0085] The index information disclosure conditions refer to the constraints for disclosing the index information of the target file. Specifically, these conditions could be that the storage progress of the target file meets a preset schedule, or that an upload completion command triggered by the object storage device based on the upload completion interface has been received.

[0086] Specifically, during the storage of a target file, the index server checks whether the target file meets the conditions for making its index information public. If the conditions are met, the index server publishes the index information of the target file based on the file storage control interface.

[0087] In this embodiment, the index server can adjust the state of the index information based on the file storage control interface. During the storage of the target file, the index information is adjusted to a hidden state to conceal it. When the conditions for making the index information of the target file public are met, the index server can adjust the index information to a public state based on the file storage control interface to make the index information of the target file public.

[0088] In one embodiment, the index information includes hidden index information and public index information. The metadata service creates hidden index information for the target file based on the file storage control interface, and stores the target file according to the hidden index information. When the conditions for publicizing the target file's index information are met, the file storage control interface is invoked to publicly generate public index information for the target file. Further, when the conditions for publicizing the target file's index information are met, the file storage control interface is invoked to generate public index information for the target file based on the hidden index information.

[0089] In this embodiment, the index server determines the preset directory structure corresponding to the file storage system, creates index information for the target file in the preset directory structure based on the file storage control interface, stores the target file according to the index information, and hides the index information during the storage process of the target file. When the conditions for publicizing the index information of the target file are met, the index information is made public in the preset directory structure based on the file storage control interface.

[0090] The default directory structure can be a directory tree, where each node is an inode (indicator). Each inode has attribute information and index information. The attribute information includes various metadata, such as modification time, permissions, and size, but is not limited to these. The attribute information is also used to identify the type of the inode, such as whether the inode is a directory or a file.

[0091] Furthermore, such as Figure 4As shown, during the storage process of the target file, the index server sets the index information of the target file to a hidden state. When the conditions for making the index information of the target file public are met, the index information is set to a public state based on the file storage control interface, thus making the index information of the target file public.

[0092] Traditional object storage systems and traditional file storage systems offer visibility during file uploads. However, the visibility of traditional object storage systems means that the file's index information is hidden during the upload process and made public after the upload is complete. In contrast, the visibility of traditional file storage systems means that the file's index information is visible in real-time during the upload process.

[0093] Traditional file storage systems are incompatible with the visibility of traditional object storage systems. However, in the data processing method described above, the file storage system possesses portable operating system interface semantics, while the object storage system possesses object storage interface semantics. The index server of the file storage system receives target files uploaded by object storage devices in the object storage system via a file upload interface. It performs interface semantic conversion on the file upload interface conforming to object storage interface semantics, transforming it into a file storage control interface conforming to portable operating system interface semantics. This interface semantic conversion enables the file storage system to support the object storage system's interface, allowing object storage devices to request the file storage system to upload target files according to the file upload methods available in the object storage system. Based on the file storage control interface, index information for the target file is created. The target file is stored according to the index information, and the index information is hidden during storage. When the conditions for publicizing the index information are met, the index information is made public based on the file storage control interface. This allows the file storage system to achieve the functionality of hiding index information during file upload and making it public after upload, thus ensuring compatibility with the file upload functionality unique to object storage systems.

[0094] In one embodiment, the file storage control interface includes an index creation interface and an index public interface, and the index information includes hidden index information and public index information; creating index information for a target file based on the file storage control interface, storing the target file according to the index information, and hiding the index information during the storage process of the target file includes:

[0095] Create hidden index information for the target file based on the index creation interface, and store the target file according to the hidden index information;

[0096] When the conditions for publicizing the index information of the target file are met, the index information is disclosed based on the file storage control interface, including:

[0097] When the conditions for making the index information of the target file public are met, the hidden index information of the target file will be adjusted to public index information based on the index public interface.

[0098] The file storage control interface includes an index creation interface and an index public interface. The index creation interface is used to create hidden index information for the target file, while the index public interface is used to generate public index information for the target file. Hidden index information refers to index information that is hidden from the user, i.e., index information that the user cannot view. Public index information refers to index information that is publicly available to the user, i.e., index information that the user can view.

[0099] In this embodiment, the file upload type of the target file is recorded in the publicly available index information.

[0100] Specifically, the index server performs interface semantic translation on the file upload interface to obtain the index creation interface and the index public interface. Based on the index creation interface, the index server creates hidden index information for the target file. The metadata service stores the target file according to the hidden index information. Further, the index server stores the target file in the file storage server according to the hidden index information.

[0101] During the storage of the target file, the index server checks whether the target file meets the conditions for making the index information public. If it does, it adjusts the hidden index information of the target file to public index information based on the index public interface.

[0102] In this embodiment, the hidden index information of the target file is adjusted to public index information based on the public index interface, including: adjusting the hidden index information from a hidden state to a public state based on the public index interface to obtain the public index information of the target file.

[0103] In this embodiment, the hidden index information of the target file is adjusted to public index information based on the public index interface, including: generating public index information of the target file based on the hidden index information of the target file according to the public index interface.

[0104] In one embodiment, the hidden index information includes global hidden index information; creating hidden index information for the target file based on the index creation interface, and storing the target file according to the hidden index information, includes:

[0105] Determine the file upload type corresponding to the file upload interface; when the file upload type is global upload, create global hidden index information for the target file based on the index creation interface, and store the target file according to the global hidden index information.

[0106] In this embodiment, the hidden index information also includes local hidden index information; the method further includes: when the upload type of the target file is chunked upload, creating global hidden index information for the target file and local hidden index information for each file chunk of the target file based on the index creation interface; and storing each file chunk according to the local hidden index information of each file chunk.

[0107] In this embodiment, when the conditions for publicizing the index information of the target file are met, the hidden index information of the target file is adjusted to public index information based on the index public interface. This includes: when the conditions for publicizing the index information of the target file are met, the public index information of the target file is generated based on the global hidden index information and the local hidden index information of each file block, according to the index public interface.

[0108] In this embodiment, when the conditions for publicizing the index information of the target file are met, based on the index publicizing interface, public index information of the target file is generated according to the global hidden index information and the local hidden index information of each file block, including:

[0109] When the conditions for making the index information of the target file public are met, a mapping relationship is established between the global hidden index information and the local hidden index information of each file block based on the index public interface; based on the index public interface, the global hidden index information after the mapping relationship is established is adjusted to public index information.

[0110] In this embodiment, following the portable operating system interface semantics of the file storage system, the file upload interface is converted into an index creation interface and an index disclosure interface. Hidden index information for the target file is created based on the index creation interface, and the target file is stored according to this hidden index information, thus hiding the file's index information during the file upload process. When the conditions for making the target file's index information public are met, the hidden index information is adjusted to public index information based on the index disclosure interface, making it accessible to users. By converting one file upload interface of the object storage system into two interfaces, the function of hiding and then revealing index information during the file upload process is implemented using two different interfaces, enabling the file storage system to possess the file upload functionality unique to object storage systems.

[0111] In one embodiment, such as Figure 5 As shown, the process of creating hidden index information for the target file based on the index creation interface includes steps S502 to S504, wherein:

[0112] Step S502: Determine the preset directory structure corresponding to the file storage system. The preset directory structure includes hidden index branches and public index branches.

[0113] The preset directory structure is a data structure used to record the index information of files stored in the file storage system. The preset directory structure can be a preset directory tree.

[0114] The preset directory structure includes at least two branches, namely, at least a hidden index branch and a public index branch, such as... Figure 6 As shown, the a.txt file in the hidden index branch contains hidden index information, while the A.txt file in the public index branch contains public index information.

[0115] Hidden index branches are used to record hidden index information, while public index branches are used to record public index information. Hidden index branches are not visible to users, while public index branches are visible to users.

[0116] Specifically, the index server determines the preset directory structure corresponding to the file storage system, and determines the hidden index branches and public index branches in the preset directory structure.

[0117] Step S504: Based on the index creation interface, create hidden index information for the target file in the hidden index branch.

[0118] Specifically, the index server calls the index creation interface, and the index creation interface is called through the index creation interface.

[0119] In this embodiment, the hidden index branch includes a global hidden index branch and a local hidden index branch, and the hidden index information includes global hidden index information and local hidden index information; based on the index creation interface, hidden index information for the target file is created in the hidden index branch, including:

[0120] When the upload type of the target file is chunked upload, based on the index creation interface, global hidden index information for the target file is created in the global hidden index branch; based on the index creation interface, local hidden index information for each file chunk of the target file is created in the local hidden index branch.

[0121] Furthermore, the index creation interface includes a global index creation interface and a local index creation interface. When the target file is uploaded in chunks, the global index creation interface creates global hidden index information for the target file in the global hidden index branch; the local index creation interface creates local hidden index information for each file chunk of the target file in the local hidden index branch.

[0122] When the conditions for making the index information of the target file public are met, the hidden index information of the target file is adjusted to public index information based on the index public interface, including step S506:

[0123] Step S506: When the conditions for publicizing the index information of the target file are met, based on the index publicizing interface, public index information for the target file is generated in the public indexing branch according to the hidden index information in the hidden indexing branch.

[0124] Specifically, during the file chunk upload process of the target file, the index server checks whether the conditions for publicizing the index information of the target file are met. If they are met, the index publicizing interface is called. Through the called index publicizing interface, the public index information for the target file in the public index branch is generated based on the hidden index information in the hidden index branch.

[0125] In this embodiment, when the upload type of the target file is chunked upload, the conditions for disclosing the index information of the target file can be that each file chunk of the target file has been uploaded, or the upload progress of each file chunk has reached a preset progress, or the upload completion instruction triggered by the object storage device based on the upload end interface has been received.

[0126] In this embodiment, when the conditions for publicizing the index information of the target file are met, based on the index publicizing interface, public index information for the target file in the public index branch is generated according to the hidden index information in the hidden index branch, including:

[0127] When the conditions for making the index information of the target file public are met, based on the index public interface, public index information for the target file is generated in the public index branch according to the global hidden index information and the local hidden index information of each file block of the target file.

[0128] In this embodiment, in response to a file to be written uploaded by a client in the file storage system via the file write interface, public index information for the file to be written is created in the public index branch. The file write interface conforms to the portable operating system interface semantics of the file storage system. That is, files uploaded by clients in the file storage system can have public index information directly created in the public index branch.

[0129] In this embodiment, the file upload type of the target file is recorded in the globally hidden index information and the publicly available index information.

[0130] In this embodiment, the preset directory structure of the file storage system is divided into a hidden index branch and a public index branch to achieve the hiding of index information during file upload and the public disclosure of index information after upload. Based on the index creation interface, hidden index information for the target file is created in the hidden index branch, and the target file is stored according to the hidden index information, making the index information invisible to the outside world during the upload process. When the conditions for public disclosure of the target file's index information are met, based on the index disclosure interface, public index information for the target file is generated in the public index branch according to the hidden index information in the hidden index branch. Thus, when the file is uploaded to a level where it can be made public, the file index information is made public through the public index branch for users to query. Through the hidden index branch and public index branch of the preset directory structure, the file storage system can maintain the original file upload function and also be compatible with the file upload function of the object storage system, thereby expanding the functionality of the file storage system.

[0131] In one embodiment, when the conditions for publicizing the index information of the target file are met, based on the index publicizing interface, public index information for the target file in the public index branch is generated according to the hidden index information in the hidden index branch, including:

[0132] When the conditions for making the index information of the target file public are met, the parent directory information is obtained from the public index branch based on the index public interface; based on the parent directory information, the hidden index information in the hidden index branch is transferred to the public index branch to obtain the public index information for the target file.

[0133] Specifically, when the conditions for public access to the index information of the target file are met, the index server obtains the parent directory information from the public index branch based on the public index interface, transfers the hidden index information in the hidden index branch to the public index branch, and adjusts the parent directory information of the hidden index information in the hidden index branch to the parent directory information in the public index branch, thereby obtaining the public index information for the target file in the public index branch.

[0134] In this embodiment, when the conditions for making the index information of the target file public are met, the index server obtains the parent directory information from the public index branch based on the index public interface. Based on the index creation interface, it obtains the parent directory information of the hidden index information in the hidden index branch. Based on the index public interface, it adjusts the parent directory information of the hidden index information in the hidden index branch to the parent directory information in the public index branch, thus obtaining the public index information for the target file in the public index branch.

[0135] In this embodiment, when the conditions for publicizing the index information of the target file are met, the parent directory information is obtained from the public index branch based on the public index interface. Based on the parent directory information, the hidden index information in the hidden index branch is transferred to the public index branch to obtain the public index information for the target file. Thus, the index information is first hidden during the file storage process, and the parent directory information is adjusted to form public index information for users after the public conditions are met.

[0136] In one embodiment, the hidden index branch includes a global hidden index branch and a local hidden index branch, and the hidden index information includes global hidden index information and local hidden index information; based on the index creation interface, hidden index information for the target file is created in the hidden index branch, including:

[0137] Determine the file upload type corresponding to the file upload interface; when the file upload type is chunked upload, based on the index creation interface, create global hidden index information for the target file in the global hidden index branch, and create local hidden index information for each file chunk of the target file in the local hidden index branch;

[0138] The target file is stored according to the hidden index information, including: storing each file block of the target file according to the local hidden index information of each file block of the target file.

[0139] Specifically, the index server determines the interface type corresponding to the file upload interface, and then determines the file upload type of the target file based on the interface type. Interface types include, for example, a default upload interface and a chunked upload interface; file upload types include global upload and chunked upload. When the interface type is the default upload interface, it means the target file's file upload type is global upload. When the interface type is the chunked upload interface, it means the target file's file upload type is chunked upload.

[0140] When the file upload type is chunked upload, the index server calls the index creation interface to create global hidden index information for the target file in the global hidden index branch. The index server also calls the index creation interface to create local hidden index information for each file chunk of the target file in the local hidden index branch.

[0141] This preset directory structure includes globally hidden index branches, locally hidden index branches, and publicly accessible index branches, such as... Figure 7 As shown, the file a.txt in the global hidden index branch contains the global hidden index information, while the file A.txt in the public index branch contains the public index information. The files a.txt.1, a.txt.2, and a.txt.3 in the local hidden index branch contain the local hidden index information for file block 1, file block 2, and file block N of the target file, respectively.

[0142] For each file block of the target file, the index server determines the local hidden index information of the target file block and stores the target file block according to the local hidden index information.

[0143] In this embodiment, the local hidden index information of the file block includes the storage path corresponding to each file block. The index server stores the target file block in the storage location pointed to by the storage path according to the storage path corresponding to the target file block.

[0144] In this embodiment, the index creation interface includes a global index creation interface and a local index creation interface. When the file upload type is chunked upload, based on the global index creation interface, global hidden index information for the target file is created in the global hidden index branch; based on the local index creation interface, local hidden index information for each file chunk of the target file is created in the local hidden index branch.

[0145] In one embodiment, based on the index creation interface, hidden index information for the target file is created in the hidden index branch, and the target file is stored according to the hidden index information, including:

[0146] The hidden index branch includes the global hidden index branch, and the hidden index information includes the global hidden index information; the index server determines the file upload type corresponding to the file upload interface; when the file upload type is global upload, based on the index creation interface, global hidden index information for the target file is created in the global hidden index branch; the target file is stored according to the global hidden index information;

[0147] When the conditions for making the index information of the target file public are met, based on the index public interface and the hidden index information in the hidden index branch, the public index information for the target file in the public index branch is generated, including: when the conditions for making the index information of the target file public are met, based on the index public interface and the global hidden index information in the global hidden index branch, the public index information for the target file in the public index branch is generated.

[0148] In this embodiment, the hidden index branch also includes a locally hidden index branch, and the hidden index information also includes locally hidden index information.

[0149] In this embodiment, the preset directory structure of the file storage system is divided into a global hidden index branch, a local hidden index branch, and a public index branch. When the file upload type of the target file is chunked upload, indicating that the object storage device will upload multiple file chunks of the target file, the index server, based on the index creation interface, creates global hidden index information for the target file in the global hidden index branch to record the global index of the target file, and creates local hidden index information for each file chunk of the target file in the local hidden index branch to record the chunk index of each file chunk. Each file chunk of the target file is stored according to the local hidden index information of each file chunk, so that the target file can be uploaded to the file storage system in the form of file chunks, making the file storage system compatible with the chunked upload function of the object storage system.

[0150] In one embodiment, when the conditions for publicizing the index information of the target file are met, based on the index publicizing interface, public index information for the target file in the public index branch is generated according to the hidden index information in the hidden index branch, including:

[0151] When the conditions for making the index information of the target file public are met, a mapping relationship is established between the global hidden index information in the global hidden index branch and the local hidden index information of each file block in the local hidden index branch. Based on the index public interface, the global hidden index information after the mapping relationship is established is transferred from the global hidden index branch to the public index branch to obtain the public index information of the target file in the public index branch.

[0152] Specifically, during the upload process of each file block of the target file, the index server checks whether the target file meets the conditions for public access to index information. If it does, a mapping relationship is established between the global hidden index information in the global hidden index branch and the local hidden index information of each file block in the local hidden index branch.

[0153] After establishing the mapping relationship, the index server calls the public index interface to determine the parent directory information in the public index branch. It then transfers the globally hidden index information after establishing the mapping relationship from the global hidden index branch to the public index branch, and modifies the parent directory information of the globally hidden index information in the global hidden index branch to the parent directory information in the public index branch, thereby obtaining the public index information for the target file in the public index branch.

[0154] In this embodiment, the conditions for disclosing the index information include the completion of uploading each file block of the target file. When the index server detects that the file block upload of the target file is complete, it establishes a mapping relationship between the globally hidden index information and the locally hidden index information of each file block of the target file.

[0155] In this embodiment, the conditions for disclosing index information include the number of file blocks uploaded to the target file reaching a preset number. For example, if the target file is divided into 20 file blocks, and 15 file blocks have been uploaded, it means that the conditions for disclosing index information are met.

[0156] In this embodiment, when the condition for publicizing the index information of the target file is met, a mapping relationship is established between the global hidden index information in the global hidden index branch and the local hidden index information of each file block in the local hidden index branch. This mapping relationship determines the index information of each file block, maintaining the integrity of the target file stored in blocks. Based on the index public interface, the global hidden index information after the mapping relationship is established is transferred from the global hidden index branch to the public index branch. This obtains the public index information for the target file in the public index branch, ensuring that the target file uploaded in blocks only exposes the global index and hides the block indexes, guaranteeing the security and integrity of file block storage. Furthermore, this enables the file storage system to possess the functionality of an object storage system in a block upload scenario, exposing the global index and hiding the block indexes, thus achieving compatibility between the file storage system and the object storage system.

[0157] In one embodiment, the file upload interface includes a chunked upload interface and an upload completion interface, and the file storage control interface further includes a relationship establishment interface. The index information disclosure condition includes receiving an upload completion instruction triggered by the object storage device. Interface semantic conversion is performed on the file upload interface to obtain the file storage control interface, including:

[0158] When the file upload type is chunked upload, the chunked upload interface is semantically converted to obtain the index creation interface and the index public interface.

[0159] When the conditions for making the index information of the target file public are met, establish a mapping relationship between the global hidden index information in the global hidden index branch and the local hidden index information of each file block in the local hidden index branch, including:

[0160] In response to the upload completion command of the target file triggered by the object storage device based on the upload completion interface, the upload completion interface is converted into a relationship establishment interface; the upload completion command conforms to the semantics of the object storage interface, and the relationship establishment interface conforms to the semantics of the portable operating system interface; based on the relationship establishment interface, a mapping relationship is established between the global hidden index information in the global hidden index branch and the local hidden index information of each file block in the local hidden index branch.

[0161] The file upload interface includes a chunked upload interface and an upload completion interface. The file storage control interface includes: an index creation interface and an index public interface corresponding to the chunked upload interface, and a relationship establishment interface corresponding to the upload completion interface.

[0162] The upload completion interface is an object storage interface. It indicates that the upload of all file chunks of the target file is complete.

[0163] The relationship establishment interface conforms to the interface semantics of a portable operating system. It represents the mapping relationship that needs to be established between the global hidden index information of a target file and the local hidden index information of all file blocks within that target file.

[0164] Specifically, when uploading a target file using a chunked upload method, the object storage device splits the target file into multiple file chunks and calls the chunked upload interface to upload these chunks sequentially to the index server. The index server performs interface semantic translation on the chunked upload interface, converting it into an index creation interface and an index public interface.

[0165] The index server receives file chunks uploaded from the object storage device and calls the index creation interface to create corresponding locally hidden index information for each received file chunk in the locally hidden index branch. Each file chunk has its own locally hidden index information.

[0166] In this embodiment, the index server can call the index creation interface to first generate global hidden index information, and when a file block is received, it can create corresponding local hidden index information for the received file block.

[0167] In this embodiment, the index server generates an upload identifier corresponding to the globally hidden index information and feeds the upload identifier back to the object storage device, so that the object storage device binds the upload identifier to each file block, which is used to identify the upload of all file blocks of the same target file.

[0168] After the object storage device upload is complete, the upload completion interface is called to trigger the upload completion command for the target file. This upload completion interface conforms to the object storage interface semantics. The index server performs interface semantic conversion on the upload completion interface to obtain the relationship establishment interface, thus converting the upload completion interface, which conforms to the object storage interface semantics, into a relationship establishment interface that conforms to the portable operating system interface semantics, for use in the file storage system.

[0169] The index server calls the relationship establishment interface to establish a mapping relationship between the global hidden index information in the global hidden index branch and the local hidden index information of each file block in the local hidden index branch.

[0170] In this embodiment, the file upload interface further includes an identifier acquisition interface, and the file storage control interface further includes an identifier generation interface; receiving a target file uploaded by an object storage device in the object storage system based on the file upload interface includes: receiving an upload instruction for the target file uploaded by an object storage device in the object storage system based on the identifier acquisition interface;

[0171] Perform interface semantic conversion on the file upload interface to obtain the file storage control interface, including: when the file upload type is chunked upload, perform interface semantic conversion on the identifier acquisition interface and the chunked upload interface to obtain the identifier generation interface corresponding to the identifier acquisition interface, as well as the index creation interface and index public interface corresponding to the chunked upload interface.

[0172] The index server receives an upload command for a target file from the object storage device based on the identifier acquisition interface, converts the identifier acquisition interface into an identifier generation interface, and calls the identifier generation interface to generate an upload identifier for the target file. This upload identifier is then sent back to the object storage device.

[0173] After receiving the upload identifier, the object storage device can see that each file chunk is bound to the upload identifier, and calls the chunk upload interface to upload multiple file chunks to the index server in sequence.

[0174] The method also includes: the index server also calls the index creation interface to bind each partially hidden index information to the upload identifier.

[0175] In this embodiment, in response to the upload completion command of the target file triggered by the object storage device based on the upload completion interface, the upload completion interface, which conforms to the semantics of the object storage interface, is converted into a relationship establishment interface, which conforms to the semantics of the portable operating system interface. This interface is then invoked in the file storage system. This interface semantic conversion enables the file storage system to support the object storage system's interface, allowing the object storage device to request the file storage system to perform the upload of the target file according to the chunked upload method available in the object storage system. Based on the relationship establishment interface, a mapping relationship is established between the global hidden index information in the global hidden index branch and the local hidden index information of each file chunk in the local hidden index branch. This determines the correspondence between the global index of the target file and the chunk indexes of each file chunk, ensuring the integrity and accuracy of the target file. Furthermore, this allows the file storage system to be compatible with the file chunked upload function unique to the object storage system.

[0176] In one embodiment, the method further includes:

[0177] The system receives a file deletion command triggered by the object storage device based on the file deletion interface, and converts the file deletion interface into an index marking interface. The file deletion interface conforms to the semantics of the object storage interface, and the index marking interface conforms to the semantics of the portable operating system interface. Based on the index marking interface, the system marks the public index information of the target file. The marking is used to instruct the memory reclamation server of the file storage system to delete the public index information of the target file and the target file.

[0178] In this embodiment, the file deletion interface conforms to the semantics of an object storage interface. The file deletion interface indicates that the uploaded target file should be deleted. Further, when the target file's upload type is global upload, the file deletion interface indicates that the uploaded target file should be deleted. When the target file's upload type is chunked upload, the file deletion interface indicates that the uploaded file chunks, which belong to the target file, should be deleted.

[0179] The index tagging interface conforms to the semantics of a portable operating system interface. The index tagging interface indicates that a tag should be added to the public index information to await deletion of the public index information and the stored target file by the memory reclamation server. Furthermore, when the target file's upload type is global upload, the index tagging interface indicates that a tag should be added to the public index information of the target file to await deletion of the public index information and the stored target file by the memory reclamation server. When the target file's upload type is chunked upload, the index tagging interface indicates that a tag should be added to the public index information of the target file to await deletion of the public index information of the target file, the hidden index information of each file chunk, and the stored file chunks by the memory reclamation server.

[0180] In this embodiment, the user uploads a target file through an object storage device. During the upload process, the user can cancel the upload at any time. When cancellation is required, the object storage device can call the file deletion interface to trigger a file deletion command for the target file.

[0181] The index server performs interface semantic conversion on the file deletion interface to convert the file deletion interface, which conforms to the object storage interface semantics, into the index tag interface, which conforms to the portable operating system interface semantics.

[0182] The index server calls the index tagging interface to mark the public index information of the target file. This tag is used to instruct the memory reclamation server of the file storage system to delete the public index information of the target file and the target file itself.

[0183] In this embodiment, a file deletion command triggered by the object storage device based on the file deletion interface is received. The file deletion interface, conforming to the semantics of the object storage interface, is converted into an index marking interface, conforming to the semantics of the portable operating system interface. This semantic conversion enables the file storage system to support the file deletion interface of the object storage system. The public index information of the target file is marked using the index marking interface, thereby instructing the memory reclamation server of the file storage system to delete the public index information and the target file itself. Furthermore, for target files uploaded using the file upload function of the object storage system, deletion is still performed using the file deletion function of the object storage system. This ensures that the file storage system is compatible with both the file upload and file deletion functions of the object storage system, effectively improving its compatibility.

[0184] like Figure 8 The diagram illustrates the correspondence between interfaces conforming to object storage interface semantics and interfaces conforming to portable operating system interface semantics in one embodiment. The file upload interface is converted into an index creation interface and an index public interface; the upload completion interface is converted into a relationship establishment interface; the file deletion interface is converted into an index marking interface; and the file access interface is converted into a public index access interface and a hidden index access interface. Specifically, the file upload interface, upload completion interface, file deletion interface, and file access interface all conform to object storage interface semantics. The index creation interface, index public interface, relationship establishment interface, index marking interface, public index access interface, and hidden index access interface all conform to portable operating system interface semantics.

[0185] In one embodiment, the index information includes global hidden index information, local hidden index information, and public index information of the target file, and the preset directory structure corresponding to the file storage system includes a global hidden index branch, a local hidden index branch, and a public index branch; the hidden index information is located in the global hidden index branch, the local hidden index information is located in the local hidden index branch, and the public index information is located in the public index branch;

[0186] The public index information of the target file is marked based on the index marking interface, including:

[0187] When the file upload type of the target file is chunked upload, the public index information of the target file in the public index branch is marked based on the index marking interface;

[0188] The flag is used to instruct the memory reclamation server to delete each stored file block of the target file, the locally hidden index information in the locally hidden index branch, the globally hidden index information in the globally hidden index branch, and the publicly exposed index information in the publicly exposed index branch in order of priority.

[0189] Specifically, the file storage system has a corresponding preset directory structure, which includes a global hidden index branch, a local hidden index branch, and a public index branch. The global hidden index branch records the global hidden index information of the target file, while the public index branch records the public index information of the target file.

[0190] When the file upload type of the target file is chunked upload, the local hidden index branch also records the local hidden index information of each file chunk of the target file.

[0191] When the target file is uploaded in chunks, the index server marks the public index information of the target file in the public index branch based on the index marking interface. This is used to instruct the memory reclamation server to delete each stored file chunk of the target file, the local hidden index information of each file chunk of the target file in the local hidden index branch, the global hidden index information of the target file in the global hidden index branch, and the public index information of the target file in the public index branch, in order of priority.

[0192] Stored file blocks have a higher priority than locally hidden index information, locally hidden index information has a higher priority than globally hidden index information, and globally hidden index information has a higher priority than publicly available index information.

[0193] In this embodiment, when the file upload type of the target file is chunked upload, the public index information of the target file in the public index branch is marked based on the index marking interface. The stored file chunks, the local hidden index information of the file chunks, the global hidden index information of the target file, and the public index information are deleted in order of priority. This can effectively delete all information related to the target file and avoid omissions.

[0194] In one embodiment, such as Figure 9 As shown, the method also includes:

[0195] Step S902: Receive the file index identifier sent by the object storage device based on the file access interface, where the file access interface conforms to the semantics of the object storage interface.

[0196] The file access interface is an interface that conforms to the object storage interface semantics of an object storage system. The file access interface indicates a need to read a file from the file storage system.

[0197] Specifically, when an object storage device needs to read a file from a file storage system, it can call the file access interface and send the file index identifier to the index server through the called file access interface.

[0198] Step S904: Perform interface semantic conversion on the file access interface to obtain the index access interface, which conforms to the interface semantics of the portable operating system.

[0199] The index access interface is an interface that conforms to the interface language of a portable operating system. The index access interface indicates a need to query index information and read the file based on that information.

[0200] Specifically, the index server performs interface semantic conversion on the file access interface to convert the file access interface that conforms to the object storage interface semantics into the index access interface that conforms to the portable operating system interface semantics.

[0201] In this embodiment, the escaping device receives a file index identifier sent by the object storage device based on the file access interface. The escaping device performs interface semantic conversion on the file access interface to obtain the index access interface. The escaping device provides the file index identifier and the index access interface to the index server, which then calls the index access interface to read the file corresponding to the file index identifier.

[0202] Furthermore, the escape device converts the file access interface into an index access interface and provides the file index identifier and index access interface to the proxy server in the file storage system, which then feeds back to the index server.

[0203] Furthermore, the object storage system includes object storage devices and proxy servers. The object storage devices send file index identifiers to the proxy servers. The proxy servers in the object storage system invoke a file access interface, and through this interface, upload the file index identifiers to the escape device in the file storage system.

[0204] Step S906: Based on the index access interface, query the index information corresponding to the file index identifier.

[0205] Specifically, the index server calls the index access interface, and through the called index access interface, it queries the index information corresponding to the file index identifier in the index information corresponding to the stored file.

[0206] Furthermore, the index information includes hidden index information and public index information. The index server calls the index access interface, and through the called index access interface, it queries the public index information corresponding to the file index identifier in the public index information corresponding to the stored file.

[0207] In this embodiment, the file storage system has a corresponding preset directory structure, which records hidden index information and public index information corresponding to the stored files. The index server can call the index access interface to query the public index information corresponding to the file index identifier in the preset directory structure.

[0208] In this embodiment, the preset directory structure includes a hidden index branch and a public index branch. The index server can call the index access interface to query the public index information corresponding to the file index identifier in the public index branch.

[0209] Step S908: Based on the queried index information, obtain the file corresponding to the file index identifier, and send the obtained file back to the object storage device.

[0210] Specifically, when the index information corresponding to the file index identifier is found, the file is retrieved from the storage location pointed to by the index information. The file at the storage location pointed to by the index information is the file corresponding to the file index identifier. The index server then sends the retrieved file back to the object storage device.

[0211] Furthermore, when the public index information corresponding to the file index identifier is found, the file is retrieved from the storage location pointed to by the public index information.

[0212] In this embodiment, the receiving object storage device sends a file index identifier based on the file access interface. The file access interface, conforming to the semantics of the object storage interface, is converted into an index access interface conforming to the semantics of the portable operating system interface, thereby enabling support for the file access interface of the object storage system. Based on the index access interface, the index information corresponding to the file index identifier is queried. The file corresponding to the file index identifier is then retrieved based on the queried index information, allowing the retrieved file to be returned to the object storage device. This enables interaction between devices in the file storage system and the object storage system, and facilitates file upload and retrieval between the file storage system and the object storage system.

[0213] In one embodiment, the index access interface includes a public index access interface and a hidden index access interface. The index information includes public index information and multiple locally hidden index information that have a mapping relationship with the public index information. Based on the index access interface, querying the public index information corresponding to the file index identifier includes:

[0214] Based on the public index access interface, query the public index information corresponding to the file index identifier;

[0215] Based on the retrieved index information, the file corresponding to the file index identifier is obtained, and the obtained file is returned to the object storage device, including:

[0216] Based on the hidden index access interface, query multiple local hidden index information that have a mapping relationship with the public index information; read file blocks according to each local hidden index information, merge the read file blocks, and return the merged file to the object storage device.

[0217] The index access interface includes a public index access interface and a hidden index access interface. Both public and hidden index access interfaces conform to the interface semantics of a portable operating system. The public index access interface is used to access public index information, while the hidden index access interface is used to access locally hidden index information.

[0218] Specifically, the index server queries the public index information corresponding to the file index identifier based on the public index access interface. The index server also queries multiple local hidden index information that have a mapping relationship with the public index information based on the hidden index access interface.

[0219] The index server reads file blocks according to each locally hidden index information, merges the read file blocks, and obtains a merged file. The index server then sends the merged file back to the object storage device.

[0220] like Figure 10 As shown, file block 1 is read according to local hidden index information 1, file block 2 is read according to local hidden index information 2, ..., file block N is read according to local hidden index information N, and file blocks 1, 2, ..., and N are merged to obtain a merged file.

[0221] In this embodiment, the file storage system has a corresponding preset directory structure, which includes a global hidden index branch, a local hidden index branch, and a public index branch. The global hidden index branch records the global hidden index information of the stored files, and the public index branch records the public index information of the stored files.

[0222] When the file upload type of the stored file is chunked upload, the local hidden index branch also records the local hidden index information of each file chunk of the stored file.

[0223] When the file upload type of the stored file is chunked upload, the system uses the public index access interface to query the public index information corresponding to the file index identifier in the public index branch; and uses the hidden index access interface to query multiple local hidden index information that have a mapping relationship with the public index information in the local hidden index branch. The system then reads the file chunks according to each local hidden index information, merges the read file chunks, and sends the merged file back to the object storage device.

[0224] When the file upload type of the stored file is global upload, the public index access interface is used to query the public index information corresponding to the file index identifier in the public index branch. Once the public index information corresponding to the file index identifier is found, the file is retrieved from the storage location pointed to by the public index information.

[0225] In this embodiment, the file access interface conforming to the semantics of the object storage interface is converted into a public index access interface and a hidden index access interface conforming to the semantics of the portable operating system interface, thereby enabling support for the file access interface of the object storage system. Based on the public index access interface, public index information corresponding to the file index identifier is queried. Based on the hidden index access interface, multiple local hidden index information that have a mapping relationship with the public index information are queried to find the storage path of each file block, thus reading the file block according to the storage path. The read file blocks are merged, and the merged file is fed back to the object storage device. This achieves compatibility between the file storage system and the file block reading function of the object storage system, enabling the file storage system to adapt not only to file reading on devices conforming to the semantics of the portable operating system interface, but also to file reading on object storage devices conforming to the semantics of the object storage interface.

[0226] In one embodiment, a data processing method is provided, applied to an index server of a file storage system, comprising:

[0227] Receives target files uploaded by object storage devices in the object storage system via a file upload interface, where the file upload interface conforms to the object storage interface semantics of the object storage system.

[0228] Next, the file upload interface is semantically transformed to obtain the index creation interface and the index public interface; the index creation interface and the index public interface conform to the interface semantics of the portable operating system of the file storage system.

[0229] Furthermore, the preset directory structure corresponding to the file storage system is determined, which includes a global hidden index branch, a local hidden index branch, and a public index branch.

[0230] Next, determine the file upload type corresponding to the file upload interface.

[0231] Optionally, when the file upload type is global upload, based on the index creation interface, global hidden index information for the target file is created in the global hidden index branch, and the target file is stored according to the global hidden index information; based on the index public interface, the public parent directory information for the global hidden index information in the public index branch is determined, and the global parent directory information of the global hidden index information in the global hidden index branch is determined; based on the index public interface, the global hidden index information is moved from the global hidden index branch to the public index branch, and the global parent directory information is modified to the public parent directory information, thereby obtaining the public index information for the target file in the public index branch.

[0232] Optionally, when the file upload type is chunked upload, based on the index creation interface, global hidden index information for the target file is created in the global hidden index branch, and local hidden index information for each file chunk of the target file is created in the local hidden index branch.

[0233] Next, each file block of the target file is stored according to the local hidden index information of each file block.

[0234] Next, in response to the upload completion instruction of the target file triggered by the object storage device based on the upload completion interface, the upload completion interface is converted into a relationship establishment interface; the upload completion instruction conforms to the semantics of the object storage interface, and the relationship establishment interface conforms to the semantics of the portable operating system interface.

[0235] Furthermore, based on the relationship, an interface is established to create a mapping relationship between the global hidden index information in the global hidden index branch and the local hidden index information of each file block in the local hidden index branch.

[0236] Furthermore, based on the index public interface, the public parent directory information of the public index branch for the globally hidden index information after the mapping relationship is established is determined, and the global parent directory information of the globally hidden index information after the mapping relationship is established in the globally hidden index branch is determined;

[0237] Next, based on the index public interface, the global parent directory information of the globally hidden index information after the mapping relationship is established is modified to the public parent directory information, and the globally hidden index information after the parent directory information is modified is moved to the public index branch to obtain the public index information for the target file in the public index branch.

[0238] Optionally, the file deletion command triggered by the object storage device based on the file deletion interface is received, and the file deletion interface is converted into an index tagging interface; the file deletion interface conforms to the semantics of the object storage interface, and the index tagging interface conforms to the semantics of the portable operating system interface.

[0239] Next, the public index information of the target file is marked based on the index marking interface. When the file upload type of the target file is global upload, the mark is used to instruct the memory reclamation server of the file storage system to delete the public index information of the target file and the target file.

[0240] Optionally, when the file upload type of the target file is chunked upload, this flag is used to instruct the memory reclamation server to delete each stored file chunk of the target file, the locally hidden index information in the locally hidden index branch, the globally hidden index information in the globally hidden index branch, and the publicly exposed index information in the publicly exposed index branch in order of priority.

[0241] Optionally, the receiving object storage device sends a file index identifier based on a file access interface, the file access interface conforming to the semantics of an object storage interface; the file access interface is then subjected to interface semantic conversion to obtain a public index access interface and a hidden index access interface; the public index access interface and the hidden index access interface conform to the semantics of a portable operating system interface.

[0242] Next, based on the public index access interface, query the public index information corresponding to the file index identifier; determine the file upload type of the file corresponding to the file index identifier based on the public index information; when the file upload type is global upload, obtain the file corresponding to the file index identifier based on the public index information, and return the obtained file to the object storage device.

[0243] Optionally, when the file upload type is chunked upload, multiple local hidden index information that have a mapping relationship with the public index information are queried based on the hidden index access interface. File chunks are read according to each local hidden index information, the read file chunks are merged to obtain a merged file, and the merged file is sent back to the object storage device.

[0244] In one embodiment, an application scenario for a data processing method is provided, specifically in the interaction scenario between a file storage system and an object storage system.

[0245] The architecture of traditional file storage systems is as follows: Figure 11 As shown, the file storage system is a distributed file system (CloudHDFS, CHDFS). This file storage system includes a client, a root server (RS), a metadata service (MDS), a proxy server, and a backend metadata storage cluster.

[0246] The client backend provides a POSIX semantic interface, encapsulating file system semantic interfaces for user use; common methods include HCFS and FUSE. The client's data stream directly accesses object storage. The root server provides management interfaces, including creating and deleting file systems, creating and deleting mount points, and configuring system properties. It also ensures balanced distribution of file systems across different metadata servers. High availability and master-slave disaster recovery are achieved through Raft election.

[0247] The proxy server provides metadata interfaces, including creating and deleting file handles and updating metadata. Clients access the proxy server directly for metadata, and the proxy server performs protocol conversion and forwards the requests to the metadata server where the file system resides. The metadata server achieves high availability and master-slave disaster recovery through Raft election, maintains a directory tree structure, and enables efficient access and manipulation of metadata information. The backend metadata storage cluster (DB Cluster) can be of any type of database and can be expanded into a key-value storage structure.

[0248] The interactions between devices in a file storage system are as follows:

[0249] Metadata servers store metadata information in a DB / KV cluster, maintain their own directory tree and metadata information, and provide POSIX semantic interfaces to the outside world. Through Raft or election, they can maintain state machines for multiple file systems, while simultaneously maintaining various metadata structure mappings to ensure strong consistency and correctness of metadata.

[0250] The root directory server provides a management interface, assigning user-created file systems and their configured attributes to designated MDSs. It also records file system routing information for user access.

[0251] The proxy server provides a metadata access interface to the outside world, and routes user metadata requests to the MDS recorded by the root server.

[0252] The client side encapsulates the POSIX semantic interface provided by the proxy server to access file system data and metadata.

[0253] The metadata server maintains the directory tree and the metadata block information corresponding to the index nodes in the directory tree.

[0254] The architecture of the file storage system and object storage system in this embodiment is as follows: Figure 12 As shown, an escaping layer is added to the file storage system. The escaping layer has an escaping device. The object storage system includes an object storage client, an object storage access layer, an object storage layer, and an accelerator. The object storage client is the object storage device.

[0255] Common interfaces in object storage systems include regular upload interfaces, chunked upload interfaces, download interfaces, HTTP Range download interfaces, file deletion interfaces, Head interfaces, and List interfaces. These interfaces conform to the semantics of object storage interfaces. File upload interfaces include regular upload interfaces and chunked upload interfaces. File access interfaces include download interfaces, HTTP Range download interfaces, Head interfaces, and List interfaces.

[0256] File chunked upload is a combined operation. When uploading files in a chunked manner within an object storage system, the upload identifier (uploadId) is first obtained through the InitMPU interface. Then, each file chunk is uploaded using the uploadId and its respective chunk ID. Finally, the CompleteMPU interface is called to complete the chunked upload. During this process, the AbortMPU interface can be used to cancel the current chunked upload; uploaded file chunks become fragmented files, and their uploadIds become invalid. The visibility of an object storage system differs from that of a file storage system. In an object storage system, the file's index information is not visible during the upload process but becomes visible after the upload is complete. In contrast, the file storage system's index information is visible in real-time during the upload process. In this embodiment, by adding three namespace branches—Tmp, Part, and User_Data—to the preset directory tree, the file storage system can maintain its own visibility while remaining compatible with the visibility of the object storage system.

[0257] Object storage systems offer two file upload types: normal upload and chunked upload. Normal upload, also known as global upload, directly uploads the corresponding data via the HTTP RESTful PUT method through the Body. Chunked upload (MPU) obtains the UploadId of the chunked upload task through the InitMPU interface. Users can then customize the chunk size, dividing the data into file chunks of the same or different sizes. Each file chunk is uploaded via the HTTP RESTful PUT method, setting a chunk identifier (partNumber) and corresponding upload identifier parameters (uploadId params). Each upload part request returns the corresponding part's ETag. Finally, the complete MPU interface is called to upload the chunk number (partNumber) and partEtag for each part to the backend. After successful backend verification, all file chunks are obtained. If the upload fails, users can delete the uploaded file chunks through the lifecycle method or the AbortMPU interface. Figure 13 As shown, once the first three file chunks have been successfully uploaded, the upload can be completed by calling the Complete MPU interface. If the upload of the fourth file chunk fails and the user wants to terminate the upload, all file chunks can be deleted by calling the abortMPU interface.

[0258] In an object storage system, a normal file upload is a single atomic interface call, while a chunked file upload involves multiple non-atomic interface calls. In this embodiment, by adding an escape layer to the file storage system and adding three namespace branches—Tmp, Part, and User_Data—to the preset directory tree, the visibility of the file storage system is maintained while ensuring compatibility with the visibility and atomicity of the object storage system.

[0259] Specifically, in the file storage system of this embodiment, an escape layer is added, and three namespace branches—Tmp, Part, and User_Data—are added to the preset directory tree. The Tmp branch is the globally hidden index branch, the Part branch is the locally hidden index branch, and the User_Data branch is the publicly accessible index branch. The User_Data branch is used to store the user's actual data structure, and the _Tmp branch is added to store temporary data. Under the Tmp branch, hashing is performed sequentially to ensure that temporary data is evenly distributed across different hash directories, thus dispersing the distribution of temporary data and preventing all data from being concentrated in the _Tmp directory, facilitating data partitioning.

[0260] In a typical upload scenario, the file upload interface is the same as the standard upload interface. When a user initiates an upload command for a target file using an object storage client, the object storage access layer receives the command and calls the standard upload interface to send the target file to the escape layer in the file storage system.

[0261] The escaping layer in the file storage system receives the target file and escapes the ordinary upload interface, which conforms to the semantics of the object storage interface, into the POSIX-compliant createTmpFile and renameTmpFile interfaces for the metadata server to call. The createTmpFile interface is the index creation interface, and the renameTmpFile interface is the index public interface.

[0262] The escape layer feeds back the target file, the createTmpFile interface, and the renameTmpFile interface to the access layer of the file storage system, which then feeds back the information to the metadata server.

[0263] The metadata server calls the createTmpFile interface to generate an a.txt file located in the Hash directory within the Tmp branch, which contains globally hidden index information. The a.txt file records the target file's metadata and its storage path in the file storage system. The target file is then stored in the file storage system's database according to the storage path.

[0264] When object storage clients concurrently upload the same file, the metadata server simultaneously generates another file, a.txt_version2, located in the same hash directory within the Tmp branch. After either a.txt or a.txt_version2 is stored in the Tmp branch via the createTmpFile interface, the metadata server calls the RenameTmpFile interface to rename either file to the User_Data branch. The renameTmpFile interface involves two steps: first, recursively creating the parent directory; then, whichever file (a.txt or a.txt_version2) modifies the parent directory and moves to the User_Data branch first is considered successfully uploaded, while the other is considered to have failed. Files that fail to upload are treated as redundant data and handled asynchronously by the memory reclamation server in the file storage system. The User_Data branch is public and visible to users, while the Tmp branch is hidden and invisible to users. Only after the file upload is complete will the a.txt file or a.txt_version2 file be moved to the User_Data branch for public access. This enables the file storage system to keep the index information hidden during the file upload process and make it public after the upload is complete, thus making the file storage system compatible with the visibility of object storage systems for ordinary upload types.

[0265] In terms of atomicity, if the a.txt file fails to be generated, fails to modify the parent directory, or fails to be moved, the relevant redundant data will be deleted. The a.txt file remaining in the Tmp branch will be the same as a.txt_version2, waiting for the memory reclamation server to automatically clean it up according to the asynchronous task.

[0266] In a chunked upload scenario, the file upload interface includes an identifier retrieval interface and a chunked upload interface. The identifier retrieval interface is the Init MPU interface, and the chunked upload interface is the UploadPart interface. When a user initiates a chunked upload command for a target file using an object storage client, the object storage access layer receives the command and calls the identifier retrieval interface to send the chunked upload command to the escaping layer in the file storage system.

[0267] The escaping layer in the file storage system receives the chunked upload command and escapes the identifier retrieval interface, which conforms to the semantics of the object storage interface, into the POSIX-compliant createTmpFile and renameTmpFile interfaces for the metadata server to use. The escaping layer then feeds back the target file, the createTmpFile interface, and the renameTmpFile interface to the access layer of the file storage system, which in turn feeds them back to the metadata server.

[0268] The metadata server generates an `a.txt` MPU inode file in the `Hash` directory of the `Tmp` branch, indicating that the file type is an MPU inode to distinguish between normal uploads and chunked uploads. The `a.txt` MPU inode file contains globally hidden index information. Similarly, if the target file has concurrent chunked uploads, an MPU type inode, `a.txt._version2`, will also be generated in the corresponding `Hash` directory. Finally, the metadata server randomly generates a unique `uploadId` and persists the relationship between this `uploadId` and the `a.txt` MPU inode file. In other words, one `uploadId` is used to identify only one file upload, thus establishing a mapping between the `uploadId` and the `a.txt` MPU inode file.

[0269] Next, the object storage client, based on the UploadId returned by the metadata server and its own file chunks of the target file, calls the UploadPart interface to upload the file chunks. The metadata server creates a Hash directory in the Part branch to store the a.txt.Part files corresponding to different file chunks. The a.txt.Part files are the locally hidden index information of the file chunks. Furthermore, the metadata server records the mapping relationship between the corresponding a.txt MPU inode file and all a.txt.Part files to preserve the mapping relationship between the uploadId and the a.txt.Part file of each file chunk, facilitating indexing when reading data. In other words, a mapping relationship is established between the a.txt MPU inode file in the Tmp branch and all a.txt.Part files in the Part branch.

[0270] Therefore, all file chunks uploaded by the object storage client via UploadId have an independent index in the Hash directory of the Part branch. The Part branch is also invisible to the user, but its metadata content can be returned via the listParts interface (metadata content includes size, etag, partNumber, etc.). Files in the Part branch can participate in lifecycle fragment deletion. To avoid namespace write expansion, concurrent writes with the same UploadId and PartNumber are controlled by adding a PartNumber lock to the parent directory of a.txt.Part.

[0271] Regarding atomicity, all operations on the Part branch are atomic, and operations on the Tmp branch are similar to ordinary upload logic and also have atomicity guarantees. Regarding visibility, neither the Tmp nor the Part branch is visible to the user. Only after the object storage client calls the Complete MPU interface, and the metadata server establishes the mapping relationship between the a.txt MPU inode file in the Tmp branch and all a.txt.Part files in the Part branch, does the metadata server create public index information under the User_Data branch by calling the RenameTmpFile interface. This User_Data branch is then visible to the user.

[0272] When an object storage client calls the Abort MPU interface to terminate a chunked upload, the parts file corresponding to the UploadId in the Part branch is considered a fragmented file and is handed over to the memory reclamation service for deletion through a marked-for-deletion process. Similarly, the corresponding MPU inode a.txt in the Tmp branch, or the MPU inode a.txt_version2 that failed due to concurrent uploads, is also handed over to the memory reclamation service for deletion. The memory reclamation service follows this procedure: first deletes the underlying Block data, then deletes the Parts file corresponding to the Block, and finally deletes the MPU inode file under the Tmp branch.

[0273] Whether it's a regular upload or a chunked upload, since the file storage system has its own metadata management layer, file deletion can be performed using a mark-and-delete approach. This involves marking the inodes in the User_Data branch and having them asynchronously deleted by a bypass memory reclamation service. The difference is that a regular upload only requires deleting the underlying Block data and the corresponding Inode record. However, for MPU files uploaded in chunks, the underlying Block must be deleted first, followed by the Parts file in the Part branch, then the corresponding UploadId index, and finally the inodes in the User_Data branch. Object storage's coverage principle refers to allowing clients to overwrite uploaded files with the same name. This is achieved by using transactions in the final RenameTmpFile step, first marking the deletion of existing Inodes in the User_Data branch, and then creating new Inodes with the same name to ensure coverage. Therefore, regardless of whether a regular upload or a chunked upload is used, the Tmp branch, Part branch, and User_Data branch can be added in this embodiment to satisfy the corresponding object storage semantics.

[0274] In file reading scenarios, files uploaded via regular uploads are only stored in the User_Data branch. The file system backend maintains a complete directory tree structure and retains the metadata information of all inodes in the tree. Therefore, files uploaded via regular uploads can be directly indexed using the inodeId to access the corresponding block and inode information, thus allowing the file to be read.

[0275] Files uploaded in chunks are also stored in the inode under the User_Data branch, and the mapping relationship between the MPUInode file and the parts file in the Part branch is also saved, so the file can be read according to the corresponding index information.

[0276] Here, file reading via chunked upload utilizes two HTTP RESTful API interfaces: List MPUs and ListParts. List MPUs provides public index access, while ListParts provides hidden index access. List MPUs returns all UploadIds and their corresponding index information, while ListParts returns the metadata mapping index between UploadIds and all corresponding Parts. This allows the reading of each file chunk, which is then merged into a single file and sent back to the object storage client.

[0277] In this embodiment, the User_Data branch maintains a complete metadata directory tree relationship. Each Inode node can be retrieved using InodeId or a secondary index (Parent inode id, name). The corresponding Inode metadata can be obtained through storage path resolution and returned to the HEAD interface. The directory traversal List interface obtains the inodeId A of the target directory through path resolution, and then retrieves all files and 'directories' under the directory in batches according to the format Parent inode id = A.

[0278] In this embodiment, a namespace abstraction based on POSIX semantic features is used to support the basic interface for object storage semantics. This fully leverages the tree-like metadata structure of the file storage system, enabling it to possess the atomicity, visibility, and coverage of object storage in high-concurrency scenarios. Externally, this method provides an object storage interface based on the file system; internally, it breaks down the directory tree structure to increase the scalability of the file storage system when supporting object storage semantics.

[0279] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0280] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data processing apparatus embodiments provided below can be found in the limitations of the data processing method described above, and will not be repeated here.

[0281] In one embodiment, such as Figure 14 As shown, a data processing apparatus 1400 is provided, applied to an index server of a file storage system, including: a receiving module 1402, a conversion module 1404, a creation module 1406, and a publishing module 1408, wherein:

[0282] The receiving module 1402 is used to receive the target file uploaded by the object storage device in the object storage system based on the file upload interface, wherein the file upload interface conforms to the object storage interface semantics of the object storage system.

[0283] The conversion module 1404 is used to perform interface semantic conversion on the file upload interface to obtain the file storage control interface; the file storage control interface conforms to the interface semantics of the portable operating system of the file storage system.

[0284] Create module 1406, which is used to create index information for the target file based on the file storage control interface, store the target file according to the index information, and hide the index information during the storage process of the target file.

[0285] The public module 1408 is used to disclose index information based on the file storage control interface when the conditions for disclosing the index information of the target file are met.

[0286] In this embodiment, the file storage system possesses portable operating system interface semantics, and the object storage system possesses object storage interface semantics. The index server of the file storage system receives target files uploaded by object storage devices in the object storage system via a file upload interface. It performs interface semantic conversion on the file upload interface conforming to object storage interface semantics, transforming it into a file storage control interface conforming to portable operating system interface semantics. This interface semantic conversion enables the file storage system to support the object storage system's interface, allowing the object storage device to request the file storage system to upload the target file according to the file upload method provided by the object storage system. Based on the file storage control interface, index information for the target file is created. The target file is stored according to the index information, and the index information is hidden during the storage process. When the conditions for publicizing the target file's index information are met, the index information is made public based on the file storage control interface. Thus, based on the converted interface, the file storage system can achieve the functionality of the object storage system hiding index information during file upload and making the index information public after the upload is complete, thereby ensuring that the file storage system is compatible with the file upload functionality unique to the object storage system.

[0287] In one embodiment, the file storage control interface includes an index creation interface and an index public interface, and the index information includes hidden index information and public index information; the creation module 1406 is also used to create hidden index information for the target file based on the index creation interface, and store the target file according to the hidden index information;

[0288] The public module 1408 is also used to adjust the hidden index information of the target file to public index information based on the index public interface when the conditions for publicizing the index information of the target file are met.

[0289] In this embodiment, following the portable operating system interface semantics of the file storage system, the file upload interface is converted into an index creation interface and an index disclosure interface. Hidden index information for the target file is created based on the index creation interface, and the target file is stored according to this hidden index information, thus hiding the file's index information during the file upload process. When the conditions for making the target file's index information public are met, the hidden index information is adjusted to public index information based on the index disclosure interface, making it accessible to users. By converting one file upload interface of the object storage system into two interfaces, the function of hiding and then revealing index information during the file upload process is implemented using two different interfaces, enabling the file storage system to possess the file upload functionality unique to object storage systems.

[0290] In one embodiment, the creation module 1406 is further configured to determine a preset directory structure corresponding to the file storage system, the preset directory structure including a hidden index branch and a public index branch; and based on the index creation interface, to create hidden index information for the target file in the hidden index branch;

[0291] The public module 1408 is also used to generate public index information for the target file in the public index branch based on the hidden index information in the hidden index branch, when the conditions for publicizing the index information of the target file are met.

[0292] In this embodiment, the preset directory structure of the file storage system is divided into a hidden index branch and a public index branch to achieve the hiding of index information during file upload and the public disclosure of index information after upload. Based on the index creation interface, hidden index information for the target file is created in the hidden index branch, and the target file is stored according to the hidden index information, making the index information invisible to the outside world during the upload process. When the conditions for public disclosure of the target file's index information are met, based on the index disclosure interface, public index information for the target file is generated in the public index branch according to the hidden index information in the hidden index branch. Thus, when the file is uploaded to a level where it can be made public, the file index information is made public through the public index branch for users to query. Through the hidden index branch and public index branch of the preset directory structure, the file storage system can maintain the original file upload function and also be compatible with the file upload function of the object storage system, thereby expanding the functionality of the file storage system.

[0293] In one embodiment, the public module 1408 is further configured to, when the conditions for publicizing the index information of the target file are met, determine the parent directory information of the public index branch for the hidden index information based on the index public interface; and, based on the parent directory information, transfer the hidden index information in the hidden index branch to the public index branch to obtain the public index information for the target file.

[0294] In this embodiment, when the conditions for publicizing the index information of the target file are met, the parent directory information is obtained from the public index branch based on the public index interface. Based on the parent directory information, the hidden index information in the hidden index branch is transferred to the public index branch to obtain the public index information for the target file. Thus, the index information is first hidden during the file storage process, and the parent directory information is adjusted to form public index information for users after the public conditions are met.

[0295] In one embodiment, the hidden index branch includes a global hidden index branch and a local hidden index branch, and the hidden index information includes global hidden index information and local hidden index information; the creation module 1406 is further configured to determine the file upload type corresponding to the file upload interface; when the file upload type is chunked upload, based on the index creation interface, global hidden index information for the target file is created in the global hidden index branch, and local hidden index information for each file chunk of the target file is created in the local hidden index branch; each file chunk of the target file is stored according to the local hidden index information of each file chunk of the target file.

[0296] In this embodiment, the preset directory structure of the file storage system is divided into a global hidden index branch, a local hidden index branch, and a public index branch. When the file upload type of the target file is chunked upload, indicating that the object storage device will upload multiple file chunks of the target file, the index server, based on the index creation interface, creates global hidden index information for the target file in the global hidden index branch to record the global index of the target file, and creates local hidden index information for each file chunk of the target file in the local hidden index branch to record the chunk index of each file chunk. Each file chunk of the target file is stored according to the local hidden index information of each file chunk, so that the target file can be uploaded to the file storage system in the form of file chunks, making the file storage system compatible with the chunked upload function of the object storage system.

[0297] In one embodiment, the public module 1408 is further configured to, when the conditions for publicizing the index information of the target file are met, establish a mapping relationship between the global hidden index information in the global hidden index branch and the local hidden index information of each file block in the local hidden index branch; and, based on the index public interface, transfer the global hidden index information after establishing the mapping relationship from the global hidden index branch to the public index branch to obtain the public index information for the target file in the public index branch.

[0298] In this embodiment, when the condition for publicizing the index information of the target file is met, a mapping relationship is established between the global hidden index information in the global hidden index branch and the local hidden index information of each file block in the local hidden index branch. This mapping relationship determines the index information of each file block, maintaining the integrity of the target file stored in blocks. Based on the index public interface, the global hidden index information after the mapping relationship is established is transferred from the global hidden index branch to the public index branch. This obtains the public index information for the target file in the public index branch, ensuring that the target file uploaded in blocks only exposes the global index and hides the block indexes, guaranteeing the security and integrity of file block storage. Furthermore, this enables the file storage system to possess the functionality of an object storage system in a block upload scenario, exposing the global index and hiding the block indexes, thus achieving compatibility between the file storage system and the object storage system.

[0299] In one embodiment, the file upload interface includes a chunked upload interface and an upload completion interface, and the file storage control interface further includes a relationship establishment interface. The index information disclosure condition includes receiving an upload completion instruction triggered by the object storage device. The creation module 1406 is further configured to, when the file upload type is chunked upload, perform interface semantic conversion on the chunked upload interface to obtain an index creation interface and an index disclosure interface; in response to the upload completion instruction of the target file triggered by the object storage device based on the upload completion interface, convert the upload completion interface into a relationship establishment interface; and based on the relationship establishment interface, establish a mapping relationship between the global hidden index information in the global hidden index branch and the local hidden index information of each file chunk in the local hidden index branch.

[0300] In this embodiment, in response to the upload completion command of the target file triggered by the object storage device based on the upload completion interface, the upload completion interface, which conforms to the semantics of the object storage interface, is converted into a relationship establishment interface, which conforms to the semantics of the portable operating system interface. This interface is then invoked in the file storage system. This interface semantic conversion enables the file storage system to support the object storage system's interface, allowing the object storage device to request the file storage system to perform the upload of the target file according to the chunked upload method available in the object storage system. Based on the relationship establishment interface, a mapping relationship is established between the global hidden index information in the global hidden index branch and the local hidden index information of each file chunk in the local hidden index branch. This determines the correspondence between the global index of the target file and the chunk indexes of each file chunk, ensuring the integrity and accuracy of the target file. Furthermore, this allows the file storage system to be compatible with the file chunked upload function unique to the object storage system.

[0301] In one embodiment, the device further includes:

[0302] The deletion module receives file deletion instructions triggered by the object storage device based on the file deletion interface and converts the file deletion interface into an index marking interface. The file deletion interface conforms to the semantics of the object storage interface, and the index marking interface conforms to the semantics of the portable operating system interface. Based on the index marking interface, the public index information of the target file is marked. The marking is used to instruct the memory reclamation server of the file storage system to delete the public index information of the target file and the target file.

[0303] In this embodiment, a file deletion command triggered by the object storage device based on the file deletion interface is received. The file deletion interface, conforming to the semantics of the object storage interface, is converted into an index marking interface, conforming to the semantics of the portable operating system interface. This semantic conversion enables the file storage system to support the file deletion interface of the object storage system. The public index information of the target file is marked using the index marking interface, thereby instructing the memory reclamation server of the file storage system to delete the public index information and the target file itself. Furthermore, for target files uploaded using the file upload function of the object storage system, deletion is still performed using the file deletion function of the object storage system. This ensures that the file storage system is compatible with both the file upload and file deletion functions of the object storage system, effectively improving its compatibility.

[0304] In one embodiment, the index information includes global hidden index information, local hidden index information, and public index information of the target file, and the preset directory structure corresponding to the file storage system includes a global hidden index branch, a local hidden index branch, and a public index branch; the hidden index information is located in the global hidden index branch, the local hidden index information is located in the local hidden index branch, and the public index information is located in the public index branch;

[0305] The deletion module is also used to mark the public index information of the target file in the public index branch based on the index marking interface when the file upload type of the target file is chunked upload. The marking is used to instruct the memory reclamation server to delete each stored file chunk of the target file, the local hidden index information in the local hidden index branch, the global hidden index information in the global hidden index branch, and the public index information in the public index branch in order of priority.

[0306] In this embodiment, when the file upload type of the target file is chunked upload, the public index information of the target file in the public index branch is marked based on the index marking interface. The stored file chunks, the local hidden index information of the file chunks, the global hidden index information of the target file, and the public index information are deleted in order of priority. This can effectively delete all information related to the target file and avoid omissions.

[0307] In one embodiment, the device further includes:

[0308] The receiving module 1402 is used to receive the file index identifier sent by the object storage device based on the file access interface, wherein the file access interface conforms to the semantics of the object storage interface.

[0309] The conversion module 1404 is used to perform interface semantic conversion on the file access interface to obtain the index access interface, which conforms to the interface semantics of the portable operating system.

[0310] The query module is used to query index information corresponding to file index identifiers based on the index access interface.

[0311] The feedback module is used to retrieve the file corresponding to the file index identifier based on the queried index information and to return the retrieved file to the object storage device.

[0312] In this embodiment, the receiving object storage device sends a file index identifier based on the file access interface. The file access interface, conforming to the semantics of the object storage interface, is converted into an index access interface conforming to the semantics of the portable operating system interface, thereby enabling support for the file access interface of the object storage system. Based on the index access interface, the index information corresponding to the file index identifier is queried. The file corresponding to the file index identifier is then retrieved based on the queried index information, allowing the retrieved file to be returned to the object storage device. This enables interaction between devices in the file storage system and the object storage system, and facilitates file upload and retrieval between the file storage system and the object storage system.

[0313] In one embodiment, the index access interface includes a global index access interface and a local index access interface, and the index information includes public index information and multiple local hidden index information that have a mapping relationship with the public index information; the query module is also used to query the public index information corresponding to the file index identifier based on the public index access interface;

[0314] The feedback module is also used to query multiple local hidden index information that have a mapping relationship with the public index information based on the hidden index access interface; read file blocks according to each local hidden index information, merge the read file blocks, and feed back the merged file to the object storage device.

[0315] In this embodiment, the file access interface conforming to the semantics of the object storage interface is converted into a public index access interface and a hidden index access interface conforming to the semantics of the portable operating system interface, thereby enabling support for the file access interface of the object storage system. Based on the public index access interface, public index information corresponding to the file index identifier is queried. Based on the hidden index access interface, multiple local hidden index information that have a mapping relationship with the public index information are queried to find the storage path of each file block, thus reading the file block according to the storage path. The read file blocks are merged, and the merged file is fed back to the object storage device. This achieves compatibility between the file storage system and the file block reading function of the object storage system, enabling the file storage system to adapt not only to file reading on devices conforming to the semantics of the portable operating system interface, but also to file reading on object storage devices conforming to the semantics of the object storage interface.

[0316] Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0317] In one embodiment, a computer device is provided, which may be a terminal or a server. Taking a server as an example, its internal structure diagram may be as follows: Figure 15 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a data processing method.

[0318] Those skilled in the art will understand that Figure 15 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0319] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0320] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0321] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0322] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0323] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0324] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0325] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A data processing method, characterized in that, An index server applied to a file storage system, the method comprising: Receive a target file uploaded by an object storage device in the object storage system via a file upload interface, wherein the file upload interface conforms to the object storage interface semantics of the object storage system. The file upload interface is semantically transformed to obtain a file storage control interface; the file storage control interface conforms to the interface semantics of the portable operating system of the file storage system. An index information for the target file is created based on the file storage control interface, the target file is stored according to the index information, and the index information is hidden during the storage of the target file. When the conditions for disclosing the index information of the target file are met, the index information is disclosed based on the file storage control interface.

2. The method according to claim 1, characterized in that, The file storage control interface includes an index creation interface and an index public interface. The index information includes hidden index information and public index information. The process of creating index information for the target file based on the file storage control interface, storing the target file according to the index information, and hiding the index information during the storage of the target file includes: Hidden index information for the target file is created based on the index creation interface, and the target file is stored according to the hidden index information; The step of disclosing the index information based on the file storage control interface when the index information disclosure conditions of the target file are met includes: When the conditions for making the index information of the target file public are met, the hidden index information of the target file is adjusted to public index information based on the index public interface.

3. The method according to claim 2, characterized in that, The process of creating hidden index information for the target file based on the index creation interface includes: Determine the preset directory structure corresponding to the file storage system, wherein the preset directory structure includes hidden index branches and public index branches; Based on the index creation interface, hidden index information for the target file is created in the hidden index branch; When the conditions for making the index information of the target file public are met, adjusting the hidden index information of the target file to public index information based on the index public interface includes: When the conditions for publicizing the index information of the target file are met, based on the index publicizing interface, public index information for the target file in the public indexing branch is generated according to the hidden index information in the hidden indexing branch.

4. The method according to claim 3, characterized in that, When the index information disclosure conditions of the target file are met, based on the index disclosure interface, and according to the hidden index information in the hidden index branch, the public index information for the target file in the public index branch is generated, including: When the conditions for publicizing the index information of the target file are met, the parent directory information of the public index branch for the hidden index information is determined based on the index public interface. Based on the parent directory information, the hidden index information in the hidden index branch is transferred to the public index branch to obtain the public index information for the target file.

5. The method according to claim 3, characterized in that, The hidden index branch includes a global hidden index branch and a local hidden index branch, and the hidden index information includes global hidden index information and local hidden index information; the step of creating hidden index information for the target file in the hidden index branch based on the index creation interface includes: Determine the file upload type corresponding to the file upload interface; When the file upload type is chunked upload, based on the index creation interface, global hidden index information for the target file is created in the global hidden index branch, and local hidden index information for each file chunk of the target file is created in the local hidden index branch; The step of storing the target file according to the hidden index information includes: Each file block of the target file is stored according to the local hidden index information of each file block of the target file.

6. The method according to claim 5, characterized in that, When the index information disclosure conditions of the target file are met, based on the index disclosure interface, and according to the hidden index information in the hidden index branch, the public index information for the target file in the public index branch is generated, including: When the conditions for public access to the index information of the target file are met, a mapping relationship is established between the global hidden index information in the global hidden index branch and the local hidden index information of each file block in the local hidden index branch; Based on the index public interface, the globally hidden index information after the mapping relationship is established is transferred from the globally hidden index branch to the public index branch, thereby obtaining the public index information for the target file in the public index branch.

7. The method according to claim 6, characterized in that, The file upload interface includes a chunked upload interface and an upload end interface; the file storage control interface also includes a relationship establishment interface; and the index information disclosure condition includes receiving an upload completion instruction triggered by the object storage device. The step of performing interface semantic conversion on the file upload interface to obtain the file storage control interface includes: When the file upload type is chunked upload, the chunked upload interface is semantically converted to obtain the index creation interface and the index public interface. When the index information disclosure condition of the target file is met, establishing a mapping relationship between the global hidden index information in the global hidden index branch and the local hidden index information of each file block in the local hidden index branch includes: In response to the upload completion instruction of the target file triggered by the object storage device based on the upload end interface, the upload end interface is converted into a relationship establishment interface; Based on the aforementioned relationship, an interface is established to create a mapping relationship between the global hidden index information in the global hidden index branch and the local hidden index information of each file block in the local hidden index branch.

8. The method according to claim 1, characterized in that, The method further includes: The system receives a file deletion command triggered by the object storage device based on the file deletion interface, and converts the file deletion interface into an index tagging interface; the file deletion interface conforms to the semantics of the object storage interface, and the index tagging interface conforms to the semantics of the portable operating system interface. The public index information of the target file is marked based on the index marking interface. The mark is used to instruct the memory reclamation server of the file storage system to delete the public index information of the target file and the target file.

9. The method according to claim 8, characterized in that, The index information includes the global hidden index information, local hidden index information and public index information of the target file, and the preset directory structure corresponding to the file storage system includes a global hidden index branch, a local hidden index branch and a public index branch; The hidden index information is located in the global hidden index branch, the local hidden index information is located in the local hidden index branch, and the public index information is located in the public index branch; The step of marking the public index information of the target file based on the index marking interface includes: When the file upload type of the target file is chunked upload, the public index information of the target file in the public index branch is marked based on the index marking interface; The marker is used to instruct the memory reclamation server to delete each stored file block of the target file, the local hidden index information in the local hidden index branch, the global hidden index information in the global hidden index branch, and the public index information in the public index branch in order of priority.

10. The method according to any one of claims 1 to 9, characterized in that, The method further includes: Receive a file index identifier sent by the object storage device based on a file access interface, wherein the file access interface conforms to the semantics of the object storage interface; The file access interface is subjected to interface semantic conversion to obtain an index access interface, which conforms to the interface semantics of the portable operating system. Based on the index access interface, query the index information corresponding to the file index identifier; Based on the retrieved index information, the file corresponding to the file index identifier is obtained, and the obtained file is returned to the object storage device.

11. The method according to claim 10, characterized in that, The index access interface includes a public index access interface and a hidden index access interface, and the index information includes public index information and multiple local hidden index information that have a mapping relationship with the public index information; The step of querying the public index information corresponding to the file index identifier based on the index access interface includes: Based on the public index access interface, query the public index information corresponding to the file index identifier; The step of retrieving the file corresponding to the file index identifier based on the queried index information and feeding back the retrieved file to the object storage device includes: Based on the hidden index access interface, query multiple local hidden index information that have a mapping relationship with the public index information; The file blocks are read according to each of the local hidden index information, the read file blocks are merged, and the merged file is sent back to the object storage device.

12. A data processing apparatus, characterized in that, An index server for use in a file storage system, the device comprising: The receiving module is used to receive a target file uploaded by an object storage device in the object storage system based on a file upload interface, wherein the file upload interface conforms to the object storage interface semantics of the object storage system. The conversion module is used to perform interface semantic conversion on the file upload interface to obtain a file storage control interface; the file storage control interface conforms to the portable operating system interface semantics of the file storage system. A creation module is used to create index information for the target file based on the file storage control interface, store the target file according to the index information, and hide the index information during the storage process of the target file; The public module is used to publicly disclose the index information based on the file storage control interface when the conditions for disclosing the index information of the target file are met.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images