Data access method, device and system

By assigning unique file identification codes and using a file index table, the method enhances access efficiency and compatibility for file applications accessing object storage service nodes.

CN119201850APending Publication Date: 2024-12-27HUAWEI CLOUD COMPUTING TECHNOLOGIES CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202311791307.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-06-25
Filing Date
2023-12-22
Publication Date
2024-12-27

AI Technical Summary

Technical Problem

Object storage service nodes require full file paths for file access, leading to low access efficiency when interacting with file applications.

Method used

Assign a globally unique file identification code to files and directories in object storage, allowing access without parsing the full file path, and utilize a file index table to manage metadata and access methods.

Benefits of technology

Improves access efficiency by enabling direct file access through identification codes, reducing the need to traverse directory structures and enhancing compatibility with file protocols.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119201850A_ABST
    Figure CN119201850A_ABST
Patent Text Reader

Abstract

The invention provides a data access method and related equipment, the method is applied to an object storage service node, the object storage service node comprises a hierarchical namespace, the hierarchical namespace is used for mapping at least one object into a file or a directory on a directory tree, and the method comprises the following steps: receiving a file creation request, the file creation request carries a first file name and a first file identification code of a parent directory of a first file; obtaining metadata of a parent directory of the first file corresponding to the first file identification code; according to metadata of a parent directory of the first file, the first file is created under the parent directory, a second file identification code is distributed to the first file, and the first file is a file or a directory on a directory tree; and recording a corresponding relation between the second file identification code and the first file name into a file index table. Therefore, the user uses the file identification code to access the file or the catalog stored by the object storage service node, the file full path does not need to be analyzed layer by layer, and the access efficiency of the user is improved.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application claims priority to a Chinese patent application filed with the State Intellectual Property Office of China on June 25, 2023, with application number 202310752941.X, and priority to a Chinese patent application entitled “A method, device and other equipment for implementing file semantics in object storage”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of storage technology, and in particular to a data access method, device and system. Background Art

[0003] File storage is to store data in the form of files on the file storage service node. Users can access the data on the file storage service node through the file interface provided by the file storage service node. Object storage is to store data in the form of objects on the object storage service node. Users can access the data on the object storage service node through the object interface provided by the object storage service node.

[0004] Currently, in order to be compatible with some applications that can only access data through file interfaces (referred to as file applications), object storage service nodes provide file services based on object storage, that is, file applications can access data on object storage service nodes through the file interfaces provided by object storage service nodes. However, since object storage service nodes use the full file path to identify a file, when accessing files stored by object storage service nodes, file application clients also need to use the full file path to access, resulting in low access efficiency. Summary of the invention

[0005] The present application provides a data access method, device and other equipment, which allocates a file identification code to a file or directory stored in an object storage service node through a file identification code allocator. The file identification code is globally unique and is used to identify the file or directory. Users use the file identification code to access files or directories stored in the object storage service node without having to parse the full path of the file layer by layer, thereby improving the user's access efficiency.

[0006] In a first aspect, a data access method is provided, which is applied to an object storage service node, wherein at least one object is stored in a storage bucket of the object storage service node, and the object storage service node includes a hierarchical namespace, wherein the hierarchical namespace is used to map at least one object to a file or directory on a directory tree, and the method includes: receiving a file creation request, the file creation request carrying a first file name and a first file identification code of a parent directory of the first file; obtaining metadata of the parent directory of the first file corresponding to the first file identification code; creating a first file under the parent directory according to the metadata of the parent directory of the first file, and assigning a second file identification code to the first file, wherein the first file is a file or directory on the directory tree; and recording the correspondence between the second file identification code and the first file name in a file index table.

[0007] In this embodiment, the object storage service node adds a file identification code distributor and a file index table. When a user creates a file or directory, the object storage service node generates a globally unique file identification code for the created file or directory through the file identification code distributor. The file identification code is the basic attribute of the file or directory and will not change once it is created. At the same time, when the file identification code is generated, the correspondence between the file identification code and the file information will be recorded in the file index table. Through the file index table, the user can query the metadata information of the file or directory through the file identification code. In this way, the user uses the file identification code to access the file or directory stored in the object storage service node without having to parse the full path of the file layer by layer, thereby improving the user's access efficiency.

[0008] In combination with the first aspect, in certain implementations of the first aspect, a file write request is received, the file write request carries a first file to be written and a second file identification code; a first file name corresponding to the second file identification code is obtained from a file index table, the first file name is used to indicate metadata of the first file; metadata of the first file is obtained, the metadata of the first file includes a data layout of the first file; the first file is written in a first storage unit according to the data layout of the first file; and the metadata of the first file is updated.

[0009] In this implementation, the file storage client uses the file identification code to write the file to the object storage service node. The object storage service node obtains the metadata information corresponding to the file identification code from the file index table, and then writes the file according to the data layout of the file. In this way, there is no need to parse the full path of the file layer by layer, thereby improving data access efficiency.

[0010] In combination with the first aspect, in certain implementations of the first aspect, the metadata of the first file includes one or more of the following: the file size of the first file, the creation time of the first file, the access rights of the first file, and the data layout of the first file, wherein the data layout of the first file includes an identifier of a first storage unit storing the first file, an offset of the first file in the first storage unit, and a length of the first file in the first storage unit.

[0011] In combination with the first aspect, in certain implementations of the first aspect, a first file read request is received, the first file read request carries a second file identification code; a first file name corresponding to the second file identification code is obtained from a file index table; metadata of the first file is obtained, the metadata of the first file includes a data layout of the first file; and the first file is read from a first storage unit according to the data layout of the first file.

[0012] In this implementation, after the object storage service node writes a file, the user can directly read the data with the file identification code, obtain the metadata information of the file based on the file identification code, and read the file based on the data layout of the file, without having to parse the full path of the file layer by layer, thereby improving data access efficiency.

[0013] In combination with the first aspect, in certain implementations of the first aspect, a second file read request is received, the second file read request carrying an access path to the first file; the access path to the first file is parsed through a hierarchical namespace to obtain a second file identification code for the first file; the first file name corresponding to the second file identification code is obtained from a file index table; metadata of the first file is obtained; and the first file is read from a first storage unit according to a data layout of the first file.

[0014] In this implementation, when the file storage client uses the full file path to access the object storage service node to read a file, the object storage service node can recognize two access methods: the file identification code or the full file path, and is still compatible with accessing the file using the full file path. It parses the full file path through the hierarchical namespace to obtain the file identification code of the file, and then reads the file based on the file identification code.

[0015] In combination with the first aspect, in certain implementations of the first aspect, a third file read request is received, the third file read request includes a file handle; a second file identification code in the file handle is identified through a gateway; a first file name corresponding to the second file identification code is obtained from a file index table; metadata of the first file is obtained; and the first file is read from a first storage unit according to a data layout of the first file.

[0016] In this implementation, the file storage client reads files from the object storage service node based on the file protocol, and can achieve the conversion of the file protocol to the object protocol that the object storage service node can recognize through the gateway. Even if the file client uses the file protocol, it can still access the files stored in the object storage service node based on the file identification code, which improves the data access efficiency. In addition, using the file identification code as the access identifier makes it easier to be compatible with the file protocol.

[0017] In a second aspect, a data access method is provided, which is applied to a file storage client, the file storage client accesses an object storage service node through a file interface, at least one object is stored in a storage bucket of the object storage service node, the object storage service node includes a hierarchical namespace, the hierarchical namespace is used to map at least one object to a file or directory on a directory tree, the method includes: receiving a file creation instruction for a first file sent by an application, the first file is a file or directory on the directory tree; obtaining a first file identification code of a parent directory of the first file from a mapping relationship table; sending a file creation request to the object storage service node, the file creation request carries the first file name and the first file identification code; receiving a second file identification code assigned by the object storage service node, and recording the correspondence between the second file identification code and the first file name in the mapping relationship table.

[0018] In this embodiment, the file storage client constructs a mapping table between the file name and the file identification code. When interacting with the object storage service node, the file identification code in the mapping table cached by the file storage client can be directly queried, reducing the access volume and access pressure of the object storage service node. At the same time, for the file storage client, the process of building the cache layer by layer of the full path of the file is also reduced, thereby improving data access efficiency.

[0019] In combination with the second aspect, in certain implementations of the second aspect, a file write instruction for a first file sent by an application is received; a second file identification code corresponding to the first file is obtained from a mapping relationship table of the file storage client; a file write request is sent to an object storage service node, the file write request carrying the second file identification code; and a processing result of the file write request is received.

[0020] In combination with the second aspect, in certain implementations of the second aspect, a first file read instruction for a first file sent by an application is received; a second file identification code corresponding to the first file is obtained from a mapping relationship table of the file storage client; a first file read request is sent to an object storage service node, the first file read request carrying the second file identification code; and a processing result of the first file read request is received.

[0021] In a third aspect, a data access device is provided, which is applied to an object storage service node, wherein at least one object is stored in a storage bucket of the object storage service node, and the object storage service node includes a hierarchical namespace, which is used to map at least one object to a file or directory on a directory tree, and the device includes: a receiving module, used to receive a file creation request, the file creation request carries a first file name and a first file identification code of a parent directory of the first file; an acquisition module, used to obtain metadata of the parent directory of the first file corresponding to the first file identification code; a processing module, used to create a first file under the parent directory according to the metadata of the parent directory of the first file, and assign a second file identification code to the first file, wherein the first file is a file or directory on the directory tree; the processing module is also used to record the correspondence between the second file identification code and the first file name in a file index table.

[0022] In a fourth aspect, a data access device is provided, which is applied to a file storage client, the file storage client accesses an object storage service node through a file interface, at least one object is stored in a storage bucket of the object storage service node, the object storage service node includes a hierarchical namespace, the hierarchical namespace is used to map at least one object to a file or directory on a directory tree, the device includes: a receiving module, used to receive a file creation instruction for a first file sent by an application, the first file is a file or directory on the directory tree; an acquisition module, used to obtain a first file identification code of a parent directory of the first file from a mapping relationship table; a sending module, used to send a file creation request to the object storage service node, the file creation request carries the first file name and the first file identification code; the receiving module is also used to receive a second file identification code assigned by the object storage service node, and record the correspondence between the second file identification code and the first file name in the mapping relationship table.

[0023] Regarding the technical principles and beneficial effects of the third and fourth aspects, reference may be made to the relevant descriptions of the first and second aspects, which will not be repeated here.

[0024] In a fifth aspect, the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method of the above-mentioned first aspect or any one of the implementation methods of the first aspect.

[0025] In a sixth aspect, the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method of the above-mentioned second aspect or any one of the implementation methods of the second aspect.

[0026] In a seventh aspect, the present application provides a computer program product comprising instructions, which, when executed by a computer device cluster, enables the computer device cluster to execute the method of the above-mentioned first aspect or any one of the implementation modes of the first aspect.

[0027] In an eighth aspect, the present application provides a computer program product comprising instructions, which, when executed by a computer device cluster, enables the computer device cluster to execute the method of the second aspect or any one of the implementation modes of the second aspect.

[0028] In a ninth aspect, the present application provides a computer-readable storage medium, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method of the above-mentioned first aspect or any one of the implementation methods of the first aspect.

[0029] In a tenth aspect, the present application provides a computer-readable storage medium, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method of the above-mentioned second aspect or any one of the implementation methods of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0031] Figure 1 This is a schematic diagram of a data access scenario involved in this application.

[0032] Figure 2 This is a schematic diagram of the architecture of an object storage client involved in this application.

[0033] Figure 3 This is a schematic diagram of the architecture of an object storage service node involved in this application.

[0034] Figure 4 This is a schematic diagram of a data access architecture involved in this application.

[0035] Figure 5 This is a schematic diagram of a mapping between an object storage space and a file storage space involved in this application.

[0036] Figure 6 This is a schematic diagram of a data access architecture provided by this application.

[0037] Figure 7 This is one of the flow charts of data access provided by this application.

[0038] Figure 8This is a schematic diagram of mapping object storage space and file storage space provided by this application.

[0039] Fig. 9 This is one of the flow charts of data access provided by this application.

[0040] Fig.10 This is a flowchart of a file writing process provided by this application.

[0041] Fig.11 This is one of the flowcharts of a file reading process provided by this application.

[0042] Fig.12 This is one of the flowcharts of a file reading process provided by this application.

[0043] Fig.13 This is one of the flowcharts of a file reading process provided by this application.

[0044] Fig.14 It is a structural diagram of a data access device provided by this application.

[0045] Fig.15 It is a structural diagram of another data access device provided by the present application.

[0046] Fig.16 It is a structural schematic diagram of a computing device provided by this application.

[0047] Fig.17 It is a structural diagram of a computing device cluster provided by this application.

[0048] Fig.18 It is a structural diagram of a network connection between computing devices provided by the present application. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0050] In the description of the embodiments of the present application, unless otherwise specified, the term "and / or" herein is a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " herein indicates that the associated objects are in an or relationship, for example, A / B means A or B. A and B may be singular or plural.

[0051] In the description of the embodiments of the present application, unless otherwise specified, the term "multiple" herein refers to two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0052] In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second", etc. are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit the difference.

[0053] Meanwhile, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0054] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more. In order to make the technical solution provided by the present application clearer, the relevant terms are first explained before the technical solution provided by the present application is specifically described.

[0055] (1) File System: It is the software in the operating system responsible for managing and storing file information. The file system includes the basic file system and several file subsystems. Users can remove and install file subsystems as needed. The target tree provided by the file system includes the root directory, subdirectories, and files under the subdirectories. The root directory or subdirectory can include files and the next-level subdirectories.

[0056] (2) File Storage Service (FSS): It stores data in the form of files on a file storage service node. The file storage service node provides a multi-level tree directory storage method based on the file system. The file storage service node is used to provide users with operations that conform to file semantics (i.e., file services), such as: reading files, writing files, creating files, deleting files, opening files, closing files, renaming files, and other operations. Among them, the file storage service node provides a file interface, and the file interface can indicate operations that conform to file semantics. For example, the file interface can be a portable operating system interface (POSIX), and users can access data on the file storage service node through the POSIX interface. It should be noted that when a user accesses data through a file interface, the data can be called a file.

[0057] (3) Object Storage Service (OBS): It stores data in the form of objects in object storage nodes. Object storage nodes provide a flat storage method based on buckets and objects. All objects in the bucket are at the same logical level, removing the multi-level tree directory structure in the file storage service node. The object storage system is used to provide users with operations that conform to object semantics (i.e., object services), such as: uploading objects, downloading objects, listing each object in a bucket, copying objects, moving objects, renaming objects, deleting objects, deleting each object in a bucket, deleting specified multiple objects, etc. Among them, the object storage node provides an object interface, and the object interface can indicate operations that conform to object semantics. For example, the object interface can be a Representational State Transfer (REST) ​​application programming interface (API), and users can access data on the object storage service node through the REST API interface. It should be noted that when users access data through the object interface, the data can be called an object.

[0058] (4) Object: An object is the basic unit of data storage in an object storage system. An object is actually a collection of a file's data and its related attribute information (metadata). The data uploaded by tenants to OBS is stored in buckets in the form of objects. Objects consist of three parts: key, metadata, and data. The key, or object name, is a UTF-8-encoded character sequence with a length greater than 0 and less than 1024. Each object in a bucket has a unique object key.

[0059] (5) Bucket: A bucket is a container for storing objects in OBS. Object storage provides a flat storage method based on buckets and objects. All objects in a bucket are at the same logical level, eliminating the multi-level tree directory structure in the file system. Each bucket has its own storage category, access rights, region, and other attributes. Tenants can create buckets with different storage categories and access rights, and configure more advanced attributes to meet the storage requirements of different scenarios.

[0060] (6) Object Storage Device (OSD): OSD is the basic storage unit of the object storage system. It is set on the physical disk and is specifically a fixed-size storage space of the physical disk. The object storage system manages the physical disks of multiple computing devices in the form of OSD.

[0061] With the development of storage technology, object storage has been proposed in related technologies to more conveniently store unstructured data such as movies, office documents, and images. Object storage uses a flat address space to store data. There is no hierarchy of directories and files in this address space. Therefore, an object storage system can store a large amount of data.

[0062] In object storage, tenants can log in to the cloud management platform on the public cloud access page with a pre-registered account and password, and after successful login, select and purchase the corresponding public cloud services on the public cloud access page, such as OBS, virtual machine services, container services, etc. For OBS, tenants can further create multiple buckets through the configuration interface or API on the public cloud access page provided by the cloud platform. There is no limit on the total number and size of objects stored in each bucket. Tenants do not need to consider the scalability of data. OBS is a service based on REST API and Hypertext Transport Protocol (HTTP) or Hypertext Transfer Protocol Secure (HTTPS). Tenants can locate bucket resources through Uniform Resource Identifier (URI), which is also called access domain name in this application, or domain name for short. Among them, in OBS, the bucket name is globally unique and cannot be modified, that is, the bucket created by the tenant cannot be the same as other bucket names created by the tenant, nor can it be the same as bucket names created by other tenants.

[0063] Figure 1 A schematic diagram of at least one application scenario in the present application is shown. Figure 1As shown, each bucket may include multiple objects, and the objects between buckets are isolated from each other. The tenant logs in to the cloud management platform 10 through the object storage client, selects and purchases the cloud service of the object storage service on the cloud management platform 10. After the purchase, the tenant can store objects based on the functions provided by the object storage service. Among them, the cloud management platform 10 is mainly used to manage the infrastructure for running the object storage service. Exemplarily, the infrastructure for running the object storage service may include multiple data centers set up in different regions, and each data center includes multiple servers. The data center can provide basic resources for the object storage service, such as computing resources, storage resources, etc. Therefore, when the tenant purchases and uses the object storage service, it mainly pays for the resources used. Specifically, the object storage service provides the domain name of the bucket, and the tenant can access the domain name of the bucket through the object storage client, thereby uploading data to the bucket or downloading data from the bucket, wherein the uploaded data is stored in the bucket in the form of objects.

[0064] When a tenant uploads an object, he or she can specify the storage category of the object. If not specified, the default is the same as the storage category of the bucket. After uploading, the storage category of the object can be modified, and the bucket can be accessed through the object storage client. The object storage client can be a browser used locally by the tenant or a dedicated object storage client provided by the cloud platform. For example, a tenant can access a bucket through a browser set up in a local computer. After entering the account and passing the verification, the tenant can use the browser to upload objects to the bucket, or modify or delete objects in the bucket, etc., where the local computer is connected to the Internet.

[0065] Combined with the following Figure 2 For a description of the system architecture applicable to this application, see Figure 2 In this embodiment, the object storage client 20 can be deployed on the user side, such as a terminal device 21 outside the data center. The terminal device 21 can also be deployed with an application 22. The present application does not limit the specific type of the terminal device. For example, the terminal device can be a mobile phone, a laptop, a tablet computer, a PDA, a wireless terminal in a smart city, or a wireless device in a smart home. The object storage client 20 can interact with the application 22. For example, the application 22 can send a data modification request to the object storage client 20 under the triggering of the tenant. After receiving the data modification request, the object storage client 20 can send a data write request to the application 22 according to the data modification request.

[0066] The cloud management platform 10 is set in a data center and is connected to the object storage service node (the object storage service node can be simply referred to as the object storage service node. In this application, only the object storage service node 201 and the object storage service node 202 are drawn as examples) and the remote connection gateway 204 through the switching device 203.

[0067] The tenant can use the account to log in to the cloud management platform 10 on the object storage client 20, create a bucket, configure the bucket name, and obtain the bucket domain name through the cloud management platform 10. After detecting the tenant's operation (such as creating a bucket, configuring a bucket name, etc.), the cloud management platform 10 can send a creation instruction to the object storage service node (such as the object storage service node 201 or the object storage service node 202) through the switching device 203. The creation instruction includes information such as the bucket name and the bucket domain name, and notifies the object storage service node to create a bucket and save the information such as the bucket name and the bucket domain name.

[0068] After creating a bucket, the tenant can also access the bucket domain name by operating the object storage client 20, locate the bucket in the object storage service node, upload data (such as uploading objects) and download data (such as downloading objects) in the bucket. In this application, the tenant can also modify the objects in the bucket by operating the object storage client 20. That is, the tenant can trigger the object storage client 20 to send a data write request to the object storage service node, indicating that the data to be modified in the target object stored in the bucket of the object storage service node is modified.

[0069] After receiving a data write request, the object storage service node can determine the target object in the bucket based on the bucket name and the name of the target object carried in the data write request, and modify the data to be modified to the modified data based on the offset of the data to be modified in the target object and the data length of the data to be modified carried in the data write request.

[0070] For example, the remote connection gateway 204 can be implemented through a virtual private network (VPN) or a dedicated network, and the object storage service node can be implemented by a server equipped with multiple physical disks.

[0071] The following specifically describes the structure of the object storage service node, taking the object storage service node 201 as an example. Figure 3The object storage service node 201 includes a software layer and a hardware layer. The hardware layer includes a disk controller 2015, a physical network card 2016, a physical disk 1, and a physical disk 2 (here, the object storage service node 201 includes two physical disks as an example. The present application does not limit the number of physical disks in the object storage service node). The software layer includes an object storage device control unit 2011 and an operating system 2012. The control unit 2011 runs on the operating system 2012. The operating system 2012 includes a disk driver 2013 and a physical network card driver 2014. The cloud management platform 10 can communicate with the control unit 2011 through the physical network card 2016. The control unit 2011 controls the disk controller 2015 through the disk driver 2013 to set the physical disk 1 and the physical disk 2 as multiple persistent storage units (Persistence Log, PLOG). The present application only takes PLOG as a basic storage unit as an example. All storage spaces that can be used as basic storage units are applicable to the present application. After receiving the creation instruction of bucket 1, the cloud management platform 10 notifies the control unit 2011 to create bucket 1. The control unit 2011 creates bucket 1 on physical disk 1 and physical disk 2 in the hardware layer through the operating system. The PLOG of bucket 1 is distributed in physical disk 1 and physical disk 2, and the bucket name, bucket domain name and other information of bucket 1 are saved.

[0072] When the tenant needs to upload an object to bucket 1 through the object storage client 20, the tenant can trigger the object storage service node 201 to write the object into the bucket through the object storage client 20. For example, the control unit 2011 in the object storage service node 201 can write the object into the bucket. When the tenant needs to download an object in bucket 1 through the object storage client 20, the tenant can trigger the object storage service node 201 to read the object from the bucket through the object storage client 20. For example, the control unit 2011 in the object storage service node 201 can obtain the object from the bucket and feed it back to the object storage client 20.

[0073] The above description is based on the granularity of the object storage service node. From a functional perspective, the interaction within the object storage service node can be understood as the interaction between functional layers.

[0074] OBS stores objects in a flat namespace and provides an object storage service in the form of a key-value pair. Users can uniquely obtain the content of an object based on the name (Key) of the object. Usually, objects can be managed as files. However, in order to make it easier for tenants to manage data, the object storage service provides a way to simulate folders: by adding " / " to the name of the object, for example, "test / 123.jpg", "test" is simulated as a folder, and "123.jpg" is simulated as the file name under the "test" folder. In fact, the object name (Key) is "test / 123.jpg" and the data (Data) is the 123.jpg file itself. Although users can use names like test1 / test.jpg, this does not mean that the user's object is saved in the test1 directory. For OBS, test1 / test.jpg is just a string, and there is no essential difference from a.jpg.

[0075] However, many current big data, data analysis applications, and artificial intelligence simulations store data in a hierarchical namespace with infinite nesting depth and atomic operations. For example, many big data and data analysis applications are configured with file systems. The file system is a typical tree index structure. To access a file named test1 / test.jpg, you need to first access the test1 directory and then search for the file named test.jpg in the directory. Therefore, the file system can easily support folder operations, such as renaming, deleting, and moving directories.

[0076] Therefore, in order to be compatible with file applications and implement the use of a hierarchical directory structure in an object storage system with a flat namespace, the object storage service node provides a hierarchical namespace for file system operations of the object storage structure, that is, to build a file service based on object storage. That is, file applications can access data on the object storage service node through the file interface provided by the object storage service node.

[0077] Figure 4 The schematic diagram of the architecture of a hierarchical namespace in the present application is shown, and the hierarchical namespace in the object storage service node is described by taking the object storage service node 201 as an example. Figure 4, the object storage service node 201 stores multiple objects. The hierarchical namespace 400 maps the object identifier stored in the object storage space 401 to the file or directory identifier (or path) in the file storage space 402, and the mapping is stored in the hierarchical namespace 400. That is, the multiple objects in the object storage space 401 are organized into a multi-level directory hierarchy structure of the file storage space 402. Among them, the object storage space 401 is a flat namespace, and the file storage space 402 is a hierarchical namespace. The two namespaces can coexist in the object storage service node 201, and allow users to access the data stored in the object storage service node 201 in two different ways. Exemplarily, the user can access the object 405 stored in the bucket 403 in the object storage space 401 through the object interface based on the REST API and HTTP protocol through the object storage client 20, and can also access the file 407 stored under the directory 406 in the file system 404 in the file storage space 402 through the file storage client 40 based on the Network File System (NFS) protocol.

[0078] Among them, the object storage space 401 uses a two-layer data model of bucket-object (which can also be considered as directory-object) to store data, and the file namespace 402 uses a directory tree structure of file system-directory-subdirectory-file to store data. Therefore, the hierarchical namespace 400 can map the bucket 403 in the object storage space 401 to the file system 404 in the file storage space 402, and can map the object 405 to the directory 406 or the file 407. In other words, the hierarchical namespace 400 maps multiple objects in the bucket 403 to multiple nodes on the directory tree.

[0079] It should be noted that when a node is a file, the node may also be referred to as a file node. When a node is a directory, the node may also be referred to as a directory node. When an object is mapped to a file node, the object may include user data, metadata, and an object ID. When an object is mapped to a directory node, the object may include metadata and an object ID.

[0080] For example, Figure 5 As shown, the object storage service node 201 can map the object " / " in the bucket 403 to the root directory " / " in the file system 404, map the object " / b" to the directory "b", and map the object " / b / c" to the file "c", that is, map the object to the directory or file in the file system in a full path manner.

[0081] The object storage service node 201 maps the bucket-object representation in the flat namespace of the object storage space to the forward slash " / +file name" in the file namespace to simulate a folder, and organizes it into a multi-level directory tree structure, which can implement a single atomic operation of renaming a directory or file. However, the object storage service node identifies a file by the full path. When a user accesses a file, the object storage service node needs to parse the full path layer by layer to access the files stored in the directory in the file system. The deeper the directory accessed, the more resources are consumed and the lower the access efficiency. In addition, for the client's cache, it is also necessary to traverse the directory tree structure layer by layer to construct the full path, resulting in low access efficiency when the client and service node access files.

[0082] In order to solve the problem of inefficient access between clients and object storage service nodes, a data access method is proposed in the present application. A file identification code allocator is used to assign a file identification code to a file or directory stored in an object storage service node. The file identification code is globally unique and is used to identify the file or directory. Users use the file identification code to access the files or directories stored in the object storage service node without having to parse the full file path layer by layer, thereby improving user access efficiency.

[0083] To facilitate understanding, at least one application scenario of an embodiment of the present application is first introduced below.

[0084] For example, Figure 6 What is shown is a schematic diagram of at least one application scenario in an embodiment of the present application. Figure 6 As shown, a file identification code distributor 610 and a file index table 620 are added to the object storage service node 201. When a user creates a file or directory, the object storage service node 201 generates a globally unique file identification code for the created file or directory through the file identification code distributor 610. The file identification code is a basic attribute of the file or directory and will not change once it is created. At the same time, when the file identification code is generated, the corresponding relationship between the file identification code and the file information will be recorded in the file index table 620. Through the file index table 620, the user can query the metadata information of the file or directory through the file identification code.

[0085] Next, based on the above description, a data access method provided by the present application is introduced. It can be understood that the method is proposed based on the above description, and part or all of the contents of the method can be referred to the above description.

[0086] For example, Figure 7 FIG. 1 is a flow chart of a data access method provided by the present application. It is understood that the method can be implemented in any suitable device, equipment, platform or device cluster with computing, processing and storage capabilities. Figure 7 As shown, the method may include steps S701 to S704:

[0087] Step S701: receiving a file creation request.

[0088] In this embodiment, the object storage service node receives a file creation request sent by a file storage client, and the file creation request is used to request the object storage service node to create a first file, wherein the first file is created in the parent directory of the first file. The object storage service node may be the object storage service node 201, which is also used as an example for explanation below.

[0089] Optionally, when sending a file creation request, the file storage client can query the first file identification code corresponding to the parent directory of the first file in the file client's cache, and then send the first file name to be created and the first file identification code of the parent directory of the first file to the object storage service node.

[0090] In this embodiment, after receiving a file creation request, the object storage service node first determines the access method of the file creation request. For example, the file creation request uses a file identification code to access the object storage service node, or uses the full path of the file to access the object storage service node. The access identifier of the file creation request can be used to distinguish whether the file creation request uses a file identification code or a full path of the file to access the object storage service node. The access identifier can be in various forms, such as being carried using a query parameter, or being carried through the header field of the file creation request, or being carried through a special prefix or special suffix of the URI, but is not limited thereto.

[0091] In one example, the file creation request may carry the identification type of the file identification code in the query parameter, such as using the query parameter to carry the file identification code 100 / c, that is, to create a file named c in the parent directory of the file identification code 100. For example, the file creation request may be as follows:

[0092] create 100 / c

[0093] Host:xxxx

[0094] Connection:xxx

[0095] In one example, the file creation request may be carried in the header field of the file creation request, such as using Uritype to indicate that the file creation request is accessed through the full path of the file. For example, the file creation request may be as follows:

[0096] Uritype:path

[0097] In one example, the file creation request may be carried by a special prefix or special suffix of the URI, such as using a special prefix FIFEID_200 / c to indicate that the file creation request is accessed through a file identification code. For example, the file creation request may be as follows:

[0098] create / FILEID_200 / c

[0099] Step S702: Obtain metadata of a parent directory of a first file corresponding to the first file identification code.

[0100] In this embodiment, the object storage service node receives a file creation request sent by the file storage client, where the file creation request carries the first file name and the first file identification code of the parent directory of the first file, queries the metadata corresponding to the parent directory of the first file from the file index table 620, obtains the layout information of the parent directory, queries whether the parent directory is a valid directory, that is, determines whether the directory has not been deleted, and whether the first file under the parent directory exists.

[0101] Step S703: Create the first file in the parent directory according to the metadata of the parent directory of the first file, and assign a second file identification code to the first file.

[0102] In this embodiment, if the first file does not exist in the parent directory corresponding to the first file identification code, and the parent directory is a valid directory, the object storage service node creates the first file in the parent directory, and allocates a second file identification code to the first file through the file identification code distributor 610 in the object storage service node. Among them, the file identification code distributor 610 can allocate a globally unique file identification code to the first file based on a logical clock or a cluster ID distributor. The allocation method only needs to ensure the global uniqueness of the file identification code and cannot be repeated, but the specific allocation method is not limited to this. In addition, the length, type, and composition of the file identification code are not specifically limited in the embodiments of the present application.

[0103] In one example, if Figure 5 As shown, when creating directory b, the file identification code assignor 610 may assign a file identification code of 100 to it; when creating file c under parent directory b, the file identification code assignor 610 may assign a file identification code of 200 to it.

[0104] Step S704: Record the correspondence between the second file identification code and the first file name in the file index table.

[0105] In this embodiment, the object storage service node creates the first file in the parent directory, assigns the second file identification code to the first file, and then records the correspondence between the second file identification code and the first file name in the file index table. Through the file index table, the metadata information of the first file can be queried by the second file identification code.

[0106] In one example, if Figure 8 As shown, the object storage service node maps the objects in the bucket 803 of the object storage space 801 to the files or directories in the file system 804 of the file storage space 802, and the file identification code allocator 610 allocates a file identification code of 1024 to the root directory, a file identification code of 50 to the directory a, a file identification code of 100 to the directory a / b, a file identification code of 200 to the file a / b / c, and a file identification code of 300 to the file a / b / d. After the file identification code allocator 610 allocates the file identification code to the file or directory, the objects in the bucket 803 are stored with the file identification code of the parent directory + the file name, such as the directory a is stored with "1024 / a", and the file a / b / c is stored with "100 / c".

[0107] In one example, after the file identification code assigner 610 assigns a file identification code to a file or directory, the corresponding relationship between the file identification code and the name of the file or directory is stored in the file index table 620, as shown in Table 1. The file identification code is the key in the file index table, and the name of the file or directory is the value in the file index table.

[0108] Table 1 File index table

[0109] Key Value 50 1024 / a 100 50 / b 200 100 / c 300 100 / d

[0110] In one example, the metadata information corresponding to the file name can be obtained from the metadata index table in the object storage service node through the file name in the file index table, as shown in Table 2. Therefore, the metadata information of a file or directory can be directly obtained through the file identification code. The metadata of a file can be used to describe the attributes of a file. The metadata of a file may include at least one of the following: the file size of the file, the creation time of the file, the modification time of the file, the access rights of the file, and the data layout of the file, wherein the data layout of the file includes the identifier of the storage unit storing the file, the offset of the file in the storage unit, and the length of the file in the storage unit. Secondly, the metadata of the file may also include the file index code of the file.

[0111] Table 2 Metadata index table

[0112] Key Value 1024 / a File metadata information, 50 50 / b File metadata information, 100 100 / c File metadata information, 200 100 / d File metadata information, 300

[0113] In this way, in this embodiment, the file identification code allocator assigns a file identification code to a file or directory, so that when a user accesses a file, the file identification code is globally unique and is used to identify the file or directory. The user uses the file identification code to access the file or directory stored in the object storage service node without having to parse the full file path layer by layer, thereby improving the user's access efficiency.

[0114] In some possible implementations, in order to improve data access efficiency, the file storage client can also use the file identification code as the key to construct a mapping relationship table as shown in Table 1. When interacting with the object storage service node, the file identification code in the mapping relationship table cached by the file storage client can be directly queried, which reduces the access volume and access pressure of the object storage service node. At the same time, it reduces the layer-by-layer construction process of the full path of the file, thereby improving data access efficiency. In this embodiment, Fig. 9 As shown, the data access method may further include steps S901 to S904:

[0115] Step S901: receiving a file creation instruction for a first file sent by an application.

[0116] In this embodiment, the file storage client may receive a file creation instruction for a first file sent from an application, and the file creation instruction is used to instruct the object storage service node to create the first file. The first file is a hierarchical namespace that maps at least one object stored in a storage bucket of the object storage service node to a file or directory on a directory tree. The file storage client may be a file storage client 40, and the object storage service node may be an object storage service node 201, which is also used as an example for explanation below.

[0117] In one example, an application program calls a file creation instruction create(c) for a file "c".

[0118] Alternatively, the file storage client can be implemented based on a private file client, such as a file system in userspace (Fuse), which can export the file system to the Linux kernel interface through a userspace program, allowing unprivileged users to create their own file systems without editing the kernel code. The FUSE-based file storage client provides two APIs: a "high-level" synchronous API and a "low-level" asynchronous API. Both APIs receive requests from the kernel and pass them to the main program (fuse_main function), which uses the corresponding callback function for processing. When using the high-level API, the callback function works with file names and paths instead of inodes, and the callback function returns when a request is processed. When using the low-level API, the callback function must work with inodes, and the response must be sent explicitly using a separate set of API functions.

[0119] Optionally, the high-level API builds an access method for the full path of the file, using the full path of the file as the access identifier. However, the implementation cost of using the high-level API is low, but it is relatively inefficient. The FUSE-based file storage client needs to build it layer by layer when passing the full path of the file. In addition, the high-level API cannot build a cache of the full path of the file, and cannot support file renaming. However, using the low-level API, it can support access identifiers using file identification codes, maintain metadata caches indexed by file identification codes, support file renaming and other operations, and by building a mapping relationship table on the file storage client, it reduces the amount of access and access pressure on the object storage service node.

[0120] Step S902: Obtain a first file identification code of a parent directory of a first file from a mapping relationship table.

[0121] In this embodiment, after receiving the file creation instruction for the first file sent by the application, the file storage client obtains the first file identification code of the parent directory of the first file from the mapping relationship table cached by the file storage client. The mapping relationship table stores the correspondence between the file name and the file identification code. When the file storage client interacts with the object storage service node, it can directly query the file identification code in the mapping relationship table cached by the file storage client to determine the first file identification code of the parent directory of the first file.

[0122] For example, the parent directory of the file "c" queried in the cache of the file storage client is / a / b, and its file identification code is 200.

[0123] Step S903: Send a file creation request to the object storage service node.

[0124] In this embodiment, after the file storage client obtains the first file identification code of the parent directory of the first file from the mapping relationship table, it sends a file creation request to the object storage service node. The file creation request is used to request the creation of the first file in the object storage service node. The file creation request carries the first file name and the first file identification code of the parent directory.

[0125] In one example, the file storage client sends a file write request PUT / 100 / c?uritype=fileid, which indicates to access the object storage service node with the file identification code fileid and create a file c in the parent directory with the file identification code 100.

[0126] Optionally, the file storage client may carry an access identifier in the file creation request to identify whether the file creation request is based on the full file path access or the file identification code access. The access identifier may be in various forms, such as being carried using a query parameter, or being carried through the header field of the file creation request, or being carried through a special prefix or special suffix of the URI, but is not limited thereto.

[0127] Step S904: Receive the second file identification code assigned by the object storage service node, and record the corresponding relationship between the second file identification code and the first file name in a mapping relationship table.

[0128] In this embodiment, after the object storage service node creates the first file under the parent directory and assigns the second file identification code to the first file through the file identification code assignor 610 in the object storage service node, the object storage service node sends the second file identification code of the first file to the file storage client. After receiving the second file identification code assigned by the object storage service node, the file storage client records the corresponding relationship between the second file identification code and the first file name in the mapping relationship table.

[0129] Thus, in this embodiment, the file storage client can directly query the file identification code in the mapping table cached by the file storage client when interacting with the object storage service node by caching the mapping table of the file identification code and the file name, thereby reducing the access volume and access pressure of the object storage service node. At the same time, for the file storage client, the process of building the cache layer by layer of the full path of the file is reduced, thereby improving the data access efficiency.

[0130] In some possible implementations, such as Fig.10 As shown, the data access method further includes steps 1001 to 1007:

[0131] Step S1001: The application sends a file write instruction for a first file.

[0132] In this embodiment, after the object storage service node creates the first file in the parent directory of the first file, it needs to write the data content of the first file. The application sends a file write instruction for the first file to the file storage client, and the file write instruction is used to instruct the file storage client to write the data content of the first file in the object storage service node.

[0133] In one example, an application program calls a file write instruction write(c) for a file "c".

[0134] Step S1002: The file storage client obtains a second file identification code corresponding to the first file from a mapping relationship table.

[0135] In this embodiment, after receiving the file write instruction for the first file, the file storage client queries the second file identification code corresponding to the first file from the mapping relationship table cached by the file storage client. For example, the second file identification code of the file "c" queried in the cache is 200.

[0136] Step S1003: The file storage client sends a file writing request, where the file writing request carries a second file identification code.

[0137] In this embodiment, after the file storage client queries the second file identification code corresponding to the first file, it sends a file write request to the object storage service node. The file write request is used to request to write the data content of the first file to the object storage service node. The file write request carries the second file identification code queried by the file storage client in the cache, indicating access to the object storage service node based on the file identification code.

[0138] In one example, the file storage client sends a file write request PUT / 200?uritype=fileid, which indicates to access the object storage service node with the file identification code fileid and write the file with the file identification code 200 to the object storage service node.

[0139] Step S1004: The object storage service node obtains the first file name corresponding to the second file identification code from the file index table.

[0140] In this embodiment, after the object storage service node receives a file write request sent by the file storage client, it determines that it is a file identification code access through the access identifier of the file write request, parses the second file identification code in the file write request, obtains the first file name corresponding to the second file identification code from the file index table, and obtains the metadata of the first file based on the first file name.

[0141] In an example, the object storage service node receives a file write request PUT / 200?uritype=fileid, determines through uritype that the file write request is accessed based on the file identification code fileid, and parses the file identification code to obtain 200.

[0142] Step S1005: The object storage service node obtains metadata of the first file.

[0143] In this embodiment, the object storage service node obtains the first file name corresponding to the second file identification code from the file index table, and obtains metadata information of the first file from the metadata index table based on the first file name. The metadata of the first file includes one or more of the following: the file size of the first file, the creation time of the first file, the access rights of the first file, and the data layout of the first file, wherein the data layout of the first file includes an identifier of a first storage unit storing the first file, an offset of the first file in the first storage unit, and a length of the first file in the first storage unit.

[0144] Step S1006: The object storage service node writes the first file into the first storage unit according to the data layout of the first file.

[0145] In this embodiment, after the object storage service node obtains the metadata of the first file based on the second file identification code, it determines the first storage unit for storing the first file according to the data layout of the first file, and writes the first file in the first storage unit.

[0146] Optionally, the file creation request carries an offset of the first storage unit where the first file is to be written, indicating that the first file is to be written at the offset address of the first storage unit.

[0147] Step S1007: The object storage service node updates the metadata of the first file.

[0148] In this embodiment, after the object storage service node writes the first file to the first storage unit, it needs to update the metadata information of the first file, such as the size of the first file, the modification time of the first file, and the data layout of the first file.

[0149] Thus, in this embodiment, the file storage client uses the file identification code to write the file in the object storage service node, without having to parse the full file path layer by layer, thereby improving data access efficiency.

[0150] In some possible implementations, such as Fig.11 As shown, the data access method further includes steps 1101 to 1106:

[0151] Step S1101: The application sends a first file reading instruction for a first file.

[0152] In this embodiment, after the object storage service node writes the first file, the application may send a first file read instruction for the first file to the file storage client, where the first file read instruction is used to instruct the file storage client to read the first file from the object storage service node.

[0153] In one example, an application program calls a first file read instruction read(c) for a file “c”.

[0154] Step S1102: The file storage client obtains a second file identification code corresponding to the first file from the mapping relationship table.

[0155] In this embodiment, after receiving the first file read instruction for the first file, the file storage client queries the second file identification code corresponding to the first file from the mapping relationship table cached by the file storage client. For example, the second file identification code of the file "c" queried in the cache is 200.

[0156] Step S1103: The file storage client sends a first file reading request, where the file reading request carries a second file identification code.

[0157] In this embodiment, after the file storage client queries the second file identification code corresponding to the first file, it sends a first file read request to the object storage service node, and the first file read request is used to instruct to read the first file from the object storage service node. The first file read request carries the queried second file identification code, indicating accessing the object storage service node based on the file identification code.

[0158] In an example, the file storage client sends a first file read request GET / 200?uritype=fileid, which indicates to access the object storage service node with the file identification code fileid and read the file with the file identification code 200 from the object storage service node.

[0159] Step S1104: The object storage service node obtains the first file name corresponding to the second file identification code from the file index table.

[0160] In this embodiment, after the object storage service node receives the first file read request sent by the file storage client, it determines that it is a file identification code access through the access identifier of the first file read request, parses the second file identification code in the first file read request, obtains the first file name corresponding to the second file identification code from the file index table, and obtains the metadata of the first file based on the first file name.

[0161] In an example, the object storage service node receives a first file read request GET / 200?uritype=fileid, determines through uritype that the first file read request is accessed based on the file identification code fileid, and parses to obtain the file identification code as 200.

[0162] Step S1105: The object storage service node obtains metadata of the first file.

[0163] In this embodiment, the object storage service node obtains the first file name corresponding to the second file identification code from the file index table, and obtains metadata information of the first file from the metadata index table based on the first file name. The metadata of the first file includes one or more of the following: the file size of the first file, the creation time of the first file, the access rights of the first file, and the data layout of the first file, wherein the data layout of the first file includes an identifier of a first storage unit storing the first file, an offset of the first file in the first storage unit, and a length of the first file in the first storage unit.

[0164] Step S1106: The object storage service node reads the first file from the first storage unit according to the data layout of the first file.

[0165] In this embodiment, after the object storage service node obtains the metadata of the first file based on the second file identification code, it determines the first storage unit storing the first file according to the data layout of the first file, and reads the first file from the first storage unit.

[0166] Optionally, the first file read request carries an offset and a length of the first file to be read in the first storage unit, indicating reading the first file at the offset address and the specified length in the first storage unit.

[0167] Thus, in this embodiment, the file storage client uses the file identification code to read the file from the object storage service node, without having to parse the full file path layer by layer, thereby improving data access efficiency.

[0168] In some possible implementations, such as Fig.12 As shown, the data access method further includes steps 1201 to 1205:

[0169] Step 1201: Receive a second file reading request.

[0170] In this embodiment, after receiving the second file read request sent by the file storage client, the object storage service node determines that the second file read request is a full path access to the file based on the access identifier of the second file read request.

[0171] In one example, the file storage client sends a second file read request GET / a / b / c?uritype=path, which indicates to access the object storage service node with the full file path path and read the file with the full file path / a / b / c from the object storage service node.

[0172] Step 1202: parse the access path of the first file through the hierarchical namespace to obtain a second file identification code of the first file.

[0173] In this embodiment, if Figure 8 As shown, since the object storage service node maps the objects in bucket 803 of object storage space 801 to files or directories in file system 804 of file storage space 802, after the file identification code allocator 610 allocates the file identification code to the file or directory, the objects in bucket 803 are stored with the file identification code + file name of the parent directory, so the full path of the file cannot be used to directly match the file. Therefore, it is necessary to parse the access path of the first file through the hierarchical namespace to obtain the second file identification code of the first file. The hierarchical namespace can be the hierarchical namespace 400, which is also used as an example for explanation below.

[0174] In one example, the hierarchical namespace resolves the full file path / a / b / c and obtains a file identification code of 200 for the file c.

[0175] Step 1203, obtaining the first file name corresponding to the second file identification code from the file index table.

[0176] In this embodiment, after the second file identification code of the first file is obtained through parsing, the first file name corresponding to the second file identification code is obtained from the file index table, and metadata of the first file is obtained based on the first file name.

[0177] Step 1204, obtaining metadata of the first file.

[0178] In this embodiment, the object storage service node obtains the first file name corresponding to the second file identification code from the file index table, and obtains metadata information of the first file from the metadata index table based on the first file name.

[0179] Step 1205: read the first file from the first storage unit according to the data layout of the first file.

[0180] In this embodiment, after the object storage service node obtains the metadata of the first file based on the second file identification code, it determines the first storage unit storing the first file according to the data layout of the first file, and reads the first file from the first storage unit.

[0181] Thus, in this embodiment, when the file storage client uses the full file path to access the object storage service node to read the file, the object storage service node can recognize the two access methods of file identification code or file full path, and is still compatible with accessing the file in the full file path method.

[0182] In some possible implementations, such as Fig.13 As shown, the data access method further includes steps 1301 to 1305:

[0183] Step 1301: Receive a third file read request, where the third file read request includes a file handle.

[0184] In this embodiment, after receiving the third file read request sent by the file storage client, the object storage service node determines that the access is based on the file handle through the access identifier of the third file read request, that is, the third file read request is to access the object storage service node based on the file protocol. Among them, the file handle is a data structure used to access files in the operating system, usually an integer or a pointer. The file handle is used to identify an open file, and each open file has a unique file handle.

[0185] Optionally, the file protocol may include a Server Message Block (SMB), a Network File System (NFS) protocol, or a Common Internet System (CIFS) protocol. The SMB protocol is an application-level network protocol, primarily used for sharing printers, file access, serial ports, and other communications between nodes on a network. SMB is primarily used by Windows systems and is a proven inter-process communication mechanism. NFS allows a system to share directories and files with others on a network. By using NFS, users and programs can access files on a remote system as if they were local files. The CIFS protocol is a network file system access protocol through the Internet File System. Through the CIFS protocol, network file sharing between Windows system hosts can be achieved. Common Internet File System (CIFS) is an enhanced version of Microsoft's Server Message Block protocol (SMB) and is a standard method for computer users to share files on corporate intranets and the Internet. CIFS enables collaboration on the Internet by defining a remote file access protocol that is compatible with the way applications share data on local disks and network file servers. CIFS runs on TCP / IP and uses the global domain name service (DNS) on the Internet to enhance its scalability and optimize for the slow dial-up connections that are common on the Internet. CIFS uses redirection packets to be sent to remote devices over the network, and the redirector also uses CIFS to send requests to the local computer's protocol stack.

[0186] Optionally, NFS or SMB protocol both define file handles under the protocol, for example, the handle of NFSv3 is 64 bytes, and the handle of SMB2 is 16 bytes. For example, in the NFS protocol, the file identification code can be encapsulated in the file handle.

[0187] In one example, the file storage client sends a third file read request Rpc: read(200), which encapsulates the file identification code 200 into the file handle of the NFS or SMB protocol, wherein the Remote Procedure Call (RPC) protocol is a protocol for establishing an appropriate framework, which a program can use to request services from a program located on another computer on the network without having to understand the details of the network.

[0188] Step 1302: Identify the second file identification code in the file handle through the gateway.

[0189] In this embodiment, the object storage service node includes a gateway. After receiving the third file read request sent by the file storage client, the gateway can identify the second file identification code encapsulated in the file handle.

[0190] In one example, after receiving the third file read request Rpc:read(200), the gateway converts the file protocol request format into an HTTP protocol format recognizable by the object storage service node, that is, converts it into GET / 200?uritype=fileid, and then converts it into accessing the object storage service node with the file identification code fileid, and reads the file with the file identification code 200 from the object storage service node.

[0191] Step 1303: Obtain the first file name corresponding to the second file identification code from the file index table.

[0192] In this embodiment, after the object storage service node receives the third file read request sent by the file storage client, it determines that it is a file identification code access through the access identifier of the third file read request, parses the second file identification code in the third file read request, obtains the first file name corresponding to the second file identification code from the file index table, and obtains the metadata of the first file based on the first file name.

[0193] In an example, the object storage service node receives GET / 200?uritype=fileid converted by the gateway, determines through uritype that the third file read request is accessed based on the file identification code fileid, and parses to obtain the file identification code as 200.

[0194] Step 1304, obtaining metadata of the first file.

[0195] In this embodiment, the object storage service node obtains the first file name corresponding to the second file identification code from the file index table, and obtains metadata information of the first file from the metadata index table based on the first file name.

[0196] Step 1305: read the first file from the first storage unit according to the data layout of the first file.

[0197] In this embodiment, after the object storage service node obtains the metadata of the first file based on the second file identification code, it determines the first storage unit storing the first file according to the data layout of the first file, and reads the first file from the first storage unit.

[0198] Optionally, the third file read request carries an offset and a length of the first file in the first storage unit, indicating that the first file is read at the offset address and the specified length in the first storage unit.

[0199] Thus, in this embodiment, the file storage client reads files from the object storage service node based on the file protocol, and can implement the conversion of the file protocol to the HTTP protocol that the object storage service node can recognize through the gateway, that is, "NAS over HTTP" is implemented. Even if the file client uses the file protocol, it can still access the files stored in the object storage service node based on the file identification code, thereby improving data access efficiency. In addition, using the file identification code as an access identifier makes it easier to be compatible with the file protocol.

[0200] The present application also provides a data access device, such as Fig.14 As shown, the device 1400 includes:

[0201] A receiving module 1401 is used to receive a file creation request, where the file creation request carries a first file name and a first file identification code of a parent directory of the first file;

[0202] An acquisition module 1402 is used to acquire metadata of a parent directory of a first file corresponding to a first file identification code;

[0203] The processing module 1403 is used to create a first file in the parent directory according to the metadata of the parent directory of the first file, and assign a second file identification code to the first file, wherein the first file is a file or a directory on the directory tree;

[0204] The processing module 1403 is further configured to record the corresponding relationship between the second file identification code and the first file name in the file index table.

[0205] In some possible implementations, the receiving module 1401 is also used to receive a file write request, the file write request carries the first file to be written and the second file identification code; the obtaining module 1402 is also used to obtain the first file name corresponding to the second file identification code from the file index table, the first file name is used to indicate the metadata of the first file; the obtaining module 1402 is also used to obtain the metadata of the first file, the metadata of the first file includes the data layout of the first file; the processing module 1403 is also used to write the first file in the first storage unit according to the data layout of the first file; the processing module is also used to update the metadata of the first file.

[0206] In some possible implementations, the metadata of the first file includes one or more of the following: the file size of the first file, the creation time of the first file, the access permissions of the first file, and the data layout of the first file, where the data layout of the first file includes an identifier of a first storage unit storing the first file, an offset of the first file in the first storage unit, and a length of the first file in the first storage unit.

[0207] In some possible implementations, the receiving module 1401 is also used to receive a first file read request, the first file read request carries a second file identification code; the obtaining module 1402 is also used to obtain the first file name corresponding to the second file identification code from the file index table; the obtaining module 1402 is also used to obtain metadata of the first file, the metadata of the first file includes the data layout of the first file; the processing module 1403 is also used to read the first file from the first storage unit according to the data layout of the first file.

[0208] In some possible implementations, the receiving module 1401 is also used to receive a second file read request, which carries an access path to the first file; the processing module 1403 is also used to parse the access path of the first file through a hierarchical namespace to obtain a second file identification code for the first file; the acquisition module 1402 is also used to obtain a first file name corresponding to the second file identification code from a file index table; the acquisition module 1402 is also used to obtain metadata of the first file; the processing module 1403 is also used to read the first file from the first storage unit according to the data layout of the first file.

[0209] In some possible implementations, the receiving module 1401 is also used to receive a third file read request, which includes a file handle; the processing module 1403 is also used to identify the second file identification code in the file handle through the gateway; the acquisition module 1402 is also used to obtain the first file name corresponding to the second file identification code from the file index table; the acquisition module 1402 is also used to obtain the metadata of the first file; the processing module 1403 is also used to read the first file from the first storage unit according to the data layout of the first file.

[0210] Among them, the receiving module 1401, the acquisition module 1402 and the processing module 1403 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the processing module 1403 is introduced below by taking the processing module 1403 as an example. Similarly, the implementation of the receiving module 1401 and the acquisition module 1402 can refer to the implementation of the processing module 1403.

[0211] As an example of a software functional unit, the processing module 1403 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the processing module 1403 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same AZ or in different AZs, and each AZ includes a data center or multiple data centers with similar geographical locations. Among them, usually a region can include multiple AZs.

[0212] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud VPC or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0213] As an example of a hardware functional unit, the processing module 1403 may include at least one computing device, such as a server, etc. Alternatively, the processing module 1403 may also be a device implemented by an application-specific integrated circuit ASIC, or a programmable logic device PLD, etc. The PLD may be implemented by a CPLD, an FPGA, a GAL, or any combination thereof.

[0214] The multiple computing devices included in the processing module 1403 can be distributed in the same region or in different regions. The multiple computing devices included in the processing module 1403 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the processing module 1403 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0215] It should be noted that, in other embodiments, the receiving module 1401 can be used to execute any step in the data access method, the acquisition module 1402 can be used to execute any step in the data access method, and the processing module 1403 can be used to execute any step in the data access method. The steps that the receiving module 1401, the acquisition module 1402 and the processing module 1403 are responsible for implementing can be specified as needed. The full functions of the data processing device are realized by respectively implementing different steps in the data access method through the receiving module 1401, the acquisition module 1402 and the processing module 1403.

[0216] The present application also provides a data access device, such as Fig.15 As shown, the device 1500 includes:

[0217] The receiving module 1501 is used to receive a file creation instruction for a first file sent by an application program, where the first file is a file or a directory on a directory tree;

[0218] An acquisition module 1502 is used to acquire a first file identification code of a parent directory of a first file from a mapping relationship table;

[0219] A sending module 1503 is used to send a file creation request to the object storage service node, where the file creation request carries a first file name and a first file identification code;

[0220] The receiving module 1501 is further used to receive a second file identification code assigned by the object storage service node, and record the corresponding relationship between the second file identification code and the first file name in the mapping relationship table.

[0221] In some possible implementations, the receiving module 1501 is also used to receive a file write instruction for the first file sent by an application; the obtaining module 1502 is also used to obtain a second file identification code corresponding to the first file from a mapping relationship table of the file storage client; the sending module 1503 is also used to send a file write request to the object storage service node, and the file write request carries the second file identification code; the receiving module 1501 is also used to receive the processing result of the file write request.

[0222] In some possible implementations, the receiving module 1501 is also used to receive a first file read instruction for a first file sent by an application; the obtaining module 1502 is also used to obtain a second file identification code corresponding to the first file from a mapping relationship table of the file storage client; the sending module 1503 is also used to send a first file read request to the object storage service node, and the first file read request carries the second file identification code; the receiving module 1501 is also used to receive the processing result of the first file read request.

[0223] Among them, the receiving module 1501, the acquisition module 1502 and the sending module 1503 can all be implemented by software, or can be implemented by hardware. Exemplarily, the following takes the acquisition module 1502 as an example to introduce the implementation of the acquisition module 1502. Similarly, the implementation of the receiving module 1501 and the sending module 1503 can refer to the implementation of the acquisition module 1502.

[0224] As an example of a software functional unit, the acquisition module 1502 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the acquisition module 1502 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same AZ or in different AZs, and each AZ includes a data center or multiple data centers with similar geographical locations. Among them, usually a region can include multiple AZs.

[0225] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud VPC or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0226] As an example of a hardware functional unit, the acquisition module 1502 may include at least one computing device, such as a server, etc. Alternatively, the acquisition module 1502 may also be a device implemented by an application-specific integrated circuit ASIC, or a programmable logic device PLD, etc. The PLD may be implemented by a CPLD, an FPGA, a GAL, or any combination thereof.

[0227] The multiple computing devices included in the acquisition module 1502 can be distributed in the same region or in different regions. The multiple computing devices included in the acquisition module 1502 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the acquisition module 1502 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0228] It should be noted that, in other embodiments, the receiving module 1501 can be used to execute any step in the data access method, the acquiring module 1502 can be used to execute any step in the data access method, and the sending module 1503 can be used to execute any step in the data access method. The steps that the receiving module 1501, the acquiring module 1502, and the sending module 1503 are responsible for implementing can be specified as needed, and the receiving module 1501, the acquiring module 1502, and the sending module 1503 respectively implement different steps in the data access method to realize the full functions of the data processing device.

[0229] The present application also provides a computing device 100. Fig.16 As shown, the computing device 100 includes: a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate with each other through the bus 102. The computing device 100 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 100.

[0230] The bus 102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.16 The bus 104 is represented by only one line, but does not mean that there is only one bus or one type of bus. The bus 104 may include a path for transmitting information between various components of the computing device 100 (eg, the memory 106, the processor 104, and the communication interface 108).

[0231] The processor 104 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0232] The memory 106 may include a volatile memory, such as a random access memory (RAM). The processor 104 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0233] The memory 106 stores executable program codes, and the processor 104 executes the executable program codes to respectively implement the functions of the aforementioned receiving module 1401, the acquisition module 1402, and the processing module 1403, thereby implementing the data access method. That is, the memory 106 stores instructions for executing the data access method. Fig.16 It is merely exemplarily shown that the memory 106 stores program codes for implementing the functions of the aforementioned receiving module 1401 , obtaining module 1402 , and processing module 1403 .

[0234] The communication interface 103 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

[0235] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0236] like Fig.17 As shown, the computing device cluster includes at least one computing device 100. The memory 106 in one or more computing devices 100 in the computing device cluster may store the same instructions for executing the data access method.

[0237] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the data access method. In other words, the combination of one or more computing devices 100 may jointly execute instructions for executing the data access method.

[0238] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the data processing apparatus. That is, the instructions stored in the memory 106 in different computing devices 100 may implement the functions of one or more of the receiving module 1401, the obtaining module 1402 and the processing module 1403, or implement the functions of one or more of the receiving module 1501, the obtaining module 1502 and the sending module 1503.

[0239] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.18 A possible implementation is shown. Fig.18 As shown, two computing devices 100A and 100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 106 in the computing device 100A stores instructions for executing the functions of the receiving module 1401. Fig.18 In the example, the memory 106 in the computing device 100A stores instructions for executing the functions of the receiving module 1401. Meanwhile, the memory 106 in the computing device 100B stores instructions for executing the functions of the acquiring module 1402 and the processing module 1403. Fig.18 In the example, the memory 106 in the computing device 100B stores instructions for executing the functions of the acquisition module 1402 and the processing module 1403.

[0240] Fig.18 The connection method between the computing device clusters shown may be that considering that the data access method provided in the present application requires a large amount of data calculation, the functions implemented by the acquisition module 1402 and the processing module 1403 are considered to be executed by the computing device 100B.

[0241] It should be understood that Fig.18 The functions of the computing device 100A shown in FIG. 1 may also be completed by multiple computing devices 100. Similarly, the functions of the computing device 100B may also be completed by multiple computing devices 100.

[0242] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes a data access method or a data access method.

[0243] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct a computing device to execute a data access method, or instructs a computing device to execute a data access method.

[0244] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data access method, characterized in that: The method is applied to an object storage service node, wherein at least one object is stored in a storage bucket of the object storage service node, and the object storage service node includes a hierarchical namespace, wherein the hierarchical namespace is used to map the at least one object to a file or directory on a directory tree, and the method includes: Receive a file creation request, wherein the file creation request carries a first file name and a first file identification code of a parent directory of the first file; Obtaining metadata of a parent directory of the first file corresponding to the first file identification code; Creating the first file under the parent directory according to metadata of the first file, and assigning a second file identification code to the first file, wherein the first file is a file or a directory on the directory tree; The correspondence between the second file identification code and the first file name is recorded in a file index table.

2. The method according to claim 1, further comprising: receiving a file writing request, wherein the file writing request carries the first file to be written and the second file identification code; Acquire the first file name corresponding to the second file identification code from the file index table, where the first file name is used to indicate metadata of the first file; Acquire metadata of the first file, where the metadata of the first file includes a data layout of the first file; Writing the first file into a first storage unit according to the data layout of the first file; The metadata of the first file is updated.

3. The method according to claim 1 or 2, characterized in that: The metadata of the first file includes one or more of the following: the file size of the first file, the creation time of the first file, the access rights of the first file, and the data layout of the first file. The data layout of the first file includes the identifier of the first storage unit storing the first file, the offset of the first file in the first storage unit, and the length of the first file in the first storage unit.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: receiving a first file reading request, wherein the first file reading request carries the second file identification code; Acquire the first file name corresponding to the second file identification code from the file index table; Acquire metadata of the first file, where the metadata of the first file includes a data layout of the first file; The first file is read from the first storage unit according to the data layout of the first file.

5. The method according to claim 1, characterized in that The method further comprises: receiving a second file read request, where the second file read request carries an access path to the first file; Parsing the access path of the first file through the hierarchical namespace to obtain the second file identification code of the first file; Acquire the first file name corresponding to the second file identification code from the file index table; Obtaining metadata of the first file; The first file is read from the first storage unit according to the data layout of the first file.

6. The method according to claim 1, characterized in that The object storage service node includes a gateway, and the method further includes: receiving a third file read request, wherein the third file read request includes a file handle; identifying, by the gateway, the second file identification code in the file handle; Acquire the first file name corresponding to the second file identification code from the file index table; Obtaining metadata of the first file; The first file is read from the first storage unit according to the data layout of the first file.

7. A data access method, characterized in that: The method is applied to a file storage client, the file storage client accesses an object storage service node through a file interface, at least one object is stored in a storage bucket of the object storage service node, the object storage service node includes a hierarchical namespace, and the hierarchical namespace is used to map the at least one object to a file or directory on a directory tree. The method includes: receiving a file creation instruction for a first file sent by an application, where the first file is a file or a directory on the directory tree; Acquire a first file identification code of a parent directory of the first file from the mapping relationship table; Sending a file creation request to the object storage service node, wherein the file creation request carries the first file name and the first file identification code; Receive a second file identification code assigned by the object storage service node, and record the corresponding relationship between the second file identification code and the first file name in the mapping relationship table.

8. The method according to claim 7, characterized in that The method further comprises: receiving a file write instruction for the first file sent by the application; Acquire the second file identification code corresponding to the first file from the mapping relationship table of the file storage client; Sending a file write request to the object storage service node, wherein the file write request carries the second file identification code; A processing result of the file writing request is received.

9. The method according to claim 7 or 8, characterized in that: The method further comprises: receiving a first file reading instruction for the first file sent by the application; Acquire the second file identification code corresponding to the first file from the mapping relationship table of the file storage client; Sending a first file read request to the object storage service node, where the first file read request carries the second file identification code; Receive a processing result of the first file read request.

10. A data access device, characterized in that: The device is applied to an object storage service node, wherein at least one object is stored in a storage bucket of the object storage service node, and the object storage service node includes a hierarchical namespace, wherein the hierarchical namespace is used to map the at least one object to a file or directory on a directory tree, and the device includes: A receiving module, configured to receive a file creation request, wherein the file creation request carries a first file name and a first file identification code of a parent directory of the first file; An acquisition module, used to acquire metadata of a parent directory of the first file corresponding to the first file identification code; a processing module, configured to create the first file in the parent directory according to metadata of the parent directory of the first file, and assign a second file identification code to the first file, wherein the first file is a file or a directory on the directory tree; The processing module is further used to record the corresponding relationship between the second file identification code and the first file name in a file index table.

11. The device according to claim 10, characterized in that The device also includes: The receiving module is further used to receive a file writing request, wherein the file writing request carries the first file to be written and the second file identification code; The acquisition module is further used to acquire the first file name corresponding to the second file identification code from the file index table, where the first file name is used to indicate metadata of the first file; The acquisition module is further used to acquire metadata of the first file, where the metadata of the first file includes a data layout of the first file; The processing module is further used to write the first file into the first storage unit according to the data layout of the first file; The processing module is further used to update the metadata of the first file.

12. The device according to claim 10 or 11, characterized in that The metadata of the first file includes one or more of the following: the file size of the first file, the creation time of the first file, the access rights of the first file, and the data layout of the first file. The data layout of the first file includes the identifier of the first storage unit storing the first file, the offset of the first file in the first storage unit, and the length of the first file in the first storage unit.

13. The device according to any one of claims 10 to 12, characterized in that The device also includes: The receiving module is further configured to receive a first file reading request, where the first file reading request carries the second file identification code; The acquisition module is further used to acquire the first file name corresponding to the second file identification code from the file index table; The acquisition module is further used to acquire metadata of the first file, where the metadata of the first file includes a data layout of the first file; The processing module is further configured to read the first file from the first storage unit according to the data layout of the first file.

14. The device according to claim 10, characterized in that The device also includes: The receiving module is further configured to receive a second file reading request, where the second file reading request carries an access path to the first file; The processing module is further used to parse the access path of the first file through the hierarchical namespace to obtain the second file identification code of the first file; The acquisition module is further used to acquire the first file name corresponding to the second file identification code from the file index table; The acquisition module is further used to acquire metadata of the first file; The processing module is further configured to read the first file from the first storage unit according to the data layout of the first file.

15. The device according to claim 10, characterized in that The object storage service node includes a gateway, and the device also includes: The receiving module is further used to receive a third file reading request, wherein the third file reading request includes a file handle; The processing module is further used to identify the second file identification code in the file handle through the gateway; The acquisition module is further used to acquire the first file name corresponding to the second file identification code from the file index table; The acquisition module is further used to acquire metadata of the first file; The processing module is further configured to read the first file from the first storage unit according to the data layout of the first file.

16. A data access device, characterized in that: The device is applied to a file storage client, the file storage client accesses an object storage service node through a file interface, at least one object is stored in a storage bucket of the object storage service node, the object storage service node includes a hierarchical namespace, and the hierarchical namespace is used to map the at least one object to a file or directory on a directory tree, the device includes: A receiving module, configured to receive a file creation instruction for a first file sent by an application, where the first file is a file or a directory on the directory tree; An acquisition module, used for acquiring a first file identification code of a parent directory of the first file from the mapping relationship table; A sending module, configured to send a file creation request to the object storage service node, wherein the file creation request carries the first file name and the first file identification code; The receiving module is further used to receive a second file identification code assigned by the object storage service node, and record the corresponding relationship between the second file identification code and the first file name in the mapping relationship table.

17. The device according to claim 16, characterized in that The device also includes: The receiving module is further used to receive a file writing instruction for the first file sent by the application; The acquisition module is further used to acquire the second file identification code corresponding to the first file from the mapping relationship table of the file storage client; The sending module is further used to send a file write request to the object storage service node, where the file write request carries the second file identification code; The receiving module is further used to receive the processing result of the file writing request.

18. The device according to claim 16 or 17, characterized in that The device also includes: The receiving module is further configured to receive a first file reading instruction for the first file sent by the application; The acquisition module is further used to acquire the second file identification code corresponding to the first file from the mapping relationship table of the file storage client; The sending module is further used to send a first file reading request to the object storage service node, where the first file reading request carries the second file identification code; The receiving module is further used to receive a processing result of the first file reading request.

19. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 6.

20. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 7 to 9.

21. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 6.

22. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 7 to 9.

23. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6.

24. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 7 to 9.

Citation Information

Cited By

  • Data access method, apparatus and system

    EP4715649A1

  • Data access method, apparatus and system

    WO2025001213A1