File creation method, data access device, storage device, and system

Abstract

A file creation method, a data access device, a storage device, and a system. The file creation method comprises: a data access device (31) sends an index acquisition request to a storage device (32) (S310); in response to the index acquisition request, the storage device (32) sends an identifier index of a first storage area to the data access device (31) (S320); the data access device (31) acquires a file creation request (S330); the data access device (31) determines whether the identifier index cached by the data access device (31) has stored an identifier of data to be processed (S340); the data access device (31) sends a data write instruction to the storage device (32) (S350); and in response to the data write instruction, the storage device (32) creates a file containing the data to be processed (S360). A data query duration in a file creation process can be reduced, thereby improving the file creation efficiency.

Description

File creation method, data access device, storage device and system

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on September 21, 2023, with application number 202311228731.7 and application name “A file creation method, data access device, storage device and system”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of data storage, and in particular to a file creation method, a data access device, a storage device, and a system. Background Art

[0003] With the continuous development of cloud computing technology, the amount of unstructured data is increasing, and efficient management of unstructured data is becoming increasingly important. Among them, unstructured data refers to data with irregular or incomplete data structure, such as multimedia data, non-standardized reports, etc. Usually, when the user needs to store unstructured data in the form of files on a storage device, the storage device will first determine whether there is a file with the same name in the stored file directory. If there is no file with the same name in the directory, the storage device will create the corresponding file to realize the storage of the data. In the case where the storage device needs to store a large amount of unstructured data, due to the limited computing power of the storage device, it takes a long time for the storage device to query the file name, and the efficiency of file creation is low.

[0004] Summary of the Invention

[0005] The present application provides a file creation method, data access device, storage device and system, which solves the problem that file creation of large amounts of data occupies the computing power of the storage device, resulting in a long file creation time and low efficiency.

[0006] In a first aspect, the present application provides a file creation method. The file creation method is executed by a data access device, and the method includes: the data access device obtains a file creation request, the file creation request is used to instruct to write the data to be processed to the storage device, and the file creation request carries the identifier of the data to be processed. Then, the data access device determines whether the identifier of the data to be processed is already stored in the identifier index cached by the data access device, and the identifier index includes the identifier of the data stored in the storage device; if the identifier of the data to be processed is not stored in the identifier index, the data access device sends a data write instruction to the storage device. The data write instruction carries the data to be processed, and the data write instruction is used to instruct the storage device to create a file containing the data to be processed.

[0007] In the present application, a data access device caches an identification index, and this identification index includes the identification of data already stored in the storage device. During the file creation process, the data access device can query the storage device based on this identification index to see whether the identification of the data is already stored. This offloads the data identification query process from the conventional storage device to the data access device. In other words, the data access device provides the computing power required for the data identification query process, thus preventing the data identification query process from occupying a large amount of the storage device's computing power. This helps reduce the data query time during the file creation process, thereby improving file creation efficiency.

[0008] In an optional implementation, the data access device includes: one or both of a user terminal and a smart network card that communicate with the above storage device.

[0009] Exemplarily, the data access device includes a user terminal, which may refer to a client that accesses a storage device, such as a mobile phone, a computer, a tablet computer, or a server.

[0010] In another exemplary embodiment, the data access device includes a smart network interface card (Smart NIC or SNIC), which is a flexible and programmable network card used in conjunction with a server.

[0011] Also illustratively, the data access device includes a user terminal and a smart network card.

[0012] In an optional implementation, the storage device includes multiple storage areas, and the identifiers of the data stored in different storage areas are different. The data access device also caches a partition index, and the partition index includes an index value range corresponding to each storage area in the multiple storage areas. Before the aforementioned data access device determines the storage status of the identifier of the data to be processed based on the identifier index cached by the data access device, the file creation method provided by the present application also includes: the data access device processes the identifier of the data to be processed according to a set algorithm to obtain an index value corresponding to the identifier of the data to be processed; the set algorithm includes: a hash algorithm and a model-based key-value mapping algorithm or a combination of one or two. In addition, the data access device queries the partition index based on the index value, determines the target storage area that matches the data to be processed in the multiple storage areas, and sends an index acquisition request to the storage device; the index acquisition request is used to instruct: the storage device sends the identifier of the data stored in the target storage area to the data access device. The data access device receives the identifier index sent by the storage device based on the index acquisition request, and the identifier index includes: the identifier of the data stored in the aforementioned target storage area.

[0013] In the present application, the data access device may only need to cache the identification index of one or a few storage areas in the storage device, so that the data transmission delay of the identification index between the data access device and the storage device is low, and the storage space required for the identification index in the data access device is small, which is conducive to further reducing the query time of data during the file creation process, thereby improving the efficiency of file creation.

[0014] In an optional implementation, the target address of the data to be processed, and the target storage area include a storage space corresponding to the target address.

[0015] In an optional implementation, the file creation method provided by the present application further includes: the data access device receiving index update information sent by the storage device, and updating the partition index according to the index update information. The index update information is used to indicate changes in the partition index.

[0016] Exemplarily, the data access device may send an index update request to the storage device and receive the aforementioned index update information.

[0017] As another example, the storage device may periodically send index update information to the data access device, so that the data access device updates the partition index.

[0018] In the present application, the data access device can update the partition index according to the changes in multiple storage areas in the storage device, so that the data access device can obtain the identification index based on the latest partition index during the query process of file creation, thereby avoiding the problem of the storage device creating duplicate files due to data with the same identification already stored in the storage device, and improving the accuracy of file creation.

[0019] In a second aspect, the present application provides a file creation method. The file creation method is executed by a storage device, and the file creation method includes: the storage device receives an index acquisition request sent by a data access device, and the index acquisition request carries the address of a first storage area to be requested, and the first storage area is one of multiple storage areas included in the storage device. In response to the index acquisition request, the storage device sends an identification index of the first storage area to the data access device; the identification index of the first storage area includes: an identification of the data stored in the first storage area. And, the storage device receives a data write instruction sent by the data access device, and the data write instruction carries data to be processed; the storage device creates a file containing the aforementioned data to be processed in response to the data write instruction.

[0020] In the present application, a storage device sends an identification index of one or more storage areas to a data access device. Subsequently, during the file creation process, the data access device can query the storage device based on the identification index to determine whether the data identifier is already stored. The data identifier query process is offloaded from the conventional storage device to the data access device. In other words, the data access device provides the computing power required for the data identifier query process, thus preventing the data identifier query process from occupying a large amount of the storage device's computing power. The storage device only needs to create a file containing the data to be processed based on a data write instruction sent by the data access device that carries the data to be processed. This helps reduce the data query time during the file creation process, thereby improving the efficiency of file creation.

[0021] In an optional implementation, the data write instruction further carries a target address of the data to be processed, and the first storage area includes storage space corresponding to the target address. In response to the data write instruction, the storage device creates a file containing the data to be processed, including: the storage device creates the file containing the data to be processed in the first storage area.

[0022] In an optional implementation, each storage area in the plurality of storage areas corresponds to a continuous address space included in a leaf node of a tree-shaped storage structure in the storage device.

[0023] In an optional implementation, before the storage device receives the index acquisition request sent by the data access device, the file creation method provided by the present application further includes: the storage device sends a partition index to the data access device, the partition index including an index value range corresponding to each storage area in a plurality of storage areas, wherein different index values ​​in the index value range correspond to identifiers of different data. Based on this, the storage device sends the identification index of the storage area to the data access device only after receiving the storage area to be accessed determined by the data access device based on the partition index. This allows the data access device to cache only the identification indexes of one or a few storage areas in the storage device, resulting in a low data transmission delay between the data access device and the storage device for the identification index, and a small amount of storage space required for the identification index in the data access device, which is conducive to further reducing the data query time during the file creation process, thereby improving the efficiency of file creation.

[0024] In a third aspect, the present application provides a data access device. The data access device includes: a transceiver and a processor. The transceiver is used to: obtain a file creation request, the file creation request is used to instruct to write the data to be processed to a storage device, and the file creation request carries an identifier of the data to be processed; the processor is used to: determine whether the identifier of the data to be processed is already stored in the identifier index cached by the data access device, the identifier index includes the identifier of the data stored in the storage device; if the identifier of the data to be processed is not stored in the identifier index, send a data write instruction to the storage device, the data write instruction carries the data to be processed, and the data write instruction is used to instruct the storage device to create a file containing the data to be processed.

[0025] In an optional implementation, the data access device includes: one or both of a user terminal and a smart network card.

[0026] In a fourth aspect, the present application provides a storage device. The storage device includes: a memory configured to provide multiple storage areas; a processor configured to receive an index acquisition request sent by a data access device, the index acquisition request carrying an address of a first storage area to be requested, the first storage area being one of the multiple storage areas included in the storage device; and executing the operating steps of the method of any optional implementation of the second aspect according to the index acquisition request.

[0027] In a fifth aspect, the present application provides a data access system, comprising: one or more storage devices provided in the fourth aspect, for storing data; and one or more data access devices provided in any optional implementation of the third aspect, for accessing the storage devices.

[0028] In a sixth aspect, the present application provides a computer-readable storage medium having computer software instructions stored therein, which, when executed on a computer, causes the computer to execute the operating steps of the method of any optional implementation of the first aspect or the second aspect.

[0029] In a seventh aspect, the present application provides a computer program product. When the computer program product is run on a computer, the computer executes the operating steps of the method of any optional implementation method in the first aspect or the second aspect.

[0030] In an eighth aspect, the present application provides a chip system. The chip system includes a processor configured to implement the functions of the data access device or storage device in the method of the first or second aspect. In one possible design, the chip system also includes a memory configured to store program instructions and / or data. The chip system may be composed solely of a chip or may include a chip and other discrete components.

[0031] Regarding the beneficial effects of the third to eighth aspects, reference may be made to the description of any optional implementation in the first or second aspects, which will not be repeated here. Based on the implementation provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG1 is a schematic diagram of the structure of a data access system provided by this application;

[0033] FIG2 is a schematic diagram of the structure and organization of a file storage system provided by the present application;

[0034] FIG3 is a flowchart of a file creation method provided by the present application;

[0035] FIG4 is a second flow chart of a file creation method provided by the present application;

[0036] FIG5 is a third flow chart of a file creation method provided by the present application;

[0037] FIG6 is a schematic diagram of two index updates provided by this application;

[0038] FIG7 is a fourth flow chart of a file creation method provided by the present application;

[0039] FIG8 is a schematic structural diagram of a data access device provided by this application. DETAILED DESCRIPTION

[0040] The present application provides a file creation method that offloads the data identification query process from the storage device to the data access device (such as a user terminal or a smart network card, etc.), avoiding the data identification query process occupying a large amount of computing power of the storage device, which is beneficial to reducing the data query time during the file creation process, thereby improving the efficiency of file creation.

[0041] Specifically, the data access device caches an identification index, which includes the identification of data already stored in the storage device. During the file creation process, the data access device can query the storage device based on the identification index to determine whether the data identification is already stored. This offloads the data identification query process from the typical storage device to the data access device. In other words, the data access device provides the computing power required for the data identification query process, preventing the data identification query process from occupying a large amount of the storage device's computing power. This helps reduce the data query time during the file creation process, thereby improving file creation efficiency.

[0042] The technical solutions involved in this application may be applicable not only to current data access devices or storage devices, but also to future data access devices and storage devices, or storage systems including the aforementioned data access devices or storage devices. The terms used in the implementation methods of this application are only used to explain the specific embodiments of this application and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in this application.

[0043] Key-Value pair (KV): A data storage structure consisting of a key and a value corresponding to the key.

[0044] An index node (inode) is used to record file metadata, such as the inode number, file size, access permissions, creation time, modification time, and the data's location on disk. An index node uniquely identifies a file and has a one-to-one correspondence with each other. These inodes are also stored on the hard drive, so they also take up disk space.

[0045] A directory entry (dentry) records a file's name, an inode pointer, and its hierarchical relationship to other directory entries. Multiple directory entries are linked together to form a directory structure. However, unlike inodes, a directory entry is a data structure maintained by the kernel and cached in memory rather than on disk.

[0046] In order to make the description of the following embodiments clear and concise, a brief introduction to the relevant technology is first given.

[0047] Figure 1 is a schematic diagram of the structure of a data access system provided by this application. The data access system includes: a data access device 100 and a storage device 120. In the application scenario shown in Figure 1, users access data through applications. The computers running these applications can be referred to as "computing devices."

[0048] The data access device 100 may be a physical machine or a virtual machine. The physical machine may include but is not limited to: one or both of a user terminal and a smart network interface card (smart NIC).

[0049] For example, the data access device 100 includes a user end, which may be a client, such as a host, a desktop computer, a server, a notebook computer, a mobile device, and the like.

[0050] For another example, the data access device 100 includes a smart network card (smart NIC). The smart network card, also known as an intelligent network adapter, not only can complete the network transmission function of a standard network card, but also provides a built-in programmable and configurable hardware acceleration engine to improve the performance of applications and significantly reduce the CPU consumption of the host connected to the smart network card in communication, providing more CPU resources for applications. For example, in a highly virtualized environment, the CPU in the host needs to run tasks related to the open virtual switch (OVS). At the same time, the CPU in the host also needs to process storage, online encryption and decryption of data packets or offline encryption and decryption, deep inspection of data packets, firewalls, complex routing and other operations. These operations not only consume a large amount of CPU resources, but also due to the competition for CPU resources between different services, the performance of the service cannot be optimized. As a hub connecting various services, the smart network card accelerates the above services on the smart network card.

[0051] For another example, the data access device 100 includes a client and a smart network card.

[0052] In a possible example, the data access device 100 accesses the storage device 120 through a network to access data. For example, the network may include a switch 110 .

[0053] In another possible example, the data access device 100 may also communicate with the storage device 120 through a wired connection, such as a universal serial bus (USB) or a peripheral component interconnect express (PCIe) bus.

[0054] The storage device 120 shown in Figure 1 can be a centralized storage system. A centralized storage system is characterized by a unified entry point through which all data from external devices must pass. This entry point is the centralized storage system's engine 121. Engine 121 is the core component of the centralized storage system, implementing many of the system's advanced functions.

[0055] As shown in Figure 1 , engine 121 may include one or more controllers. Figure 1 illustrates an example in which engine 121 includes a single controller. In one possible example, if engine 121 includes multiple controllers, any two controllers may have a mirroring channel, enabling mutual backup of the two controllers, thereby preventing hardware failures from causing unavailability of the entire storage device 120. It should be understood that if engine 121 includes multiple controllers, engine 121 may also be referred to as the array controller of storage device 120.

[0056] The engine 121 further includes a front-end interface 1211 and a back-end interface 1214. The front-end interface 1211 is used to communicate with the data access device 100, thereby providing data access services for the data access device 100. The back-end interface 1214 is used to communicate with the hard disk to expand the capacity of the storage device 120. Through the back-end interface 1214, the engine 121 can connect to more hard disks, thereby forming a very large storage resource pool.

[0057] In terms of hardware, as shown in FIG1 , the controller includes at least a processor 1212 and a memory 1213. The processor 1212 is a central processing unit (CPU) that is used to process data access requests from outside the storage device 120 (servers or other storage systems), and is also used to process requests generated within the storage device 120. Exemplarily, when the processor 1212 receives write data requests sent by the data access device 100 through the front-end interface 1211, it temporarily stores the data in these write data requests in the memory 1213. When the total amount of data in the memory 1213 reaches a certain threshold, the processor 1212 sends the data stored in the memory 1213 to at least one of the mechanical hard disk 1221, the mechanical hard disk 1222, the solid state drive (SSD) 1223, or the other hard disk 1224 through the back-end port for persistent storage.

[0058] Memory 1213 refers to an internal memory that directly exchanges data with the processor. It can read and write data at any time and at a high speed, and serves as a temporary data storage for the operating system or other running programs. Memory includes at least two types of memory. For example, memory can be either random access memory or read-only memory (ROM). For example, random access memory is DRAM or SCM. DRAM is a semiconductor memory that, like most random access memories (RAM), is a volatile memory device. However, DRAM and SCM are only exemplary in this embodiment. Memory can also include other random access memories, such as static random access memory (SRAM). As for read-only memory, for example, it can be programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), etc.

[0059] In addition, the memory 1213 can also be a dual in-line memory module or a dual in-line memory module (DIMM), that is, a module composed of dynamic random access memory (DRAM), or an SSD. In actual applications, multiple memories 1213 and different types of memories 1213 can be configured in the controller. This embodiment does not limit the number and type of memory 1213. In addition, the memory 1213 can be configured to have a power-saving function. The power-saving function means that when the system loses power and then powers on again, the data stored in the memory 1213 will not be lost. A memory with a power-saving function is called a non-volatile memory.

[0060] Memory 1213 stores software programs, and processor 1212 executes the software programs in memory 1213 to manage the hard disks. For example, the hard disks can be abstracted into a storage resource pool, which is then provided to the server in the form of a logical unit number (LUN). The LUN in this context is actually the hard disk seen by the server. Of course, some centralized storage systems are also file servers, providing shared file services to the server.

[0061] As shown in Figure 1, in this system, the engine 121 may not have a hard drive slot. The hard drive must be placed in the hard drive enclosure 122, and the back-end interface 1214 communicates with the hard drive enclosure 122. The back-end interface 1214 exists in the form of an adapter card in the engine 121. Two or more back-end interfaces 1214 can be used simultaneously on one engine 121 to connect to multiple hard drive enclosures. Alternatively, the adapter card can be integrated into the motherboard, in which case the adapter card can communicate with the processor 1212 via the PCIe bus.

[0062] It should be noted that FIG1 only shows one engine 121 . However, in actual applications, the storage system may include two or more engines 121 , and redundancy or load balancing may be performed between the multiple engines 121 .

[0063] The hard disk enclosure 122 includes a control unit 1225 and several hard disks. The control unit 1225 can have various forms. In one case, the hard disk enclosure 122 is an intelligent disk enclosure, as shown in Figure 1. The control unit 1225 includes a CPU and memory. The CPU is used to perform operations such as address translation and reading and writing data. The memory is used to temporarily store data to be written to the hard disk or read from the hard disk to be sent to the controller. In another case, the control unit 1225 is a programmable electronic component, such as a data processing unit (DPU). The DPU has the versatility and programmability of a CPU, but is more specialized and can efficiently operate on network data packets, storage requests, or analysis requests. The DPU is distinguished from the CPU by its high degree of parallelism (it needs to process a large number of requests). Optionally, the DPU can be replaced with a processing chip such as a graphics processing unit (GPU) or an embedded neural network processing unit (NPU). Typically, the number of control units 1225 can be one, two, or more. The functions of the control unit 1225 can be offloaded to the network interface card 1226. In other words, in this embodiment, the hard disk frame 122 does not have a control unit 1225 inside, but the network card 1226 completes data reading and writing, address conversion and other computing functions. At this time, the network card 1226 is an intelligent network card. It can include a CPU and memory. The CPU is used to perform operations such as address conversion and reading and writing data. The memory is used to temporarily store data to be written to the hard disk, or data read from the hard disk to be sent to the controller. It can also be a programmable electronic component, such as a DPU. There is no ownership relationship between the network card 1226 and the hard disk in the hard disk frame 122. The network card 1226 can access any hard disk in the hard disk frame 122 (such as the mechanical hard disk 1221, mechanical hard disk 1222, solid-state drive 1223 and other hard disks 1224 shown in Figure 1), so it is more convenient to expand the hard disk when storage space is insufficient.

[0064] Depending on the type of communication protocol between the engine 121 and the hard disk frame 122, the hard disk frame 122 may be a serial attached small computer system interface (SAS) hard disk frame, or it may be an NVMe (Non-Volatile Memory express) hard disk frame or other types of hard disk frames. The SAS hard disk frame uses the SAS3.0 protocol, and each frame supports 25 SAS hard disks. The engine 121 is connected to the hard disk frame 122 through an onboard SAS interface or a SAS interface module. The NVMe hard disk frame is more like a complete computer system, and the NVMe hard disk is inserted into the NVMe hard disk frame. The NVMe hard disk frame is then connected to the engine 121 through the RDMA port.

[0065] For example, the storage device 120 may refer to a storage array, such as an all-flash storage array in which all storage media are flash memories.

[0066] In an optional implementation, the storage device 120 is a centralized storage system with integrated disk and controller. The storage device 120 does not have the aforementioned disk enclosure 122. The engine 121 is used to manage multiple hard disks connected via the disk slots. The function of the disk slots can be implemented by the backend interface 1214.

[0067] In another optional implementation, the storage device 120 shown in FIG1 is a distributed storage system comprising a computing device cluster and a storage device cluster. The computing device cluster includes one or more computing devices, each of which can communicate with each other. The computing device can be a computing device, such as a server, desktop computer, or storage array controller. In terms of hardware, the computing device can include a processor, memory, and a network interface card. The processor is a CPU, which processes data access requests from outside the computing device or requests generated within the computing device. For example, when the processor receives a write data request from a user, it temporarily stores the data in the write data request in memory. When the total amount of data in memory reaches a certain threshold, the processor sends the data stored in memory to the storage device for persistent storage. In addition, the processor is used to perform data calculations or processing, such as metadata management, data deduplication, data compression, virtualized storage space, and address translation. In one example, any computing device can access any storage device in the storage device cluster via a network. The storage device cluster includes multiple storage devices. A storage device includes one or more controllers, a network interface card, and multiple hard disks. The network interface card is used to communicate with the computing device.

[0068] It is worth noting that the above examples are merely possible implementations of the data access system provided in this embodiment and should not be construed as limiting the present application. It should be understood that in the storage device 120 shown in FIG1 , data is stored in the form of files on each hard disk. The files stored on each hard disk constitute a file storage system, which may be, for example, a distributed file system. For example, a Network File System (NFS) is both a distributed file system and a network protocol used to access and share files between devices on the same local area network. For example, a NAS system can be implemented using the NFS protocol. The Network File System is a low-cost network file sharing option that enables users and applications to access, store, and update files on remote computers, much like using direct-attached storage. The Network File System uses the Remote Procedure Call protocol to route requests between clients and servers. While participating devices must support the Network File System, they do not need to understand the detailed network information. It is worth noting that remote procedure calls can be insecure, so the Network File System should only be deployed on trusted networks behind firewalls. Although Windows supports this protocol, it is primarily used in Linux environments.

[0069] As shown in Figure 2, Figure 2 is a schematic diagram of the structure and organization of a file storage system provided by this application. The hardware implementation of file storage system 200 can refer to the relevant content of storage device 120 in Figure 1 and is not described in detail here. As shown in Figure 2, file storage system 200 includes one or more index nodes, and the data in an index node is processed and accessed by a controller node.

[0070] A file storage system, also known as a file system, is a structured data file storage and organization format. As shown in Figure 2, an index node includes multiple directory sets, such as Directory Set 1, Directory Set 2, and Directory Set 3. A directory set consists of one or more directories, each of which stores multiple files. Each file can be used to store one or more sets of data.

[0071] In computers, all data consists of 0s and 1s. The series of 0s and 1s stored on hardware media is completely incomprehensible and impossible for computers to understand or manage. Therefore, computers use the concept of "files" to organize this data. Data used for the same purpose can be organized into different types of files according to the structure required by different applications. Different suffixes are typically used to denote different types of data, and the computer then assigns each file a name that is easy to understand and remember.

[0072] One possible storage structure is as shown in Figure 2 (Storage Structure 1). The computer groups these files (the black squares in Figure 2) according to a certain partitioning method, and each group of files is placed in the same directory (or folder). Furthermore, in addition to files, directories can also contain sub-directories (called sub-directories or sub-folders), forming a tree-like structure. This tree structure has a special name: file system. There are many types of file systems, including FAT / FAT32 / NTFS in Windows and EXT2 / EXT3 / EXT4 / XFS / BtrFS in Linux. To facilitate searching, starting from the root node and working down through the directories, all the way to the file itself, the names of these directories, sub-directories, and files are concatenated using special characters (such as "\" in Windows / DOS and " / " in Unix-like systems). This string of characters is called a file path, such as " / etc / systemd / system.conf" in Linux or "C:\Windows\System32\taskmgr.exe" in Windows. A path is a unique identifier for accessing a specific file. For example, D:\data\file.exe in Windows is a file path, which represents the file.exe file in the data directory under the D partition.

[0073] File systems are built on top of block devices. They not only record file paths but also which blocks make up a file and which blocks contain directory / subdirectory information. Different file systems have different organizational structures. For ease of management, block devices like hard drives are typically divided into multiple logical block devices, known as partitions. Conversely, due to the limited capacity and performance of a single medium, various technologies can be used to combine multiple physical block devices into a single logical block device, such as various levels of redundant arrays of independent disks (RAID) and just a bunch of disks (JBOD). File systems can also be built on top of these logical block devices. Regardless, the host application does not need to know the specific location of the file it wants to access on the underlying block device. It only needs to send the file's file name / identifier (ID) to the file system, which then retrieves the file path based on the file name / ID.

[0074] Common file access protocols include NFS, Common Internet of File System (CIFS), or Server Message Block (SMB) protocols, which are not limited in this embodiment.

[0075] For example, file-level tiered storage can be configured with various policies. For example, it can use various attributes such as the associated application, directory, user, group, call method, size, and access frequency as classification criteria and trigger conditions, allowing migration to any volume or directory. Block-level tiered storage, on the other hand, typically migrates the entire LUN or a portion of its blocks based on the access frequency of the entire LUN or a specific block or blocks within the LUN.

[0076] Another possible storage structure is shown in Figure 2 (Storage Structure 2). The file system can use a tree-like storage structure (also known as a tree-like data structure, tree-like index structure, etc.) that supports offloading. This solves the problem that traditional dentry tables cannot be offloaded using heterogeneous computing power. For example, the file system stores complete tree intermediate nodes (index nodes) and leaf nodes, with the leaf nodes storing dentry metadata.

[0077] In the (storage structure 2) shown in FIG2 , index node 0 includes multiple other index nodes, such as index node 1 to index node 3. Each index node includes only a key and does not include data content. In some optional cases, the keys of the index nodes are arranged in ascending order. For the key of a specified index node, the keys of all index nodes in the left subtree of the specified index node are smaller than the key of the specified index node (e.g., the key of index node 1 is smaller than the key of index node 2), and the keys of all index nodes in the right subtree of the specified index node are greater than or equal to the key of the specified index node (e.g., the key of index node 3 is smaller than the key of index node 2).

[0078] Each leaf node may include one or more storage partitions (partitions), each of which may include a continuous address space. For example, leaf node 1 includes partitions 1 to m, and leaf node 2 includes partitions 1 to k.

[0079] In this embodiment, the data access device can directly calculate the address of the leaf node to be accessed through the cached index node, and then directly read the leaf node of the tree from the storage end using a one-way read method and parse it to obtain the target dentry metadata.

[0080] The tree-based storage structure uses a large-leaf design, aiming to store the same number of records using fewer leaf nodes (hereinafter referred to as leaves). This results in fewer internal index nodes in the tree. This means that the client or SmartNIC cache requires less memory resources to cache the internal index nodes in the tree, or, with the same resources, more internal index nodes can be cached (e.g., index nodes 0 to 3 in Figure 2). This improves cache efficiency and ultimately query performance: the leaf address of the target record stored on the server can be directly calculated from the cached index nodes, allowing for direct unilateral DMA reads of the records in the leaf nodes.

[0081] The large leaves are partitioned to improve the performance of one-way reading. As an optional example, the size of each index node (inode nodesize) is 8 kilo bytes (KB), the size of each leaf node is 1 million bytes (MB), and the storage capacity of each partition in the leaf node is 16KB. For example, a 1MB leaf is divided into 64 16KB partitions. The leaf partition corresponding to the target key can be calculated through the intermediate index node, and only a single 16KB target partition is read when reading the content of the leaf node. The purpose of doing this is because the performance of one-way reading is related to the size of the read content. Directly reading a large leaf will cause the one-way reading performance to be significantly reduced, while reading a single small leaf partition can ensure the high performance of one-way reading.

[0082] The file creation method provided by the present application is described in detail below in combination with the data access system shown in Figure 1 and the storage structure 2 of the file storage system 200 shown in Figure 2, as shown in Figure 3, which is a flow chart 1 of a file creation method provided by the present application.

[0083] The file creation method shown in Figure 3 can be applied to the data access system shown in Figure 1. The data access device 31 can be used to perform the functions of the data access device 100 in Figure 1, and the storage device 32 can be used to implement the functions of the storage device 120 in Figure 1. Regarding the hardware implementation of the data access device 31 and the storage device 32, please refer to the description of Figures 1 and 2 above and will not be repeated here.

[0084] Please refer to FIG. 3 . The file creation method shown in FIG. 3 includes the following steps S310 to S360 .

[0085] S310 : The data access device 31 sends an index acquisition request to the storage device 32 .

[0086] Corresponding to S310 , the storage device 32 receives the index acquisition request sent by the data access device 31 .

[0087] The index acquisition request carries the address of the first storage area to be requested, and the first storage area is one of the multiple storage areas included in the storage device 32, such as any one of storage area 1 to storage area n, where n is a positive integer.

[0088] In this embodiment, each of the plurality of storage areas corresponds to a continuous address space included in a leaf node of the tree storage structure in the storage device 32. For a description of each leaf node of the tree storage structure, please refer to the relevant content of FIG. 2 and will not be repeated here.

[0089] Exemplarily, a storage area includes: all address spaces in a leaf node, such as the address space included in leaf node 1 or the address space included in leaf node 6 in FIG. 2 .

[0090] As another example, a storage area includes: a portion of the address space in a leaf node, which may refer to the address space of a partition in the leaf node, or may refer to the address space of multiple consecutive partitions in the leaf node. For example, referring to (storage structure 2) in FIG2 , the multiple consecutive partitions may refer to: partition 1 to partition 2, or partition 2 to partition i (i is an integer greater than 2 and less than k) in leaf node 6, etc., and this application does not limit this.

[0091] The above two examples are merely possible implementations of the storage area provided in this embodiment and should not be construed as limitations on this application.

[0092] In this embodiment, the data access device 31 can only obtain index information of one or a few storage areas among the multiple storage areas, avoiding the data access device 31 from needing to obtain index information of a large amount of data, thereby reducing the amount of data that needs to be transmitted between the data access device 31 and the storage device 32, and also reducing the amount of data that needs to be stored in the memory of the data access device 31, that is, reducing the storage resources of the data access device 31 that the above-mentioned index information needs to occupy.

[0093] S320 : The storage device 32 sends the identification index (identification index 1 ) of the first storage area (storage area 1 ) to the data access device 31 in response to the index acquisition request.

[0094] The identification index of the first storage area includes: an identification of data stored in the first storage area.

[0095] The first storage area may refer to one of the multiple storage areas included in the storage device 32, such as storage area 1. For hardware implementation of the multiple storage areas, reference may be made to the description of FIG1 . For example, FIG2 is provided for illustration, and the storage area 1 may include one or more partitions in a leaf node. For details, reference may be made to the description of S310 , which is not detailed here.

[0096] The identifier of the data stored in the storage area 1 may refer to: the file name of the file containing the stored data.

[0097] Alternatively, in an application scenario where a large number of small files are created, the identifier of the data stored in the storage area 1 may be dentry metadata. The dentry metadata is stored in a dentry table and is a pointer to the data stored in the storage device 32 .

[0098] Hereinafter, storage area 1 may be referred to as the first storage area, storage area 2 may be referred to as the second storage area, and storage area n may be referred to as the nth storage area.

[0099] S330: The data access device 31 obtains a file creation request.

[0100] The query file req carries the identifier of the data to be processed. The contents of the identifier can be found in the description of S320 above and will not be repeated here. The query file req is used to instruct the data to be processed to be written into the storage device 32 .

[0101] In a possible example, the file creation request is a request generated by the data access device 31 according to a running service.

[0102] In another possible example, the file creation request is a request sent by another client or a smart network card (SNIC) to the data access device 31. In this case, S330 includes: the data access device 31 receives the file creation request.

[0103] The above two examples are merely possible ways of obtaining a file creation request provided in this embodiment and should not be construed as limiting the present application.

[0104] S340 : The data access device 31 determines whether the identifier of the data to be processed is stored in the identifier index cached by the data access device 31 .

[0105] The identification index includes the identification of the data stored in the storage device 32 .

[0106] In an optional example, the identification index includes: keys (identifications) stored in all index nodes in the storage device 32 .

[0107] In another optional example, the identification index includes: keys (identifiers) stored in some index nodes in the storage device 32. Exemplarily, the identification index only includes keys stored in index nodes of one storage area or a few storage areas in the storage device 32, such as the keys stored in the storage area 1 above, which are used to indicate files stored in a continuous address space included in a leaf node 1 in the storage device 32.

[0108] Regarding the process of the data access device 31 caching the above identification index, S310 and S320 are merely examples provided in this implementation and should not be understood as limiting the present application. The following embodiments also provide other possible implementation methods, which are not described here in detail.

[0109] If the identification index does not store the identification of the data to be processed, the data access device 31 executes S350; if the identification index already stores the identification of the data to be processed, the data access device 31 determines that a file containing the data to be stored has been stored in the storage device 32, and ends the file creation process.

[0110] S350 : The data access device 31 sends a data write instruction to the storage device 32 .

[0111] The data write instruction carries the data to be processed, and is used to instruct the storage device 32 to create a file containing the data to be processed.

[0112] Corresponding to the designated sending process of S350 , the storage device 32 receives the data write instruction sent by the data access device 31 .

[0113] S360 : The storage device 32 creates a file containing the data to be processed in response to the data write instruction.

[0114] In one optional implementation, the data write instruction also carries the target address of the data to be processed, and the aforementioned storage area 1 includes the storage space corresponding to the target address. Exemplarily, the aforementioned S360 includes the storage device 32 creating a file containing the data to be processed in the first storage area (storage area 1). That is, the storage device 32 stores the created file in storage area 1.

[0115] In this embodiment, by offloading the data identification query process from the storage device 32 to the data access device 31 (such as a user terminal or a smart network card, etc.), the data identification query process is avoided from occupying more computing power of the storage device 32, which is beneficial to reducing the query time of the data during the file creation process, thereby improving the efficiency of file creation. As described in the above embodiments, an identification index is cached in the data access device 31, and the identification index includes the identification of the data stored in the storage device 32. During the file creation process, the data access device 31 can query whether the identification of the data is already stored in the storage device 32 based on the identification index. The data identification query process is offloaded from the usual storage device 32 to the data access device 31, that is, the data access device 31 provides the computing power required for the data identification query process, which avoids the data identification query process occupying more computing power of the storage device 32, which is beneficial to reducing the query time of the data during the file creation process, thereby improving the efficiency of file creation.

[0116] When the above data access device refers to a pluggable smart network card, the smart network card has computing capabilities. In order to release the CPU computing power of the client to which the smart network card belongs, the smart network card will offload the data processing functions in network, security, and storage that are not suitable for CPU-related functions to the programmable hardware chip for execution, reducing the client's CPU consumption, enabling the client to run key applications and operating systems more effectively, and optimizing the processing efficiency of business data.

[0117] It is worth noting that the above-mentioned identification index acquisition process is only one possible implementation method provided by this embodiment. For the above-mentioned identification index acquisition process, another possible implementation method is also provided below, as shown in Figure 4, which is a flow chart of a file creation method provided by this application. The execution subject of the file creation method can refer to the description of Figure 3, which will not be repeated here; the data access device includes a memory that can read and write data at any time and at a high speed, serving as a temporary data storage for the operating system or other running programs. For example, the memory can refer to DRAM, SRAM, or SCM, etc.

[0118] Before the aforementioned S320 , the file creation method provided in this embodiment includes steps S401 to S404 .

[0119] S401 : The data access device 31 obtains a partition index of the storage device 32 .

[0120] The storage device 32 includes multiple storage areas, and different storage areas store data with different identifiers.

[0121] The partition index includes a range of index values ​​corresponding to each of the multiple storage areas, where different index values ​​within the range correspond to identifiers of different data. The index value herein may be a value calculated based on the identifier of the data according to a predetermined algorithm. In some optional examples, the amount of data in the index value is smaller than the amount of data in the identifier, thereby reducing the storage resources occupied by the partition index and reducing the storage space required by the data access device 31 to cache the partition index.

[0122] In an optional implementation, the storage device 32 periodically sends the partition index to each data access device, such as when the storage device 32 sends the partition index to the data access device 31. In this way, each data access device can quickly determine the file storage status of each storage area in the storage device 32 based on the partition index, avoiding repeated negotiations between the data access device and the storage device, and improving the efficiency of the file creation process performed by the data access device and the storage device.

[0123] In another optional implementation, the data access device 31 requests the partition index from the storage device 32 based on its own business needs. For example, the data access device 31 sends a request to the storage device 32 to obtain the partition index, and the storage device 32 sends the partition index to the data access device 31 in response to the request. In this way, when there are a large number of data access devices communicating with the storage device 32, the storage device 32 does not need to consume a large amount of broadcast resources to synchronize the partition index in the storage device 32, thereby reducing the network resources (such as data transmission bandwidth) and processor computing power consumption of the storage device 32, which is conducive to improving the efficiency of the storage device 32 in executing various data access services (such as file creation, data reading, or garbage collection, etc.).

[0124] The above two optional implementations are merely examples provided in this embodiment and should not be construed as limiting the present application. In some examples, the partition index synchronization between the data access device and the storage device may also take other forms, such as the storage device caching the partition index in a shared memory that can be read simultaneously by multiple data access devices, and each data access device querying the partition index from the shared memory in response to a received file creation request.

[0125] S402 : The data access device 31 processes the identifier of the data to be processed according to a set algorithm to obtain an index value corresponding to the identifier of the data to be processed.

[0126] The above-mentioned set algorithm includes: a hash algorithm and a model-based key-value mapping algorithm or a combination of the two.

[0127] For example, the algorithm includes a hash algorithm. A hash algorithm transforms an input of arbitrary length (also called a pre-image) into an output of fixed length through a hash algorithm. The output is the hash value. The hash algorithm operation process is a compression mapping.

[0128] For another example, the set algorithm includes: a model-based key-value mapping algorithm. For example, the model may refer to a neural network model implemented based on a deep neural network (DNN) or a convolutional neural network (CNN). DNN, also known as a multi-layer neural network, can be understood as a neural network with many hidden layers. The "many" here does not have a special metric standard; CNN is a deep neural network with a convolutional structure. For more information about DNN and CNN, please refer to the description of the general technology and will not be elaborated here.

[0129] For example, the set algorithms include: a hash algorithm and a model-based key-value mapping algorithm.

[0130] The above examples are only possible implementations of the set algorithm provided in this embodiment and should not be understood as limitations on this application. Other algorithms that can determine a unique index value based on the identification of the data may also be included in the set algorithm.

[0131] S403 , the data access device 31 queries the partition index according to the index value, determines a target storage area among the multiple storage areas that matches the data to be processed, and sends an index acquisition request to the storage device 32 .

[0132] The index acquisition request is used to instruct the storage device 32 to send the identifier of the data stored in the target storage area to the data access device 31. For example, the target storage area may be storage area 1. For more details about the index acquisition request, please refer to S310 and will not be repeated here.

[0133] Exemplarily, the partition index includes the content of the following Table 1, which is a possible implementation method of the partition index provided by this embodiment.

[0134] Table 1

[0135] Among them, the index value range included in storage partition 1 is 0 to 9, the index value range included in storage partition 2 is 10 to 19, and the index value range included in storage partition n is (n-1)×10 to (n-1)×10+9. That is, the index value range corresponding to a single storage partition includes 10 index values, and each storage partition can store 10 files. Of course, in the partition index, each storage area can also correspond to a larger or smaller index value range, that is, each storage partition can also be used to store more or fewer files, which is not limited in this application.

[0136] S404: The data access device 31 receives the identification index sent by the storage device 32 according to the index acquisition request.

[0137] The identification index includes: an identification of data stored in the target storage area, for example, the identification index is identification index 1. For more details about the identification index, please refer to the description of S330 above, which will not be repeated here.

[0138] In the present application, the data access device may only need to cache the identification index of one or a few storage areas in the storage device 32, so that the data transmission delay of the identification index between the data access device 31 and the storage device 32 is low, and the storage space required for the identification index in the data access device 31 is small, which is conducive to further reducing the query time of data during the file creation process, thereby improving the efficiency of file creation.

[0139] Optionally, based on the aforementioned Figures 3 and 4, the present application further provides an optional implementation method for the partition index content, as shown in Figure 5, which is a flowchart diagram of a file creation method provided by the present application. After the aforementioned S404, the file creation method provided in this embodiment also includes the following S405 and S406.

[0140] S405 : The data access device 31 obtains index update information of the storage device 32 .

[0141] The index update information is used to indicate changes in the partition index in the storage device 32 .

[0142] Regarding the process of the data access device 31 obtaining index update information, the following takes an example in which one storage area corresponds to one leaf node, the index value range of each storage area is the same as the index node to which the leaf node belongs, and the above-mentioned partition index includes each index node in the storage device 32. In combination with Figure 6, two possible scenarios are provided. Figure 6 is a schematic diagram of two index updates provided in this application.

[0143] In the first optional scenario, if the number of data access devices communicating with the storage device 32 is small, the storage device 32 may adopt an active push method, namely: 1. The partition index in the storage device 32 changes; 2. The storage device 32 actively sends the updated partition index or index update information to each data access device; accordingly, the data access device receives the partition index or index update information sent by the storage device, and 3. The data access device updates the partition index according to the index update information. In this scenario, the storage device adopts a broadcast method to one or more data access devices to synchronize and update the partition index cached by each data access device. Each data access device can obtain the latest partition index in a short time after the partition index in the storage device changes, avoiding errors in the subsequent file creation process, such as repeated creation of the same file, etc., which is conducive to improving the accuracy of file creation while ensuring the timeliness of partition index updates.

[0144] In the second optional scenario, if there are a large number of data access devices communicating with the storage device 32, the data access device can be used to actively synchronize, that is: 1. The data access device triggers the partition index synchronization process periodically or through a statistical index cache hit rate (or miss rate) strategy. For example, when the data access device receives an input / output (IO) request, it sends an index update request to the storage device 32; accordingly, 2. The storage device 32 responds to the index update request and sends the partition index of the storage device 32 to the data access device; thus, 3. The data access device receives the partition index and updates the old partition index. In this scenario, each data access device obtains the partition index from the storage device on demand, so that only the data access devices that actually have IO access among the multiple data access devices will synchronize and cache the partition index, and the data access devices without IO access will not cache the partition index, which reduces the waste of storage resources of these data access devices without IO access and avoids the broadcast overhead caused by the storage device having to push the partition index to a large number of data access devices, which is conducive to improving the data management performance within the storage device.

[0145] In the above two scenarios, the number of data access devices can be set according to the user's usage needs or the performance requirements of the data access system. For example, if the number of data access devices is greater than or equal to the quantity threshold, it means a large number, and if it is less than the quantity threshold, it means a small number. The quantity threshold can be 100, 1000 or other possible values, and this application does not limit this.

[0146] S406 : The data access device 31 updates the partition index according to the index update information.

[0147] In a possible example, if the index update information includes all contents of the updated partition index, the data access device 31 caches the index update information as the updated partition index of the storage device 32 in the memory of the data access device 31 and deletes the old partition index stored in the memory.

[0148] In another possible example, if the index update information includes part of the content of the updated partition index, the data access device 31 overwrites the content in the index update information that is different from the old partition index into the old partition index to obtain the updated partition index.

[0149] The above two possible examples are possible implementations of the partition index update method provided in this embodiment and should not be understood as limitations on this application.

[0150] In the present application, the data access device can update the partition index according to the changes in multiple storage areas in the storage device, so that the data access device can obtain the identification index based on the latest partition index during the query process of file creation, thereby avoiding the problem of the storage device creating duplicate files due to data with the same identification already stored in the storage device, and improving the accuracy of file creation.

[0151] The following describes the file creation method provided in an embodiment of the present application, assuming that the data identifier refers to the file name (filename), the identifier index refers to the dentry metadata, the storage area corresponds to the leaf node, and the partition index corresponds to the index node. As shown in FIG7 , FIG7 is a fourth flowchart of a file creation method provided in the present application. The file creation method provided in this embodiment includes the following steps S701 to S704.

[0152] S701: The data access device 31 obtains a file creation request.

[0153] For a more detailed description of S701, please refer to the content of S330, which will not be repeated here.

[0154] S702 , the data access device 31 queries the cached partition index according to the file name (filename1) of the data in the file creation request, determines the target leaf node, and unilaterally reads the identification index of the target leaf node in the storage device 32 .

[0155] Exemplarily, the input for the query operation performed by the data access device 31 is filename1, and the filename1 is used as a key to query the address of the target leaf node in the cached partition index (such as a tree index node).

[0156] If the target leaf node refers to leaf node 6, the identification index refers to the identification index 6 of the leaf node 6, and the identification index 6 includes: the file name (filename) of the file stored in the leaf node 6. In the identification index 6, each file name exists in the form of dentry metadata: the dentry metadata includes multiple KV pairs, such as filename->inode ID, and the inode ID refers to the ID of the index node where the leaf node to which the filename belongs is located, such as index node 2.

[0157] After the data access device 31 queries and determines that the file name of the data in the file creation request is not stored in the identification index 6, S703 is executed. The query method can be a binary search or a traversal method. Taking the traversal query as an example, the data access device 31 compares the identification index 6 (dentry metadata) one by one to see whether there is a KV pair containing filename1. If not, S703 is executed.

[0158] S703 : The data access device 31 sends a data write instruction carrying the data in the file creation request to the storage device 32 .

[0159] S704 : The storage device 32 creates a file in response to the data write instruction.

[0160] Furthermore, the storage device 32 creates a KV pair including filename1 and writes the newly created KV pair into the identification index 6 of the storage device 32. Accordingly, the storage device 32 updates the partition index according to the identification index 6 after the new KV pair has been written.

[0161] The index update process between the data access device and the storage device can refer to the processes in Figures 4 to 6 above, and will not be described in detail here.

[0162] In the accompanying drawings corresponding to the above embodiments, only one data access device is shown. However, in some optional implementations, a single storage device can be provided for access by multiple different data access devices, which can still implement the file creation method provided by this application and will not be described in detail here.

[0163] It is understood that, in order to implement the functions described in the above embodiments, the data access device and storage device include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily appreciate that, in conjunction with the various exemplary units and method steps described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in hardware or in a manner driven by computer software depends on the specific application scenario and design constraints of the technical solution.

[0164] The file creation method provided in accordance with the present embodiment is described in detail above with reference to the accompanying drawings. The data access device provided in accordance with the present embodiment will be described below with reference to FIG. 8 .

[0165] FIG8 is a schematic diagram of the structure of a data access device provided by the present application. This data access device can be used to implement the functions of the data access device in the above-described method embodiments, thereby also achieving the beneficial effects of the above-described method embodiments. In this embodiment, the data access device can be the data access device 100 shown in FIG1 , or the data access device 31 in subsequent embodiments, or a module (e.g., a chip) used in a data access device.

[0166] As shown in FIG8 , the data access device 800 may include a processor 820. Optionally, the data access device 800 may further include a memory 830 and / or a transceiver 810. The processor 820 is coupled to the memory 830 and the transceiver 810, for example, by a communication bus. The communication bus may include, but is not limited to, a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (UBus or UB), a Compute Express Link (CXL), a Cache Coherent Interconnect for Accelerators (CCIX), and the like.

[0167] The following is a detailed introduction to the various components of the data access device 800 in conjunction with FIG8 :

[0168] The processor 820 is the control center of the data access device 800 and can be a single processor or a collective term for multiple processing elements. For example, the processor 820 can be one or more CPUs, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application, such as one or more digital signal processors (DSPs) or one or more field programmable gate arrays (FPGAs).

[0169] Optionally, the processor 820 may execute various functions of the data access device 800 by running or executing software programs stored in the memory 830 and calling data stored in the memory 830. In a specific implementation, as an embodiment, the processor 820 may include one or more CPUs.

[0170] Optionally, the data access device 800 may also include multiple processors. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0171] Wherein, the memory 830 is used to store the software program for executing the solution of the present application, and is controlled by the processor 820 to execute. The specific implementation method can refer to the above method embodiment and will not be repeated here. Exemplarily, the memory 830 can be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this. The memory 830 can be integrated with the processor 820, or it can exist independently and be coupled to the processor 820 through the interface circuit of the data access device 800 (not shown in Figure 8). The embodiment of the present application does not specifically limit this.

[0172] Transceiver 810 is used for communication with other devices. For example, if data access device 800 is a user end (such as a client) or an application server, transceiver 810 can be used to communicate with a storage device or another data access device. For another example, if data access device 800 is a Smart Network Interface Card (SNIC), transceiver 810 can be used to communicate with a storage device or another Smart Network Interface Card. For another example, if data access device 800 is a multi-core chip, transceiver 810 can be used to communicate with another multi-core chip.

[0173] Optionally, the transceiver 810 may include a receiver and a transmitter (not shown separately in FIG8 ). The receiver is used to implement a receiving function, and the transmitter is used to implement a transmitting function. Optionally, the transceiver 810 may be integrated with the processor 820, or may exist independently and be coupled to the processor 820 via an interface circuit (not shown in FIG8 ) of the data access device 800. This embodiment of the present application does not specifically limit this.

[0174] In this embodiment, the transceiver 810 is used to obtain a file creation request, where the file creation request is used to instruct writing the data to be processed into a storage device, and the file creation request carries an identifier of the data to be processed.

[0175] The processor 820 is configured to determine whether an identifier of data to be processed is stored in an identifier index cached by the data access device, where the identifier index includes identifiers of data stored in the storage device.

[0176] The transceiver 810 is further configured to: if the identification index does not store the identification of the data to be processed, send a data write instruction to the storage device, the data write instruction carrying the data to be processed, and instructing the storage device to create a file containing the data to be processed.

[0177] The memory 830 is used to store information such as partition indexes and identification indexes. For more information about the partition indexes and identification indexes, please refer to the description of the aforementioned embodiments and will not be repeated here.

[0178] The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in RAM, flash memory, ROM, PROM, EPROM, EEPROM, registers, hard disk, mobile hard disk, CD-ROM or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium can be located in an ASIC. In addition, the ASIC can be located in a computing device. Of course, the processor and storage medium can also exist as discrete components in a network device or a terminal device.

[0179] The present application also provides a chip system, which includes a processor for implementing the functions of the data processing unit in the above method. In one possible design, the chip system also includes a memory for storing program instructions and / or data. The chip system can be composed of a chip alone or can include a chip and other discrete devices.

[0180] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is performed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive (SSD).

[0181] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A file creation method, characterized in that: The method is executed by a data access device, and comprises: Obtaining a file creation request, where the file creation request is used to instruct writing the data to be processed into a storage device, and the file creation request carries an identifier of the data to be processed; Determine whether the identification index of the data access device cache has stored the identification of the data to be processed, wherein the identification index includes the identification of the data stored in the storage device; If the identification index does not store the identification of the data to be processed, a data write instruction is sent to the storage device, the data write instruction carries the data to be processed, and the data write instruction is used to instruct the storage device to create a file containing the data to be processed.

2. The method according to claim 1, characterized in that The data access device includes: one or both of a user terminal and a smart network card that communicate with the storage device.

3. The method according to claim 1 or 2, characterized in that: The storage device includes multiple storage areas, and different storage areas store data with different identifiers; the data access device also caches a partition index, and the partition index includes an index value range corresponding to each storage area in the multiple storage areas; Before determining the storage state of the identification of the to-be-processed data according to the identification index cached by the data access device, the method further includes: Processing the identifier of the data to be processed according to a set algorithm to obtain an index value corresponding to the identifier of the data to be processed; the set algorithm includes: a hash algorithm and a model-based key-value mapping algorithm or a combination of both; Query the partition index according to the index value, determine the target storage area matching the data to be processed among the multiple storage areas, and send an index acquisition request to the storage device; the index acquisition request is used to instruct: the storage device sends the identifier of the data stored in the target storage area to the data access device; The identification index sent by the storage device according to the index acquisition request is received, where the identification index includes: an identification of data stored in the target storage area.

4. The method according to claim 3, characterized in that The data write instruction also carries: a target address of the data to be processed, and the target storage area includes a storage space corresponding to the target address.

5. The method according to claim 3 or 4, characterized in that: The method further comprises: Receiving index update information sent by the storage device, where the index update information is used to indicate a change in the partition index; The partition index is updated according to the index update information.

6. A file creation method, characterized in that: The method is performed by a storage device, and the method includes: receiving an index acquisition request sent by a data access device; the index acquisition request carries an address of a first storage area to be requested, the first storage area being one of a plurality of storage areas included in the storage device; In response to the index acquisition request, sending the identification index of the first storage area to the data access device; the identification index of the first storage area includes: an identification of the data stored in the first storage area; receiving a data write instruction sent by the data access device, wherein the data write instruction carries data to be processed; In response to the data writing instruction, a file containing the data to be processed is created.

7. The method according to claim 6, characterized in that The data write instruction also carries a target address of the data to be processed, and the first storage area includes a storage space corresponding to the target address; The step of creating a file containing the data to be processed in response to the data writing instruction comprises: A file containing the data to be processed is created in the first storage area.

8. The method according to claim 6 or 7, characterized in that: Each storage area in the plurality of storage areas corresponds to: a continuous address space included in a leaf node of a tree storage structure in the storage device.

9. The method according to any one of claims 6 to 8, characterized in that: Before receiving the index acquisition request sent by the data access device, the method further includes: A partition index is sent to the data access device, where the partition index includes an index value range corresponding to each storage area in the plurality of storage areas, and different index values ​​in the index value range correspond to identifiers of different data.

10. A data access device, characterized in that: include: The transceiver is used to: obtain a file creation request, wherein the file creation request is used to instruct to write the data to be processed into a storage device, and the file creation request carries an identifier of the data to be processed; The processor is used to: determine whether the identification index of the data access device cache has stored the identification of the data to be processed, wherein the identification index includes the identification of the data stored in the storage device; The transceiver is further used to: if the identification index does not store the identification of the data to be processed, send a data write instruction to the storage device, the data write instruction carries the data to be processed, and the data write instruction is used to instruct the storage device to create a file containing the data to be processed.

11. The data access device according to claim 10, characterized in that: The data access device includes: one or both of a user terminal and a smart network card.

12. A storage device, characterized in that: include: A memory, for providing a plurality of storage areas; A processor, configured to receive an index acquisition request sent by a data access device, wherein the index acquisition request carries the address of a first storage area to be requested, and the first storage area is one of a plurality of storage areas included in the storage device; and execute the method described in any one of claims 6 to 9 according to the index acquisition request.

13. A data access system, characterized in that: include: One or more storage devices according to claim 12, for storing data; One or more data access devices as claimed in claim 10 or 11, for accessing the storage device.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer software instructions. When the computer software instructions are executed in a computer, the computer executes the method according to any one of claims 1 to 9.

15. A computer program product, characterized in that When the computer program product is run on a computer, the computer executes the method according to any one of claims 1 to 9.

Images