File creation method, data access device, storage device and system

Abstract

The invention discloses a file creation method, a data access device, a storage device and a system, relates to the technical field of data storage, and solves the problems of long time consumption and low efficiency of file creation due to the fact that file creation of mass data occupies the computing power of the storage device. According to the method, the identification query process of the data is unloaded from the storage device to the data access device (such as a user side or an intelligent network card), so that the situation that the identification query process of the data occupies more computing power of the storage device is avoided, the query duration of the data in the file creation process is shortened, and the file creation efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and in particular to a file creation method, a data access device, a storage device and a system. Background Art

[0002] With the continuous development of cloud computing technology, the amount of unstructured data is increasing, and efficient management of unstructured data is becoming increasingly important. Among them, unstructured data is data with irregular or incomplete data structure, such as multimedia data, non-standardized reports, etc. Usually, when the user needs to store unstructured data in the form of files on a storage device, the storage device will first determine whether there is a file with the same name in the stored file directory. If there is no file with the same name in the directory, the storage device will create the corresponding file to store the data. In the case where the storage device needs to store a large amount of unstructured data, due to the limited computing power of the storage device, it takes a long time for the storage device to query the file name, and the efficiency of file creation is low. Summary of the invention

[0003] The present application provides a file creation method, a data access device, a storage device and a system, which solves the problem that the file creation of large amounts of data occupies the computing power of the storage device, resulting in a long time-consuming and low efficiency file creation.

[0004] In a first aspect, the present application provides a file creation method. The file creation method is executed by a data access device, and the method includes: the data access device obtains a file creation request, the file creation request is used to instruct to write the data to be processed to a storage device, and the file creation request carries an identifier of the data to be processed. Then, the data access device determines whether the identifier of the data to be processed is stored in the identifier index cached by the data access device, and the identifier index includes the identifier of the data stored in the storage device; if the identifier of the data to be processed is not stored in the identifier index, the data access device sends a data write instruction to the storage device. The data write instruction carries the data to be processed, and the data write instruction is used to instruct the storage device to create a file containing the data to be processed.

[0005] In the present application, an identification index is cached in the data access device, and the identification index includes the identification of the data stored in the storage device. During the file creation process, the data access device can query whether the identification of the data has been stored in the storage device according to the identification index. The data identification query process is unloaded from the usual storage device to the data access device, that is, the data access device provides the computing power required for the data identification query process, which avoids the data identification query process occupying more computing power of the storage device, which is conducive to reducing the query time of the data during the file creation process, thereby improving the efficiency of file creation.

[0006] In an optional implementation, the data access device includes: one or both of a user terminal and a smart network card that communicate with the above storage device.

[0007] Exemplarily, the data access device includes a user end, which may refer to a client that accesses a storage device, such as a mobile phone, a computer, a tablet computer, or a server.

[0008] In another exemplary embodiment, the data access device includes a smart network interface card (Smart NIC or SNIC) which is a flexible and programmable network card used in conjunction with a server.

[0009] Also exemplarily, the data access device includes a user terminal and a smart network card.

[0010] In an optional implementation, the storage device includes multiple storage areas, and the identifiers of the data stored in different storage areas are different. The data access device also caches a partition index, and the partition index includes an index value range corresponding to each storage area in the multiple storage areas. Before the aforementioned data access device determines the storage state of the identifier of the data to be processed according to the identifier index cached by the data access device, the file creation method provided by the present application also includes: the data access device processes the identifier of the data to be processed according to a set algorithm to obtain an index value corresponding to the identifier of the data to be processed; the set algorithm includes: a hash algorithm and a model-based key-value mapping algorithm. One or a combination of two. And, the data access device queries the partition index according to the index value, determines the target storage area that matches the data to be processed in the multiple storage areas, and sends an index acquisition request to the storage device; the index acquisition request is used to indicate: the storage device sends the identifier of the data stored in the target storage area to the data access device. The data access device receives the identifier index sent by the storage device according to the index acquisition request, and the identifier index includes: the identifier of the data stored in the aforementioned target storage area.

[0011] In the present application, the data access device may only need to cache the identification index of one or a few storage areas in the storage device, so that the data transmission delay of the identification index between the data access device and the storage device is low, and the storage space required for the identification index in the data access device is small, which is conducive to further reducing the query time of data during the file creation process, thereby improving the efficiency of file creation.

[0012] In an optional implementation, the target address of the data to be processed, the target storage area includes a storage space corresponding to the target address.

[0013] In an optional implementation, the file creation method provided by the present application further includes: the data access device receives index update information sent by the storage device, and updates the partition index according to the index update information. The index update information is used to indicate changes in the partition index.

[0014] Exemplarily, the data access device may send an index update request to the storage device, and receive the aforementioned index update information.

[0015] As another example, the storage device may periodically send index update information to the data access device, so that the data access device updates the partition index.

[0016] In the present application, the data access device can update the partition index according to the changes in multiple storage areas in the storage device, so that the data access device can obtain the identification index based on the latest partition index during the query process of file creation, thereby avoiding the problem of data with the same identification already stored in the storage device, causing the storage device to create duplicate files, and improving the accuracy of file creation.

[0017] In a second aspect, the present application provides a file creation method. The file creation method is executed by a storage device, and the file creation method includes: the storage device receives an index acquisition request sent by a data access device, and the index acquisition request carries the address of a first storage area to be requested, and the first storage area is one of multiple storage areas included in the storage device. In response to the index acquisition request, the storage device sends an identification index of the first storage area to the data access device; the identification index of the first storage area includes: an identification of data stored in the first storage area. And, the storage device receives a data write instruction sent by the data access device, and the data write instruction carries data to be processed; the storage device responds to the data write instruction and creates a file containing the aforementioned data to be processed.

[0018] In the present application, the storage device sends the identification index of one or more storage areas to the data access device. Then, during the process of file creation, the data access device can query whether the identification of the data has been stored in the storage device according to the identification index, and the data identification query process is unloaded from the usual storage device to the data access device, that is, the data access device provides the computing power required for the data identification query process, which avoids the data identification query process occupying more computing power of the storage device. The storage device only needs to create a file containing the aforementioned data to be processed according to the data write instruction carrying the data to be processed sent by the data access device, which is conducive to reducing the data query time during the file creation process, thereby improving the efficiency of file creation.

[0019] In an optional implementation, the data write instruction also carries a target address of the data to be processed, and the first storage area includes a storage space corresponding to the target address. The storage device creates a file containing the data to be processed in response to the data write instruction, including: the storage device creates a file containing the data to be processed in the first storage area.

[0020] In an optional implementation, each storage area in the plurality of storage areas corresponds to: a continuous address space included in a leaf node of a tree storage structure in the storage device.

[0021] In an optional implementation, before the storage device receives the index acquisition request sent by the data access device, the file creation method provided by the present application also includes: the storage device sends a partition index to the data access device, the partition index includes an index value range corresponding to each storage area in a plurality of storage areas, and different index values ​​in the index value range correspond to the identification of different data. Based on this, the storage device sends the identification index of the storage area to the data access device only after receiving the storage area to be accessed determined by the data access device according to the partition index, so that the data access device only needs to cache the identification index of one or a few storage areas in the storage device, so that the data transmission delay of the identification index between the data access device and the storage device is low, and the storage space required for the identification index in the data access device is small, which is conducive to further reducing the query time of data in the file creation process, thereby improving the efficiency of file creation.

[0022] In a third aspect, the present application provides a data access device. The data access device includes: a transceiver and a processor. The transceiver is used to: obtain a file creation request, the file creation request is used to instruct to write the data to be processed to a storage device, and the file creation request carries an identifier of the data to be processed; the processor is used to: determine whether the identifier of the data to be processed is stored in the identifier index cached by the data access device, and the identifier index includes the identifier of the data stored in the storage device; if the identifier of the data to be processed is not stored in the identifier index, send a data write instruction to the storage device, the data write instruction carries the data to be processed, and the data write instruction is used to instruct the storage device to create a file containing the data to be processed.

[0023] In an optional implementation, the data access device includes: one or both of a user terminal and a smart network card.

[0024] In a fourth aspect, the present application provides a storage device. The storage device includes: a memory for providing multiple storage areas; a processor for receiving an index acquisition request sent by a data access device, the index acquisition request carrying an address of a first storage area to be requested, the first storage area being one of the multiple storage areas included in the storage device; and performing the operation steps of the method of any optional implementation method in the second aspect according to the index acquisition request.

[0025] In a fifth aspect, the present application provides a data access system, which includes: one or more storage devices provided in the fourth aspect, used to store data; and one or more data access devices provided in any optional implementation of the third aspect, used to access the storage device.

[0026] In a sixth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium stores computer software instructions, and when the computer software instructions are executed in a computer, the computer executes the operation steps of the method of any optional implementation of the first aspect or the second aspect.

[0027] In a seventh aspect, the present application provides a computer program product. When the computer program product runs on a computer, the computer executes the operating steps of the method of any optional implementation method in the first aspect or the second aspect.

[0028] In an eighth aspect, the present application provides a chip system. The chip system includes a processor for implementing the functions of the data access device or storage device in the method of the first aspect or the second aspect. In a possible design, the chip system also includes a memory for storing program instructions and / or data. The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0029] Regarding the beneficial effects of the third to eighth aspects, reference may be made to the description of any optional implementation in the first or second aspect, and no further description is given here. Based on the implementation provided in the above aspects, the present application may be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A structural diagram of a data access system provided for this application;

[0031] Figure 2 A schematic diagram of the structure and organization of a file storage system provided for this application;

[0032] Figure 3 A schematic diagram of a file creation method provided for this application Figure 1 ;

[0033] Figure 4 A schematic diagram of a file creation method provided for this application Figure 2 ;

[0034] Figure 5 A schematic diagram of a file creation method provided for this application Figure 3 ;

[0035] Figure 6 Schematic diagram of two index updates provided for this application;

[0036] Figure 7 A schematic diagram of a file creation method provided for this application Figure 4 ;

[0037] Figure 8 A structural diagram of a data access device provided in this application. DETAILED DESCRIPTION

[0038] The present application provides a file creation method, which offloads the data identification query process from the storage device to the data access device (such as a user terminal or a smart network card, etc.), thereby avoiding the data identification query process occupying more computing power of the storage device, which is beneficial to reducing the data query time during the file creation process, thereby improving the efficiency of file creation.

[0039] Specifically, an identification index is cached in the data access device, and the identification index includes the identification of the data stored in the storage device. During the file creation process, the data access device can query whether the identification of the data has been stored in the storage device according to the identification index. The data identification query process is unloaded from the usual storage device to the data access device, that is, the data access device provides the computing power required for the data identification query process, which avoids the data identification query process occupying more computing power of the storage device, which is conducive to reducing the data query time during the file creation process, thereby improving the efficiency of file creation.

[0040] The technical solutions involved in this application may not only be applied to current data access devices or storage devices, but may also be applied to future data access devices and storage devices, or storage systems including the aforementioned data access devices or storage devices. The terms used in the implementation methods of this application are only used to explain the specific embodiments of this application, and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in this application.

[0041] Key-Value pair (KV): A data storage structure consisting of a key and a value corresponding to the key.

[0042] Index node (inode): used to record file meta information, such as inode number, file size, access rights, creation time, modification time, data location on disk, etc. Index nodes are unique identifiers of files, they correspond one to one, and are also stored on the hard disk, so index nodes also take up disk space.

[0043] Directory entry (dentry): used to record the file name, index node pointer and hierarchical relationship with other directory entries. Multiple directory entries are associated to form a directory structure, but the difference between directory entry and index node is that directory entry is a data structure maintained by the kernel and is not stored on disk, but cached in memory.

[0044] In order to make the description of the following embodiments clear and concise, a brief introduction to the related technology is first given.

[0045] Figure 1 1 is a schematic diagram of a data access system provided by the present application. The data access system includes: a data access device 100 and a storage device 120. Figure 1 In the application scenario shown, users access data through applications. Computers running these applications can be called "computing devices."

[0046] The data access device 100 may be a physical machine or a virtual machine. The physical machine may include but is not limited to: one or both of a user terminal and a smart network card (smart NIC).

[0047] For example, the data access device 100 includes a user end, which may be a client, such as a host, a desktop computer, a server, a notebook computer, a mobile device, and the like.

[0048] For another example, the data access device 100 includes a smart network card (smart NIC). The smart network card is also called an intelligent network adapter. In addition to being able to complete the network transmission function of a standard network card, it also provides a built-in programmable and configurable hardware acceleration engine to improve the performance of applications and greatly reduce the CPU consumption in the communication of the host connected to the smart network card, providing more CPU resources for the application. For example, in a highly virtualized environment, the CPU in the host needs to run tasks related to an open virtual switch (open virtual switch, OVS), and at the same time, the CPU in the host also needs to handle storage, online encryption and decryption of data packets or offline encryption and decryption, deep inspection of data packets, firewalls, complex routing and other operations. These operations not only consume a large amount of CPU resources, but also due to the competition for CPU resources between different services, the performance of the service cannot be optimized. As a hub connecting various services, the smart network card accelerates the above services on the smart network card.

[0049] For another example, the data access device 100 includes a client and a smart network card.

[0050] In a possible example, the data access device 100 accesses the storage device 120 through a network to access data. For example, the network may include a switch 110 .

[0051] In another possible example, the data access device 100 may also communicate with the storage device 120 through a wired connection, such as a universal serial bus (USB) or a peripheral component interconnect express (PCIe) bus.

[0052] Figure 1 The storage device 120 shown may be a centralized storage system. The centralized storage system is characterized by having a unified entrance through which all data from external devices must pass, and this entrance is the engine 121 of the centralized storage system. The engine 121 is the core component of the centralized storage system, and many advanced functions of the storage system are implemented in it.

[0053] like Figure 1 As shown, the engine 121 may have one or more controllers. Figure 1The example of the engine 121 including one controller is used for explanation. In a possible example, if the engine 121 has multiple controllers, any two controllers may have a mirror channel to achieve the function of any two controllers backing up each other, thereby avoiding the unavailability of the entire storage device 120 due to hardware failure. It should be understood that if the engine 121 includes multiple controllers, the engine 121 may also be called an array controller of the storage device 120.

[0054] The engine 121 further includes a front-end interface 1211 and a back-end interface 1214, wherein the front-end interface 1211 is used to communicate with the data access device 100, thereby providing data access services for the data access device 100. The back-end interface 1214 is used to communicate with the hard disk to expand the capacity of the storage device 120. Through the back-end interface 1214, the engine 121 can connect more hard disks, thereby forming a very large storage resource pool.

[0055] In hardware, such as Figure 1 As shown, the controller includes at least a processor 1212 and a memory 1213. The processor 1212 is a central processing unit (CPU) for processing data access requests from outside the storage device 120 (server or other storage system), and is also used to process requests generated inside the storage device 120. Exemplarily, when the processor 1212 receives a write data request sent by the data access device 100 through the front-end interface 1211, the data in the write data request will be temporarily stored in the memory 1213. When the total amount of data in the memory 1213 reaches a certain threshold, the processor 1212 sends the data stored in the memory 1213 to at least one of the mechanical hard disk 1221, the mechanical hard disk 1222, the solid state drive (SSD) 1223 or other hard disks 1224 through the back-end port for persistent storage.

[0056] Memory 1213 refers to an internal memory that directly exchanges data with the processor. It can read and write data at any time and at a very fast speed. It serves as a temporary data storage for the operating system or other running programs. The memory includes at least two types of memories. For example, the memory can be either a random access memory or a read-only memory (ROM). For example, the random access memory is DRAM or SCM. DRAM is a semiconductor memory, and like most random access memories (RAM), it is a volatile memory device. However, DRAM and SCM are only exemplary descriptions in this embodiment, and the memory can also include other random access memories, such as static random access memory (SRAM), etc. As for the read-only memory, for example, it can be a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), etc.

[0057] In addition, the memory 1213 may also be a dual in-line memory module or a dual in-line memory module (DIMM), that is, a module composed of a dynamic random access memory (DRAM), or an SSD. In practical applications, multiple memories 1213 and different types of memories 1213 may be configured in the controller. This embodiment does not limit the number and type of memory 1213. In addition, the memory 1213 may be configured to have a power-saving function. The power-saving function means that when the system loses power and then powers on again, the data stored in the memory 1213 will not be lost. A memory with a power-saving function is called a non-volatile memory.

[0058] The memory 1213 stores software programs, and the processor 1212 runs the software programs in the memory 1213 to manage the hard disk. For example, the hard disk is abstracted into a storage resource pool, and the storage resource pool is provided to the server in the form of a logical unit number (LUN). The LUN here is actually the hard disk seen on the server. Of course, some centralized storage systems are also file servers themselves, which can provide shared file services for the server.

[0059] like Figure 1As shown, in this system, the engine 121 may not have a hard disk slot, and the hard disk needs to be placed in the hard disk frame 122, and the back-end interface 1214 communicates with the hard disk frame 122. The back-end interface 1214 exists in the form of an adapter card in the engine 121, and two or more back-end interfaces 1214 can be used simultaneously on one engine 121 to connect multiple hard disk frames. Alternatively, the adapter card can also be integrated on the motherboard, in which case the adapter card can communicate with the processor 1212 via the PCIe bus.

[0060] It should be noted that Figure 1 Only one engine 121 is shown in the figure. However, in actual applications, the storage system may include two or more engines 121, and redundancy or load balancing is performed between the multiple engines 121.

[0061] The hard disk enclosure 122 includes a control unit 1225 and a plurality of hard disks. The control unit 1225 may have various forms. In one case, the hard disk enclosure 122 is an intelligent disk enclosure, such as Figure 1 As shown, the control unit 1225 includes a CPU and a memory. The CPU is used to perform operations such as address conversion and reading and writing data. The memory is used to temporarily store data to be written to the hard disk, or read from the hard disk to send data to the controller. In another case, the control unit 1225 is a programmable electronic component, such as a data processing unit (DPU). The DPU has the versatility and programmability of the CPU, but is more specialized and can run efficiently on network data packets, storage requests or analysis requests. The DPU is distinguished from the CPU by a large degree of parallelism (needing to process a large number of requests). Optionally, the DPU here can also be replaced by a processing chip such as a graphics processing unit (GPU) and an embedded neural network processor (NPU). Generally, the number of control units 1225 can be one, or two or more. The functions of the control unit 1225 can be offloaded to the network card 1226. In other words, in this embodiment, the hard disk frame 122 does not have a control unit 1225 inside, but the network card 1226 completes data reading and writing, address conversion and other computing functions. At this time, the network card 1226 is an intelligent network card. It can include a CPU and a memory. The CPU is used to perform operations such as address conversion and reading and writing data. The memory is used to temporarily store data to be written to the hard disk, or data read from the hard disk to be sent to the controller. It can also be a programmable electronic component, such as a DPU. There is no ownership relationship between the network card 1226 and the hard disk in the hard disk frame 122. The network card 1226 can access any hard disk in the hard disk frame 122 (such as Figure 1The mechanical hard disk 1221, mechanical hard disk 1222, solid state disk 1223 and other hard disks 1224 are shown, so it is more convenient to expand the hard disk when the storage space is insufficient.

[0062] According to the type of communication protocol between the engine 121 and the hard disk frame 122, the hard disk frame 122 may be a serial attached small computer system interface (SAS) hard disk frame, or it may be an NVMe (Non-Volatile Memory express) hard disk frame and other types of hard disk frames. The SAS hard disk frame adopts the SAS3.0 protocol, and each frame supports 25 SAS hard disks. The engine 121 is connected to the hard disk frame 122 through the onboard SAS interface or the SAS interface module. The NVMe hard disk frame is more like a complete computer system, and the NVMe hard disk is inserted in the NVMe hard disk frame. The NVMe hard disk frame is then connected to the engine 121 through the RDMA port.

[0063] For example, the storage device 120 may refer to a storage array, such as an all-flash storage array in which all storage media are flash memories.

[0064] In an optional implementation, the storage device 120 is a centralized storage system with integrated disk and controller, and the storage device 120 does not have the above-mentioned hard disk frame 122, and the engine 121 is used to manage multiple hard disks connected through the hard disk slots. The function of the hard disk slots can be implemented by the back-end interface 1214.

[0065] In another optional implementation, Figure 1The storage device 120 shown is a distributed storage system, which includes a computing device cluster and a storage device cluster. The computing device cluster includes one or more computing devices, and each computing device can communicate with each other. The computing device can be a computing device, such as a server, a desktop computer, or a controller of a storage array. In terms of hardware, the computing device can include a processor, a memory, and a network card. Among them, the processor is a CPU, which is used to process data access requests from outside the computing device, or requests generated inside the computing device. Exemplarily, when the processor receives a write data request sent by a user, the data in these write data requests will be temporarily stored in the memory. When the total amount of data in the memory reaches a certain threshold, the processor sends the data stored in the memory to the storage device for persistent storage. In addition, the processor is also used for data calculation or processing, such as metadata management, deduplication, data compression, virtualized storage space, and address conversion. In an example, any computing device can access any storage device in the storage device cluster through a network. The storage device cluster includes multiple storage devices. A storage device includes one or more controllers, a network card, and multiple hard disks, and the network card is used to communicate with the computing device.

[0066] It is worth noting that the above examples are only possible implementations of the data access system provided in this embodiment and should not be construed as limiting the present application. Figure 1 In the storage device 120 shown, data is stored in the form of files on each hard disk. The files stored in each hard disk constitute a file storage system, which can be a distributed file system, for example. For example, the Network File System (NFS), NFS is both a distributed file system and a network protocol for accessing and sharing files between devices on the same local area network. Exemplarily, the NAS system can be implemented by NFS protocol support. The Network File System is a low-cost network file sharing option that enables users and applications to access, store and update files on remote computers, just like using direct attached storage. The Network File System uses the Remote Procedure Call protocol to route requests between clients and servers. Although the participating devices need to support the Network File System, they do not need to understand the details of the network. It is worth noting that remote procedure calls may be unsafe, so the Network File System should only be deployed on a trusted network behind a firewall. Although Windows supports this protocol, it is mainly used in Linux environments.

[0067] like Figure 2 As shown, Figure 2 This is a schematic diagram of the structure and organization of a file storage system provided in this application. The hardware implementation of the file storage system 200 can refer to Figure 1The relevant contents of the storage device 120 are not described in detail here. Figure 2 As shown, the file storage system 200 includes one or more index nodes, and the data in an index node is processed and accessed by a controller node.

[0068] A file storage system is also called a file system. A file system is a structured data file storage and organization form. Figure 2 As shown, an index node includes multiple directory sets, such as Figure 2 The illustrated directory set 1, directory set 2, and directory set 3. A directory set includes one or more directories, a directory stores multiple files, and each file can be used to store one or more sets of data.

[0069] In a computer, all data is 0 and 1. A series of 01 combinations stored on hardware media is completely indistinguishable and unmanageable for a computer. Therefore, computers use the concept of "files" to organize these data. Data for the same purpose is organized into different types of files according to the structure required by different applications. Different suffixes are usually used to refer to different types, and then the computer gives each file a name that is easy to understand and remember.

[0070] A possible storage structure is: Figure 2 As shown in (storage structure 1), the computer divides these files (such as Figure 2 The black squares shown in the figure are grouped into groups, and each group of files is placed in the same directory (or folder). In addition to files, there can also be a lower-level directory (called a subdirectory or subfolder) under the directory, and all files and directories form a tree structure. This tree structure has a special name: file system. There are many types of file systems, the most common ones are FAT / FAT32 / NTFS in Windows, EXT2 / EXT3 / EXT4 / XFS / BtrFS in Linux, etc. In order to facilitate searching, start from the root node and go down the directory level by level until the file itself, and splice the names of these directories, subdirectories, and files with special characters (for example, "\" for Windows / DOS and " / " for Unix-like systems). Such a string of characters is called a file path, such as " / etc / systemd / system.conf" in Linux or "C:\Windows\System32\taskmgr.exe" in Windows. The path is a unique identifier for accessing a specific file. For example, D:\data\file.exe in Windows is a file path, which represents the file.exe file in the data directory under the D partition.

[0071] The file system is built on the block device. The file system not only records the file path, but also records which blocks make up a file and which blocks record directory / subdirectory information. Different file systems have different organizational structures. For the convenience of management, block devices such as hard disks can usually be divided into multiple logical block devices, that is, hard disk partitions. Conversely, the capacity and performance of a single medium are limited, and multiple physical block devices can be combined into a logical block device through certain technical means, such as various levels of redundant array of independent disks (RAID), disk clusters (Just a Bundle Of Disks, JBOD), etc. The file system can also be built on these logical block devices. In any case, the host application does not need to care about the specific location of the file to be accessed in the underlying block device. It only needs to send the file name / identifier (ID) of the file to the file system, and the file system will query the file path based on the file name / ID.

[0072] Common file access protocols include NFS, Common Internet of File System (CIFS), or Server Message Block (SMB) protocols, which are not limited in this embodiment.

[0073] For example, file-level tiered storage can be configured with multiple policies, such as using various attributes of the file such as associated application, directory, user, group, calling method, size, access frequency, etc. as classification conditions and trigger conditions, and performing migration with any volume or directory as the migration target. Block-level tiered storage usually migrates the entire LUN or some blocks in it according to the access frequency of the entire LUN or one or some blocks in the LUN.

[0074] Another possible storage structure is: Figure 2 As shown in (storage structure 2), the file system can use a tree storage structure (or tree data structure, tree index structure, etc.) that supports unloading to solve the problem that the traditional dentry table does not support unloading and cannot be accessed by using heterogeneous computing power. For example, the file system stores a complete tree intermediate node (index node) and leaf node (leaf node), and the leaf node stores dentry metadata.

[0075] exist Figure 2In the (storage structure 2) shown, index node 0 includes multiple other index nodes, such as index node 1 to index node 3. Each index node includes only a key, but not data content; in some optional situations, the keys of the index nodes are arranged in ascending order, and for the key of a specified index node, the keys of all index nodes on the left subtree of the specified index node are smaller than the key of the specified index node (such as the key of index node 1 is smaller than the key of index node 2), and the keys of all index nodes on the right subtree of the specified index node are greater than or equal to the key of the specified index node (such as the key of index node 3 is smaller than the key of index node 2).

[0076] Each leaf node may include one or more storage partitions (partitions for short), and each partition may include a continuous address space. For example, leaf node 1 includes partitions 1 to m, and leaf node 1 includes partitions 1 to k.

[0077] In this embodiment, the data access device can directly calculate the address of the leaf node to be accessed through the cached index node, and then directly read the leaf node of the tree from the storage end in a one-way read manner and parse to obtain the target dentry metadata.

[0078] The tree storage structure adopts a large leaf design, with the goal of using fewer leaf nodes (hereinafter referred to as leaves) to store the same amount of records, and the corresponding tree internal index nodes are also fewer. This means that the client or smart card side caches the tree internal index nodes, which requires less memory resources, or the same resources can cache more tree internal index nodes (such as Figure 2 Index nodes 0 to 3 in the cache), improving cache efficiency and ultimately improving query performance: all or most records can be directly calculated through the cached index nodes to obtain the leaf addresses of the target records stored on the server side, and then directly read the records in the leaf nodes through unilateral DMA.

[0079] The large leaves are partitioned to improve the performance of one-way reading. As an optional example, the size of each index node (inode nodesize) is 8 kilo bytes (KB), the size of each leaf node is 1 million bytes (MB), and the storage capacity of each partition in the leaf node is 16KB. For example, a 1MB leaf is divided into 64 16KB partitions. The leaf partition corresponding to the target key can be calculated through the intermediate index node, and only a single 16KB target partition is read when reading the content of the leaf node. The purpose of doing this is because the performance of one-way reading is related to the size of the read content. Directly reading large leaves will cause a significant decrease in one-way reading performance, while reading a single small leaf partition can ensure high performance of one-way reading.

[0080] Combine the following Figure 1 The data access system shown and Figure 2 The storage structure 2 of the file storage system 200 shown in the figure is used to describe the file creation method provided by the present application in detail. Figure 3 As shown, Figure 3 A schematic diagram of a file creation method provided for this application Figure 1 .

[0081] Figure 3 The file creation method shown can be applied to Figure 1 In the data access system shown, the data access device 31 can be used to perform Figure 1 The function of the data access device 100, the storage device 32 can be used to implement Figure 1 The functions of the storage device 120 in the data access device 31 and the hardware implementation of the storage device 32 can be referred to in the foregoing Figure 1 and Figure 2 The description is not repeated here.

[0082] Please refer to Figure 3 , Figure 3 The illustrated file creation method includes the following steps S310 to S360.

[0083] S310 , the data access device 31 sends an index acquisition request to the storage device 32 .

[0084] Corresponding to S310 , the storage device 32 receives the index acquisition request sent by the data access device 31 .

[0085] The index acquisition request carries the address of the first storage area to be requested, and the first storage area is one of the multiple storage areas included in the storage device 32, such as any one of storage area 1 to storage area n, where n is a positive integer.

[0086] In this embodiment, each storage area in the plurality of storage areas corresponds to a continuous address space included in a leaf node of the tree storage structure in the storage device 32. For a description of each leaf node of the tree storage structure, refer to Figure 2 The relevant content will not be elaborated here.

[0087] Exemplarily, a storage area includes: all address spaces in a leaf node. Figure 2 The address space included in leaf node 1, or the address space included in leaf node 6.

[0088] In another exemplary embodiment, a storage area includes: a portion of the address space in a leaf node, where the portion of the address space may refer to the address space of a partition in the leaf node, or may refer to the address space of multiple consecutive partitions in the leaf node. Figure 2 The (storage structure 2) in is used as an example. The multiple continuous partitions may refer to the leaf node 6: partition 1 to partition 2, or partition 2 to partition i (i is an integer greater than 2 and less than k), etc. This application is not limited to this.

[0089] The above two examples are merely possible implementations of the storage area provided in this embodiment and should not be construed as limitations on the present application.

[0090] In this embodiment, the data access device 31 can only obtain index information of one or a few storage areas among the multiple storage areas, avoiding the data access device 31 from needing to obtain index information of a large amount of data, thereby reducing the amount of data to be transmitted between the data access device 31 and the storage device 32, and also reducing the amount of data that needs to be stored in the memory of the data access device 31, that is, reducing the storage resources of the data access device 31 that the above-mentioned index information needs to occupy.

[0091] S320 , the storage device 32 sends an identification index (identification index 1 ) of the first storage area (storage area 1 ) to the data access device 31 in response to the index acquisition request.

[0092] The identification index of the first storage area includes: an identification of data stored in the first storage area.

[0093] The first storage area may refer to one of the multiple storage areas included in the storage device 32, such as storage area 1. For hardware implementation of multiple storage areas, refer to Figure 1 Description of. Combined Figure 2 For example, the storage area 1 may include one or more partitions in a leaf node. For details, please refer to the description of S310, which will not be described here.

[0094] The identifier of the data stored in the storage area 1 may refer to: the file name of the file containing the stored data.

[0095] Alternatively, in an application scenario where a large number of small files are created, the identifier of the data stored in the storage area 1 may refer to: dentry metadata. The dentry metadata is stored in a dentry table, and the dentry metadata is: a pointer to the data stored in the storage device 32.

[0096] In the following, storage area 1 may be referred to as the first storage area, storage area 2 may be referred to as the second storage area, and storage area n may be referred to as the nth storage area, which will not be described in detail later. The above is only an example provided in this embodiment and should not be construed as limiting the present application.

[0097] S330: The data access device 31 obtains a file creation request.

[0098] The query file req carries the identifier of the data to be processed. The contents of the identifier can be referred to the description of S320 above, which will not be repeated here. The query file req is used to instruct to write the data to be processed into the storage device 32 .

[0099] In a possible example, the file creation request is a request generated by the data access device 31 according to a running service.

[0100] In another possible example, the file creation request is a request sent by other clients or smart network cards (SNIC) to the data access device 31. Corresponding to this situation, S330 includes: the data access device 31 receives the file creation request.

[0101] The above two examples are merely possible ways of obtaining a file creation request provided in this embodiment and should not be construed as limitations on the present application.

[0102] S340 : The data access device 31 determines whether the identifier of the data to be processed is stored in the identifier index cached by the data access device 31 .

[0103] The identification index includes the identification of the data stored in the storage device 32 .

[0104] In an optional example, the identification index includes: keys (identifications) stored in all index nodes in the storage device 32 .

[0105] In another optional example, the identification index includes: keys (identifiers) stored in some index nodes in the storage device 32. Exemplarily, the identification index only includes keys stored in index nodes of one storage area or a few storage areas in the storage device 32, such as the keys stored in the storage area 1 above, which are used to indicate files stored in a continuous address space included in a leaf node 1 in the storage device 32.

[0106] Regarding the process of the data access device 31 caching the above identification index, S310 and S320 are only examples provided in this implementation and should not be understood as limiting the present application. The following embodiments also provide other possible implementation methods, which will not be elaborated here.

[0107] If the identification index does not store the identification of the data to be processed, the data access device 31 executes S350; if the identification index has stored the identification of the data to be processed, the data access device 31 determines that a file containing the data to be stored has been stored in the storage device 32, and ends the file creation process.

[0108] S350 , the data access device 31 sends a data write instruction to the storage device 32 .

[0109] The data writing instruction carries the data to be processed, and is used to instruct the storage device 32 to create a file containing the data to be processed.

[0110] Corresponding to the designated sending process of S350 , the storage device 32 receives the data write instruction sent by the data access device 31 .

[0111] S360: The storage device 32 creates a file containing the data to be processed in response to the data write instruction.

[0112] In an optional implementation, the data write instruction also carries: a target address of the data to be processed, and the aforementioned storage area 1 includes a storage space corresponding to the target address. Exemplarily, the above S360 includes: the storage device 32 creates a file containing the data to be processed in the first storage area (storage area 1). That is, the storage device 32 stores the created file in the storage area 1.

[0113] In this embodiment, by unloading the data identification query process from the storage device 32 to the data access device 31 (such as a user terminal or an intelligent network card, etc.), the data identification query process is avoided from occupying more computing power of the storage device 32, which is beneficial to reducing the query time of the data during the file creation process, thereby improving the efficiency of file creation. As described in the above embodiments, the data access device 31 has a cached identification index, and the identification index includes the identification of the data stored in the storage device 32. During the file creation process, the data access device 31 can query whether the identification of the data has been stored in the storage device 32 according to the identification index. The data identification query process is unloaded from the usual storage device 32 to the data access device 31, that is, the data access device 31 provides the computing power required for the data identification query process, which avoids the data identification query process occupying more computing power of the storage device 32, which is beneficial to reducing the query time of the data during the file creation process, thereby improving the efficiency of file creation.

[0114] When the above data access device refers to a pluggable smart network card, the smart network card has computing capabilities. In order to release the CPU computing power of the client to which the smart network card belongs, the smart network card will offload the data processing functions that are not suitable for CPU in network, security, and storage to the programmable hardware chip for execution, thereby reducing the CPU consumption of the client, enabling the client to run key applications and operating systems more effectively, and optimizing the processing efficiency of business data.

[0115] It is worth noting that the above identification index acquisition process is only a possible implementation method provided in this embodiment. For the above identification index acquisition process, another possible implementation method is also given below, such as Figure 4 As shown, Figure 4 A schematic diagram of a file creation method provided for this application Figure 2 For details about how to create this file, refer to Figure 3 The description is not repeated here; the data access device includes a memory, which can read and write data at any time and at a very high speed, and serves as a temporary data storage for an operating system or other running programs. For example, the memory can refer to DRAM, SRAM or SCM, etc.

[0116] Before the aforementioned S320 , the file creation method provided in this embodiment includes steps S401 to S404 .

[0117] S401 : The data access device 31 obtains a partition index of the storage device 32 .

[0118] The storage device 32 includes multiple storage areas, and different storage areas store data with different identifiers.

[0119] The partition index includes an index value range corresponding to each storage area in a plurality of storage areas, and different index values ​​in the index value range correspond to the identification of different data. The index value here may refer to a value obtained by calculating the identification of the data according to a set algorithm. In some optional examples, the amount of data of the index value is less than the amount of data of the identification, so that the storage resources occupied by the partition index are smaller, and the storage space required for the data access device 31 to cache the partition index is reduced.

[0120] In an optional implementation, the storage device 32 periodically sends the partition index to each data access device, such as the storage device 32 sending the partition index to the data access device 31. In this way, each data access device can quickly determine the file storage status of each storage area in the storage device 32 based on the partition index, avoiding repeated negotiations between the data access device and the storage device, and improving the efficiency of the data access device and the storage device in executing the file creation process.

[0121] In another optional implementation, the data access device 31 requests the partition index from the storage device 32 according to its own business needs. For example, the data access device 31 sends a request for obtaining the partition index to the storage device 32, and the storage device 32 sends the partition index to the data access device 31 in response to the request. In this way, when there are a large number of data access devices communicating with the storage device 32, the storage device 32 does not need to consume a large amount of broadcast resources to synchronize the partition index in the storage device 32, which reduces the network resources (such as data transmission bandwidth) and processor computing power consumption of the storage device 32, which is conducive to improving the efficiency of the storage device 32 in executing various data access services (such as file creation, data reading or garbage collection, etc.).

[0122] The above two optional implementations are only examples provided in this embodiment and should not be construed as limiting the present application. In some examples, the partition index synchronization method between the data access device and the storage device may also adopt other forms, such as the storage device caching the partition index on a shared memory that can be read by multiple data access devices at the same time, and each data access device responds to the received file creation request and queries the partition index from the shared memory, etc.

[0123] S402 : The data access device 31 processes the identifier of the data to be processed according to a set algorithm to obtain an index value corresponding to the identifier of the data to be processed.

[0124] The above-mentioned set algorithm includes: a hash algorithm and a model-based key-value mapping algorithm or a combination of both.

[0125] For example, the set algorithm includes: hash algorithm. The hash algorithm transforms an input of any length (also called pre-image) into an output of fixed length through a hash algorithm, and the output is a hash value (hash value). The operation process of the hash algorithm refers to a compression mapping.

[0126] For another example, the set algorithm includes: a model-based key-value mapping algorithm. For example, the model may refer to a neural network model implemented based on a deep neural network (DNN) or a convolutional neural network (CNN). DNN, also known as a multi-layer neural network, can be understood as a neural network with many hidden layers. There is no special metric for "many" here; CNN is a deep neural network with a convolutional structure. For more information about DNN and CNN, please refer to the description of the general technology, which will not be repeated here.

[0127] For example, the set algorithms include: a hash algorithm and a model-based key-value mapping algorithm.

[0128] The above examples are only possible implementations of the set algorithm provided in this embodiment and should not be understood as limitations on the present application. Other algorithms that can determine a unique index value based on the identification of the data may also be included in the set algorithm.

[0129] S403 , the data access device 31 queries the partition index according to the index value, determines a target storage area among multiple storage areas that matches the data to be processed, and sends an index acquisition request to the storage device 32 .

[0130] The index acquisition request is used to indicate that the storage device 32 sends the identifier of the data stored in the target storage area to the data access device 31. Exemplarily, the target storage area may refer to storage area 1. For more information about the index acquisition request, please refer to S310, which will not be repeated here.

[0131] Exemplarily, the partition index includes the content of the following Table 1, which is a possible implementation method of the partition index provided in this embodiment.

[0132] Table 1

[0133] Index value range - minimum value Index value range-maximum value Storage Partition 1 0 9 Storage Partition 2 10 19 ··· ··· ··· Storage partition n (n-1)×10 (n-1)×10+9

[0134] Among them, the index value range included in storage partition 1 is 0 to 9, the index value range included in storage partition 2 is 10 to 19, and the index value range included in storage partition n is (n-1)×10 to (n-1)×10+9. That is, the index value range corresponding to a single storage partition includes 10 index values, and each storage partition can store 10 files. Of course, in the partition index, each storage area can also correspond to a larger or smaller index value range, that is, each storage partition can also be used to store more or fewer files, which is not limited in this application.

[0135] S404: The data access device 31 receives the identification index sent by the storage device 32 according to the index acquisition request.

[0136] The identification index includes: the identification of the data stored in the target storage area, for example, the identification index refers to identification index 1. For more information about the identification index, please refer to the description of S330 above, which will not be described in detail here.

[0137] In the present application, the data access device may only need to cache the identification index of one or a few storage areas in the storage device 32, so that the data transmission delay of the identification index between the data access device 31 and the storage device 32 is low, and the storage space required for the identification index in the data access device 31 is small, which is conducive to further reducing the query time of data during the file creation process, thereby improving the efficiency of file creation.

[0138] Optionally, in the aforementioned Figure 3 and Figure 4 Based on this, the present application also provides an optional implementation method for the content of the partition index, such as Figure 5 As shown, Figure 5 A schematic diagram of a file creation method provided for this application Figure 3 After the aforementioned S404, the file creation method provided in this embodiment further includes the following S405 and S406.

[0139] S405 : The data access device 31 obtains index update information of the storage device 32 .

[0140] The index update information is used to indicate changes in the partition index in the storage device 32 .

[0141] Regarding the process of the data access device 31 obtaining the index update information, the following takes a storage area corresponding to a leaf node, the index value range of each storage area and the index node to which the leaf node belongs, and the above partition index includes the index nodes in the storage device 32 as an example, combined with Figure 6 Two possible scenarios are provided. Figure 6 Schematic diagram of two index updates provided in this application.

[0142] In the first optional scenario, if the number of data access devices communicating with the storage device 32 is small, the storage device 32 can adopt an active push method, that is: 1. The partition index in the storage device 32 changes; 2. The storage device 32 actively sends the updated partition index or index update information to each data access device; accordingly, the data access device receives the partition index or index update information sent by the storage device, and 3. The data access device updates the partition index according to the index update information. In this scenario, the storage device adopts a broadcasting method to one or more data access devices to synchronize and update the partition index cached by each data access device. Each data access device can obtain the latest partition index in a short time after the partition index in the storage device changes, avoiding errors in the subsequent file creation process, such as repeated creation of the same file, etc., which is conducive to improving the accuracy of file creation on the basis of ensuring the timeliness of the update of the partition index.

[0143] In the second optional scenario, if the number of data access devices communicating with the storage device 32 is large, the data access device can be used to actively synchronize, that is: 1. The data access device triggers the synchronization process of the partition index by timing or by using strategies such as the statistical index cache hit rate (or miss rate). Exemplarily, when the data access device receives an input output (IO) request, it sends an index update request to the storage device 32; accordingly, 2. The storage device 32 responds to the index update request and sends the partition index of the storage device 32 to the data access device; thus, 3. The data access device receives the partition index and updates the old partition index. In this scenario, each data access device obtains the partition index in the storage device on demand, so that only the data access device that truly has IO access among the multiple data access devices will synchronize and cache the partition index, and the data access device without IO access will not cache the partition index, which reduces the waste of storage resources of these data access devices without IO access, and avoids the broadcast overhead caused by the storage device pushing the partition index to a large number of data access devices, which is conducive to improving the data management performance inside the storage device.

[0144] In the above two scenarios, the number of data access devices can be set according to the user's usage requirements or the performance requirements of the data access system. For example, if the number of data access devices is greater than or equal to the quantity threshold, it means a large number, and if it is less than the quantity threshold, it means a small number. The quantity threshold can refer to 100, 1000 or other possible values, and this application is not limited to this.

[0145] S406: The data access device 31 updates the partition index according to the index update information.

[0146] In a possible example, if the index update information includes all contents of the updated partition index, the data access device 31 caches the index update information as the updated partition index of the storage device 32 in the memory of the data access device 31 and deletes the old partition index stored in the memory.

[0147] In another possible example, if the index update information includes part of the content of the updated partition index, the data access device 31 writes the content in the index update information that is different from the old partition index over the old partition index to obtain the updated partition index.

[0148] The above two possible examples are possible implementations of the partition index update method provided in this embodiment and should not be understood as limitations on the present application.

[0149] In the present application, the data access device can update the partition index according to the changes in multiple storage areas in the storage device, so that the data access device can obtain the identification index based on the latest partition index during the query process of file creation, thereby avoiding the problem of data with the same identification already stored in the storage device, causing the storage device to create duplicate files, and improving the accuracy of file creation.

[0150] The following is an exemplary description of the file creation method provided in the embodiment of the present application, with the data identifier referring to the file name (filename), the identifier index referring to the dentry metadata, the storage area corresponding to the leaf node, and the partition index corresponding to the index node. Figure 7 As shown, Figure 7 A schematic diagram of a file creation method provided for this application Figure 4 The file creation method provided in this embodiment includes the following steps S701 to S704.

[0151] S701: The data access device 31 obtains a file creation request.

[0152] For a more detailed description of S701, please refer to the content of S330, which will not be repeated here.

[0153] S702 , the data access device 31 queries the cached partition index according to the file name (filename1) of the data in the file creation request, determines the target leaf node, and unilaterally reads the identification index of the target leaf node in the storage device 32 .

[0154] Exemplarily, the input for the query operation performed by the data access device 31 is: filename1, and the filename1 is used as a key to query the address of the target leaf node in the cached partition index (such as tree index node).

[0155] If the target leaf node refers to leaf node 6, the identification index refers to the identification index 6 of the leaf node 6, and the identification index 6 includes: the file name (filename) of the file stored in the leaf node 6. In the identification index 6, each file name exists in the form of dentry metadata: the dentry metadata includes multiple KV pairs, such as filename->inode ID, and the inode ID refers to the ID of the index node where the leaf node to which the filename belongs is located, such as index node 2.

[0156] After the data access device 31 queries and determines the identification index 6, and determines that the file name of the data in the file creation request is not stored in the identification index 6, S703 is executed. The query method may refer to a binary search or a traversal method. Taking the traversal query as an example, the data access device 31 compares the identification index 6 (dentry metadata) one by one to see whether there is a KV pair containing filename1. If not, S703 is executed.

[0157] S703 : The data access device 31 sends a data write instruction carrying the data in the file creation request to the storage device 32 .

[0158] S704: The storage device 32 creates a file in response to the data write instruction.

[0159] And, the storage device 32 also creates a KV pair including filename1, and writes the newly created KV pair into the identification index 6 of the storage device 32. Accordingly, the storage device 32 also updates the partition index according to the identification index 6 after the new KV pair has been written.

[0160] The index update process between the data access device and the storage device can refer to the above Figures 4 to 6 The process will not be described in detail here.

[0161] In the accompanying drawings corresponding to the above embodiments, only one data access device is shown. However, in some optional implementations, a single storage device can be provided to be accessed by multiple different data access devices, which can still implement the file creation method provided by the present application and will not be described in detail here.

[0162] It is understandable that, in order to implement the functions in the above embodiments, the data access device and the storage device include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0163] The above text describes in detail the file creation method provided by the present embodiment in conjunction with the accompanying drawings. Figure 8 , describing the data access device provided according to this embodiment.

[0164] Figure 8 Schematic diagram of the structure of a data access device provided by the present application. The data access device can be used to implement the functions of the data access device in the above method embodiment, and thus can also achieve the beneficial effects of the above method embodiment. In this embodiment, the data access device can be as follows Figure 1 The data access device 100 shown may also be the data access device 31 in the subsequent embodiments, or a module (such as a chip) applied to a data access device.

[0165] like Figure 8 As shown, the data access device 800 may include a processor 820. Optionally, the data access device 800 may also include a memory 830 and / or a transceiver 810. The processor 820 is coupled to the memory 830 and the transceiver 810, such as by a communication bus. The communication bus may include, but is not limited to: a Peripheral Component Interconnect Express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc.

[0166] Combine the following Figure 8 The components of the data access device 800 are described in detail:

[0167] The processor 820 is the control center of the data access device 800, and may be a processor or a general term for multiple processing elements. For example, the processor 820 is one or more CPUs, or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application, such as one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA).

[0168] Optionally, the processor 820 may perform various functions of the data access device 800 by running or executing software programs stored in the memory 830 and calling data stored in the memory 830. In a specific implementation, as an embodiment, the processor 820 may include one or more CPUs.

[0169] Optionally, the data access device 800 may also include multiple processors. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0170] The memory 830 is used to store the software program for executing the scheme of the present application, and is controlled by the processor 820 to execute. The specific implementation method can refer to the above method embodiment, which will not be repeated here. Exemplarily, the memory 830 can be a ROM or other types of static storage devices that can store static information and instructions, RAM or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this. The memory 830 can be integrated with the processor 820, or it can exist independently and access the computer through the interface circuit ( Figure 8(not shown) is coupled to the processor 820, which is not specifically limited in the embodiment of the present application.

[0171] The transceiver 810 is used for communication with other devices. For example, the data access device 800 is a user end (such as a client) or an application server, and the transceiver 810 can be used to communicate with a storage device, or with another data access device. For another example, the data access device 800 is a smart network card (SNIC), and the transceiver 810 can be used to communicate with a storage device, or with another smart network card. For another example, the data access device 800 is a multi-core chip, and the transceiver 810 can be used to communicate with another multi-core chip.

[0172] Optionally, the transceiver 810 may include a receiver and a transmitter ( Figure 8 The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function. Optionally, the transceiver 810 can be integrated with the processor 820, or can exist independently and access the data through the interface circuit ( Figure 8 (not shown) is coupled to the processor 820, which is not specifically limited in the embodiment of the present application.

[0173] In this embodiment, the transceiver 810 is used to obtain a file creation request, where the file creation request is used to instruct to write the data to be processed into a storage device, and the file creation request carries an identifier of the data to be processed.

[0174] The processor 820 is used to determine whether the identification index of the data access device cache has stored the identification of the data to be processed, and the identification index includes the identification of the data stored in the storage device.

[0175] The transceiver 810 is also used to: if the identification index does not store the identification of the data to be processed, send a data write instruction to the storage device, the data write instruction carries the data to be processed, and the data write instruction is used to instruct the storage device to create a file containing the data to be processed.

[0176] The memory 830 is used to store information such as partition indexes and identification indexes. For more information about the partition indexes and identification indexes, please refer to the description of the aforementioned embodiments, which will not be repeated here.

[0177] The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in RAM, flash memory, ROM, PROM, EPROM, EEPROM, registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a computing device. Of course, the processor and the storage medium can also be present in a network device or a terminal device as discrete components.

[0178] The present application also provides a chip system, which includes a processor for implementing the functions of the data processing unit in the above method. In one possible design, the chip system also includes a memory for storing program instructions and / or data. The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0179] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0180] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A file creation method, characterized in that: The method is executed by a data access device, and comprises: Obtaining a file creation request, where the file creation request is used to instruct writing the data to be processed into a storage device, and the file creation request carries an identifier of the data to be processed; Determine whether the identification index of the data access device cache has stored the identification of the data to be processed, wherein the identification index includes the identification of the data stored in the storage device; If the identification index does not store the identification of the data to be processed, a data write instruction is sent to the storage device, the data write instruction carries the data to be processed, and the data write instruction is used to instruct the storage device to create a file containing the data to be processed.

2. The method according to claim 1, characterized in that The data access device includes: one or both of a user terminal and a smart network card that communicate with the storage device.

3. The method according to claim 1 or 2, characterized in that: The storage device includes multiple storage areas, and different storage areas store data with different identifiers; the data access device also caches a partition index, and the partition index includes an index value range corresponding to each storage area in the multiple storage areas; Before determining the storage state of the identification of the to-be-processed data according to the identification index cached by the data access device, the method further includes: Processing the identifier of the data to be processed according to a set algorithm to obtain an index value corresponding to the identifier of the data to be processed; the set algorithm includes: a hash algorithm and a model-based key-value mapping algorithm or a combination of both; Query the partition index according to the index value, determine the target storage area matching the data to be processed in the multiple storage areas, and send an index acquisition request to the storage device; the index acquisition request is used to instruct: the storage device sends the identifier of the data stored in the target storage area to the data access device; The identification index sent by the storage device according to the index acquisition request is received, where the identification index includes: an identification of data stored in the target storage area.

4. The method according to claim 3, characterized in that The data write instruction also carries: a target address of the data to be processed, and the target storage area includes a storage space corresponding to the target address.

5. The method according to claim 3 or 4, characterized in that: The method further comprises: Receiving index update information sent by the storage device, where the index update information is used to indicate a change in the partition index; The partition index is updated according to the index update information.

6. A file creation method, characterized in that: The method is performed by a storage device, and the method includes: receiving an index acquisition request sent by a data access device; the index acquisition request carries an address of a first storage area to be requested, the first storage area being one of a plurality of storage areas included in the storage device; In response to the index acquisition request, sending the identification index of the first storage area to the data access device; the identification index of the first storage area includes: an identification of the data stored in the first storage area; receiving a data write instruction sent by the data access device, wherein the data write instruction carries data to be processed; In response to the data writing instruction, a file containing the data to be processed is created.

7. The method according to claim 6, characterized in that The data write instruction also carries a target address of the data to be processed, and the first storage area includes a storage space corresponding to the target address; The step of creating a file containing the data to be processed in response to the data writing instruction comprises: A file containing the data to be processed is created in the first storage area.

8. The method according to claim 6 or 7, characterized in that: Each storage area in the plurality of storage areas corresponds to: a continuous address space included in a leaf node of a tree storage structure in the storage device.

9. The method according to any one of claims 6 to 8, characterized in that: Before receiving the index acquisition request sent by the data access device, the method further includes: A partition index is sent to the data access device, where the partition index includes an index value range corresponding to each storage area in the plurality of storage areas, and different index values ​​in the index value range correspond to identifiers of different data.

10. A data access device, characterized in that: include: The transceiver is used to: obtain a file creation request, wherein the file creation request is used to instruct to write the data to be processed into a storage device, and the file creation request carries an identifier of the data to be processed; The processor is used to: determine whether the identification index of the data access device cache has stored the identification of the data to be processed, wherein the identification index includes the identification of the data stored in the storage device; The transceiver is further used to: if the identification index does not store the identification of the data to be processed, send a data write instruction to the storage device, the data write instruction carries the data to be processed, and the data write instruction is used to instruct the storage device to create a file containing the data to be processed.

11. The data access device according to claim 10, characterized in that: The data access device includes: one or both of a user terminal and a smart network card.

12. A storage device, characterized in that: include: A memory, for providing a plurality of storage areas; A processor, configured to receive an index acquisition request sent by a data access device, wherein the index acquisition request carries the address of a first storage area to be requested, and the first storage area is one of a plurality of storage areas included in the storage device; and execute the method described in any one of claims 6 to 9 according to the index acquisition request.

13. A data access system, characterized in that: include: One or more storage devices according to claim 12, for storing data; One or more data access devices as claimed in claim 10 or 11, for accessing the storage device.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer software instructions. When the computer software instructions are executed in a computer, the computer executes the method according to any one of claims 1 to 9.

15. A computer program product, characterized in that When the computer program product is run on a computer, the computer executes the method according to any one of claims 1 to 9.

Images