Object type data processing method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The application provides a processing method and device of object type data, electronic equipment, computer readable storage medium and computer program product. The method comprises the following steps: receiving a first conversion instruction, wherein the first conversion instruction is used for indicating storage type conversion of a target file system object; converting the storage type of a plurality of storage objects in a first storage space from a first storage type to a second storage type, wherein the plurality of storage objects are used for distributing metadata of the target file system object; converting the storage type of the target file system object in a second storage space from the first storage type to the second storage type; and in response to the target file system object of the second storage type meeting a storage type conversion condition, converting the storage type of the target file system object from the second storage type to the first storage type. Through the application, the reliability of data storage can be improved.

Description

Technical Field

[0001] This application relates to cloud storage technology, and more particularly to a method, apparatus, electronic device, computer-readable storage medium, and computer program product for processing object-type data. Background Technology

[0002] In the storage process of file system objects, the data and metadata of the file system object are generally stored in two different spaces (e.g., a first storage space and a second storage space). When certain file system objects are accessed intermittently or are not accessed again after being accessed, the storage type of the file system object often changes with the access frequency. However, in the daily operation of the file system, there may be situations where the storage type of the file system object changes, but the storage type of the corresponding metadata remains unchanged, that is, the storage type of data in different storage spaces is inconsistent.

[0003] There is currently no effective solution for ensuring consistent data storage types across different storage spaces. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, computer-readable storage medium, and computer program product for processing object-type data, which can improve the reliability of data storage.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a method for processing object-type data, the method comprising:

[0007] Receive a first conversion instruction, wherein the first conversion instruction is used to instruct the storage type conversion of the target file system object;

[0008] The storage type of multiple storage objects in the first storage space is changed from the first storage type to the second storage type, wherein the multiple storage objects are used to distribute the metadata of the target file system object;

[0009] Convert the storage type of the target file system object in the second storage space from the first storage type to the second storage type;

[0010] In response to the target file system object of the second storage type satisfying the storage type conversion condition, the storage type of the target file system object is converted from the second storage type to the first storage type.

[0011] This application provides an object-type data processing apparatus, the apparatus comprising:

[0012] The instruction receiving module is used to receive a first conversion instruction, wherein the first conversion instruction is used to instruct the storage type conversion of the target file system object;

[0013] The first conversion module is used to convert the storage type of multiple storage objects in the first storage space from a first storage type to a second storage type, wherein the multiple storage objects are used to distribute the metadata of the target file system object;

[0014] The second conversion module is used to convert the storage type of the target file system object in the second storage space from the first storage type to the second storage type;

[0015] The third conversion module is used to convert the storage type of the target file system object from the second storage type to the first storage type in response to the second storage type target file system object meeting the storage type conversion condition.

[0016] This application provides an electronic device, the electronic device comprising:

[0017] Memory is used to store executable instructions or computer programs.

[0018] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the object-type data processing method provided in the embodiments of this application.

[0019] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the object-type data processing method provided in this application.

[0020] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the object data processing method provided in this application.

[0021] The embodiments of this application have the following beneficial effects:

[0022] Upon receiving the first conversion instruction, the storage type of the metadata of the target file system object in the first storage space is first converted from the first storage type to the second storage type. Then, the storage type of the target file system object in the second storage space is converted from the first storage type to the second storage type. This conversion of the file system object's storage type tightly associates the storage type of the metadata with the storage type of the file system object, ensuring that the storage type of the file system object's metadata is always consistent with the storage type of the file system object. This avoids the extra time consumption caused by modifying the storage type to be consistent when the storage types are inconsistent in different storage spaces. When the target file system object of the second storage type meets the storage type conversion conditions, the storage type of the target file system object is converted back to the first storage type, forming a closed loop of data storage types. This allows for flexible conversion between the first and second storage types, ensuring that data in different storage spaces has a consistent storage type during any storage type conversion process, thus improving data reliability. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the architecture of the object-type data processing system 100 provided in the embodiments of this application;

[0024] Figure 2 This is a schematic diagram of the structure of metadata service 200-1 provided in an embodiment of this application;

[0025] Figure 3A This is a first flowchart illustrating the object-type data processing method provided in the embodiments of this application;

[0026] Figure 3B This is a second flowchart illustrating the object-type data processing method provided in the embodiments of this application;

[0027] Figure 3C This is a schematic diagram of the third process of the object-type data processing method provided in the embodiments of this application;

[0028] Figure 3D This is a schematic diagram of the fourth process of the object-type data processing method provided in the embodiments of this application;

[0029] Figure 3E This is a fifth flowchart illustrating the object-type data processing method provided in the embodiments of this application;

[0030] Figure 3F This is a sixth flowchart illustrating the object-type data processing method provided in the embodiments of this application;

[0031] Figure 3G This is a seventh flowchart illustrating the object-type data processing method provided in the embodiments of this application;

[0032] Figure 3H This is the eighth flowchart of the object-type data processing method provided in the embodiments of this application;

[0033] Figure 3I This is a ninth flowchart illustrating the object-type data processing method provided in the embodiments of this application;

[0034] Figure 3J This is a schematic diagram of the tenth process of the object-type data processing method provided in the embodiments of this application;

[0035] Figure 4 This is a schematic diagram of the directory tree structure provided in an embodiment of this application;

[0036] Figure 5 This is a schematic diagram illustrating the partitioning of the target file system object provided in an embodiment of this application;

[0037] Figure 6 This is an application illustration of generating new storage objects provided in an embodiment of this application;

[0038] Figure 7 This is a schematic diagram of the state change of the conversion task provided in the embodiments of this application;

[0039] Figure 8 This is a schematic diagram showing the state of the target index node in a state to be converted, as provided in the embodiments of this application;

[0040] Figure 9 This is a schematic diagram of the state detection process provided in an embodiment of this application;

[0041] Figure 10 This is a schematic diagram illustrating the storage type conversion of multiple file system objects provided in this application embodiment;

[0042] Figure 11 This is a detailed process diagram of accessing the first storage space provided in the embodiments of this application.

[0043] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0046] In the following description, the terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0047] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0048] Unless otherwise specified, "at least one" as used below refers to one or more cases, and "multiple" can refer to two or more cases.

[0049] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0050] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0051] 1) A file system object is a data entity stored in the file system. File system objects are the basic units in the file system, and their types include at least one of the following: file, directory, device file, socket, and pipe.

[0052] 2) A storage object is an entity in the file system used to actually store the metadata of file system objects. Metadata is descriptive information about a file or directory and may include at least one of the following: creation time, modification time, size, and permissions. The metadata of a file system object can be stored in a single storage object or distributed across multiple storage objects.

[0053] 3) Storage type is a classification used in a file system to represent the data storage method and characteristics. Different storage types are usually associated with data usage patterns, access frequency, and storage costs. File system objects have the following four storage types: Standard, Low-Frequency, Archive, and Deep Archive.

[0054] Standard storage type: This typically refers to a high-performance, high-reliability storage type, suitable for data accessed more frequently than a first frequency threshold. This storage type generally provides fast read and write operations, ensuring data real-time performance and availability. Standard storage is relatively expensive and suitable for storing critical business data or frequently used data. The first frequency threshold can be determined by performing the following process: sorting the access frequency of data within a specific time period in descending order, and using the mean, minimum, or maximum value of the access frequency of a first preset number or a first preset proportion of data in the sorted results as the first frequency threshold. As an example, sorting the access frequency of data within a week in descending order, the access frequency of the last data in the top 25% (the proportion can be customized according to business needs) of the sorted results is used as the first frequency threshold.

[0055] Low-frequency storage type: This storage type is designed for data whose access frequency is lower than a first frequency threshold but higher than a second frequency threshold. The second frequency threshold is lower than the first frequency threshold and can be determined by performing the following process: Sort the access frequency of data within a specific time period in descending order, and use the average, minimum, or maximum value of the access frequency of a second preset quantity or a second preset proportion of data in the sorted results as the second frequency threshold. For example, sort the access frequency of data within a week in descending order, and use the access frequency of the last data in the top 50% (the proportion can be customized according to business needs) of the sorted results as the second frequency threshold. This type of data may not require the high-speed access of data in standard storage types, therefore, lower-cost storage media can be used. Low-frequency storage is suitable for storing backup data, historical records, or seasonal data.

[0056] Archive storage type: Used for long-term data preservation. This data has an access frequency below a second frequency threshold but above a third frequency threshold, but has retention value for the organization. The third frequency threshold is lower than the second frequency threshold. The third frequency threshold can be determined by performing the following process: Sort the access frequency of data within a specific time period in descending order, and use the average, minimum, or maximum value of the access frequency of the data in the sorted results (a third preset number or a third preset proportion) as the first frequency threshold. For example, sort the access frequency of data within a week in descending order, and use the access frequency of the last data in the top 70% (the proportion can be customized according to business needs) as the third frequency threshold. Archive storage is generally the lowest cost, but access speed may be slower. It is suitable for storing legal documents, historical records, or log files, etc.

[0057] Deep archive storage is for data that needs to be stored long-term and accessed less than the third frequency threshold. This type typically has the lowest cost but the slowest access speed, and may require manual recovery or the use of specific tools to access the data. Deep archive storage is suitable for storing disaster recovery data, long-term backups, or other long-term storage needs.

[0058] 4) The first storage space refers to the physical space used to store the actual data content in the file system. This space contains the actual content of file system objects, such as text in a text file, pixel data in an image file, and video streams in a video file. The first storage space typically consists of multiple storage nodes, which can be storage devices such as hard disk drives, solid-state drives, or tape libraries.

[0059] 5) Secondary storage space refers to the space used to store descriptive information (such as metadata) about the data itself in the file system. This information includes the file's name, size, creation and modification time, permissions, owner, or the location of data blocks. In a distributed file system, secondary storage space is typically maintained by a dedicated metadata server or metadata management node. These nodes are responsible for handling metadata read and write requests and ensuring the integrity and consistency of the metadata.

[0060] 6) Lifecycle refers to the entire existence of a file system object within a certain storage type, from its inception to its termination. The concept of lifecycle is usually closely related to storage type and storage policy, used to indicate the duration of a file system object's existence within a certain storage type, and how it can be migrated or deleted as needed.

[0061] 7) An inode is a data structure in the file system, such as any node in a directory tree. One inode corresponds to one file system object. Inodes store key information about file system objects, i.e., metadata, enabling the operating system to quickly locate and manage these file system objects. Each inode has a unique identifier called the inode identifier, which the operating system uses to uniquely identify and access objects in the file system.

[0062] 8) A data block is a basic unit of storage in a file system used to store at least a portion of the contents of a file system object (such as a file). In a file system, large file system objects are usually divided into multiple data blocks for easier management and storage. Each data block contains a certain number of bytes, which is usually fixed in size but can vary depending on the design of the file system.

[0063] 9) Metadata service is an information service in a distributed file system responsible for managing and providing metadata about file system objects. It is a system service that maintains, organizes, and provides access to the metadata of file system objects. This metadata includes key information such as file size, creation time, modification time, access permissions, owner information, and data block location. Metadata service can be implemented by a server's metadata service layer or by a separate metadata server.

[0064] 10) Responding to: used to indicate the conditions or states on which the operation performed depends. When the conditions or states on which it depends are met, one or more operations performed may be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0065] 11) Human-computer interaction interface, which is used to provide human-computer interaction functions / interface to display file system objects.

[0066] For example, graphical user interfaces (GUIs) include augmented reality (AR) interfaces, virtual reality (VR) interfaces, voice user interfaces (VUIs), interactive projection interfaces (using projection technology to display information on a flat surface), eye-tracking interfaces (interfaces controlled by detecting the user's gaze), holographic interfaces (three-dimensional holograms formed by projecting images using holographic projection technology, allowing users to see stereoscopic images without wearing special glasses), multimodal interfaces (interfaces that combine multiple interaction methods, such as tactile, visual, and auditory interaction), and brain-machine interfaces (BMIs).

[0067] During the daily operation of a file system, the storage type of a file system object may change, but the storage type of the corresponding metadata may remain the same, meaning that the storage type of data in different storage spaces may be inconsistent.

[0068] Based on the above analysis, the applicant found that the object-type data processing methods of related technologies cannot control the consistency of data storage types in different storage spaces, resulting in the inability to synchronize data management in different storage spaces, which increases the cost of data storage and management. In order to address the above problems, this application provides an object-type data processing method that can improve the reliability of data storage.

[0069] The following describes exemplary applications of the electronic devices provided in the embodiments of this application. These electronic devices can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals, or as servers. The following will describe exemplary applications when the electronic device is implemented as a server.

[0070] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the object data processing system 100 provided in the embodiments of this application. In order to support an object data processing application, the terminal (terminal 400-1 and terminal 400-2 are shown as examples) connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0071] Taking a software development scenario as an example, when writing a graphics editing software, the target file system object can be an object of the Shape class. The Shape class object includes the following attributes: color, position, and size. Terminal 400 sends a first conversion instruction to server 200. Server 200 converts the storage type of the Shape class object according to the received first conversion instruction and sends the converted Shape class object back to terminal 400 for display on the human-computer interaction interface 410.

[0072] Taking a database management scenario as an example, when storing and retrieving library book information, the target file system object can be a Book object, which includes the following attributes: title, author, ISBN, and publisher. Terminal 400 sends a first conversion instruction to server 200, and server 200 converts the storage type of the Book object according to the received first conversion instruction, and sends the converted Book object to terminal 400 for display on the human-computer interaction interface 410.

[0073] Taking a graphical user interface design scenario as an example, when designing a text editor, the target file system object can be a TextField object. The TextField object includes the following properties: text, font, and color. Terminal 400 sends a first conversion instruction to server 200. Server 200 converts the storage type of the TextField object according to the received first conversion instruction and sends the converted TextField object back to terminal 400 for display on the human-computer interaction interface 410.

[0074] Taking a game development scenario as an example, when developing a role-playing game, the target file system object can be a Character object. The Character object includes the following attributes: health, strength, and inventory. Terminal 400 sends a first conversion instruction to server 200. Server 200 converts the storage type of the Character object according to the received first conversion instruction and sends the converted Character object back to terminal 400 for display on the human-computer interaction interface 410.

[0075] Taking an enterprise application scenario as an example, when managing customer information, the target file system object can be a Customer object, which includes the following attributes: name, email, phone, and purchaseHistory. Terminal 400 sends a first conversion instruction to server 200, and server 200 converts the storage type of the Customer object according to the received first conversion instruction, and sends the converted Customer object to terminal 400 for display on the human-computer interaction interface 410.

[0076] Taking a network application scenario as an example, when creating an online reservation system, the target file system object can be a Reservation object. The Reservation object includes the following attributes: customerID, date, time, and seatNumber. Terminal 400 sends a first conversion instruction to server 200. Server 200 converts the storage type of the Reservation object according to the received first conversion instruction and sends the converted Reservation object back to terminal 400 for display on the human-computer interaction interface 410.

[0077] Taking an artificial intelligence and machine learning scenario as an example, when building a recommendation system, the target file system object can be a User object, which includes the following attributes: preferences, purchaseHistory, and ratings. Terminal 400 sends a first conversion instruction to server 200, and server 200 converts the storage type of the User object according to the received first conversion instruction, and sends the converted User object to terminal 400 for display on the human-computer interaction interface 410.

[0078] Taking a simulation scenario as an example, when simulating urban traffic flow, the target file system object can be a Vehicle object, which includes the following attributes: position, speed, and destination. Terminal 400 sends a first conversion instruction to server 200. Server 200 converts the storage type of the Vehicle object according to the received first conversion instruction and sends the converted Vehicle object to terminal 400 for display on the human-computer interaction interface 410.

[0079] In different scenarios, the embodiments of this application ensure that the storage type of the metadata of the file system object is always consistent with the storage type of the file system object during the storage type conversion process, thereby improving the reliability of the data and enhancing the stability of the corresponding business processing.

[0080] In some embodiments, server 200 may be a standalone physical server, a server cluster, or a distributed system.

[0081] When server 200 is an independent physical server, terminal 400 generates a first conversion instruction and sends it to server 200. Server 200 includes metadata service 200-1, public gateway service 200-2, protocol conversion service 200-3, data access service 200-4, and data management service 200-5. Among them, public gateway service 200-2 can be a public gateway interface, used to receive the first metadata request and send it to protocol conversion service 200-3. Protocol conversion service 200-3 can be a protocol conversion interface, used to convert the first metadata request from object storage protocol (S3) based to Portable Operating System Interface (POSIX) semantics and send it to data access service 200-4. Data access service 200-4 can be a data access interface, used to send the first metadata request based on POSIX semantics to metadata service 200-1. Metadata service 200-1 can be a metadata service layer in server 200, used to implement the object data processing method provided in the embodiments of this application. When terminal 400 generates a second conversion instruction and sends it to server 200, public gateway service 200-2 can be a public gateway interface used to receive the second metadata request and send it to data access service 200-4. Data access service 200-4 can be a data access service layer in server 200 used to receive the second metadata request sent by public gateway service 200-2 and send it to metadata service 200-1. Metadata service 200-1 responds to the second metadata request and retrieves the metadata of candidate data blocks from the first storage space. After converting the storage type of the target file system object from the first storage type to the second storage type, or from the second storage type to the first storage type, metadata service 200-1 sends the target file system object of the second storage type or the target file system object of the first storage type to terminal 400 via network 300 for display on human-computer interaction interface 410.

[0082] When server 200 is a server cluster or a distributed system, terminal 400 generates a first conversion instruction and sends it to server 200. Server 200 includes metadata service 200-1, public gateway service 200-2, protocol conversion service 200-3, data access service 200-4, and data management service 200-5. Among them, public gateway service 200-2 can be a public gateway server, used to receive the first metadata request and send it to protocol conversion service 200-3. Protocol conversion service 200-3 can be a protocol conversion server, used to convert the first metadata request from object storage protocol (S3) based to Portable Operating System Interface (POSIX) semantics and send it to data access service 200-4. Data access service 200-4 can be a data access server, used to send the first metadata request based on POSIX semantics to metadata service 200-1. Metadata service 200-1 can be a metadata server, used to implement the object data processing method provided in the embodiments of this application. When terminal 400 generates a second conversion instruction and sends it to server 200, public gateway service 200-2 can be a public gateway server, used to receive the second metadata request and send it to data access service 200-4. Data access service 200-4 can be a data access server, used to receive the second metadata request sent by public gateway service 200-2 and send it to metadata service 200-1. Metadata service 200-1 responds to the second metadata request and retrieves the metadata of the candidate data block from the first storage space. After converting the storage type of the target file system object from the first storage type to the second storage type, or from the second storage type to the first storage type, metadata service 200-1 sends the target file system object of the second storage type or the target file system object of the first storage type to terminal 400 via network 300 for display on human-computer interaction interface 410.

[0083] In some embodiments, server 200 may be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, and this embodiment does not impose any restrictions.

[0084] Taking metadata service 200-1 as an example, which is a metadata server, see [link / reference]. Figure 2 , Figure 2 This is a schematic diagram of the structure of the metadata service 200-1 provided in the embodiments of this application. Figure 2 The metadata service 200-1 shown includes at least one processor 210, memory 230, and at least one network interface 220. The various components in the metadata service 200-1 are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 240.

[0085] The processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0086] The memory 230 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 230 may optionally include one or more storage devices physically located away from the processor 210.

[0087] The memory 230 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 230 described in this application embodiment is intended to include any suitable type of memory.

[0088] In some embodiments, memory 230 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0089] Operating system 231 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0090] The network communication module 232 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0091] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A processing device 233 for object-type data stored in memory 230 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: instruction receiving module 2331, first conversion module 2332, second conversion module 2333, and third conversion module 2334. These modules are logically connected and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0092] The method for processing object-type data provided in this application will be described in conjunction with exemplary applications and implementations of the server provided in the embodiments of this application.

[0093] For ease of explanation, this application embodiment uses the above-described metadata service 200-1 as an example of the metadata service layer in server 200. Server 200 includes a first storage space and a second storage space. The first storage space is used to store metadata, and the second storage space is used to store entity data, such as file system objects.

[0094] See Figure 3A , Figure 3A This is a first flowchart illustrating the object-type data processing method provided in this application embodiment. Taking the metadata service 200-1 as the executing entity as an example, it will be combined with... Figure 3A The steps shown are explained.

[0095] In step 101, a first conversion instruction is received, wherein the first conversion instruction is used to instruct the storage type conversion of the target file system object.

[0096] In some embodiments, the first conversion instruction may be generated based on user operations or by another entity outside the metadata server. The target file system object type includes at least one of the following: file, directory, device file, socket, and pipe. The storage type of the target file system object includes standard, low-frequency, archive, and deep archive. Different storage types represent different read frequencies of the target file system object, and the storage types are ordered from highest to lowest read frequency as: standard, low-frequency, archive, and deep archive.

[0097] For example, the first conversion instruction can be generated based on user actions, such as responding to a user's open operation on a file that has been in archive storage for a long time, generating a first conversion instruction for that file. The first conversion instruction can also be generated by other entities outside the metadata server. For instance, an automation policy might be set up to automatically generate a first conversion instruction for a set of files that have not been accessed in the past six months, instructing them to be converted from the standard storage type to a low-frequency storage type. Alternatively, when storage space is detected to be approaching quota limits, a first conversion instruction can be generated, instructing files used less frequently than a frequency threshold to be converted to a smaller storage type, such as deep archive storage. The decision to convert the storage type can be based on preset rules or policies, such as factors like file access patterns, size, creation time, or storage duration. The first conversion instruction can include the identity document (ID), lifecycle, original storage type, and target storage type of the target file system object.

[0098] In some embodiments, a file system is generally represented as a directory tree structure, where each node is called an index node (Inode). Each Inode has attributes that identify the type of the current node. The Portable Operating System Interface (POSIX) semantics can include unary and binary operations. The file system can quickly find index nodes using the index node identifier (InodeId). In the Hadoop Distributed File System (HDFS) scenario, the index node identifier is found by path, and then the specific index node is located using the index node identifier. Each index node mainly contains the following three pieces of information: parent node identifier (ParentId), index node identifier, and index node name. The parent node identifier is used to directly index its parent node information.

[0099] For example, see Figure 4 , Figure 4 This is a schematic diagram of the directory tree structure provided in the embodiments of this application. Figure 4The directory tree 901 contains three directories and two files. Files are generally represented as leaf nodes, such as leaf nodes 905 and 906, while intermediate path nodes are represented as directories, such as intermediate path nodes 902 (root directory / ), intermediate directory node 903, and intermediate path node 904. Directory paths include / directory1 and / directory1 / directory2, and file paths include / directory1 / file2 and / file1. Each leaf node or intermediate path node is an index node. Unary operations include adding, deleting, modifying, or querying index nodes, such as modifying index node attributes. Binary operations commonly involve renaming, such as moving file 2 to directory 2, which involves modifying the attributes of two parent directories, making file 2's parent directory point to directory 2, and deleting content related to file 2 from the child items of directory 1.

[0100] In some embodiments, see Figure 3B , Figure 3B This is a schematic diagram of the second process of the object-type data processing method provided in this application embodiment. Before step 101 "receiving the first conversion instruction", the following steps are executed: Figure 3B Steps 1001 to 1002 are explained in detail below.

[0101] In step 1001, the target file system object is divided into multiple data blocks, which are used for distributed storage of the contents of the target file system object.

[0102] In some embodiments, the data block size can be set in any of the following ways: setting the data block size based on empirical values; optimizing the data block size by testing the impact of different data block sizes on system performance, where the data block size is positively correlated with the performance value; setting the data block size to an integer multiple of the disk sector in order to improve the utilization of storage space; or setting the data block size to be positively correlated with the size of the target file system object.

[0103] For example, see Figure 5 , Figure 5 This is a schematic diagram illustrating the partitioning of the target file system object provided in an embodiment of this application. Figure 5 In the target file system object, index node 907 in the directory tree divides the target file system object into multiple data blocks. If the size of the target file system object is 15M and the maximum size of the data block is 4M, then the multiple data blocks are data block 1 (4M), data block 2 (4M), data block 3 (4M), and data block 4 (3M). Each data block is used to store a portion of the contents of the target file system object.

[0104] In step 1002, for each data block, the metadata of the data block is stored in a storage object.

[0105] For examples, see below. Figure 5 The metadata of data block 1, data block 2, data block 3 and data block 4 are each stored through a storage object. For example, the metadata of data block 1 is stored in storage object 1, the metadata of data block 2 is stored in storage object 2, the metadata of data block 3 is stored in storage object 3, and the metadata of data block 4 is stored in storage object 4.

[0106] Here, if the target file system object is refreshed (the buffer is refreshed, and the data in the buffer is immediately written to the target file system object) and a close operation is performed on the target file system object, then the upload of the target file system object is achieved.

[0107] This application's embodiments divide the target file system object into multiple data blocks and store the metadata of each data block through a storage object, thereby efficiently managing storage resources and improving storage efficiency. Allowing distributed storage of the target file system object's metadata helps achieve load balancing in a distributed environment, improving system reliability and scalability.

[0108] In some embodiments, see Figure 3C , Figure 3C This is a schematic diagram of the third process of the object-type data processing method provided in this application embodiment. Before step 1002 "Storing the metadata of the data block in a storage object", the following steps are executed: Figure 3C Steps 201 to 203 are explained in detail below.

[0109] In step 201, a second metadata request based on the object storage protocol is received through the protocol conversion interface.

[0110] For example, when retrieving metadata from the first storage space, the second metadata request is first sent to the common gateway interface (CGI) of the second storage space through the object storage application programming interface of the second storage space. The common gateway interface can be the common gateway service 200-2 mentioned above. Then, the second metadata request is forwarded to the protocol conversion interface (Gateway) for protocol conversion through the common gateway interface of the second storage space. The protocol conversion interface can be the protocol conversion service 200-3 mentioned above.

[0111] In step 202, the second metadata request based on the object storage protocol is converted into a second metadata request based on the semantics of the portable operating system interface.

[0112] In some embodiments, the protocol conversion interface is used to convert a second metadata request based on the object storage protocol (S3) into a second metadata request based on the Portable Operating System Interface Semantics (POSIX).

[0113] For example, S3 is an object-oriented storage protocol primarily used for cloud storage services, manipulating objects via Hypertext Transfer Protocol (HTTP) or Hypertext Transfer Protocol Secure (HTTPS) requests. POSIX is a specification for defining operating system interfaces, defining a series of Application Programming Interfaces (APIs) and functions to provide consistent file system operations across different systems. In S3, metadata requests can be transmitted via HTTP headers or specially encoded URLs; while in POSIX, metadata requests are typically completed via system calls (such as stat()). The aforementioned "converting a second metadata request based on an object storage protocol into a second metadata request based on portable operating system interface semantics" specifically involves: receiving a second metadata request using the S3 protocol; parsing the second metadata request and extracting metadata information; encapsulating the extracted metadata information according to the POSIX protocol format; and sending the encapsulated POSIX second metadata request to a POSIX-enabled file system or service.

[0114] In step 203, the metadata of the data block is obtained from the first storage space through the data access interface according to the second metadata request based on the semantics of the portable operating system interface.

[0115] For example, a POSIX-based second metadata request is sent to the metadata service layer (MetaServer) of the first storage space through a data access interface (Proxy). The metadata service layer of the first storage space then obtains the metadata of the data block. The data access interface can be the data access service 200-4 mentioned above, and the metadata service layer can be the metadata service 200-1 mentioned above.

[0116] Here, in implementing POSIX semantics, the conversion process of storage types supported by the object storage protocol under this metadata structure is also supported. The aforementioned metadata service layer can provide the standard S3 protocol to extend metadata services; that is, it can be accessed using either POSIX semantics or S3 semantics. Due to the extended S3 semantic interface, the metadata service layer supports calls to interfaces such as atomic rename or recursive deletion.

[0117] This application embodiment can receive second metadata requests based on object storage protocols and convert them into second metadata requests based on portable operating system interface semantics through a protocol conversion interface, thereby enhancing compatibility between different semantics.

[0118] In some embodiments, see Figure 3D , Figure 3D This is a schematic diagram of the fourth process of the object-type data processing method provided in this application embodiment. The process can be executed during step 1002, "Storing the metadata of the data block in a storage object," or after step 1002, "Storing the metadata of the data block in a storage object." Figure 3D Steps 301 to 302 are explained in detail below.

[0119] In step 301, for any data block, if the data block is truncated at least once in order from head to tail, multiple sub-data blocks are obtained.

[0120] For example, see Figure 6 , Figure 6 This is an application illustration of generating new storage objects provided in an embodiment of this application. Figure 6 In this context, for any data block, such as data block 908 (data block 2), the data block can be truncated at least once in order from head to tail according to the size of the required file system object. If the size of the required file system object is 6M and the size of data block 1 is 4M, then data block 2 is truncated at the 2M position to obtain two 2M sub-data blocks. For example, the 4M data block 908 is truncated into two 2M sub-data blocks, such as sub-data block 909.

[0121] In step 302, candidate data blocks are added after the last sub-data block, and the metadata of the last sub-data block and the metadata of the candidate data blocks are stored together in a new storage object.

[0122] In some embodiments, the last sub-data block refers to the last sub-data block in the order from head to tail among the multiple sub-data blocks obtained after any data block is truncated.

[0123] For examples, see below. Figure 6After adding a candidate data block (e.g., candidate data block 910) after the last sub-data block (e.g., sub-data block 909), the metadata of the last sub-data block 909 and the metadata of the candidate data block 910 are stored together in a new storage object, such as storage object 911. This new storage object is different from the storage objects corresponding to the multiple data blocks of the file system object. It is a newly created storage object based on the original storage object and is used to store the metadata of the last sub-data block and the candidate data block. While constructing the new storage object, the storage object of the original data block 908 is retained.

[0124] This application allows data blocks to be truncated as needed to adapt to different storage and access patterns. By truncating data blocks, storage resources can be used more efficiently, avoiding waste. By supplementing candidate data blocks after the last sub-data block, unused space can be reduced, improving the overall utilization of storage space. Combining and storing the metadata of the last sub-data block with that of the candidate data blocks can reduce the number of accesses and optimize performance.

[0125] In some embodiments, see Figure 3E , Figure 3E This is a schematic diagram of the fifth process of the object-type data processing method provided in this application embodiment. Before step 302 "Storing the metadata of the data block in a storage object", the following steps are executed: Figure 3E Steps 401 to 402 are explained in detail below.

[0126] In step 401, a first metadata request is received via the public gateway interface and the data management interface, wherein the first metadata request is used to indicate the acquisition of metadata for candidate data blocks.

[0127] For example, see Figure 1 The public gateway interface can be Figure 1 The public gateway service 200-2 in the middle, the data management interface can be Figure 1 In the data management service 200-5, after the first metadata request arrives at the server (e.g., server 200), it can first pass through the public gateway service 200-2, and then through the data management service 200-5, before being received by the metadata service 200-1. When adding or deleting some data in the first storage space, such as adding metadata for candidate data blocks, the first metadata request can be forwarded to the data management interface (RootServer) through the public gateway interface of the second storage space. The data management interface can be the aforementioned data management service 200-4, and the metadata service layer can be the aforementioned metadata service 200-1.

[0128] This application embodiment receives requests through a public gateway interface. The public gateway interface can provide a security mechanism to ensure that only authorized requests can access metadata, and to ensure that the metadata of candidate data blocks obtained from the first storage space is complete and accurate. Metadata is obtained efficiently through the data management interface, reducing processing time.

[0129] In step 402, the metadata of the candidate data block is obtained from the first storage space according to the first metadata request.

[0130] For example, the metadata service layer responds to the first metadata request, which may include an identifier of the candidate data block, and determines the storage location of the candidate data block's metadata from the first storage space based on the identifier, and retrieves the candidate data block's metadata from that storage location.

[0131] See also Figure 3A In step 102, the storage type of multiple storage objects in the first storage space is changed from the first storage type to the second storage type, wherein the multiple storage objects are used to distribute the metadata of the target file system object.

[0132] Here, each storage object is used to store a portion of the metadata of the target file system object, and there is no overlap between the metadata stored in each storage object. The conversion from the first storage type to the second storage type can be a conversion from a storage type with a read frequency below a frequency threshold to a storage type with a read frequency above a frequency threshold, such as from archive to low-frequency; or it can be a conversion from a storage type with a read frequency above a frequency threshold to a storage type with a read frequency below a frequency threshold, such as from low-frequency to archive.

[0133] In some embodiments, before converting the storage type of the target file system object, a task table is constructed, and the storage type of the target file system object is converted based on the task table. The structure of the task table may include at least one of the following: the storage path, type, number of days, status, and target inode identifier of the target file system object. The storage path of the target file system object refers to the full path of the target file system object to be converted. The recovery modes for target file system objects of archive storage type or intelligent hierarchical archive type mainly include three types as follows: Expedited recovery mode, where the recovery task is completed within the first time period, for example, within 1-5 minutes; Standard recovery mode, where the recovery task is completed within the second time period, which is longer than the first time period, for example, within 3-5 hours; and Bulk recovery mode, where the recovery task is completed within the third time period, which is longer than the second time period, for example, within 5-12 hours. For target file system objects of deep archive storage type or intelligent tiered deep archive type, there are two recovery modes: Standard recovery mode, with a recovery time of the fourth time period, which is longer than the third time period, for example, 12-24 hours; and Bulk recovery mode, with a recovery time of the fifth time period, which is longer than the fourth time period, for example, 24-48 hours.

[0134] Here, taking the storage type change from archive (first storage type) to low-frequency (second storage type) as an example, it is fully compatible with the types supported by the object storage (Post Restore) interface. The "days" parameter refers to the effective duration of the temporary copy generated during the storage type conversion process for file system objects of both ordinary archive and deep archive storage types. The unit can be "days". This parameter is not required for file system objects of intelligent tiered archive or deep archive layers. The readable file system object obtained through warm-up (storage type change from archive to low-frequency) is temporary, a temporary copy of the original target file system object. The lifecycle of the target file system object can be set to remain readable until the temporary copy is deleted. The lifecycle (expiration time) of the temporary copy can be specified using the "days" parameter. If the lifecycle is exceeded and no copying or extension operations have been performed on the target file system object during its usable period, the temporary copy will be automatically deleted by the system. The temporary copy is only a copy of the archive storage type file system object; the original archive type file system object exists throughout its lifecycle.

[0135] The storage type conversion task has four states: Binding (meaning the inode representation is being bound to the conversion task), BindSuccess, Restoring, and RestoreSuccess. The Binding and Restoring states are intermediate processes that require asynchronous detection to transition to the BindingSuccess or RestoreSuccess states once conditions are met. The execution of the conversion task is task-driven, but the intermediate process of converting the storage type of the target file system object depends on the inode state machine. The target inode's states include Async_transfering, Async_monitor, and Arch_up_restore.

[0136] In some embodiments, the first storage space includes a directory tree, which includes multiple index nodes, and the target file system object corresponds to a target index node.

[0137] For examples, see below. Figure 5 The first storage space includes a directory tree, which contains multiple index nodes. The target file system object corresponds to the target index node. For example, index node 907 is the target index node.

[0138] In some embodiments, Figure 3A Step 102, "Converting the storage type of multiple storage objects in the first storage space from the first storage type to the second storage type", can be achieved by performing the following process: when the state of the target index node is the state to be converted, convert the storage type of multiple storage objects in the first storage space from the first storage type to the second storage type, and update the type value in the label of each converted storage object to the second storage type.

[0139] For example, the process retrieves all data blocks corresponding to an inode and their corresponding storage objects. When the target inode is in a state awaiting conversion, the `PutObjectTag` method in the S3 interface is called to update the type values ​​in the tags of all storage objects to the second storage type. When the target file system object is a normal file or a part file, the process searches the directory tree to find the target inode and assigns change instructions to the storage objects corresponding to the target inode. Specifically, assigning change instructions includes determining the target inode, identifying the corresponding multiple storage objects, and tagging the storage objects. When the target file is a MultiPart Upload (MPU) file, the target inode is first located in the directory tree. Then, the partition file corresponding to the MPU file is determined, and the target sub-inodes corresponding to each partition file are obtained. The status of the target sub-inodes corresponding to each partition file is set to the pending conversion state. Change instructions are then assigned to the storage objects corresponding to each partition file. After all the storage objects corresponding to a single partition file have been assigned change instructions, the status of the target sub-inodes is set to the detection state. After all the target sub-inodes are set to the detection state, the status of the conversion task of the target inode corresponding to the MPU file is changed from the binding successful state to the conversion in progress (restoring) state.

[0140] In implementing POSIX semantics by building a directory tree based on object storage, it's crucial to ensure that during storage type conversion, the underlying storage object's storage type is converted first, followed by the inode's storage type state conversion. This ensures consistency between the underlying storage object's and the upper-level file system object's storage state. Storage type state conversion is related to billing. Different rules, such as deletion or sinking, can be configured to meet cost reduction requirements. However, since one inode corresponds to multiple storage objects, considering the non-atomic nature of the operation and the lack of immediate visibility of the state, the inode's state is split. For example, the "pending conversion" stage represents the inode hitting a rule and reversing its state. This state appears as "sinking" to the terminal (e.g., terminal 400 mentioned above). The actual conversion stage is executed asynchronously by the server (e.g., metadata service 200-1 mentioned above). This allows the terminal to quickly meet user needs, shortening the data conversion waiting time while ensuring billing.

[0141] In some embodiments, see Figure 3F , Figure 3F This is a sixth flowchart illustrating the object-type data processing method provided in this application embodiment. Before step 102, "converting the storage type of multiple storage objects in the first storage space from the first storage type to the second storage type", the following steps are executed: Figure 3FSteps 501 to 503 are explained in detail below.

[0142] In step 501, the target index node corresponding to the target file system object is queried from the directory tree, and the status of the conversion task of the target index node is set to the binding state. The binding state is the state in which the target index node and the conversion task are being bound. The target index node includes the metadata of the target file system object. The conversion task is used to convert from the first storage type to the second storage type.

[0143] Here, the "Binding in Progress" state explicitly indicates that a connection is being established between the target inode and the transformation task. It helps reflect which file system objects are undergoing preprocessing or configuration work for the transformation task. By setting the "Binding in Progress" state, you can ensure that the state of the file system object's metadata is synchronized with the state of the transformation task, avoiding inconsistencies during the transformation process. The "Binding in Progress" state can reserve necessary resources, such as CPU, memory, and storage bandwidth, for the upcoming transformation task to ensure a smooth transformation process. When a transformation task is in the "Binding in Progress" state, it prevents other processes or tasks from performing conflicting operations on the target file system object, thereby avoiding data corruption or transformation failure.

[0144] For example, see Figure 7 , Figure 7This is a schematic diagram of the status change of the conversion task provided in the embodiments of this application. The conversion task is a periodic task. The periodic task obtains the specific task through the restore task list (InstRestoreTasks). First, the status of the conversion task is determined. If it is not in the binding state, the target inode corresponding to the target file system object is queried from the directory tree (getInodeById). If the target file system object has been deleted (ifNotExistDeleteTask), the corresponding conversion task is invalid, and the conversion task corresponding to the target file system object is directly deleted. Because of the latency between the database and the data in memory, locking can be used to ensure data consistency when reading data. Specifically: before reading or modifying the transformation task data, a lock is acquired by calling the operating system's locking API or using a locking mechanism in the database management system. Lock types include mutex locks and optimistic locks. With a mutex lock, no other thread or process is accessing or modifying the transformation task data before the lock is acquired. With an optimistic lock, a version number or timestamp is recorded when reading data, and the version number or timestamp is checked before writing to ensure it hasn't been changed by other threads. After acquiring the lock, the thread can safely read or modify the transformation task data without worrying about interference from other threads. After data processing is complete, the thread releases the lock, allowing other threads to access or modify the data. The transformation task's state transitions sequentially from the binding state to the successful binding state (bindSuccess), the pending transformation state (Restoring), and the successful transformation state (RestoreSuccess).

[0145] In step 502, the level of the first storage type of the target file system object is detected.

[0146] For example, in the task-driven process, the compliance of the binding state is verified. Taking the warm-up task as an example, verifying the compliance of the task means determining whether the state of the target inode needs to be warmed up. That is, warm-up can only be performed when the type of the target inode is archive or deep archive. Among them, deep archive does not support fast warm-up.

[0147] In step 503, if the level of the first storage type of the target file system object is a preset level, the status of the conversion task of the target inode is changed from the binding state to the binding successful state, and the status of the target inode is set to the conversion state. Here, the level is positively correlated with the reading frequency of the target file system object, and the conversion state indicates that the storage type of multiple storage objects in the first storage space is to be converted from the first storage type to the second storage type.

[0148] Following the example of step 502 above, if the compliance conditions are not met, the reactivation task will be deleted. If the verification passes, a reactivation task will be generated. If the target file system object is an MPU file, the target sub-inode (parts inode) corresponding to each partition file will be obtained through the partition list (istparts) and executed. Figure 7 The same judgment logic applies, namely, determining whether the conversion task is bound to the inode identifier. If it is not bound, the inode identifier is obtained based on the path, and the status of the conversion task changes from the binding status to the binding successful status.

[0149] For example, see Figure 8 , Figure 8 This is a schematic diagram illustrating the state of the target index node in the pending conversion state provided in this application embodiment. When the target index node is in the pending conversion state, all storage objects corresponding to the target index node are obtained, and the conversion interface (postrestore) is called. After all storage objects have successfully called the postrestore interface, the conversion task information and target state (async_transferting) are recorded through the target index node.

[0150] Here, the execution of the transformation task requires task-driven triggering. However, after the node binding is successful and the target index node's state is changed to the "pending transformation" state, the state machine takes over the state-driven process. The "pending transformation" state and the subsequent state of the target index node store key information about the transformation task (transformation type, transformation days, transformation date), meaning that the state can be transformed independently without relying on the transformation task-driven information.

[0151] This application embodiment uses the "binding in progress" status to reflect the binding process between the conversion task and the target inode, ensuring that the conversion task is correctly associated with the target file system object. By detecting the storage type level, objects that meet the conditions can be filtered for conversion, improving the efficiency and accuracy of the conversion. Updating the conversion task's status from "binding in progress" to "binding successful" marks the conversion task as ready to execute, providing a clear signal for the conversion operation.

[0152] See also Figure 3A In step 103, the storage type of the target file system object in the second storage space is changed from the first storage type to the second storage type.

[0153] For example, if the target file system object corresponds to 4 storage objects, when the storage type of all 4 storage objects is changed from the first storage type to the second storage type, the storage type of the target file system object in the second storage space is changed from the first storage type to the second storage type.

[0154] In some embodiments, Figure 3A Step 103 can be achieved by performing the following processing: in response to the fact that the type value of the tags of multiple storage objects is the second storage type, the state of the target inode is changed to the detection state, and the storage type of the target file system object corresponding to the target inode is changed from the first storage type to the second storage type, wherein the detection state is the state in which the storage object is being detected.

[0155] Following the example of step 103 above, taking the process of changing the storage type of the target file system object from archive to low frequency as an example, when the type values ​​of the tags of multiple storage objects corresponding to the target file system object are all the second storage type, for example, when the type values ​​of the tags of storage object 1, storage type 2, storage object 3 and storage object 4 are all low frequency, the state of the target inode is changed to the detection state, and the storage type of the target file system object corresponding to the target inode is changed from archive to low frequency.

[0156] This application embodiment changes the state of the target inode to a "under detection" state, ensuring that the state of the storage object is checked before conversion, thus avoiding erroneous conversion of objects that do not meet the conditions. It ensures that the storage type conversion of the target file system object is performed only after the tag type values ​​of all storage objects corresponding to the target file system object are of the second storage type, guaranteeing the consistency and accuracy of the conversion.

[0157] In some embodiments, prior to step 103, the following process may be performed: the state of the target inode's transformation task is changed from a successfully bound state to a transformation in progress state, wherein the transformation in progress state is a state in which the storage type of the target file system object is being transformed.

[0158] Here, by simultaneously changing the target inode's state to "under detection" and the target inode's transformation task state from "bound successfully" to "transformation in progress," the synchronization and consistency of the system's internal states are ensured, preventing errors caused by state asynchrony. In a distributed environment, multiple transformation tasks can run concurrently; the "transformation in progress" state prevents simultaneous operations on the same target file system object, reducing the risk of concurrency conflicts. If errors occur during the transformation process, the "transformation in progress" state allows for quick problem location and appropriate error handling and recovery operations. The "transformation in progress" state provides real-time progress feedback, reflecting which transformation tasks are in progress, thereby dynamically adjusting resource allocation based on the currently ongoing transformation tasks and optimizing system resource utilization.

[0159] In some embodiments, see Figure 3G , Figure 3G This is a schematic diagram of the seventh step in the object-type data processing method provided in this application embodiment. After step 103, the following steps are executed: Figure 3GSteps 601 to 603 are explained in detail below.

[0160] In step 601, the status of each of the multiple storage objects is checked.

[0161] Here, at least one round of state checks is performed on each of the multiple storage objects.

[0162] In step 602, if multiple storage objects are in the expected state, the state of the target index node is changed from the detection state to the detected state. The expected state is the state in which the storage type of multiple storage objects is the second storage type, and the detected state indicates that multiple storage objects have been detected as being in the expected state.

[0163] In some embodiments, in any round of state detection, if at least some of the multiple storage objects are not in the expected state, then the next round of state detection is performed; until all multiple storage objects are in the expected state, the state transition of the target index node is changed from the state under detection to the state that has been detected.

[0164] For example, see Figure 9 , Figure 9 This is a schematic diagram of the state detection process provided in this application embodiment. When the state of the target index node is transitioned to the detection state, the key information of the state to be transitioned is used to determine whether the minimum time has been reached to determine the state. By obtaining the storage objects corresponding to all data blocks, the concurrent object storage interface (head object) is used to determine whether the current state is the expected state. If some storage objects have not transitioned to the expected state in the current round, the next round of detection is started until all storage objects have reached the expected state. Then, the state of the target index node is transitioned to the detected state. In the detected state, the key information is used to determine whether the lifecycle has been reached so that the target file system object can be re-archived. If necessary, the state of the target index node is directly returned to the initial state.

[0165] In step 603, the status of the target inode's conversion task is changed from the pending conversion state to the conversion successful state, wherein the conversion successful state indicates that the storage type of the target file system object has been changed from the first storage type to the second storage type.

[0166] Here, while changing the state of the target inode from the "in detection" state to the "detected" state, the state of the target inode's conversion task is changed from the "pending conversion" state to the "conversion successful" state, indicating that the target file system object has completed the storage type conversion process.

[0167] This application embodiment verifies the success of the conversion by detecting the status of multiple storage objects, ensuring that the conversion result meets expectations. Once all storage objects are detected to be in the expected state, the status of the target inode is updated, improving the system response speed. Subdividing the conversion process into multiple states allows for checks and verifications at different stages of the conversion, ensuring its correctness and integrity.

[0168] See also Figure 3A In step 104, in response to the target file system object of the second storage type satisfying the storage type conversion condition, the storage type of the target file system object is converted from the second storage type to the first storage type.

[0169] Taking the conversion of the storage type of the target file system object from archive to low-frequency via steps 101 to 103 as an example, if the target file system object meets the storage type conversion conditions, it can be converted from low-frequency to archive again. Similarly, taking the conversion of the storage type of the target file system object from low-frequency to archive via steps 101 to 103 as an example, if the target file system object meets the storage type conversion conditions, it can be converted from archive to low-frequency again.

[0170] In some embodiments, the level of the second storage type and the level of the first storage type are both positively correlated with the read frequency of the target file system object. When the level of the second storage type is lower than that of the first storage type, step 104 can be implemented by performing the following process: in response to receiving a third conversion instruction, the storage type of the target file system object is converted from the second storage type to the first storage type, wherein the third conversion instruction is used to instruct the conversion of the storage type of the target file system object.

[0171] For example, storage types include Standard, Low Frequency, Archive, and Deep Archive. The storage type is positively correlated with the read frequency of the target file system object; that is, file system objects of multiple storage types are sorted in descending order of read frequency as follows: Standard, Low Frequency, Archive, and Deep Archive. When the second storage type is at a lower level than the first storage type, the second storage type can be Archive, and the first storage type can be Low Frequency. In response to receiving a third conversion instruction, the storage type of the target file system object is changed from Archive to Low Frequency.

[0172] In some embodiments, see Figure 3H , Figure 3H This is a schematic diagram of the eighth process of the object-type data processing method provided in this application embodiment. The above-mentioned "converting the storage type of the target file system object from the second storage type to the first storage type" can be achieved through... Figure 3H Steps 1041 to 1042 are implemented, and the details are explained below.

[0173] In step 1041, the storage type of multiple storage objects in the first storage space is changed from the second storage type to the first storage type.

[0174] In some embodiments, step 1041, "converting the storage type of multiple storage objects in the first storage space from the second storage type to the first storage type", can be implemented by performing the following process: when the state of the target index node is a state to be converted, converting the storage type of multiple storage objects in the first storage space from the second storage type to the first storage type, and updating the type value in the label of each converted storage object to the first storage type.

[0175] For example, the process retrieves all data blocks corresponding to an inode and their corresponding storage objects. When the target inode is in a state awaiting conversion, the `PutObjectTag` method in the S3 interface is called to update the type values ​​in the tags of all storage objects to the first storage type. When the target file system object is a normal file or a part file, the process searches the directory tree to find the target inode and assigns change instructions to the storage objects corresponding to the target inode. Specifically, assigning change instructions includes determining the target inode, identifying the corresponding storage objects, and tagging the storage objects. When the target file is a MultiPart Upload (MPU) file, the target inode is first located in the directory tree. Then, the partition file corresponding to the MPU file is determined, and the target sub-inodes corresponding to each partition file are obtained. The status of the target sub-inodes corresponding to each partition file is set to the pending conversion state. Change instructions are then assigned to the storage objects corresponding to each partition file. After all the storage objects corresponding to a single partition file have been assigned change instructions, the status of the target sub-inodes is set to the detection state. After all the target sub-inodes are set to the detection state, the status of the conversion task of the target inode corresponding to the MPU file is changed from the binding successful state to the conversion in progress (restoring) state.

[0176] In some embodiments, before step 1041 "converting the storage type of multiple storage objects in the first storage space from the second storage type to the first storage type", the target index node corresponding to the target file system object is queried from the directory tree, and the status of the conversion task of the target index node is set to the binding state; the level of the second storage type of the target file system object is detected; if the level of the second storage type of the target file system object is a preset level, the status of the conversion task of the target index node is changed from the binding state to the binding successful state, and the status of the target index node is set to the pending conversion state.

[0177] For example, the transition task is a periodic task, and the state transition process of the transition task is similar to... Figure 7 The process shown is similar and will not be repeated here.

[0178] In step 1042, the storage type of the target file system object in the second storage space is changed from the second storage type to the first storage type.

[0179] In some embodiments, step 1042, "converting the storage type of the target file system object in the second storage space from the second storage type to the first storage type", can be implemented by performing the following process: in response to the fact that the type values ​​of the tags of multiple storage objects are all of the first storage type, the state of the target inode is changed to the detection state, and the storage type of the target file system object corresponding to the target inode is changed from the second storage type to the first storage type.

[0180] For example, taking the process of changing the storage type of a target file system object from low frequency to archive as an example, if the target file system object corresponds to 4 storage objects, when the type value of the tags of multiple storage objects corresponding to the target file system object is the first storage type, for example, when the type value of the tags of storage object 1, storage type 2, storage object 3 and storage object 4 is archive, the state of the target inode is changed to the detection state, and the storage type of the target file system object corresponding to the target inode is changed from low frequency to archive.

[0181] In some embodiments, when performing step 1042 "converting the storage type of the target file system object in the second storage space from the second storage type to the first storage type", the following process may also be performed: changing the state of the conversion task of the target inode from the binding successful state to the conversion in progress state, wherein the conversion in progress state is the state in which the storage type of the target file system object is being converted.

[0182] Here, when changing the target inode's state to the "in detection" state, changing the target inode's transformation task state from "bound successfully" to "transforming" provides a more accurate reflection of the transformation progress and reveals which file system objects are being processed. When a transformation task is in the "bound successfully" state, it means the file system object is ready for transformation but has not yet started. If another process attempts to operate on the same file system object at this time, a conflict may occur. Marking it as "transforming" prevents other processes or tasks from operating on these file system objects.

[0183] In some embodiments, after executing step 1042 "converting the storage type of the target file system object in the second storage space from the second storage type to the first storage type", the following processing may also be performed: performing status detection on multiple storage objects respectively; if multiple storage objects are all in the expected state, changing the status of the target inode from the detection state to the detected state, wherein the expected state is the state in which the storage type of multiple storage objects is the first storage type, and the detected state indicates that multiple storage objects have been detected to be in the expected state; changing the status of the conversion task of the target inode from the pending conversion state to the conversion successful state, wherein the conversion successful state indicates that the storage type of the target file system object has been converted from the second storage type to the first storage type.

[0184] For example, at least one round of state checks is performed on multiple storage objects. In any round of state checks, if at least some of the storage objects are not in the expected state, the next round of state checks is performed. This continues until all storage objects are in the expected state, at which point the state of the target index node is changed from "under detection" to "detected". Simultaneously with changing the state of the target index node from "under detection" to "detected", the state of the target index node's transformation task is changed from "pending transformation" to "transformation successful".

[0185] This application embodiment, by executing a third conversion instruction, can flexibly convert the storage type from a second storage type to a first storage type to adapt to changes in data access patterns. The conversion process from the second storage type to the first storage type is clearly defined, including conversion processing in both the first and second storage spaces, which helps ensure the correctness and consistency of the conversion process. Data storage types can be intelligently managed according to preset rules and real-time data access conditions, improving the automation and intelligence level of storage management.

[0186] In some embodiments, the target file system object includes a first lifetime, which indicates the maximum duration for which the target file system object's storage type is in the second storage type; the level of the second storage type and the level of the first storage type are both positively correlated with the read frequency of the target file system object. If the level of the second storage type is higher than the level of the first storage type, step 104 can be implemented by performing the following process: in response to the target file system object's storage type being in the first storage type for a duration equal to the first lifetime, the storage type of the target file system object is changed from the second storage type to the first storage type.

[0187] For example, if the first storage type is archive and the second storage type is low frequency, and the first lifecycle of the target file system object is 5 days, that is, the maximum duration of the target file system object's storage type being low frequency is 5 days, then when the duration of the target file system object's storage type being low frequency is equal to 5 days, the storage type of the target file system object will be changed from low frequency to archive.

[0188] In some embodiments, where the level of the second storage type is higher than that of the first storage type, the process of converting the storage type of the target file system object from the first storage type to the second storage type through steps 101 to 103 is called a warm-up process. The process of "converting the storage type of the target file system object from the second storage type to the first storage type" in step 104 is called a cooling-down process after warm-up. The warm-up process only generates a copy of the target file system object. After the duration of the second storage type reaches its lifecycle, it will be degraded back to its original state (archived or deep archive), i.e., the first storage type. When the storage type of the target file system object is in the second storage state, it is determined whether cooling is required. If so, the state of the target inode is changed from the detected state to the initial state, and the task-driven process ends.

[0189] The automated conversion process of the storage type of the target file system object based on the first lifecycle of this application can reduce manual intervention and improve the automation level of storage management. Storage strategies can be flexibly adjusted according to the actual usage of the target file system object to meet the storage needs of different data.

[0190] In some embodiments, when performing step 101, the following processing may be performed: receiving a second conversion instruction, wherein the second conversion instruction is used to instruct the conversion of the storage type of at least one candidate file system object, the candidate file system object being a file system object in the second storage space other than the target file system object, and the storage type of the target file system object being converted together with the storage type of the candidate file system object. When performing step 104, the following processing may be performed: converting the storage type of the candidate file system object from a second storage type to a first storage type.

[0191] For example, the storage type of the target file system object and the storage type of the candidate file system object can be converted simultaneously or sequentially. See also Figure 10 , Figure 10 This is a schematic diagram illustrating the storage type conversion of multiple file system objects provided in this application embodiment. Figure 10The structure of a directory tree is shown. If the target file system object corresponds to the target index node 914, there are multiple candidate file system objects. Taking the candidate file system objects as an example, including candidate file system object A and candidate file system object B, candidate file system object A corresponds to candidate index node 912, and candidate file system object B corresponds to candidate index node 913.

[0192] In some embodiments, the first storage space includes a directory tree, which includes multiple inodes. Different inodes correspond to different file system objects. Inodes are used to store metadata of the file system objects. Multiple inodes constitute multiple paths, and the target file system object and candidate file system objects are converted in a specific order. See also Figure 3I , Figure 3I This is a ninth flowchart illustrating the object-type data processing method provided in this application embodiment. A specific sequence is achieved through execution... Figure 3I The steps 701 to 703 are determined as follows, and are explained in detail below.

[0193] In step 701, the target path of the target inode corresponding to the target file system object is queried in the directory tree, and the candidate path of the candidate inode corresponding to the candidate file system object is queried.

[0194] Following the example above, see below. Figure 10 The target path of the target inode 914 corresponding to the target file system object is " / user / local / service", and the candidate path 1 of the candidate inode 912 corresponding to the candidate file system object A is " / user / local".

[0195] In step 702, the path lengths of the multiple paths are sorted in descending order to obtain the sorting result.

[0196] Here, if at least two paths among multiple paths have the same path length, then these at least two paths will be in the same order in the sorting results. For example, if the target path has the same path length as candidate path 1, then they will be in the same order in the sorting results, for example, both being 1.

[0197] Following the example of step 701 above, the path lengths of the target path and candidate path 1 are sorted in descending order to obtain the sorting result, which is: path length of the target path, path length of candidate path 1. The path length of the target path is ranked 1 in the sorting result, and since the path length of candidate path 1 is less than the path length of the target path, the path length of candidate path 1 is ranked 2 in the sorting result.

[0198] In step 703, the target path length of the target path and the candidate path length of the candidate paths are sorted in descending order in the sorting results to form a specific order.

[0199] Following the example of step 702 above, the target path is in the sorting order 1, and the candidate path 1 is in the sorting order 2. Therefore, the specific order of conversion is target file system object first, and candidate file system object A second.

[0200] This application embodiment prioritizes the conversion of deeper file system objects by sorting path lengths in descending order. Converting in a specific order avoids unnecessary search and sorting operations, improving conversion efficiency. Automated conversion processes simplify storage type conversion and reduce manual intervention. Specific sequential conversion strategies can be adjusted according to different business needs and storage strategies to adapt to different scenarios and priorities.

[0201] In some embodiments, see Figure 3J , Figure 3J This is a schematic diagram of the tenth process of the object-type data processing method provided in the embodiments of this application. The specific order can also be achieved by executing... Figure 3J The steps 801 to 803 are determined as follows, and are explained in detail below.

[0202] In step 801, the target path of the target inode corresponding to the target file system object is queried in the directory tree, and the candidate path of the candidate inode corresponding to the candidate file system object is queried.

[0203] Following the example above, see below. Figure 10 The target path of the target inode 914 corresponding to the target file system object is " / user / local / service", and the candidate path 2 of the candidate inode 913 corresponding to the candidate file system object B is " / user / local / workspace".

[0204] In step 802, in response to the target path length of the target path being the same as the candidate path length of the candidate path, multiple first timestamps in the target file system object are compared with the first lifecycle, and multiple second timestamps in the candidate file system object are compared with the second lifecycle corresponding to the candidate file system object.

[0205] In some embodiments, the multiple first timestamps and multiple second timestamps each include the file's most recently accessed time (Atime), the file's most recently modified content time (Mtime), and the file's most recently modified permissions and attributes time (Ctime). The first lifetime refers to the maximum duration for which the target file system object's storage type is in the second storage type, and the second lifetime refers to the maximum duration for which the candidate file system object's storage type is in the second storage type.

[0206] Following the example of step 801 above, the path length of the target path is the same as the path length of candidate path 2. Multiple first timestamps in the target file system object are compared with the first lifecycle, that is, the file's most recently accessed time, most recently modified content time, and most recently modified permissions and attributes time are all compared with the first lifecycle. Multiple second timestamps in candidate file system object B are compared with the corresponding second lifecycle of the candidate file system object, that is, the file's most recently accessed time, most recently modified content time, and most recently modified permissions and attributes time are all compared with the second lifecycle.

[0207] In step 803, if multiple first timestamps are greater than the first lifetime and at least one second timestamp is less than the second lifetime, a specific order indicating that the target file system object comes first and the candidate file system object comes last is generated.

[0208] Following the example of step 802 above, if the most recent access time, the most recent content modification time, and the most recent permission and attribute modification time of the target file system object are all greater than the lifecycle, and at least one of the most recent access time, the most recent content modification time, and the most recent permission and attribute modification time of the candidate file system object B is less than the second lifecycle, then the specific conversion order is target file system object first, and candidate file system object B second.

[0209] In some embodiments, if multiple first timestamps are greater than the first lifetime and multiple second timestamps are greater than the second lifetime, then the first lifetime of the target file system object is compared with the second lifetime of the candidate file system object; if the first lifetime of the target file system object is less than the second lifetime of the candidate file system object, then a specific order indicating that the target file system object comes first and the candidate file system object comes last is generated.

[0210] For example, if the most recent access time, most recent content modification time, and most recent permission and attribute modification time of the target file system object are all greater than its lifespan, and the most recent access time, most recent content modification time, and most recent permission and attribute modification time of the candidate file system object B are all greater than its second lifespan, then the first lifespan of the target file system object is compared with the second lifespan of the candidate file system object B. If the first lifespan is 10 days and the second lifespan is 15 days, then the specific conversion order is target file system object first, and candidate file system object B second.

[0211] This application's embodiments determine conversion priorities by matching target path lengths and candidate path lengths, ensuring that important file system objects are converted first. By comparing multiple first timestamps and first lifetimes, and comparing multiple second timestamps and second lifetimes, the timing of storage type conversions can be precisely controlled, ensuring that the conversion timing meets preset conditions. Utilizing the structured information of directory trees and inodes, file system objects can be managed and converted more efficiently, and structured data management makes the system easier to maintain and monitor.

[0212] As an example of the above processing procedure, see Figure 11 , Figure 11This is a detailed process diagram of accessing the first storage space provided in the embodiments of this application. Specifically, to obtain the first metadata request, the first metadata request is first sent to the data management layer (RootServer). The data management layer finds a directory tree corresponding to the first metadata request from the directory trees maintained by multiple metadata storage master services (MDS Master) and multiple metadata storage slave services (MDS Slave) in the first storage space, and sends the storage location of the directory tree to the terminal (e.g., the terminal 400 mentioned above). The data management layer is used to provide management and control interfaces to the outside world, including creating and deleting file systems, creating and deleting mount points, and configuring system attributes, etc. At the same time, it also needs to evenly distribute the file system on different metadata services and elect them through a distributed algorithm (Raft) to achieve high availability and master-slave disaster recovery. Furthermore, the storage location is provided to the data access interface (proxy) via a metadata stream. The data access interface then retrieves the corresponding metadata in the first storage space based on the storage location and sends it to the terminal (e.g., terminal 400 mentioned above). The databases storing various metadata in the first storage space can form a database cluster, including various types of data such as Cloud Database (CDB), Distributed Database (TDSQL), and Cloud-native Database (CynosDB). The data access interface provides external metadata interfaces, including creating and deleting file handles and updating metadata. First metadata requests directly access the data access interface, which performs protocol conversion and forwards the requests to the metadata service where the file system resides. Additionally, when a user wants to retrieve file data from the second storage space, they can directly send a data retrieval request to the second storage space. The data retrieval request is then transmitted to the second storage space via a data stream to obtain the corresponding file data.

[0213] In this embodiment, upon receiving a metadata access request, the specific location of the corresponding target data can be queried based on the architecture index of the file storage architecture for each directory tree in the first storage space, achieving accurate and rapid location of the directory trees in the first storage space. Furthermore, a preset data access interface is used to perform protocol conversion and forwarding based on the first metadata request, thereby enabling rapid retrieval of metadata in the first storage space and improving the convenience of metadata acquisition.

[0214] The following description continues to illustrate the exemplary structure of the object-type data processing device 233 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules in the object-type data processing device 233 stored in the memory 233 may include:

[0215] The instruction receiving module 2331 is used to receive a first conversion instruction, wherein the first conversion instruction is used to instruct the storage type conversion of the target file system object.

[0216] The first conversion module 2332 is used to convert the storage type of multiple storage objects in the first storage space from a first storage type to a second storage type, wherein the multiple storage objects are used to distribute the metadata of the target file system object.

[0217] The second conversion module 2333 is used to convert the storage type of the target file system object in the second storage space from the first storage type to the second storage type.

[0218] The third conversion module 2334 is used to convert the storage type of the target file system object from the second storage type to the first storage type in response to the target file system object meeting the storage type conversion conditions.

[0219] In some embodiments, the instruction receiving module 2331 is further configured to divide the target file system object into multiple data blocks, wherein the multiple data blocks are used for distributed storage of the contents of the target file system object; and for each data block, the metadata of the data block is stored in a storage object.

[0220] In some embodiments, the instruction receiving module 2331 is further configured to receive a first metadata request based on an object storage protocol through a protocol conversion interface; convert the first metadata request based on the object storage protocol into a first metadata request based on a portable operating system interface semantics; and obtain the metadata of a data block from a first storage space through a data access interface according to the first metadata request based on the portable operating system interface semantics.

[0221] In some embodiments, the instruction receiving module 2331 is further configured to, for any data block, if the data block is truncated at least once in order from head to tail to obtain multiple sub-data blocks; supplement the last sub-data block with a candidate data block; and store the metadata of the last sub-data block and the metadata of the candidate data block together in a new storage object.

[0222] In some embodiments, the instruction receiving module 2331 is further configured to receive a second metadata request via a data management interface through a public gateway interface, wherein the second metadata request is used to indicate the acquisition of metadata of candidate data blocks; and to acquire the metadata of candidate data blocks from the first storage space according to the second metadata request.

[0223] In some embodiments, the first storage space includes a directory tree, which includes multiple index nodes. The first conversion module 2332 is further configured to query the target index node corresponding to the target file system object from the directory tree, and set the status of the conversion task of the target index node to a binding state, wherein the binding state is the state in which the target index node and the conversion task are being bound. The target index node includes the metadata of the target file system object, and the conversion task is used to convert from a first storage type to a second storage type. The level of the first storage type of the target file system object is detected. If the level of the first storage type of the target file system object is a preset level, the status of the conversion task of the target index node is changed from the binding state to the binding successful state, and the status of the target index node is set to a pending conversion state, wherein the level is positively correlated with the reading frequency of the target file system object, and the pending conversion state indicates that the storage type of multiple storage objects in the first storage space is to be converted from the first storage type to the second storage type.

[0224] In some embodiments, the first conversion module 2332 is further configured to convert the storage type of multiple storage objects in the first storage space from the first storage type to the second storage type when the state of the target index node is the state to be converted, and update the type value in the label of each converted storage object to the second storage type.

[0225] In some embodiments, the second conversion module 2333 is further configured to, in response to the fact that the type values ​​of the tags of multiple storage objects are all of the second storage type, convert the state of the target inode to the detection state, and convert the storage type of the target file system object corresponding to the target inode from the first storage type to the second storage type, wherein the detection state is the state in which the storage object is being detected.

[0226] In some embodiments, the second conversion module 2333 is further configured to change the state of the conversion task of the target inode from the binding successful state to the conversion in progress state, wherein the conversion in progress state is the state in which the storage type of the target file system object is being converted.

[0227] In some embodiments, the second conversion module 2333 is further configured to perform state detection on multiple storage objects respectively; if multiple storage objects are all in the expected state, the state of the target index node is converted from the detection state to the detected state, wherein the expected state is the state in which the storage type of multiple storage objects is the second storage type, and the detected state indicates that multiple storage objects have been detected to be in the expected state; the state of the conversion task of the target index node is converted from the pending conversion state to the conversion successful state, wherein the conversion successful state indicates that the storage type of the target file system object has been converted from the first storage type to the second storage type.

[0228] In some embodiments, the level of the second storage type and the level of the first storage type are both positively correlated with the read frequency of the target file system object. The third conversion module 2334 is further configured to, in response to receiving a third conversion instruction, convert the storage type of the target file system object from the second storage type to the first storage type when the level of the second storage type is lower than that of the first storage type. The third conversion instruction is used to instruct the conversion of the storage type of the target file system object.

[0229] In some embodiments, the third conversion module 2334 is further configured to convert the storage type of multiple storage objects in the first storage space from the second storage type to the first storage type; and to convert the storage type of the target file system object in the second storage space from the second storage type to the first storage type.

[0230] In some embodiments, the target file system object includes a first lifetime, which is used to indicate the maximum duration for which the target file system object's storage type is in the second storage type; the level of the second storage type and the level of the first storage type are both positively correlated with the read frequency of the target file system object; the third conversion module 2334 is further used to convert the storage type of the target file system object from the second storage type to the first storage type when the level of the second storage type is higher than the level of the first storage type, in response to the duration for which the target file system object's storage type is in the first storage type being equal to the first lifetime.

[0231] In some embodiments, the instruction receiving module 2331 is further configured to receive a second conversion instruction, wherein the second conversion instruction is configured to indicate a conversion of the storage type of at least one candidate file system object, the candidate file system object being a file system object in the second storage space other than the target file system object, and the storage type of the target file system object being converted together with the storage type of the candidate file system object.

[0232] In some embodiments, the third conversion module 2334 is further configured to convert the storage type of the candidate file system object from the second storage type to the first storage type.

[0233] In some embodiments, the first storage space includes a directory tree, which includes multiple index nodes. Different index nodes correspond to different file system objects. The index nodes are used to store the metadata of the file system objects. Multiple index nodes constitute multiple paths. The target file system object and the candidate file system object are converted in a specific order. The third conversion module 2334 is also used to determine the specific order by: querying the target path where the target index node corresponding to the target file system object is located in the directory tree, and querying the candidate path where the candidate index node corresponding to the candidate file system object is located; sorting the path lengths of the multiple paths in descending order to obtain a sorting result; and sorting the target path length of the target path and the candidate path length of the candidate path in the sorting result in descending order to form a specific order.

[0234] In some embodiments, the third conversion module 2334 is further configured to determine the specific order by: querying the target path where the target inode corresponding to the target file system object is located in the directory tree, and querying the candidate path where the candidate inode corresponding to the candidate file system object is located; in response to the target path length of the target path being the same as the candidate path length of the candidate path, comparing multiple first timestamps in the target file system object with the first lifecycle, and comparing multiple second timestamps in the candidate file system object with the second lifecycle corresponding to the candidate file system object; if multiple first timestamps are all greater than the first lifecycle, and at least one second timestamp is less than the second lifecycle, then generating a specific order indicating that the target file system object comes first and the candidate file system object comes last.

[0235] This application provides a computer program product comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the object-type data processing method described in this application.

[0236] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the object-type data processing method provided in this application, for example... Figure 3A The method for processing object-type data is shown.

[0237] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0238] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0239] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0240] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0241] In summary, upon receiving the first conversion instruction, this embodiment first converts the storage type of the metadata of the target file system object in the first storage space from the first storage type to the second storage type. Then, it converts the storage type of the target file system object in the second storage space from the first storage type to the second storage type, thus realizing the conversion of the file system object's storage type. This tightly associates the storage type of the metadata with the storage type of the file system object, ensuring that the storage type of the file system object's metadata is always consistent with the storage type of the file system object. This avoids the extra time consumption caused by modifying the storage type to be consistent when the storage types are inconsistent in different storage spaces. When the target file system object of the second storage type meets the storage type conversion conditions, the storage type of the target file system object is converted back to the first storage type, forming a closed loop of data storage types. This allows for flexible conversion between the first and second storage types, ensuring that data in different storage spaces has a consistent storage type during any storage type conversion process, thereby improving data reliability.

[0242] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for processing object-type data, characterized in that, The method includes: Receive a first conversion instruction, wherein the first conversion instruction is used to instruct the storage type conversion of the target file system object; The storage type of multiple storage objects in the first storage space is changed from the first storage type to the second storage type, wherein the multiple storage objects are used to distribute the metadata of the target file system object; Convert the storage type of the target file system object in the second storage space from the first storage type to the second storage type; In response to the target file system object of the second storage type satisfying the storage type conversion condition, the storage type of the target file system object is converted from the second storage type to the first storage type.

2. The method according to claim 1, characterized in that, The target file system object includes a first lifecycle, which indicates the maximum duration for which the target file system object's storage type is in the second storage type. Both the level of the second storage type and the level of the first storage type are positively correlated with the read frequency of the target file system object. When the level of the second storage type is higher than the level of the first storage type, the target file system object, in response to the second storage type, satisfies the storage type conversion condition, and the storage type of the target file system object is converted from the second storage type to the first storage type, including: In response to the target file system object being stored in the first storage type for a duration equal to the first lifecycle, the storage type of the target file system object is changed from the second storage type to the first storage type.

3. The method according to claim 2, characterized in that, When receiving the first instruction, the method further includes: Receive a second conversion instruction, wherein the second conversion instruction is used to instruct the storage type of at least one candidate file system object to be converted, the candidate file system object being a file system object in the second storage space other than the target file system object, and the storage type of the target file system object being converted together with the storage type of the candidate file system object; When converting the storage type of the target file system object from the second storage type to the first storage type, the method further includes: Convert the storage type of the candidate file system object from the second storage type to the first storage type.

4. The method according to claim 3, characterized in that, The first storage space includes a directory tree, which includes multiple index nodes. Different index nodes correspond to different file system objects. The index nodes are used to store the metadata of the file system objects. Multiple index nodes constitute multiple paths. The target file system object and the candidate file system object are converted in a specific order. The specific order is determined by the following method: In the directory tree, query the target path where the target index node corresponding to the target file system object is located, and query the candidate path where the candidate index node corresponding to the candidate file system object is located; Sort the path lengths of the multiple paths in descending order to obtain the sorting result; The target path length of the target path and the candidate path length of the candidate paths are sorted in descending order in the sorting results to form the specific order.

5. The method according to claim 3, characterized in that, The first storage space includes a directory tree, which includes multiple index nodes. Different index nodes correspond to different file system objects. The index nodes are used to store the metadata of the file system objects. Multiple index nodes constitute multiple paths. The target file system object and the candidate file system object are converted in a specific order. The specific order is determined by the following method: In the directory tree, query the target path where the target index node corresponding to the target file system object is located, and query the candidate path where the candidate index node corresponding to the candidate file system object is located; In response to the fact that the target path length of the target path is the same as the candidate path length of the candidate path, multiple first timestamps in the target file system object are compared with the first lifecycle, and multiple second timestamps in the candidate file system object are compared with the second lifecycle corresponding to the candidate file system object. If all of the first timestamps are greater than the first lifecycle, and at least one of the second timestamps is less than the second lifecycle, then a specific order is generated indicating that the target file system object comes first and the candidate file system object comes last.

6. The method according to claim 1, characterized in that, Both the level of the second storage type and the level of the first storage type are positively correlated with the read frequency of the target file system object. When the level of the second storage type is lower than that of the first storage type, the target file system object, in response to the second storage type, meets the storage type conversion condition, and the storage type of the target file system object is converted from the second storage type to the first storage type, including: In response to receiving a third conversion instruction, the storage type of the target file system object is converted from the second storage type to the first storage type, wherein the third conversion instruction is used to instruct the conversion of the storage type of the target file system object.

7. The method according to claim 6, characterized in that, The step of converting the storage type of the target file system object from the second storage type to the first storage type includes: Convert the storage type of the plurality of storage objects in the first storage space from the second storage type to the first storage type; Convert the storage type of the target file system object in the second storage space from the second storage type to the first storage type.

8. The method according to any one of claims 1 to 7, characterized in that, The first storage space includes a directory tree, which includes multiple index nodes. Before converting the storage type of multiple storage objects in the first storage space from a first storage type to a second storage type, the method further includes: The target index node corresponding to the target file system object is queried from the directory tree, and the status of the conversion task of the target index node is set to the binding state. The binding state is the state in which the target index node is being bound to the conversion task. The target index node includes the metadata of the target file system object. The conversion task is used to convert from the first storage type to the second storage type. The level of the first storage type of the target file system object is detected; If the level of the first storage type of the target file system object is a preset level, then the status of the conversion task of the target inode is changed from the binding state to the binding successful state, and the status of the target inode is set to the conversion pending state. The level is positively correlated with the reading frequency of the target file system object, and the conversion pending state indicates that the storage type of the plurality of storage objects in the first storage space is to be converted from the first storage type to the second storage type.

9. The method according to claim 8, characterized in that, The step of converting the storage type of multiple storage objects in the first storage space from the first storage type to the second storage type includes: When the state of the target index node is the state to be converted, the storage type of the plurality of storage objects in the first storage space is converted from the first storage type to the second storage type, and the type value in the tag of each of the converted storage objects is updated to the second storage type.

10. The method according to claim 9, characterized in that, The step of converting the storage type of the target file system object in the second storage space from the first storage type to the second storage type includes: In response to the fact that the type value of the tags of the plurality of storage objects are all the second storage type, the state of the target index node is changed to the detection state, and the storage type of the target file system object corresponding to the target index node is changed from the first storage type to the second storage type, wherein the detection state is the state in which the storage object is being detected.

11. The method according to claim 8, characterized in that, When converting the storage type of the target file system object in the second storage space from the first storage type to the second storage type, the method further includes: The status of the conversion task of the target inode is changed from the binding successful status to the conversion in progress status, wherein the conversion in progress status is the status of converting the storage type of the target file system object.

12. The method according to claim 10 or 11, characterized in that, After converting the storage type of the target file system object in the second storage space from the first storage type to the second storage type, the method further includes: Perform state detection on each of the multiple storage objects; If all the multiple storage objects are in the expected state, the state of the target index node is changed from the detected state to the detected state. The expected state is when the storage type of all the multiple storage objects is the second storage type, and the detected state indicates that all the multiple storage objects have been detected to be in the expected state. The conversion task of the target inode is changed from the pending conversion state to the conversion successful state, wherein the conversion successful state indicates that the storage type of the target file system object has been changed from the first storage type to the second storage type.

13. The method according to any one of claims 1 to 12, characterized in that, Prior to receiving the first conversion instruction, the method further includes: The target file system object is divided into multiple data blocks, wherein the multiple data blocks are used for distributed storage of the content of the target file system object; For each data block, the metadata of the data block is stored in a storage object.

14. The method according to claim 13, characterized in that, The method further includes: For any of the data blocks, if the data block is truncated at least once in order from head to tail, multiple sub-data blocks are obtained; After the last sub-data block, a candidate data block is added, and the metadata of the last sub-data block and the metadata of the candidate data block are stored together in a new storage object.

15. The method according to claim 14, characterized in that, Before storing the metadata of the last sub-data block and the metadata of the candidate data block together in a new storage object, the method further includes: Through the public gateway interface and via the data management interface, a first metadata request is received, wherein the first metadata request is used to indicate the acquisition of metadata of the candidate data block; Based on the first metadata request, the metadata of the candidate data block is obtained from the first storage space.

16. The method according to claim 13 or 14, characterized in that, Before storing the metadata of the data block in one of the storage objects, the method further includes: Receive second metadata requests based on the object storage protocol through the protocol conversion interface; The second metadata request based on the object storage protocol is converted into a second metadata request based on the semantics of a portable operating system interface. Based on the second metadata request based on the semantics of the portable operating system interface, the metadata of the data block is obtained from the first storage space through the data access interface.

17. A processing apparatus for object-type data, characterized in that, The device includes: The instruction receiving module is used to receive a first conversion instruction, wherein the first conversion instruction is used to instruct the storage type conversion of the target file system object; The first conversion module is used to convert the storage type of multiple storage objects in the first storage space from a first storage type to a second storage type, wherein the multiple storage objects are used to distribute the metadata of the target file system object; The second conversion module is used to convert the storage type of the target file system object in the second storage space from the first storage type to the second storage type; The third conversion module is used to convert the storage type of the target file system object from the second storage type to the first storage type in response to the second storage type target file system object meeting the storage type conversion condition.

18. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the object-type data processing method according to any one of claims 1 to 16.

19. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the method for processing object-type data as described in any one of claims 1 to 16 is implemented.

20. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the method for processing object-type data as described in any one of claims 1 to 16 is implemented.

Images