Data processing method, system, electronic device and computer-readable storage medium

Abstract

This application discloses a data processing method, system, electronic device, and computer-readable storage medium, relating to the field of distributed storage. The data processing method includes: upon receiving an upload object request sent by a client, creating a current version file corresponding to the upload object request; recording metadata information corresponding to the current version file in a logical header file corresponding to the upload object request using extended attributes supported by the file system; and displaying the logical header file so that the client can process any version of the file based on the metadata information. This application can save multiple versions of metadata information, enabling the distributed S3-NAS converged interoperable storage system to support the multi-version function of the S3 protocol, thereby resolving the problem of users being unable to recover data after accidentally deleting it, enriching the functionality of distributed converged interoperable storage, and improving the market competitiveness of distributed storage systems.

Description

Technical Field

[0001] The present application relates to the field of distributed storage, and in particular to a data processing method, system, electronic device, and computer-readable storage medium. Background Art

[0002] The current market has a demand for interoperability between the S3 (Simple Storage Service) protocol and the NAS (Network Attached Storage) protocol, that is, files written through NAS can be subsequently operated through the S3 protocol, and data written through S3 can also be subsequently operated through NAS. The current distributed storage system already supports the integration and interoperability of the S3 protocol and the NAS protocol. There are mainly two implementation solutions: Solution 1, refer to Figure 1 As shown, with S3 object storage as the base, deploy NFS Ganesha gateway to realize the conversion of NFS file protocol to S3 protocol; Solution 2, refer to Figure 2 As shown in the figure, the distributed file system is used as the base to achieve mutual access and communication between S3 and NAS.

[0003] The S3 protocol features multi-versioning, which generates a random version number when an object is uploaded. It then processes uploaded files with the same name based on the file's version number and file name, saving multiple versions to maintain the latest version of the object. It also allows downloading any historical version of the object from the bucket. However, the current distributed S3-NAS converged interoperable storage system lacks an index pool, and there's no way to record multi-version metadata. As a result, the distributed S3-NAS converged interoperable storage system doesn't support S3's multi-versioning feature, making it impossible to recover data if users accidentally delete it.

[0004] Therefore, how to provide a solution to the above technical problems is a problem that those skilled in the art need to solve at present. Summary of the Invention

[0005] The purpose of this application is to provide a data processing method, system, electronic device, and computer-readable storage medium, so that a distributed S3-NAS fusion interoperable storage system supports the multi-version function of the S3 protocol, thereby solving the problem of users being unable to recover data after accidentally deleting it, enriching the functions of distributed fusion interoperable storage, and improving the market competitiveness of distributed storage systems.

[0006] To solve the above technical problems, this application provides a data processing method, including:

[0007] When receiving an object upload request sent by a client, creating a current version file corresponding to the object upload request;

[0008] Recording metadata information corresponding to the current version file in a logical header file corresponding to the object upload request as an extended attribute supported by the file system;

[0009] The logical header file is displayed so that the client can process any version file based on the metadata information.

[0010] Optionally, after recording the metadata information corresponding to the current version file in an extended attribute supported by the file system into a logical header file corresponding to the object upload request, the data processing method further includes:

[0011] When receiving a download object request sent by the client, determining target metadata information corresponding to the download object request in the logical header file;

[0012] Determine the target version file according to the target metadata information;

[0013] Read the data in the target version file and send the data to the client.

[0014] Optionally, the process of determining target metadata information corresponding to the download object request in the logical header file includes:

[0015] Determine whether a download version number is specified in the download object request;

[0016] If so, determining the metadata information corresponding to the downloaded version number in the logical header file as the target metadata information;

[0017] If not, the latest metadata information in the logical header file is determined as the target metadata information.

[0018] Optionally, after recording the metadata information corresponding to the current version file in an extended attribute supported by the file system into a logical header file corresponding to the object upload request, the data processing method further includes:

[0019] When receiving a delete object request sent by the client, determining whether a deletion version number is specified in the delete object request;

[0020] If so, obtaining target metadata information corresponding to the deletion version number in the logical header file, determining a target version file based on the target metadata information, and deleting the target version file;

[0021] If not, a new version number is generated, and metadata information corresponding to the version number is recorded in the logical header file using an extended attribute supported by the file system, wherein the metadata information includes an identifier of the deleted object.

[0022] Optionally, after deleting the target version file, the data processing method further includes:

[0023] The metadata information corresponding to the target version file is deleted from the logical header file.

[0024] Optionally, upon receiving an object upload request sent by a client, the process of creating a current version file corresponding to the object upload request includes:

[0025] When receiving an upload object request sent by a client, generating a version number corresponding to the upload object request;

[0026] A current version file is created, where the name of the current version file is determined based on the version number and the object name corresponding to the object upload request, and the name of each version file is unique.

[0027] Optionally, the process of recording metadata information corresponding to the current version file in a logical header file corresponding to the object upload request as an extended attribute supported by the file system includes:

[0028] The metadata information corresponding to the current version file is recorded in the logical header file in the form of xattr.

[0029] To solve the above technical problems, the present application also provides a data processing system, comprising:

[0030] A creation module, configured to, upon receiving an object upload request sent by a client, create a current version file corresponding to the object upload request;

[0031] A first recording module is configured to record metadata information corresponding to the current version file in a logical header file corresponding to the object upload request using an extended attribute supported by the file system;

[0032] The management module is used to display the logical header file so that the client can process any version file based on the metadata information.

[0033] To solve the above technical problems, the present application further provides an electronic device, comprising:

[0034] memory for storing computer programs;

[0035] A processor is configured to implement the steps of any one of the data processing methods described above when executing the computer program.

[0036] To solve the above technical problems, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data processing method described in any one of the above items are implemented.

[0037] The present application provides a data processing method. When receiving an upload object request sent by a client, the present application first creates and stores a current version file corresponding to the upload object request. The present application then records the metadata information corresponding to the current version file created by the upload object request in a logical header file as an extended attribute supported by the file system and displays it. This allows for the preservation of multiple versions of metadata information, enabling the distributed S3-NAS fusion interoperable storage system to support the multi-version functionality of the S3 protocol. This solves the problem of users being unable to recover data after accidentally deleting it, enriches the functionality of distributed fusion interoperable storage, and improves the market competitiveness of the distributed storage system. The present application also provides a data processing system, electronic device, and computer-readable storage medium, which have the same beneficial effects as the above-mentioned data processing method. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 A schematic diagram of an implementation scheme for a distributed storage system supporting the integration and interoperability of the S3 protocol and NAS;

[0040] Figure 2 A schematic diagram of another distributed storage system supporting the integration and interoperability of the S3 protocol and NAS.

[0041] Figure 3 A schematic diagram of common file upload and download in a distributed S3-NAS fusion interoperable storage system provided in this application;

[0042] Figure 4 A flowchart of the steps of a data processing method provided in this application;

[0043] Figure 5 A schematic diagram of recording data in xattr format provided by this application;

[0044] Figure 6 A flowchart of the steps of a multi-version uploading method provided by this application;

[0045] Figure 7 A flowchart of the steps of a multi-version download method provided by this application;

[0046] Figure 8 A flowchart of the steps of a multi-version deletion method provided by this application;

[0047] Figure 9 This is a structural diagram of a data processing system provided by this application. DETAILED DESCRIPTION

[0048] The core of this application is to provide a data processing method, system, electronic device and computer-readable storage medium, so that the distributed S3-NAS fusion interoperable storage system supports the multi-version function of the S3 protocol, thereby solving the problem of users being unable to recover data after accidentally deleting it, enriching the functions of distributed fusion interoperable storage, and improving the market competitiveness of the distributed storage system.

[0049] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0050] To facilitate understanding of the solution of this application, the following describes a distributed object storage system, the S3 protocol, the NAS protocol, the S3-NAS fusion interoperable storage system, and the multi-version function. A distributed object storage system refers to a distributed storage system for unstructured data objects. Currently, two object storage protocols, S3 and SWIFT, are used in this field.

[0051] S3 is an object storage service that provides a RESTful API interface for interacting with the service;

[0052] NAS is a technology that integrates distributed, independent data into a large, centrally managed data center to facilitate access to different hosts and application servers. NAS is a device connected to the network that has data storage capabilities, hence the name "network storage," and is a dedicated data storage server. With data as the center, storage devices are completely separated from servers, allowing for centralized data management. NAS itself supports multiple protocols (such as NFS, CIFS, FTP, HTTP, etc.) and various operating systems. NAS devices can be intuitively and conveniently managed using a browser from any workstation, enabling NAS file sharing.

[0053] S3-NAS converged interoperable storage system, a distributed storage system that integrates and interoperates the S3 protocol and the NAS protocol, supports multiple protocols (CIFS / NFS / S3) to simultaneously share a copy of data and access and communicate with each other. That is, files written through NAS can be subsequently operated through the S3 protocol, and data written through S3 can also be subsequently operated through NAS.

[0054] The multi-version feature is designed to address scenarios where objects with the same name may be overwritten in distributed object storage, or objects can be restored after being accidentally deleted. This feature generates a random version number when uploading an object, and processes uploaded files with the same name based on the file's version number and file name. This feature allows for the storage of multiple versions, ensuring the latest version of the object is present. Historical versions of the object at any point in time can be downloaded from the bucket.

[0055] Currently, the distributed S3-NAS fusion interoperable storage system only supports the upload and download of common files. After the common files are uploaded through the S3 object storage protocol, they will be converted into the file storage method of the file storage system and saved in the storage pool. The metadata is managed uniformly by MDS. When downloading objects through the S3 object storage system, they must be converted into requests for obtaining file data in the file storage system and the read data will be returned to the client. Figure 3 However, the current distributed S3-NAS fusion interoperable storage system does not support the original S3 multi-version function.

[0056] As mentioned above, the current distributed storage system supports the S3 protocol and NAS integration and interoperability mainly as follows: Figure 1 and Figure 2 The two solutions shown take into account Figure 2 The solution shown can avoid the loss of NAS related semantics and better realize multi-protocol intercommunication. This application is based on Figure 2 The solution shown proposes a distributed S3-NAS fusion intercommunication storage system that supports multi-version functions. The following describes a data processing solution provided by this application and applied to a distributed S3-NAS fusion intercommunication storage system.

[0057] First, please refer to Figure 4 , Figure 4 This is a flowchart of the steps of a data processing method provided in this application, which includes:

[0058] S101: upon receiving an object upload request sent by a client, creating a current version file corresponding to the object upload request;

[0059] As an optional embodiment, when receiving an object upload request sent by a client, the process of creating a current version file corresponding to the object upload request includes:

[0060] When receiving an upload object request sent by the client, generate a version number corresponding to the upload object request;

[0061] Create a current version file. The name of the current version file is determined based on the version number and the object name corresponding to the upload object request. Each version file has a unique name.

[0062] Specifically, when a client sends an object upload request, specifically an S3 client, and the multi-version feature is enabled, the S3FS gateway generates a version number corresponding to the current object upload request and creates a version file containing the object name and version number. When the file is written to disk, the current version number and object name are used as the file name, specifically in the format of bucket_ino + object name + version number. It is understood that the version file records specific version data and metadata.

[0063] S102: Record metadata information corresponding to the current version file in the logical header file corresponding to the object upload request as an extended attribute supported by the file system;

[0064] S103: Displaying the logical header file so that the client can process any version of the file based on the metadata information.

[0065] Specifically, a determination is made as to whether a logical header file corresponding to the client's upload request exists. If not, a logical header file is created for external presentation. If so, metadata information corresponding to the current version of the file is recorded in the logical header file. It is understood that the file name corresponding to the client's upload request, uploaded by the user, is referred to as the logical header file, which only records metadata information corresponding to each version of the file.

[0066] After creating the current version file, it is hidden from the public, presenting only a logical header file with the same name as the uploaded file. After creating the logical header file, the metadata corresponding to the current version file is recorded in the logical header file as extended attributes supported by the file system. Specifically, this metadata is recorded in the form of xarrt metadata. The metadata corresponding to each version file is sorted using a default scheme, allowing for easy identification of the actual data storage location during subsequent downloads and file access.

[0067] Specifically, refer to Figure 5 As shown, the key of xarrt is composed of file name, time and version number, so that it can be sorted by map to facilitate finding the latest version. The value of xarrt is version information and delete mark delete_marker. Please refer to Figure 6 , Figure 6 A flowchart of the steps of a multi-version upload method provided for this application includes:

[0068] S201: receiving an object upload request;

[0069] S202: Determine whether the multi-version function is enabled. If so, execute S203;

[0070] S203: Generate a version number and create a version file that stores specific data and has a name consisting of the object name and the version number;

[0071] S204: Determine whether there is a logical header file corresponding to the upload object request. If not, execute S205. If so, execute S206.

[0072] S205: Create a logical header file corresponding to the object upload request, and then execute S206;

[0073] S206: Record the metadata information of the version file to the logical header file;

[0074] S207: Return to view the results.

[0075] It can be seen that in this embodiment, when an upload object request sent by a client is received, a current version file corresponding to the upload object request is first created and stored, and then the metadata information corresponding to the current version file created by the upload object request is recorded in a logical header file with extended attributes supported by the file system and displayed, so as to save multiple versions of metadata information, so that the distributed S3-NAS fusion interoperable storage system supports the multi-version function of the S3 protocol, thereby solving the problem that users cannot recover data after accidentally deleting it, enriching the functions of distributed fusion interoperable storage, and improving the market competitiveness of the distributed storage system.

[0076] Based on the above embodiment:

[0077] As an optional embodiment, after recording metadata information corresponding to the current version file in an extended attribute supported by the file system into a logical header file corresponding to the object upload request, the data processing method further includes:

[0078] When receiving a download object request sent by a client, determining target metadata information corresponding to the download object request in the logical header file;

[0079] Determine the target version file according to the target metadata information;

[0080] Read the data in the target version file and send the data to the client.

[0081] As an optional embodiment, the process of determining target metadata information corresponding to the download object request in the logical header file includes:

[0082] Determine whether the download version number is specified in the download object request;

[0083] If so, determining the metadata information corresponding to the downloaded version number in the logical header file as the target metadata information;

[0084] If not, the latest metadata information in the logical header file is determined as the target metadata information.

[0085] Specifically, when the client downloads a multi-version object, there are two situations: one is downloading without specifying a version number (i.e., obtaining the latest version data), and the other is downloading with a specified version number. When downloading an object without specifying a version number, that is, when the download version number is not specified in the object download request, the latest metadata information recorded on the logical header file is first read. It is necessary to obtain the metadata stored on the object file in the fusion intercommunication storage system, and then find the target version file that stores the data based on the obtained metadata information, read the data from the target version file, and return the data to the client; when downloading with a specified version, that is, when the download version number is specified in the object download request, the metadata information stored in the logical header file is obtained based on the download version number, and then find the target version file that stores the data based on the obtained metadata information, and finally read the data from the target version file, and return the data to the client.

[0086] Please refer to Figure 7 , Figure 7 A flowchart of the steps of a multi-version download method provided in this application includes:

[0087] S301: receiving a request to download an object;

[0088] S302: Determine whether to download a specific version. If not, execute S303. If so, execute S304.

[0089] S303: Obtain the latest metadata information on the logical header file and execute S305;

[0090] S304: Obtain metadata information of the specified version on the logical header file, and execute S305;

[0091] S305: Determine the location of the target version file based on the metadata information;

[0092] S306: Read data from the target version file and return it to the client.

[0093] As an optional embodiment, after recording metadata information corresponding to the current version file in an extended attribute supported by the file system into a logical header file corresponding to the object upload request, the data processing method further includes:

[0094] When receiving a delete object request sent by a client, determine whether a delete version number is specified in the delete object request;

[0095] If so, obtain target metadata information corresponding to the deleted version number in the logical header file, determine the target version file based on the target metadata information, and delete the target version file;

[0096] If not, a new version number is generated, and metadata information corresponding to the version number is recorded in the logical header file using extended attributes supported by the file system. The metadata information includes an identifier for the deleted object.

[0097] As an optional embodiment, after deleting the target version file, the data processing method further includes:

[0098] Delete the metadata information corresponding to the target version file in the logical header file.

[0099] Specifically, when a client deletes a multi-version object, there are two cases: one is deletion without specifying a version number, and the other is deletion with a specified version number. When deleting an object without specifying a version number, that is, when the deletion version number is not specified in the object deletion request, a new version number will be generated, and the key corresponding to the xattr will be generated based on this version number. The value of the key records the deletion object mark, and the xattr metadata is recorded in the latest position of the logical header file. When deleting with a specified version number, that is, when the deletion version number is specified in the object deletion request, the metadata information saved in the logical header file is obtained according to the specified deletion version number, and then the target version file for saving the data is determined based on the obtained metadata information, and the target version file is deleted, and then the xattr metadata information is deleted, and the result is returned to the client.

[0100] Please refer to Figure 8 , Figure 8 A flowchart of the steps of a multi-version deletion method provided in this application includes:

[0101] S401: receiving a request to delete an object;

[0102] S402: Determine whether to delete the specified version. If not, execute S403. If so, execute S404.

[0103] S403: Generate a new version number, record xattr metadata information to the logical header file, record the delete mark in value, and execute S408;

[0104] S404: Obtain metadata information of the specified version on the logical header file, and execute S405;

[0105] S405: Determine the location of the target version file for storing specific data based on the metadata information;

[0106] S406: Delete the target version file;

[0107] S407: Delete the metadata information of the specified version;

[0108] S408: Return the result to the client.

[0109] In summary, this application provides a solution that supports the S3 multi-version function in a distributed S3-NAS fusion interoperable storage system, solving the problem that the fusion interoperable storage system does not support the S3 multi-version function. This allows the fusion interoperable storage system to store multiple versions of the same-name file, enabling multiple versions of data to be stored and correctly downloaded and accessed. Not only can the latest version be downloaded and accessed, but also specific versions can be downloaded and accessed, which can effectively solve the problem of users being unable to restore files after accidental deletion, providing customers with a good user experience. At the same time, it enriches the functions of distributed fusion interoperable storage and improves the market competitiveness of distributed storage products.

[0110] Second, please refer to Figure 9 , Figure 9 This is a schematic diagram of the structure of a data processing system provided in this application, which includes:

[0111] Creation module 1, for creating a current version file corresponding to the upload object request when receiving an upload object request sent by a client;

[0112] The first recording module 2 is configured to record metadata information corresponding to the current version file in a logical header file corresponding to the object upload request using extended attributes supported by the file system;

[0113] The management module 3 is used to display the logical header file so that the client can process any version file based on the metadata information.

[0114] It can be seen that in this embodiment, when an upload object request sent by a client is received, a current version file corresponding to the upload object request is first created and stored, and then the metadata information corresponding to the current version file created by the upload object request is recorded in a logical header file with extended attributes supported by the file system and displayed, so as to save multiple versions of metadata information, so that the distributed S3-NAS fusion interoperable storage system supports the multi-version function of the S3 protocol, thereby solving the problem that users cannot recover data after accidentally deleting it, enriching the functions of distributed fusion interoperable storage, and improving the market competitiveness of the distributed storage system.

[0115] As an optional embodiment, the data processing system further includes:

[0116] A first determining module is configured to, upon receiving a download object request sent by a client, determine target metadata information corresponding to the download object request in a logical header file;

[0117] A second determining module is used to determine a target version file according to target metadata information;

[0118] The reading module is used to read the data in the target version file and send the data to the client.

[0119] As an optional embodiment, the process of determining target metadata information corresponding to the download object request in the logical header file includes:

[0120] Determine whether the download version number is specified in the download object request;

[0121] If so, determining the metadata information corresponding to the downloaded version number in the logical header file as the target metadata information;

[0122] If not, the latest metadata information in the logical header file is determined as the target metadata information.

[0123] As an optional embodiment, the data processing system further includes:

[0124] The third determining module is configured to, upon receiving a deletion request from a client, determine whether a deletion version number is specified in the deletion request, and if so, trigger the deletion module; otherwise, trigger the second recording module;

[0125] A deletion module is used to obtain target metadata information corresponding to the deletion version number in the logical header file, determine the target version file based on the target metadata information, and delete the target version file;

[0126] The second recording module is used to generate a new version number and record metadata information corresponding to the version number into the logical header file using extended attributes supported by the file system. The metadata information includes an identifier of the deleted object.

[0127] As an optional embodiment, after deleting the target version file, the deletion module is further configured to:

[0128] Delete the metadata information corresponding to the target version file in the logical header file.

[0129] As an optional embodiment, when receiving an object upload request sent by a client, the process of creating a current version file corresponding to the object upload request includes:

[0130] When receiving an upload object request sent by the client, generate a version number corresponding to the upload object request;

[0131] Create a current version file. The name of the current version file is determined based on the version number and the object name corresponding to the upload object request. Each version file has a unique name.

[0132] As an optional embodiment, the process of recording metadata information corresponding to the current version file in the logical header file corresponding to the object upload request as an extended attribute supported by the file system includes:

[0133] Record the metadata information corresponding to the current version file in the logical header file in the form of xattr.

[0134] In a third aspect, the present application further provides an electronic device, comprising:

[0135] memory for storing computer programs;

[0136] A processor is configured to implement the steps of the data processing method described in any one of the above embodiments when executing a computer program.

[0137] Specifically, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer-readable instructions, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile storage medium. When the processor executes the computer program stored in the memory, the following steps can be implemented: upon receiving an upload object request sent by a client, a current version file corresponding to the upload object request is created; metadata information corresponding to the current version file is recorded in a logical header file corresponding to the upload object request with extended attributes supported by the file system; and the logical header file is displayed so that the client can process any version file based on the metadata information.

[0138] It can be seen that in this embodiment, when an upload object request sent by a client is received, a current version file corresponding to the upload object request is first created and stored, and then the metadata information corresponding to the current version file created by the upload object request is recorded in a logical header file with extended attributes supported by the file system and displayed, so as to save multiple versions of metadata information, so that the distributed S3-NAS fusion interoperable storage system supports the multi-version function of the S3 protocol, thereby solving the problem that users cannot recover data after accidentally deleting it, enriching the functions of distributed fusion interoperable storage, and improving the market competitiveness of the distributed storage system.

[0139] As an optional embodiment, when the processor executes the computer subroutine stored in the memory, the following steps can be implemented: when receiving a download object request sent by the client, determining the target metadata information corresponding to the download object request in the logical header file; determining the target version file based on the target metadata information; reading the data in the target version file, and sending the data to the client.

[0140] As an optional embodiment, when the processor executes the computer subroutine stored in the memory, the following steps can be implemented: determine whether the download version number is specified in the download object request; if so, determine the metadata information corresponding to the download version number in the logical header file as the target metadata information; if not, determine the latest metadata information in the logical header file as the target metadata information.

[0141] As an optional embodiment, when the processor executes a computer subroutine stored in the memory, the following steps can be implemented: when receiving a request to delete an object sent by the client, determine whether a deletion version number is specified in the request to delete the object; if so, obtain the target metadata information corresponding to the deletion version number in the logical header file, determine the target version file based on the target metadata information, and delete the target version file; if not, generate a new version number, and record the metadata information corresponding to the version number in the logical header file using an extended attribute supported by the file system, where the metadata information includes an identifier for the deleted object.

[0142] As an optional embodiment, when the processor executes the computer subroutine stored in the memory, the following steps may be implemented: deleting metadata information corresponding to the target version file in the logical header file.

[0143] As an optional embodiment, when the processor executes the computer subroutine stored in the memory, the following steps can be implemented: when receiving an upload object request sent by the client, a version number corresponding to the upload object request is generated; a current version file is created, and the name of the current version file is determined based on the version number and the object name corresponding to the upload object request, and the name of each version file is unique.

[0144] As an optional embodiment, when the processor executes the computer subroutine stored in the memory, the following steps may be implemented: metadata information corresponding to the current version file is recorded in the logical header file in the form of xattr.

[0145] Based on the above embodiment, as a preferred implementation manner, the electronic device further includes:

[0146] The input interface, connected to the processor, is used to obtain externally imported computer programs, parameters, and instructions, and save them to memory under the control of the processor. This input interface can be connected to an input device to receive parameters or instructions manually entered by the user. This input device can be a touch screen overlay on the display screen, or it can be a key, trackball, or touchpad provided on the terminal housing.

[0147] The display unit is connected to the processor and is used to display the data sent by the processor. The display unit can be a liquid crystal display or an electronic ink display.

[0148] The network port is connected to the processor and is used to communicate with external terminal devices. The communication technology used for the communication connection can be wired communication technology or wireless communication technology, such as Mobile High-Definition Link technology (MHL), Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), Wireless Fidelity technology (WiFi), Bluetooth communication technology, Bluetooth Low Energy communication technology, and communication technology based on IEEE802.11s.

[0149] In a fourth aspect, the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the data processing method described in any one of the above embodiments are implemented.

[0150] Specifically, the computer-readable storage medium may include various media capable of storing program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. The storage medium stores a computer program, which, when executed by a processor, implements the following steps: upon receiving an object upload request sent by a client, creates a current version file corresponding to the object upload request; records metadata information corresponding to the current version file in a logical header file corresponding to the object upload request using extended attributes supported by the file system; and displays the logical header file so that the client can process any version file based on the metadata information.

[0151] It can be seen that in this embodiment, when an upload object request sent by a client is received, a current version file corresponding to the upload object request is first created and stored, and then the metadata information corresponding to the current version file created by the upload object request is recorded in a logical header file with extended attributes supported by the file system and displayed, so as to save multiple versions of metadata information, so that the distributed S3-NAS fusion interoperable storage system supports the multi-version function of the S3 protocol, thereby solving the problem that users cannot recover data after accidentally deleting it, enriching the functions of distributed fusion interoperable storage, and improving the market competitiveness of the distributed storage system.

[0152] As an optional embodiment, when the computer subroutine stored in the computer-readable storage medium is executed by the processor, the following steps can be specifically implemented: when a download object request sent by the client is received, the target metadata information corresponding to the download object request is determined in the logical header file; the target version file is determined based on the target metadata information; the data in the target version file is read, and the data is sent to the client.

[0153] As an optional embodiment, when the computer subroutine stored in the computer-readable storage medium is executed by the processor, the following steps can be specifically implemented: determine whether the download version number is specified in the download object request; if so, determine the metadata information corresponding to the download version number in the logical header file as the target metadata information; if not, determine the latest metadata information in the logical header file as the target metadata information.

[0154] As an optional embodiment, when the computer subroutine stored in the computer-readable storage medium is executed by the processor, the following steps can be specifically implemented: when a request to delete an object is received from the client, it is determined whether a deletion version number is specified in the request to delete the object; if so, the target metadata information corresponding to the deletion version number is obtained in the logical header file, the target version file is determined based on the target metadata information, and the target version file is deleted; if not, a new version number is generated, and the metadata information corresponding to the version number is recorded in the logical header file using an extended attribute supported by the file system, and the metadata information includes an identifier for the deleted object.

[0155] As an optional embodiment, when the computer subroutine stored in the computer-readable storage medium is executed by the processor, the following steps may be specifically implemented: deleting metadata information corresponding to the target version file in the logical header file.

[0156] As an optional embodiment, when the computer subroutine stored in the computer-readable storage medium is executed by the processor, the following steps can be specifically implemented: when an upload object request sent by the client is received, a version number corresponding to the upload object request is generated; a current version file is created, and the name of the current version file is determined based on the version number and the object name corresponding to the upload object request, and the name of each version file is unique.

[0157] As an optional embodiment, when the computer subroutine stored in the computer-readable storage medium is executed by the processor, the following steps may be specifically implemented: metadata information corresponding to the current version file is recorded in the logical header file in the form of xattr.

[0158] It should also be noted that, in this specification, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0159] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing method, characterized in that: include: When receiving an object upload request sent by a client, creating a current version file corresponding to the object upload request; Recording metadata information corresponding to the current version file in a logical header file corresponding to the object upload request as an extended attribute supported by the file system; Displaying the logical header file so that the client can process any version file based on the metadata information; After recording the metadata information corresponding to the current version file in the logical header file corresponding to the upload object request as an extended attribute supported by the file system, the data processing method further includes: When receiving a download object request sent by the client, determining target metadata information corresponding to the download object request in the logical header file; Determine the target version file according to the target metadata information; Read the data in the target version file and send the data to the client; The data processing method is applied in a distributed storage system in which the S3 protocol and the NAS protocol are integrated and interoperable, so as to support the mutual access and interoperability of the S3 protocol and the NAS protocol; the client is an S3 client; When receiving an object upload request sent by a client, the process of creating a current version file corresponding to the object upload request includes: When receiving an upload object request sent by a client, generating a version number corresponding to the upload object request; Creating a current version file, where the name of the current version file is determined based on the version number and the object name corresponding to the upload object request, and the name of each version file is unique; The metadata information of the version file is recorded in the form of xarrt metadata, and the metadata information corresponding to each version file is sorted according to the default scheme; the key of the xarrt metadata is composed of the file name, time and version number, and the value of xarrt is the version information and the delete mark delete_marker.

2. The data processing method according to claim 1, wherein: The process of determining the target metadata information corresponding to the download object request in the logical header file includes: Determine whether a download version number is specified in the download object request; If so, determining the metadata information corresponding to the downloaded version number in the logical header file as the target metadata information; If not, the latest metadata information in the logical header file is determined as the target metadata information.

3. The data processing method according to claim 1, wherein: After recording the metadata information corresponding to the current version file in the logical header file corresponding to the upload object request as an extended attribute supported by the file system, the data processing method further includes: When receiving a delete object request sent by the client, determining whether a deletion version number is specified in the delete object request; If so, obtaining target metadata information corresponding to the deletion version number in the logical header file, determining a target version file based on the target metadata information, and deleting the target version file; If not, a new version number is generated, and metadata information corresponding to the version number is recorded in the logical header file using an extended attribute supported by the file system, wherein the metadata information includes an identifier of the deleted object.

4. The data processing method according to claim 3, wherein: After deleting the target version file, the data processing method further includes: The metadata information corresponding to the target version file is deleted from the logical header file.

5. The data processing method according to any one of claims 1 to 4, characterized in that: The process of recording metadata information corresponding to the current version file in the logical header file corresponding to the upload object request as an extended attribute supported by the file system includes: The metadata information corresponding to the current version file is recorded in the logical header file in the form of xattr.

6. A data processing system, characterized in that: include: A creation module, configured to, upon receiving an object upload request sent by a client, create a current version file corresponding to the object upload request; A first recording module is configured to record metadata information corresponding to the current version file in a logical header file corresponding to the object upload request using an extended attribute supported by the file system; A management module, configured to display the logical header file so that the client can process any version of the file based on the metadata information; The data processing system further includes: A first determining module is configured to, upon receiving a download object request sent by a client, determine target metadata information corresponding to the download object request in a logical header file; A second determining module is used to determine a target version file according to target metadata information; The reading module is used to read the data in the target version file and send the data to the client; The data processing method is applied in a distributed storage system in which the S3 protocol and the NAS protocol are integrated and interoperable, so as to support the mutual access and interoperability of the S3 protocol and the NAS protocol; the client is an S3 client; When receiving an object upload request sent by a client, the process of creating a current version file corresponding to the object upload request includes: When receiving an upload object request sent by a client, generating a version number corresponding to the upload object request; Creating a current version file, where the name of the current version file is determined based on the version number and the object name corresponding to the upload object request, and the name of each version file is unique; The metadata information of the version file is recorded in the form of xarrt metadata, and the metadata information corresponding to each version file is sorted according to the default scheme; the key of the xarrt metadata is composed of the file name, time and version number, and the value of xarrt is the version information and the delete mark delete_marker.

7. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the data processing method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the data processing method according to any one of claims 1 to 5.

Images