Data processing method, device, storage medium and electronic device

Abstract

The present application discloses a data processing method, device, storage medium and electronic device. The method includes: obtaining a data stream sent by a client, parsing the data stream, and obtaining initial request information; obtaining target instruction information and data information from the initial request information, and converting the target instruction information into an execution function according to a preset conversion relationship; obtaining metadata from the data information, obtaining first metadata, and determining the second metadata corresponding to the first metadata, wherein, when the first metadata is file metadata, the second metadata is object metadata, and when the first metadata is object metadata, the second metadata is file metadata; performing data processing operations according to the execution function, the first metadata and the second metadata. Through the present application, the problem of low data conversion efficiency between different protocols in the related art is solved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular, to a data processing method, device, storage medium and electronic device. Background Art

[0002] When storing data, data storage is generally implemented using multiple protocols, such as the NFS file protocol, the S3 object protocol, etc. The data storage implemented using the file protocol and the object protocol are file storage and object storage respectively. When the same data uses different protocols, the corresponding metadata used is also different. Therefore, when storing the same data using different protocols, the data needs to be stored separately using different protocols. For example, file data is stored in a file storage data pool, and object data is stored in an object storage data pool, so that different protocols can be used to read and write data. However, the data storage method used in the related art stores the same data separately according to different protocols, resulting in the same data occupying more storage space and wasting storage resources.

[0003] In the related art, although data and metadata corresponding to the data can be stored separately, data can be stored in a data pool, and metadata of different protocols can be stored in different metadata pools, so as to achieve the effect of saving storage space. However, when different protocols are used to perform data operations, the storage method in the related art cannot achieve true two-way real-time sharing of metadata. For example, when a certain data only has file metadata stored in a file storage metadata pool, when the data is operated through an object storage protocol, the file metadata needs to be stored in the object storage metadata pool, and the object metadata corresponding to the data needs to be obtained in the object storage metadata pool to complete the corresponding data operation. At the same time, storing metadata of different protocols corresponding to the same data in different data pools will take up a large amount of storage space, thereby affecting the efficiency of data operations and the utilization of storage space.

[0004] Currently, no effective solution has been proposed to address the problem of low efficiency in data conversion between different protocols in related technologies. Summary of the invention

[0005] The present application provides a data processing method, device, storage medium and electronic device to solve the problem of low data conversion efficiency between different protocols in the related art.

[0006] According to one aspect of the present application, a data processing method is provided. The method includes: obtaining a data stream sent by a client, parsing the data stream, and obtaining initial request information; obtaining target instruction information and data information from the initial request information, and converting the target instruction information into an execution function according to a preset conversion relationship, wherein the data information is file data or object data; obtaining metadata from the data information, obtaining first metadata, and determining second metadata corresponding to the first metadata, wherein when the first metadata is file metadata, the second metadata is object metadata, and when the first metadata is object metadata, the second metadata is file metadata; performing data processing operations according to the execution function, the first metadata, and the second metadata.

[0007] Optionally, obtaining the data stream sent by the client and parsing the data stream to obtain initial request information includes: determining the protocol type used by the data stream to obtain the target protocol type; sending the data stream to a service process corresponding to the target protocol type; parsing the data stream through the service process to obtain initial request information.

[0008] Optionally, when the data information is file data, the first metadata is file metadata, and determining the second metadata corresponding to the first metadata includes: converting the file metadata into metadata in an object format to obtain object metadata; and determining the obtained object metadata as the second metadata.

[0009] Optionally, when the data information is object data, the first metadata is object metadata, and determining the second metadata corresponding to the first metadata includes: converting the object metadata into metadata in a file format to obtain file metadata; and determining the obtained file metadata as the second metadata.

[0010] Optionally, when the request information is a write request, the first metadata includes a storage path, and performing data processing operations according to the execution function, the first metadata and the second metadata includes: determining whether there is data to be stored in the data information; if there is no data to be stored in the data information, storing the first metadata and the second metadata in the metadata storage pool according to the execution function; if there is data to be stored in the data information, storing the first metadata and the second metadata in the metadata storage pool according to the execution function, and storing the data to be stored in the storage path of the data storage pool.

[0011] Optionally, after storing the data to be stored in the storage path of the data storage pool according to the execution function, the method further includes: acquiring the storage result returned by the storage pool, and sending the storage result to the client.

[0012] Optionally, when the request information is a read request, the first metadata contains the data name of the data to be read, and performing data processing operations according to the execution function, the first metadata and the second metadata includes: obtaining the data name from the first metadata according to the execution function, and obtaining target metadata containing the data name from the metadata storage pool; obtaining a read path from the target metadata, and obtaining the data to be read from the data storage pool according to the read path, and sending the data to be read to the client.

[0013] According to another aspect of the present application, a data processing device is provided. The device includes: a first acquisition unit, which is used to acquire a data stream sent by a client, and parse the data stream to obtain initial request information; a second acquisition unit, which is used to acquire target instruction information and data information from the initial request information, and convert the target instruction information into an execution function according to a preset conversion relationship, wherein the data information is file data or object data; a determination unit, which is used to acquire metadata from the data information, obtain first metadata, and determine second metadata corresponding to the first metadata, wherein when the first metadata is file metadata, the second metadata is object metadata, and when the first metadata is object metadata, the second metadata is file metadata; an execution unit, which is used to perform data processing operations according to the execution function, the first metadata, and the second metadata.

[0014] According to another aspect of an embodiment of the present invention, a computer storage medium is provided. The computer storage medium is used to store a program. When the program is executed, the device where the computer storage medium is located is controlled to execute a data processing method.

[0015] According to another aspect of an embodiment of the present invention, an electronic device is provided, comprising one or more processors and a memory; the memory stores computer-readable instructions, and the processor is used to execute the computer-readable instructions, wherein a data processing method is executed when the computer-readable instructions are executed.

[0016] Through this application, the following steps are adopted: obtaining the data stream sent by the client, and parsing the data stream to obtain the initial request information; obtaining the target instruction information and data information from the initial request information, and converting the target instruction information into an execution function according to a preset conversion relationship, wherein the data information is file data or object data; obtaining metadata from the data information, obtaining the first metadata, and determining the second metadata corresponding to the first metadata, wherein, when the first metadata is file metadata, the second metadata is object metadata, and when the first metadata is object metadata, the second metadata is file metadata; performing data processing operations according to the execution function, the first metadata, and the second metadata. The problem of low efficiency of data conversion between different protocols in the related art is solved. By parsing the data instructions of different file protocols, the corresponding operation instruction information and the data to be operated are obtained, and the instruction information of different protocols is converted into a universal execution function, and at the same time, the metadata corresponding to another protocol is generated according to the metadata corresponding to one protocol, and the execution function is called to operate the data based on the metadata corresponding to the two protocols, thereby achieving the effect of improving the data conversion between different protocols. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a flow chart of a data processing method provided according to an embodiment of the present application;

[0019] Figure 2 is a system architecture diagram of an optional data processing method provided in an embodiment of the present application;

[0020] Figure 3 It is a schematic diagram of a data processing device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. For example, an interface is set between this system and the relevant user or organization. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0025] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0026] Data storage pool: the storage pool for data.

[0027] Metadata storage pool: The storage pool for the metadata corresponding to the data.

[0028] According to an embodiment of the present application, a data processing method is provided.

[0029] S3: In this application, it refers to the object storage service protocol based on Amazon Simple Storage Service.

[0030] Figure 1 is a flow chart of a data processing method according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0031] Step S102, obtaining the data stream sent by the client, and parsing the data stream to obtain initial request information.

[0032] Specifically, the data stream may be received by the gateway, and the data stream may be parsed to obtain the initial request information sent by the client, and the data storage instruction or data read / write instruction sent by the client may be identified according to the initial request information.

[0033] It should be noted that before the gateway receives the data stream, a data transmission channel needs to be established between the gateway and the client, for example, a TCP protocol connection is established between the client and the gateway, and the data stream is transmitted according to the TCP connection.

[0034] In order to ensure that the data stream can be accurately parsed, optionally, in the data processing method provided in the embodiment of the present application, the data stream sent by the client is obtained, and the data stream is parsed to obtain the initial request information including: determining the protocol type adopted by the data stream to obtain the target protocol type; sending the data stream to the service process corresponding to the target protocol type; parsing the data stream through the service process to obtain the initial request information.

[0035] It should be noted that due to the different operations required by the client, the file protocols used when transmitting file data may be different, such as NFS protocol, CIFS protocol, FTP protocol, etc. Similarly, the object protocols used when transmitting object data may be different, such as S3 protocol or Swift protocol, etc. In order to accurately parse the data stream, it is necessary to first obtain the protocol type of the object data or file data, and obtain the process corresponding to the protocol type in the gateway, so as to parse the protocol through the corresponding process and obtain the initial request information.

[0036] For example, when the data stream transmits file data of the NFS protocol, the gateway will identify the NFS protocol, obtain the corresponding service process, such as the XNFSD process, according to the file protocol, and input the data stream into the XNFSD process for parsing, thereby obtaining the initial request information in the data stream.

[0037] Step S104, obtaining target instruction information and data information from the initial request information, and converting the target instruction information into an execution function according to a preset conversion relationship, wherein the data information is file data or object data.

[0038] Specifically, after obtaining the initial request information, the target instruction information and data information in the initial request information can be obtained, wherein the target instruction information is used to record the operation instructions on the data information issued by the client, for example, storing the data information, and the data information includes metadata and data, such as file data and meta-file data corresponding to the file data.

[0039] It should be noted that since the protocols of the target instruction information parsed by different processes are different, it is necessary to convert the target instruction information into a universal execution function in the dynamic link library according to the preset conversion relationship, and store the data information according to the execution function, thereby ensuring that the data in the data pool can be operated by instructions in different processes through the same universal execution function, so that data of different protocols can be successfully operated.

[0040] Step S106, obtaining metadata from the data information to obtain first metadata, and determining second metadata corresponding to the first metadata, wherein when the first metadata is file metadata, the second metadata is object metadata, and when the first metadata is object metadata, the second metadata is file metadata.

[0041] Specifically, after determining the execution function, it is necessary to perform corresponding operations on the data information. At this time, it is necessary to first determine the metadata protocol in the data information, and obtain the metadata corresponding to another metadata protocol based on the metadata protocol. That is, before storing the data, the metadata corresponding to data of multiple protocols are generated and stored together, and then when using any protocol to operate the data, the corresponding metadata information can be obtained in the metadata storage pool, thereby achieving the effect of storing multi-protocol data in the same storage pool and sharing the data for multiple protocols.

[0042] Optionally, in the data processing method provided in an embodiment of the present application, when the data information is file data, the first metadata is file metadata, and determining the second metadata corresponding to the first metadata includes: converting the file metadata into metadata in an object format to obtain object metadata; and determining the obtained object metadata as the second metadata.

[0043] Specifically, when the first metadata is file metadata, the metadata can be converted into object metadata based on the content of the file metadata, and the file metadata and the object metadata can be used together as metadata corresponding to the data in the data information, so that both the file protocol and the object protocol can operate on the data.

[0044] Optionally, in the data processing method provided in an embodiment of the present application, when the data information is object data, the first metadata is object metadata, and determining the second metadata corresponding to the first metadata includes: converting the object metadata into metadata in a file format to obtain file metadata; and determining the obtained file metadata as the second metadata.

[0045] Similarly, when the first metadata is object metadata, the metadata can be converted into file metadata based on the content of the object metadata, and the file metadata and the object metadata can be used together as metadata corresponding to the data in the data information, so that both the file protocol and the object protocol can operate on the data.

[0046] Step S108: performing a data processing operation according to the execution function, the first metadata and the second metadata.

[0047] Specifically, after the file metadata and the object metadata are obtained, the file metadata, the object metadata, and the data in the data information may be stored or read and written accordingly according to the execution function.

[0048] The data processing method provided in the embodiment of the present application obtains the data stream sent by the client and parses the data stream to obtain the initial request information; obtains the target instruction information and data information from the initial request information, and converts the target instruction information into an execution function according to a preset conversion relationship, wherein the data information is file data or object data; obtains metadata from the data information to obtain the first metadata, and determines the second metadata corresponding to the first metadata, wherein when the first metadata is file metadata, the second metadata is object metadata, and when the first metadata is object metadata, the second metadata is file metadata; performs data processing operations according to the execution function, the first metadata and the second metadata. The problem of low efficiency of data conversion between different protocols in the related art is solved. By parsing the data instructions of different file protocols, the corresponding operation instruction information and the data to be operated are obtained, and the instruction information of different protocols is converted into a universal execution function, and at the same time, the metadata corresponding to another protocol is generated according to the metadata corresponding to one protocol, and the execution function is called to operate the data based on the metadata corresponding to the two protocols, thereby achieving the effect of improving the data conversion between different protocols.

[0049] Optionally, in the data processing method provided in an embodiment of the present application, when the request information is a write request, the first metadata includes a storage path, and performing data processing operations according to the execution function, the first metadata, and the second metadata includes: determining whether there is data to be stored in the data information; if there is no data to be stored in the data information, storing the first metadata and the second metadata in a metadata storage pool according to the execution function; if there is data to be stored in the data information, storing the first metadata and the second metadata in the metadata storage pool according to the execution function, and storing the data to be stored in the storage path of the data storage pool.

[0050] It should be noted that the operation on data information can be an operation on metadata or an operation on data. When the request information is a write request, the data storage operation can be judged by determining whether there is data to be stored in the data information.

[0051] Specifically, when there is no data to be stored in the data information, the proof instruction information only includes changes and writes to the metadata. At this time, the metadata in the data information can be directly stored or rewritten in the metadata storage pool according to the information in the metadata, thereby completing the target instruction information.

[0052] When there is data to be stored in the data information, it indicates that the instruction information not only includes changes and writes to the metadata, but also includes changes to the data to be stored. At this time, after completing the changes and writes to the metadata, it is also necessary to obtain the storage path in the metadata, and determine the location of the data storage based on the storage path, so as to write the data to be stored to the storage area corresponding to the storage path, thereby completing the writing of the data to be stored.

[0053] Optionally, in the data processing method provided in the embodiment of the present application, after storing the data to be stored in the storage path of the data storage pool according to the execution function, the method further includes: obtaining the storage result returned by the storage pool, and sending the storage result to the client.

[0054] Specifically, after writing the data, the data storage pool and the metadata storage pool will return the storage result to the gateway, and the gateway will return the storage result to the client, so that the client can know whether the data is stored successfully.

[0055] Optionally, in the data processing method provided in an embodiment of the present application, when the request information is a read request, the first metadata contains the data name of the data to be read, and performing the data processing operation according to the execution function, the first metadata and the second metadata includes: obtaining the data name from the first metadata according to the execution function, and obtaining target metadata containing the data name from the metadata storage pool; obtaining a read path from the target metadata, and obtaining the data to be read from the data storage pool according to the read path, and sending the data to be read to the client.

[0056] Specifically, when the request information is a read request, the data name in the metadata can be obtained, and then the metadata information corresponding to the data name can be obtained from the metadata storage pool according to the data name, and at the same time, the file data or object data is determined according to the data protocol in the read request, so as to obtain the metadata to be obtained corresponding to the file data or object data in the metadata storage pool, and the data to be obtained corresponding to the metadata is determined according to the storage path in the metadata information, that is, the reading path, so that the data to be obtained and the metadata to be obtained are returned from the storage pool to the gateway as the data to be read, and the data to be read is returned to the client in the gateway, thereby completing the data reading operation of any protocol.

[0057] Figure 2 is a system architecture diagram of an optional data processing method provided in an embodiment of the present application, such as Figure 2 As shown, when the data to be stored is file data, the client can be a file storage application of the NFS protocol. At this time, the file storage application sends a data stream to the gateway. The gateway determines that the file protocol corresponding to the data stream is the NFS protocol, so the data stream can be sent to the XNFSD process, and the data stream is parsed in the XNFSD process to obtain the initial request information.

[0058] Furthermore, after parsing the initial request information to obtain the target instruction information and data information, the target instruction information can be converted into a general execution function in the dynamic link library through a preset conversion relationship, and the corresponding storage operation can be performed on the data information according to the general execution function. At this time, the execution function can send the data information to the XPS process, and the metadata in the data information can be changed in the XPS process, so as to obtain the object metadata based on the file metadata, and store the file metadata and object metadata together in the metadata storage pool, and store the data in the data storage pool according to the storage path in the metadata, thereby completing the data storage.

[0059] After receiving the data stream that reads the above-mentioned data to be stored from the object storage application, the protocol of the data stream is first determined, such as the S3 protocol. The gateway determines that the file protocol corresponding to the data stream is the S3 protocol, so the data stream can be sent to the XS3D process and the data stream is parsed in the XS3D process to obtain the initial request information.

[0060] Furthermore, after parsing the initial request information to obtain the target instruction information and data information, the target instruction information can be converted into a general execution function in the dynamic link library through a preset conversion relationship, and the corresponding read operation can be performed according to the general execution function, so that the data name in the metadata in the data information can be obtained, and the metadata information corresponding to the data name can be obtained from the metadata storage pool according to the data name. At the same time, the file data or object data is determined according to the data protocol in the read request, so as to obtain the metadata to be obtained corresponding to the file data or object data in the metadata storage pool, and the data to be obtained corresponding to the metadata is determined according to the storage path in the metadata information, that is, the reading path, so that the data to be obtained and the metadata to be obtained are returned from the storage pool to the gateway as the data to be read, and the data to be read is returned to the client in the gateway, thereby completing the data reading operation of any protocol.

[0061] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0062] The embodiment of the present application also provides a data processing device. It should be noted that the data processing device of the embodiment of the present application can be used to execute the data processing method provided by the embodiment of the present application. The data processing device provided by the embodiment of the present application is introduced below.

[0063] Figure 3 is a schematic diagram of a data processing device according to an embodiment of the present application. Figure 3 As shown, the device includes: a first acquisition unit 31, a second acquisition unit 32, a determination unit 33, and an execution unit 34.

[0064] The first acquisition unit 31 is used to acquire the data stream sent by the client, and parse the data stream to obtain initial request information.

[0065] The second acquisition unit 32 is used to acquire target instruction information and data information from the initial request information, and convert the target instruction information into an execution function according to a preset conversion relationship, wherein the data information is file data or object data.

[0066] The determination unit 33 is used to obtain metadata from the data information, obtain first metadata, and determine second metadata corresponding to the first metadata, wherein when the first metadata is file metadata, the second metadata is object metadata, and when the first metadata is object metadata, the second metadata is file metadata.

[0067] The execution unit 34 is used to execute a data processing operation according to the execution function, the first metadata and the second metadata.

[0068] The data processing device provided by the embodiment of the present application is used to obtain the data stream sent by the client through the first acquisition unit 31, and parse the data stream to obtain the initial request information; the second acquisition unit 32 is used to obtain the target instruction information and data information from the initial request information, and convert the target instruction information into an execution function according to a preset conversion relationship, wherein the data information is file data or object data; the determination unit 33 is used to obtain metadata from the data information, obtain the first metadata, and determine the second metadata corresponding to the first metadata, wherein when the first metadata is file metadata, the second metadata is object metadata, and when the first metadata is object metadata, the second metadata is file metadata; the execution unit 34 is used to perform data processing operations according to the execution function, the first metadata and the second metadata. The problem of low data conversion efficiency between different protocols in the related art is solved. By parsing the data instructions of different file protocols, the corresponding operation instruction information and the data to be operated are obtained, and the instruction information of different protocols is converted into a universal execution function, and at the same time, the metadata corresponding to another protocol is generated according to the metadata corresponding to one protocol, and the execution function is called to operate the data based on the metadata corresponding to the two protocols, thereby achieving the effect of improving the data conversion between different protocols.

[0069] Optionally, in the data processing device provided in an embodiment of the present application, the first acquisition unit 31 includes: a first determination module, used to determine the protocol type adopted by the data stream to obtain the target protocol type; a sending module, used to send the data stream to a service process corresponding to the target protocol type; and a parsing module, used to parse the data stream through the service process to obtain initial request information.

[0070] Optionally, in the data processing device provided in an embodiment of the present application, when the data information is file data, the first metadata is file metadata, and the determination unit 33 includes: a first conversion module, used to convert the file metadata into metadata in an object format to obtain object metadata; and a second determination module, used to determine the obtained object metadata as second metadata.

[0071] Optionally, in the data processing device provided in an embodiment of the present application, when the data information is object data, the first metadata is object metadata, and the determination unit 33 includes: a second conversion module, used to convert the object metadata into metadata in a file format to obtain file metadata; and a third determination module, used to determine the obtained file metadata as the second metadata.

[0072] Optionally, in the data processing device provided in the embodiment of the present application, when the request information is a write request, the first metadata includes a storage path, and the execution unit 34 includes: a judgment module, used to judge whether there is data to be stored in the data information; a first storage module, used to store the first metadata and the second metadata in the metadata storage pool according to the execution function when there is no data to be stored in the data information; a second storage module, used to store the first metadata and the second metadata in the metadata storage pool according to the execution function when there is data to be stored in the data information, and store the data to be stored in the storage path of the data storage pool.

[0073] Optionally, in the data processing device provided in the embodiment of the present application, the device further includes: a third acquisition unit, configured to acquire the storage result returned by the storage pool, and send the storage result to the client.

[0074] Optionally, in the data processing device provided in the embodiment of the present application, when the request information is a read request, the first metadata contains the data name of the data to be read, and the execution unit 34 includes: a first acquisition module, used to obtain the data name from the first metadata according to the execution function, and obtain the target metadata containing the data name from the metadata storage pool; a second acquisition module, used to obtain a read path from the target metadata, and obtain the data to be read from the data storage pool according to the read path, and send the data to be read to the client.

[0075] The above-mentioned data processing device includes a processor and a memory. The above-mentioned first acquisition unit 31, second acquisition unit 32, determination unit 33, execution unit 34, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0076] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the problem of low data conversion efficiency between different protocols in the related art is solved by adjusting kernel parameters.

[0077] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0078] An embodiment of the present invention provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the data processing method is implemented.

[0079] An embodiment of the present invention provides a processor, which is used to run a program, wherein the data processing method is executed when the program is run.

[0080] An embodiment of the present invention provides an electronic device, including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: obtaining a data stream sent by a client, parsing the data stream, and obtaining initial request information; obtaining target instruction information and data information from the initial request information, and converting the target instruction information into an execution function according to a preset conversion relationship, wherein the data information is file data or object data; obtaining metadata from the data information, obtaining first metadata, and determining second metadata corresponding to the first metadata, wherein when the first metadata is file metadata, the second metadata is object metadata, and when the first metadata is object metadata, the second metadata is file metadata; performing data processing operations according to the execution function, the first metadata, and the second metadata. The device in this article may be a server, a PC, a PAD, a mobile phone, etc.

[0081] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: obtaining a data stream sent by a client, and parsing the data stream to obtain initial request information; obtaining target instruction information and data information from the initial request information, and converting the target instruction information into an execution function according to a preset conversion relationship, wherein the data information is file data or object data; obtaining metadata from the data information to obtain first metadata, and determining second metadata corresponding to the first metadata, wherein when the first metadata is file metadata, the second metadata is object metadata, and when the first metadata is object metadata, the second metadata is file metadata; performing data processing operations according to the execution function, the first metadata, and the second metadata.

[0082] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0083] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0084] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0085] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0086] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0087] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0088] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0089] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0090] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A data processing method, characterized in that: include: Obtaining the data stream sent by the client, and parsing the data stream to obtain initial request information; Acquire target instruction information and data information from the initial request information, convert the target instruction information into an execution function according to a preset conversion relationship, and store the data information according to the execution function, wherein the data information is file data or object data; Acquire metadata from the data information to obtain first metadata, and determine second metadata corresponding to the first metadata, wherein, when the first metadata is file metadata, the second metadata is object metadata, and when the first metadata is the object metadata, the second metadata is the file metadata; Performing a data processing operation according to the execution function, the first metadata, and the second metadata includes: storing the first metadata and the second metadata in a metadata storage pool according to the execution function.

2. The method according to claim 1, characterized in that Obtain the data stream sent by the client, and parse the data stream to obtain the initial request information including: Determine the protocol type adopted by the data flow to obtain a target protocol type; Sending the data stream to a service process corresponding to the target protocol type; The data stream is parsed by the service process to obtain the initial request information.

3. The method according to claim 1, characterized in that In a case where the data information is the file data, the first metadata is the file metadata, and determining the second metadata corresponding to the first metadata includes: Convert the file metadata into metadata in an object format to obtain the object metadata; The obtained object metadata is determined as the second metadata.

4. The method according to claim 1, characterized in that: In a case where the data information is the object data, the first metadata is the object metadata, and determining the second metadata corresponding to the first metadata includes: Convert the object metadata into metadata in a file format to obtain the file metadata; The obtained file metadata is determined as the second metadata.

5. The method according to claim 2, characterized in that: In a case where the request information is a write request, the first metadata includes a storage path, and performing a data processing operation according to the execution function, the first metadata, and the second metadata includes: Determining whether there is data to be stored in the data information; When the data to be stored does not exist in the data information, storing the first metadata and the second metadata in a metadata storage pool according to the execution function; In the case where the data to be stored exists in the data information, the first metadata and the second metadata are stored in the metadata storage pool according to the execution function, and the data to be stored is stored in the storage path of the data storage pool.

6. The method according to claim 5, characterized in that After storing the data to be stored in the storage path of the data storage pool according to the execution function, the method further includes: The storage result returned by the storage pool is obtained, and the storage result is sent to the client.

7. The method according to claim 1, characterized in that In the case where the request information is a read request, the first metadata includes a data name of the data to be read, and performing a data processing operation according to the execution function, the first metadata, and the second metadata includes: Acquire the data name from the first metadata according to the execution function, and acquire target metadata including the data name from a metadata storage pool; A read path is obtained from the target metadata, and the to-be-read data is obtained from a data storage pool according to the read path, and the to-be-read data is sent to the client.

8. A data processing device, characterized in that: include: A first acquisition unit, used to acquire a data stream sent by a client, and parse the data stream to obtain initial request information; a second acquisition unit, configured to acquire target instruction information and data information from the initial request information, convert the target instruction information into an execution function according to a preset conversion relationship, and store the data information according to the execution function, wherein the data information is file data or object data; a determining unit, configured to obtain metadata from the data information to obtain first metadata, and determine second metadata corresponding to the first metadata, wherein, in the case where the first metadata is file metadata, the second metadata is object metadata, and in the case where the first metadata is the object metadata, the second metadata is the file metadata; An execution unit is used to perform a data processing operation according to the execution function, the first metadata and the second metadata, including: storing the first metadata and the second metadata in a metadata storage pool according to the execution function.

9. A computer storage medium, characterized in that The computer storage medium is used to store a program, wherein when the program is run, the device where the computer storage medium is located is controlled to execute the data processing method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method described in any one of claims 1 to 7.

Images