Data processing method, apparatus, storage medium, and system

Abstract

The application discloses a data processing method and device, a storage medium and a system. The method is applied to a distributed system, the distributed system comprises at least one type of metadata management node, and the method comprises the following steps: receiving request information for processing to-be-processed data; determining a target node from a plurality of metadata management nodes according to the request information, wherein different types of metadata management nodes process different types of data; and performing running processing on the to-be-processed data based on the target node, to obtain a processing result. The application solves the technical problem that the expansion performance of the distributed system is poor due to the fact that the metadata management nodes in the distributed system need to process a plurality of types of data.

Description

TECHNICAL FIELD

[0001] The present application relates to the computer field, in particular, to a data processing method and device, a storage medium and a system. BACKGROUND

[0002] With the development of Internet technology, more and more individuals or enterprises use distributed systems for data processing. In the distributed file system, multiple discrete storage nodes are connected together through a network to provide high availability and high performance file storage semantics to external devices. Moreover, the storage capacity of the distributed system can be expanded horizontally with the increase of storage nodes.

[0003] However, in the distributed system, the horizontal expansion of the storage nodes is restricted by the memory and performance bottleneck of the metadata management nodes, so that the capacity and performance of the distributed system cannot be linearly increased with the increase of the storage nodes. The metadata management nodes are mainly used for managing metadata describing file attributes and metadata describing directory trees.

[0004] For example, Figure 2 is a data processing method flowchart in the prior art. As shown in Figure 2 , the metadata management nodes are management node 1, management node 2 and management node 3, and the storage nodes are storage node 1, storage node 2 and storage node 3. Each group of metadata management nodes corresponds to a management space, and each group of metadata management nodes manages its own corresponding management space independently and cannot coordinate with each other. Moreover, the data in the same management space forms a data pool, such as Figure 2 data pool 1, data pool 2 and data pool 3, so that each group of metadata management nodes maintains a corresponding management space and data pool respectively for processing the metadata describing the directory tree and the metadata describing the file attributes.

[0005] However, it is easy to notice that in Figure 2 , since each group of metadata management nodes is independent of each other, the performance and scalability of the distributed system are limited by the processing capacity of each metadata management node and the number of metadata management nodes. Moreover, the balance of the distributed system depends on the balance of the pressure of the management space corresponding to each group of metadata management nodes. If the pressure of the management space corresponding to a group of metadata management nodes is large, the distributed system is still prone to performance bottlenecks.

[0006] At present, there is no effective solution to the above problems. SUMMARY

[0007] Embodiments of the present application provide a data processing method and device, a storage medium, and a system to at least solve the problem of poor scalability of a distributed system caused by the need for a metadata management node in the distributed system to process multiple types of data.

[0008] According to an aspect of embodiments of the present application, a data processing method is provided, applied to a distributed system including at least one type of metadata management node. The data processing method includes: receiving request information for processing to-be-processed data; determining a target node from a plurality of metadata management nodes according to the request information, wherein different node types of metadata management nodes process different types of data; and performing run-time processing on the to-be-processed data based on the target node to obtain a processing result.

[0009] According to another aspect of embodiments of the present application, a data processing method is also provided, applied to a distributed system including at least one type of metadata management node. The data processing method includes: reading request information for processing to-be-processed data sent by a client; displaying a target node for processing the to-be-processed data, wherein the node type of the target node is determined by the request information, and different node types of metadata management nodes process different types of data; and displaying a processing result of run-time processing of the to-be-processed data by the target node.

[0010] According to another aspect of embodiments of the present application, a data processing device is also provided, applied to a distributed system including at least one type of metadata management node. The data processing device includes: a receiving module configured to receive request information for processing to-be-processed data; a determining module configured to determine a target node from a plurality of metadata management nodes according to the request information, wherein different node types of metadata management nodes process different types of data; and a processing module configured to perform run-time processing on the to-be-processed data based on the target node to obtain a processing result.

[0011] According to another aspect of embodiments of the present application, a storage medium is also provided, including a stored program. When the program is running, the storage medium controls a device in which the storage medium is located to perform the data processing method described above.

[0012] According to another aspect of the embodiments of the present application, a data processing system is also provided, which is applied to a distributed system including at least one type of metadata management node, and includes a processor and a memory connected with the processor, for providing the processor with instructions to process the following steps: receiving request information for processing data to be processed; determining a target node from the plurality of metadata management nodes according to the request information, wherein different types of metadata management nodes process different types of data; and performing running processing on the data to be processed based on the target node to obtain a processing result.

[0013] In the embodiments of the present application, the target node corresponding to the data to be processed is determined from the plurality of metadata management nodes according to the request information, so that running processing is performed on the data to be processed based on the target node to obtain a processing result after receiving the request information for processing the data to be processed. Different types of metadata management nodes process different types of data.

[0014] In the above process, the target node corresponding to the data to be processed is determined from the plurality of metadata management nodes by analyzing the request information for processing the data to be processed before performing running processing on the data to be processed. Since different types of metadata management nodes process different types of data, the metadata nodes are distributed in the process of determining the target node, that is, the original multiple data processed by one metadata management node is distributed to multiple metadata management nodes for processing, so that the memory consumption and performance consumption of one metadata management node are shared by multiple metadata management nodes. On this basis, when the distributed system increases the storage nodes, it is no longer limited by the memory and performance bottlenecks of the metadata management nodes, thereby solving the problem of poor expansion performance of the distributed system caused by the metadata management nodes in the distributed system needing to process multiple data, and achieving the effect of improving the storage capacity of the distributed system.

[0015] Therefore, the technical scheme of the embodiments of the present application achieves the purpose of avoiding the limitation of the memory and performance bottlenecks of the metadata management nodes when the distributed system increases the storage nodes, thereby achieving the effect of improving the storage capacity of the distributed system and reducing the cost, and further solving the problem of poor expansion performance of the distributed system caused by the metadata management nodes in the distributed system needing to process multiple data in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0017] Figure 1 is a hardware structure block diagram of a computer terminal for implementing a data processing method;

[0018] Figure 2 is a prior art data processing method flow chart;

[0019] Figure 3 is a data processing method flow chart according to Embodiment 1 of the application;

[0020] Figure 4 is a data processing method flow chart according to Embodiment 1 of the application;

[0021] Figure 5 is a data processing method flow chart according to Embodiment 2 of the application;

[0022] Figure 6 is a schematic diagram of a data processing device according to Embodiment 4 of the application. DETAILED DESCRIPTION

[0023] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to the process, method, product or device.

[0025] First, some of the nouns or terms that appear in the description of the embodiments of the present application are applicable to the following explanations:

[0026] Metadata: Also known as intermediary data or relay data, it is data that describes other data. It mainly describes the attributes of other data and is used to support functions such as indicating storage location, historical data, resource lookup, and file records.

[0027] URI Meta: Metadata describing the directory tree, including the name of the directory or file, parent-child relationships, etc.

[0028] File Meta: Metadata that describes file attributes, including file length, creation time, data distribution information, etc.

[0029] Example 1

[0030] According to an embodiment of this application, an embodiment of a data processing method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware structure block diagram of a computer terminal for implementing a data processing method is shown. Figure 1 As shown, the computer terminal 10 may include one or more processors (shown as 102a, 102b, ..., 102n in the figure) (the processor may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer terminal described above. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0032] It should be noted that the one or more processors and / or other data processing circuitry described above can be referred to herein generally as "data processing circuitry". The data processing circuitry can be embodied in whole or in part as software, hardware, firmware, or any combination thereof. In addition, the data processing circuitry can be a single standalone processing module, or incorporated in whole or in part within any one of the other elements of the computer terminal 10. As referred to in the embodiments of the present application, the data processing circuitry functions as a processor to control, for example, the selection of the variable resistance terminal path connected to the interface.

[0033] The memory 104 can be used to store software programs of application software and modules, such as program instructions / data storage means corresponding to the data processing method in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, i.e. implements the data processing method described above. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor, which can be connected to the computer terminal 10 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0034] The transmission device 106 is used to receive or send data via a network. Specific examples of the network can include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet in a wireless manner.

[0035] The display can be, for example, a touch screen type liquid crystal display (LCD), which can enable a user to interact with the user interface of the computer terminal 10.

[0036] In addition, it should be noted that a processor can serve as the execution subject of the data processing method in the embodiments.

[0037] Under the above operating environment, the present application provides a data processing method as shown in Figure 3 Figure 3 is a flowchart of the data processing method according to the embodiment 1 of the present application. As shown in Figure 3

[0038] ​​Step S301, receiving a request information for processing the to-be-processed data.

[0039] Optionally, in step S301, the data processing method is applied to a distributed system, wherein the distributed system comprises at least one type of metadata management node. The metadata management node can be a discrete metadata management node, and the type of each metadata management node is different. For example, the type of the metadata management node can be a first metadata management node and a second metadata management node. In addition, the processor can name the first metadata management node as a Name Node, which is used to manage the metadata describing the directory tree, such as obtaining the data block distribution, obtaining or modifying the data block length, and marking the data block state. The processor can also name the second metadata management node as a Meta Node, which is used to manage the metadata describing the file attributes, such as the operations of creating, deleting, moving, querying, and updating the file or directory.

[0040] Optionally, the to-be-processed data can be various types of data, such as the metadata describing the directory tree, such as the file name or the file directory, the metadata describing the file attributes, such as the file creation time and the file length. In addition, the request information at least includes the operation type, the operation time, and the operation object of the file corresponding to the to-be-processed data.

[0041] It should be noted that, by receiving the request information for processing the to-be-processed data before performing the running processing on the to-be-processed data, the processor can analyze the request information and determine the operation type and other information of the to-be-processed data, so as to classify the to-be-processed data according to the request information, thereby achieving the effect of improving the management efficiency of the to-be-processed data, and providing a data basis for managing different to-be-processed data through different metadata management nodes.

[0042] Step S302, determining a target node from the plurality of metadata management nodes according to the request information.

[0043] Optionally, in step S302, metadata management nodes of different node types process different types of data. A distributed system can have multiple metadata management nodes, and these nodes can have different node types. For example, a distributed system may have three metadata management nodes: metadata management node A, metadata management node B, and metadata management node C. Metadata management nodes A and B are of type 1 metadata management nodes, while metadata management node C is of type 2 metadata management nodes. It should be noted that metadata management nodes A and B, as 1 metadata management nodes, process metadata describing the directory tree of the file corresponding to the data to be processed, while metadata management node C, as 2 metadata management nodes, processes metadata describing the file attributes of the file corresponding to the data to be processed.

[0044] Furthermore, the request information must contain at least information identifying the target node. For example, it may include an identifier for the operation type performed on the file corresponding to the data to be processed. Upon receiving the request information, the processor will parse it, obtain the identifier for the corresponding operation type, and, based on this identifier, determine the corresponding metadata management node, i.e., the target node.

[0045] It is worth noting that by determining the target node from multiple metadata management nodes based on the request information, the data to be processed is classified according to different node types. This allows the processor to allocate the classified data to the corresponding metadata management node based on the node type, thereby improving data management efficiency, reducing the memory consumption of the metadata management node, and avoiding the problem of all data to be processed being managed by a single metadata management node.

[0046] Step S303: Perform processing on the data to be processed based on the target node to obtain the processing result.

[0047] Optionally, in step S303, the processor may perform corresponding processing on the data to be processed based on the target node. For example, when the target node is the first metadata management node, the processor may perform processing such as creating, deleting, moving, and querying the metadata of the directory tree of the file corresponding to the data to be processed. When the target node is the second metadata management node, the processor may perform processing such as modifying and marking the metadata of the file attributes of the file corresponding to the data to be processed.

[0048] It should be noted that, since the metadata management nodes of different node types process different types of data, the process of determining the target node realizes the distribution of the metadata nodes, that is, the multiple types of data originally processed by one metadata management node are distributed to multiple metadata management nodes for processing, so that the memory consumption and performance consumption corresponding to one metadata management node are shared by multiple metadata management nodes. On this basis, when the distributed system increases the storage nodes, it is no longer limited by the memory and performance bottlenecks of the metadata management nodes, thereby solving the problem of poor expansion performance of the distributed system caused by the metadata management nodes in the distributed system needing to process multiple types of data in the prior art.

[0049] Based on the contents of steps S301 to S303, in the embodiment of the present application, the target node corresponding to the to-be-processed data is determined from the multiple metadata management nodes according to the request information, and after receiving the request information for processing the to-be-processed data, the target node is determined from the multiple metadata management nodes according to the request information, so that the running processing of the to-be-processed data is performed based on the target node to obtain the processing result. Different node types of metadata management nodes process different types of data.

[0050] In the above process, before performing the running processing on the to-be-processed data, the request information for processing the to-be-processed data is analyzed, so that the target node corresponding to the to-be-processed data can be determined from the multiple metadata management nodes. Since different node types of metadata management nodes process different types of data, the process of determining the target node realizes the distribution of the metadata nodes, that is, the multiple types of data originally processed by one metadata management node are distributed to multiple metadata management nodes for processing, so that the memory consumption and performance consumption corresponding to one metadata management node are shared by multiple metadata management nodes. On this basis, when the distributed system increases the storage nodes, it is no longer limited by the memory and performance bottlenecks of the metadata management nodes, thereby solving the problem of poor expansion performance of the distributed system caused by the metadata management nodes in the distributed system needing to process multiple types of data in the prior art, and achieving the effect of improving the storage capacity of the distributed system.

[0051] As can be seen, the technical scheme of the embodiment of the present application achieves the purpose of avoiding the distributed system being limited by the memory and performance bottlenecks of the metadata management nodes when increasing the storage nodes, thereby achieving the effect of improving the storage capacity of the distributed system, and further solving the problem of poor expansion performance of the distributed system caused by the metadata management nodes in the distributed system needing to process multiple types of data in the prior art.

[0052] In an optional embodiment, before determining the target node from the plurality of metadata management nodes according to the request information, the processor determines an operation type of the operation on the file corresponding to the to-be-processed data from the request information, and determines a request type corresponding to the request information according to the operation type.

[0053] Optionally, the processor can determine whether the user operates on the metadata of the directory tree of the file corresponding to the to-be-processed data or on the metadata of the file attribute of the file corresponding to the to-be-processed data by identifying the operation of the user. For example, a user establishes a file with a file name A file in the distributed system, which belongs to the operation on the metadata of the directory tree of the file. The request information contains the operation type of the A file, so that the processor can identify the operation type of the A file after receiving the request information. Based on the operation type of the A file, the processor can determine a first metadata type metadata management node as the target node from the plurality of metadata management nodes.

[0054] In the above process, the request type corresponding to the request information is determined according to the operation type, which realizes the effect that the processor automatically identifies the request type corresponding to the request information when the to-be-processed data changes.

[0055] In an optional embodiment, the processor determines the first metadata management node as the target node from the plurality of metadata management nodes when detecting that the request information is the first request information, wherein the first metadata management node is used for managing the metadata describing the directory tree of the file.

[0056] Optionally, the first request information corresponds to the request information about the metadata describing the directory tree of the file. For example, when a user modifies, deletes, updates, or performs other operations on the directory, name, or parent-child relationship of a file, the distributed system generates the first request information about the file and sends it to the processor. When the processor receives the first request information, it determines the metadata management node with the first metadata management node as the target node from the plurality of metadata management nodes in the distributed system. The first metadata management node is a type of metadata management node in the distributed system, and there can be multiple first metadata management nodes. The first metadata management node is used for managing the metadata describing the directory tree of the file.

[0057] In the above process, by detecting the request information, the metadata describing the directory tree of the file is allocated to the first metadata management node for management, which is beneficial to improve the data management efficiency and reduce the memory consumption of the metadata management node.

[0058] In an optional embodiment, the processor obtains a first correspondence relationship between the first management space and at least one group of metadata management nodes, and determines a first target management space from the first management space corresponding to the target node according to the first correspondence relationship, so as to control the target node to process the to-be-processed data in the first target management space and obtain a processing result

[0059] Optionally, on the basis of the metadata of the directory tree of the description file being handed over to the first metadata management node, the plurality of first metadata management nodes can be divided into a plurality of groups. For example, in a distributed system, there are totally 10 first metadata management nodes, which can be represented by serial numbers 1 to 10 respectively, wherein the three first metadata management nodes with serial numbers 1 to 3 are a first group, and the seven first metadata management nodes with serial numbers 4 to 10 are a second group. In addition, each group of first metadata management nodes has a corresponding management space, i.e., a first management space.

[0060] Further, an administrator of the distributed system can pre-set a first correspondence relationship between the first management space and at least one group of metadata management nodes. For example, the distributed system includes a first management space 01 and a first management space 02, wherein the first management space 01 is used to correspond to the first metadata management nodes of the first group, and the first management space 02 is used to correspond to the first metadata management nodes of the second group. When the target node is a first metadata management node, the processor determines a first target management space from the first management space according to the first correspondence relationship. For example, the processor finds that the current memory resource of the first management space 02 is in an idle state, so the first management space 02 is taken as the first target management space, and the target node is controlled to process the to-be-processed data. It should be noted that the first target management space can be a plurality of first management spaces, for example, when the to-be-processed data is too large, the processor can simultaneously allocate the to-be-processed data to the first management space 01 and the first management space 02 for storage.

[0061] In the above process, by allocating the metadata of the directory tree of the description file to the management spaces corresponding to the plurality of metadata management nodes for storage management, the problem that the to-be-processed data is managed by one metadata management node is avoided, and the effect of reducing the memory consumption of the metadata management node is achieved,

[0062] In an optional embodiment, the processor determines a second metadata management node as the target node from the plurality of metadata management nodes when detecting that the request information is second request information, wherein the second metadata management node is used to manage the metadata of the file attribute of the description file.

[0063] Optionally, the second request information corresponds to request information about metadata of file attributes of the description file. For example, when a user modifies or performs other operations on file length or data distribution information of a certain file, the distributed system generates second request information about the file and sends it to the processor. When the processor receives the second request information, it determines a metadata management node of the second metadata management node type from the plurality of metadata management nodes in the distributed system as the corresponding target node. The second metadata management node is a type of metadata management node in the distributed system, and there can be multiple second metadata management nodes. The second metadata management node is used to manage metadata of file attributes of the description file.

[0064] In the above process, by detecting the request information, the metadata describing the file attributes is allocated to the second metadata management node for management, which is beneficial to improve data management efficiency and reduce memory consumption of the metadata management node.

[0065] In an optional embodiment, the processor determines a second correspondence between the second management space and the at least one group of metadata management nodes based on a preset random number, and determines a second target management space from the second management space corresponding to the target node according to the second correspondence, so as to control the target node to process the to-be-processed data in the second target management space to obtain a processing result.

[0066] Optionally, on the basis of handing over the metadata describing the file attributes of the file to the second metadata management node, the processor can divide the plurality of second metadata management nodes into a plurality of groups, and each group of second metadata management nodes has a corresponding management space, i.e., a second management space.

[0067] Further, the processor can determine a second correspondence between the second management space and the at least one group of metadata management nodes based on a preset random number. For example, the distributed system includes a second management space 01 and a second management space 02. When the target node is a second metadata management node, the processor randomly allocates the to-be-processed data to the second management space 01 or the second management space 02 for storage. In a random case, each time the to-be-processed data needs to be stored in the second management space, the probability of being allocated to any group of second metadata management nodes is also random, i.e., the second target management space is randomly determined.

[0068] It should be noted that, Figure 2 is a data processing method flowchart in the prior art. In the prior art, Figure 2In the embodiment, the management nodes are management node 1, management node 2 and management node 3, and the storage nodes are storage node 1, storage node 2 and storage node 3. In the embodiment, the management spaces corresponding to the multiple groups of metadata management nodes are directly independent of each other, and each group of metadata management nodes manages the corresponding management space independently. For example, the management space 1, the management space 2 and the management space 3 in the embodiment are independent of each other. Figure 2 In the embodiment, the management spaces are independent of each other, and thus the management spaces corresponding to each group of metadata management nodes are independent of each other.

[0069] In addition, in the embodiment, the data belonging to the same management space form a data pool. For example, the data pool 1, the data pool 2 and the data pool 3 in the embodiment form a data pool. Figure 2 In the embodiment, the data belonging to the same management space form a data pool. For example, the data pool 1, the data pool 2 and the data pool 3 in the embodiment form a data pool. Figure 2 In the embodiment, the data belonging to the same management space form a data pool. For example, the data pool 1, the data pool 2 and the data pool 3 in the embodiment form a data pool. Figure 2 In the embodiment, each group of metadata management nodes is independent of each other, and thus the data processing method in the embodiment still has the problem of single-point failure, and each group of metadata management nodes is still a single point. In addition, the performance and the expansibility of the distributed system are limited by the processing capacity of the metadata management nodes and the number of the metadata management nodes. More importantly, the balance of the distributed system depends on the balance of the pressure of the management spaces corresponding to each group of metadata management nodes. If the pressure of the management space corresponding to a group of metadata management nodes is large, the distributed system is still prone to performance bottleneck. Figure 2 In the embodiment, the metadata describing the file attributes of the files are randomly allocated and processed according to the second management space, and thus the balance of each second metadata management node can be guaranteed, which is independent of the balance of the user accessing the first management area, thereby avoiding the problem that the distributed system is prone to performance bottleneck due to the large pressure of the management space corresponding to a group of metadata management nodes, and achieving the effect of improving the balance and the expansibility of the distributed system.

[0070] In an optional embodiment, the processor detects whether the target node is in an abnormal state, and determines a target agent node from at least one agent node when it is detected that the target node is in the abnormal state, wherein the at least one agent node is a metadata management node of the same type as the target node, so as to process the to-be-processed data based on the target agent node to obtain a processing result.

[0071] Optionally, as shown in

[0072] Figure 4 ​As shown, the processor can detect the state of the target node through the root directory service program RootServer, and when the target node has an abnormality such as process blocking or logical error, the processor can discover the abnormality in the first time and determine the target proxy node from the at least one proxy node. For example, when the target node is the first metadata management node A, if the first metadata management node A has a logical error, the processor changes the to-be-processed data to be processed by the first metadata management node B or the first metadata management node C. It should be noted that the at least one proxy node is a metadata management node of the same type as the target node.

[0073] Further, as shown in the figure, Figure 4 The root directory service program RootServer can detect the first metadata management node and the second metadata management node, that is, the process of detecting the metadata describing the target number and the metadata describing the file attribute.

[0074] In the above process, the processor actively maintains when the target node is in an abnormal state, thereby improving the stability and high availability of the distributed system.

[0075] In an optional embodiment, the processor determines that the target node is in an abnormal state when detecting that a process corresponding to the target node is blocked; and / or determines that the target node is in an abnormal state when detecting that a running program corresponding to the target node has a logical abnormality.

[0076] Optionally, when the processor determines that the target node is abnormal, the processor can determine whether a process corresponding to the target node is blocked. For example, when the amount of data to be processed by the target node is too large or the data network transmission is abnormal, the process is slow and cannot be performed, the processor determines that the current target node is in an abnormal state. In addition, if the processor detects that a running program corresponding to the target node has a logical abnormality, for example, cannot correctly perform data update, the processor determines that the current target node is in an abnormal state.

[0077] In an optional embodiment, the processor obtains node state information of the at least one proxy node, wherein the node state information represents state information of system resources occupied by the at least one proxy node, and determines a target proxy node from the at least one proxy node according to the node state information, wherein node state information of the target proxy node satisfies a preset condition.

[0078] Optionally, when selecting at least one proxy node, the processor can determine it based on node status information. For example, there are currently 10 metadata management nodes that can be used as proxy nodes. However, 5 of these metadata management nodes have a memory utilization rate of 70%, 4 have a memory utilization rate of 50%, and 1 has a memory utilization rate of 30%. In addition, the preset condition is that the memory utilization rate of the metadata management node that can be used as the target proxy node must be less than 45%. Therefore, the processor can determine the metadata management node with a memory utilization rate of 30% as the target proxy node.

[0079] The above process avoids imbalances in the distributed system caused by selecting proxy nodes.

[0080] In one alternative embodiment, after the processor performs processing on the data to be processed based on the target node and obtains the processing result, it updates the data to be processed stored in the cloud node based on the processing result.

[0081] Optional, such as Figure 4 As shown, the processor can update the data to be processed stored in the cloud node based on the processing results obtained after performing the processing on the data to be processed. For example, when a user changes the name of a file named "File A" in the distributed system to "File B", after the processor changes the name of the file based on the target node, it can perform an incremental update on the corresponding data to be processed in the cloud node. That is, it only changes the file name from "File A" to "File B" without replacing the entire file.

[0082] It should be noted that the above process is a persistent, real-time update process. Using cloud nodes to store data solves the dependence of metadata management nodes on the performance of local storage space, that is, it reduces the requirements for local storage devices and improves the scalability of distributed systems.

[0083] From the above, before performing the running processing on the to-be-processed data, the request information for processing the to-be-processed data is analyzed, so that the target node corresponding to the to-be-processed data can be determined from the plurality of metadata management nodes. Since the metadata management nodes of different node types process different types of data, the process of determining the target node realizes the distribution of the metadata nodes, that is, the plurality of metadata management nodes are distributed to process the plurality of data originally processed by one metadata management node, so that the memory consumption and performance consumption of one metadata management node are shared by the plurality of metadata management nodes. On this basis, when the storage nodes of the distributed system are increased, the distributed system is no longer limited by the memory and performance bottlenecks of the metadata management nodes, thereby solving the problem of poor expansion performance of the distributed system caused by the metadata management nodes in the distributed system needing to process a plurality of data, and achieving the effect of improving the storage capacity of the distributed system.

[0084] Therefore, the technical scheme of the embodiment of the application achieves the purpose of avoiding the distributed system being limited by the memory and performance bottlenecks of the metadata management nodes when the storage nodes are increased, thereby achieving the effect of improving the storage capacity of the distributed system, and further solving the problem of poor expansion performance of the distributed system caused by the metadata management nodes in the distributed system needing to process a plurality of data.

[0085] Embodiment 2

[0086] According to the embodiment of the application, a data processing method embodiment is also provided, wherein Figure 5 is a flowchart of the data processing method according to Embodiment 2 of the application. As shown in Figure 5 :

[0087] In step S501, the request information for processing the to-be-processed data sent by the client is read.

[0088] Optionally, in step S501, the data processing method is applied to a distributed system, wherein the distributed system includes at least one type of metadata management node. The metadata management node can be a discrete metadata management node, and the type of each metadata management node is different. For example, the type of the metadata management node can be a first metadata management node and a second metadata management node. In addition, the processor can name the first metadata management node as a Name Node, which is used to manage the metadata describing the directory tree, such as obtaining the data block distribution, obtaining or modifying the data block length, and marking the data block state. The processor can also name the second metadata management node as a Meta Node, which is used to manage the metadata describing the file attributes, such as the creation, deletion, movement, query, and update of the file or directory.

[0089] Further, the to-be-processed data can be various types of data, such as file names or file directory metadata describing a directory tree, or file creation times and file lengths, metadata describing file attributes. In addition, the request information at least includes information such as an operation type, an operation time, and an operation object for operating the file corresponding to the to-be-processed data.

[0090] It should be noted that by reading the request information for processing the to-be-processed data before performing the running processing on the to-be-processed data, the request information can be analyzed to determine the operation type and other information of the to-be-processed data, and the to-be-processed data can be classified according to the request information, which is beneficial to improve the management efficiency of the to-be-processed data and provides a data basis for managing different to-be-processed data through different metadata management nodes.

[0091] In step S502, a target node for processing the to-be-processed data is displayed.

[0092] Optionally, in step S502, the node type of the target node is determined by the request information, and different metadata management nodes of different node types process different types of data. Among them, there can be multiple metadata management nodes in the distributed system, and the node types of the multiple metadata management nodes can also be different. For example, there are three metadata management nodes in a distributed system, namely metadata management node A, metadata management node B, and metadata management node C. The node types of the metadata management node A and the metadata management node B are first metadata management nodes, and the node type of the metadata management node C is a second metadata management node. It should be noted that the metadata management node A and the metadata management node B as the first metadata management nodes process metadata describing a directory tree of a file corresponding to the to-be-processed data, and the metadata management node C as the second metadata management node processes metadata describing file attributes of the file corresponding to the to-be-processed data.

[0093] Further, the request information at least contains information for determining the target node, such as identification information of an operation type for operating the file corresponding to the to-be-processed data. After the processor receives the request information, the request information is parsed to obtain the identification information of the corresponding operation type, and based on the identification information, the corresponding metadata management node, i.e., the target node, is determined and displayed on the display device. The management personnel of the distributed system can confirm whether the target node selected by the processor is correct through the target node displayed on the display device, and conveniently master the status of each metadata management node.

[0094] It should be noted that in the process of displaying the target node for processing the to-be-processed data, the processor determines the target node from the plurality of metadata management nodes according to the request information, realizes classification of the to-be-processed data according to different types, and thus selects the corresponding metadata management node according to the type for the classified to-be-processed data, which is beneficial to improve the data management efficiency and reduce the memory consumption of the metadata management node, and avoids the problem that the to-be-processed data is managed by one metadata management node.

[0095] In step S503, the processing result of the target node performing the running processing on the to-be-processed data is displayed.

[0096] Optionally, in step S503, the processor can perform the corresponding running processing on the to-be-processed data based on the target node, for example, if the target node is the first metadata management node, the processor can perform the processing such as creation, deletion, movement and query on the metadata of the directory tree of the file corresponding to the to-be-processed data, and if the target node is the second metadata management node, the processor can perform the processing such as modification and marking on the metadata of the file attribute of the file corresponding to the to-be-processed data.

[0097] Further, after obtaining the processing result, the processor can control the connected display device to display the processing result, so that the administrator of the distributed system can intuitively observe the processing result.

[0098] It should be noted that since the metadata management nodes of different node types process different types of data, the process of determining the target node realizes the distribution of the metadata nodes, that is, the plurality of metadata management nodes is distributed to process the plurality of data originally processed by one metadata management node, so that the memory consumption and performance consumption of one metadata management node are shared by the plurality of metadata management nodes, and on this basis, the distributed system can no longer be restricted by the memory and performance bottlenecks of the metadata management node when increasing the storage nodes, thereby solving the problem of poor expansion performance of the distributed system in the prior art due to the need of the metadata management node in the distributed system to process a plurality of data.

[0099] Based on the contents of steps S501 to S503, in the embodiment of the present application, the target node corresponding to the to-be-processed data is determined from the plurality of metadata management nodes according to the request information, the request information for processing the to-be-processed data sent by the client is read, and the target node for processing the to-be-processed data is displayed, so that the processing result of the target node performing the running processing on the to-be-processed data is displayed. The node type of the target node is determined by the request information, and the metadata management nodes of different node types process different types of data.

[0100] In the above process, before performing the running processing on the to-be-processed data, the request information for processing the to-be-processed data is analyzed, so that the target node corresponding to the to-be-processed data can be determined from the plurality of metadata management nodes. Since the metadata management nodes of different node types process different types of data, the process of determining the target node realizes the distribution of the metadata nodes, that is, the plurality of metadata management nodes are distributed to process the plurality of data originally processed by one metadata management node, so that the memory consumption and performance consumption of one metadata management node are shared by the plurality of metadata management nodes. On this basis, when the distributed system increases the storage nodes, it is no longer limited by the memory and performance bottlenecks of the metadata management nodes, thereby solving the problem of poor expansion performance of the distributed system caused by the metadata management nodes in the distributed system needing to process a plurality of data, and achieving the effect of improving the storage capacity of the distributed system.

[0101] Therefore, the technical scheme of the embodiment of the present application achieves the purpose of avoiding the distributed system from being limited by the memory and performance bottlenecks of the metadata management nodes when increasing the storage nodes, thereby achieving the effect of improving the storage capacity of the distributed system, and further solving the problem of poor expansion performance of the distributed system caused by the metadata management nodes in the distributed system needing to process a plurality of data.

[0102] Embodiment 3

[0103] According to another aspect of the embodiment of the present application, a data processing system is also provided, which is applied to a distributed system including at least one type of metadata management node. The data processing system includes a processor and a memory connected with the processor, for providing the processor with instructions for processing the following processing steps: receiving request information for processing to-be-processed data, and determining a target node from a plurality of metadata management nodes according to the request information, wherein different node types of metadata management nodes process different types of data, so as to perform running processing on the to-be-processed data based on the target node, and obtain a processing result.

[0104] In the embodiment of the present application, the target node corresponding to the to-be-processed data is determined from the plurality of metadata management nodes according to the request information, the request information for processing the to-be-processed data is received, and the target node is determined from the plurality of metadata management nodes according to the request information, so as to perform running processing on the to-be-processed data based on the target node, and obtain a processing result. Different node types of metadata management nodes process different types of data.

[0105] In the above process, before performing the running processing on the to-be-processed data, the request information for processing the to-be-processed data is analyzed, so that the target node corresponding to the to-be-processed data can be determined from the plurality of metadata management nodes. Since the metadata management nodes of different node types process different types of data, the process of determining the target node realizes the distribution of the metadata nodes, that is, the plurality of metadata management nodes are distributed to process the plurality of data originally processed by one metadata management node, so that the memory consumption and performance consumption of one metadata management node are shared by the plurality of metadata management nodes. On this basis, when the storage nodes of the distributed system are increased, the distributed system is no longer limited by the memory and performance bottlenecks of the metadata management nodes, thereby solving the problem of poor expansion performance of the distributed system caused by the metadata management nodes in the distributed system needing to process a plurality of data, and achieving the effect of improving the storage capacity of the distributed system.

[0106] Therefore, the technical scheme of the embodiment of the application achieves the purpose of avoiding the distributed system from being limited by the memory and performance bottlenecks of the metadata management nodes when the storage nodes are increased, thereby achieving the effect of improving the storage capacity of the distributed system, and further solving the problem of poor expansion performance of the distributed system caused by the metadata management nodes in the distributed system needing to process a plurality of data.

[0107] Embodiment 4

[0108] According to the embodiment of the application, a data processing device embodiment is also provided, which is applied to a distributed system including at least one type of metadata management node. Wherein, Figure 6 is a schematic diagram of the data processing device according to Embodiment 4 of the application, as Figure 6 shown, the device includes a receiving module 601, a determining module 602, and a processing module 603.

[0109] The receiving module 601 is configured to receive request information for processing to-be-processed data; the determining module 602 is configured to determine a target node from a plurality of metadata management nodes according to the request information, wherein metadata management nodes of different node types process different types of data; and the processing module 603 is configured to perform running processing on the to-be-processed data based on the target node to obtain a processing result.

[0110] It should be noted that the receiving module 601, the determining module 602, and the processing module 603 correspond to steps S301 to S303 in the above embodiment, and the three modules have the same examples and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1.

[0111] Optionally, the data processing apparatus further comprises a first determining module and a second determining module. The first determining module is configured to determine, from the request information, an operation type of operating on a file corresponding to the to-be-processed data; and the second determining module is configured to determine, according to the operation type, a request type corresponding to the request information.

[0112] Optionally, the determining module comprises a third determining module configured to determine, when it is detected that the request information is first request information, a first metadata management node as the target node from the plurality of metadata management nodes, wherein the first metadata management node is configured to manage metadata describing a directory tree of the file.

[0113] Optionally, the processing module comprises an obtaining module, a fourth determining module and a control module. The obtaining module is configured to obtain a first correspondence relationship between the first management space and the at least one group of metadata management nodes; the fourth determining module is configured to determine, according to the first correspondence relationship, a first target management space from the first management space corresponding to the target node; and the control module is configured to control the target node to process the to-be-processed data in the first target management space to obtain the processing result.

[0114] Optionally, the determining module comprises a fifth determining module configured to determine, when it is detected that the request information is second request information, a second metadata management node as the target node from the plurality of metadata management nodes, wherein the second metadata management node is configured to manage metadata describing file attributes of the file.

[0115] Optionally, the processing module comprises a sixth determining module, a seventh determining module and a first control module. The sixth determining module is configured to determine, based on a preset random number, a second correspondence relationship between the second management space and the at least one group of metadata management nodes; the seventh determining module is configured to determine, according to the second correspondence relationship, a second target management space from the second management space corresponding to the target node; and the first control module is configured to control the target node to process the to-be-processed data in the second target management space to obtain the processing result.

[0116] Optionally, the processing module comprises a detecting module, an eighth determining module and a first processing module. The detecting module is configured to detect whether the target node is in an abnormal state; the eighth determining module is configured to determine, when it is detected that the target node is in the abnormal state, a target proxy node from at least one proxy node, wherein the at least one proxy node is a metadata management node of the same type as the target node; and the first processing module is configured to process the to-be-processed data based on the target proxy node to obtain the processing result.

[0117] Optionally, the detection module comprises a ninth determination module configured to determine that the target node is in an abnormal state when it is detected that the process corresponding to the target node is blocked; and / or determine that the target node is in an abnormal state when it is detected that the running program corresponding to the target node has a logic exception.

[0118] Optionally, the eighth determination module comprises a first acquisition module and a tenth determination module. The first acquisition module is configured to acquire node state information of the at least one proxy node, wherein the node state information represents state information of the at least one proxy node occupying system resources; and the tenth determination module is configured to determine a target proxy node from the at least one proxy node according to the node state information, wherein the node state information of the target proxy node satisfies a preset condition.

[0119] Optionally, the data processing apparatus further comprises an updating module configured to perform an updating operation on the to-be-processed data stored in the cloud node based on the processing result.

[0120] Embodiment 5

[0121] The embodiments of the present application further provide a storage medium. Optionally, in the present embodiment, the storage medium can be used to save the program code executed by the data processing method provided in the first embodiment.

[0122] Optionally, in the present embodiment, the storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0123] Optionally, in the present embodiment, the storage medium is configured to store program code for performing the following steps: receiving request information for processing to-be-processed data; determining a target node from a plurality of metadata management nodes according to the request information, wherein the metadata management nodes of different node types process different types of data; performing running processing on the to-be-processed data based on the target node to obtain a processing result

[0124] Optionally, in the present embodiment, the storage medium is configured to store program code for performing the following steps: before determining a target node from a plurality of metadata management nodes according to request information, determining an operation type of operating a file corresponding to to-be-processed data from the request information; and determining a request type corresponding to the request information according to the operation type.

[0125] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: determining the first metadata management node as the target node from the plurality of metadata management nodes when detecting that the request information is the first request information, wherein the first metadata management node is configured to manage metadata describing a directory tree of a file.

[0126] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: obtaining a first correspondence relationship between the first management space and the at least one group of metadata management nodes; determining a first target management space from the first management space corresponding to the target node according to the first correspondence relationship; and controlling the target node to process the to-be-processed data in the first target management space to obtain the processing result.

[0127] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: determining the second metadata management node as the target node from the plurality of metadata management nodes when detecting that the request information is the second request information, wherein the second metadata management node is configured to manage metadata describing a file attribute of a file.

[0128] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: determining a second correspondence relationship between the second management space and the at least one group of metadata management nodes based on a preset random number; determining a second target management space from the second management space corresponding to the target node according to the second correspondence relationship; and controlling the target node to process the to-be-processed data in the second target management space to obtain the processing result.

[0129] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: detecting whether the target node is in an abnormal state; determining a target proxy node from at least one proxy node when detecting that the target node is in the abnormal state, wherein the at least one proxy node is a metadata management node of the same type as the target node; and processing the to-be-processed data based on the target proxy node to obtain the processing result.

[0130] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: determining that the target node is in the abnormal state when detecting that a process corresponding to the target node is blocked; and / or determining that the target node is in the abnormal state when detecting that a running program corresponding to the target node has a logical exception.

[0131] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: obtaining node state information of the at least one proxy node, wherein the node state information represents state information of the at least one proxy node occupying system resources; and determining a target proxy node from the at least one proxy node according to the node state information, wherein the node state information of the target proxy node satisfies a preset condition.

[0132] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: after performing running processing on the to-be-processed data based on the target node, and obtaining a processing result, performing an update operation on the to-be-processed data stored in the cloud node based on the processing result.

[0133] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: reading request information sent by the client for processing the to-be-processed data; displaying a target node for processing the to-be-processed data, wherein the node type of the target node is determined by the request information, and different metadata management nodes of different node types process different types of data; and displaying a processing result of the target node performing running processing on the to-be-processed data.

[0134] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0135] Those skilled in the art can clearly understand the data processing method according to the above-mentioned embodiments can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the method of each embodiment of the present application.

[0136] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0137] In the above-mentioned embodiments of the present application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0138] In several embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division mode, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between units or modules, which can be electrical or other forms.

[0139] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment.

Claims

1. A data processing method, characterized in that, Applied to a distributed system, the distributed system including at least one type of metadata management node, the method includes: A request to process data to be processed is received, wherein the data to be processed is metadata; The target node is determined from multiple metadata management nodes based on the request information. The metadata management nodes of different node types process different types of data. The multiple metadata management nodes include management nodes for managing the metadata of the directory tree of the description file and management nodes for managing the metadata of the file attributes of the description file. When the target node is detected to be in an abnormal state, the data to be processed is processed based on the target proxy node in at least one proxy node to obtain the processing result.

2. The method according to claim 1, characterized in that, Before determining the target node from the plurality of metadata management nodes based on the request information, the method further includes: Determine the operation type for the file corresponding to the data to be processed from the request information; The request type corresponding to the request information is determined based on the operation type.

3. The method according to claim 2, characterized in that, Determining the target node from the plurality of metadata management nodes based on the request information includes: When the request information is detected to be the first request information, the first metadata management node is determined from the plurality of metadata management nodes as the target node, wherein the first metadata management node is used to manage the metadata describing the directory tree of the file.

4. The method according to claim 3, characterized in that, Based on the target node, the data to be processed is processed to obtain the processing result, including: Obtain the first correspondence between the first management space and at least one set of metadata management nodes; The first target management space is determined from the first management space corresponding to the target node based on the first correspondence relationship; The target node is controlled to process the data to be processed in the first target management space to obtain the processing result.

5. The method according to claim 2, characterized in that, Determining the target node from the plurality of metadata management nodes based on the request information includes: When the request information is detected to be the second request information, the second metadata management node is determined as the target node from the plurality of metadata management nodes, wherein the second metadata management node is used to manage the metadata describing the file attributes of the file.

6. The method according to claim 5, characterized in that, Based on the target node, the data to be processed is processed to obtain the processing result, including: A second correspondence between the second management space and at least one set of metadata management nodes is determined based on a preset random number. The second target management space is determined from the second management space corresponding to the target node according to the second correspondence relationship; The target node is controlled to process the data to be processed in the second target management space to obtain the processing result.

7. The method according to claim 1, characterized in that, Based on the target node, the data to be processed is processed to obtain the processing result, including: Detect whether the target node is in an abnormal state; When the target node is detected to be in the abnormal state, a target proxy node is determined from at least one proxy node, wherein the at least one proxy node is a metadata management node of the same type as the target node; The target proxy node is used to process the data to be processed to obtain the processing result.

8. The method according to claim 7, characterized in that, Detecting whether the target node is in an abnormal state includes: When a blocked process is detected corresponding to the target node, it is determined that the target node is in the abnormal state; and / or, When a logical anomaly is detected in the running program corresponding to the target node, the target node is determined to be in the abnormal state.

9. The method according to claim 7, characterized in that, Determining the target proxy node from at least one proxy node includes: Obtain the node status information of the at least one proxy node, wherein the node status information represents the status information of the system resources occupied by the at least one proxy node; The target proxy node is determined from the at least one proxy node based on the node status information, wherein the node status information of the target proxy node satisfies a preset condition.

10. The method according to claim 1, characterized in that, After performing processing on the data to be processed based on the target node and obtaining the processing result, the method further includes: Based on the processing results, the data to be processed stored in the cloud node is updated.

11. A data processing method, characterized in that, Applied to a distributed system, the distributed system including at least one type of metadata management node, the method includes: Read the request information sent by the client to process the data to be processed, wherein the data to be processed is metadata; The target node for processing the data to be processed is displayed, wherein the node type of the target node is determined by the request information. Metadata management nodes of different node types process different types of data. The metadata management nodes of different node types include management nodes for managing the metadata of the directory tree of the description file and management nodes for managing the metadata of the file attributes of the description file. The display shows the processing result of the target node performing the operation on the data to be processed, wherein the processing result is obtained by processing the data to be processed based on the target proxy node in at least one proxy node when the target node is detected to be in an abnormal state.

12. A data processing apparatus, characterized in that, Applied to a distributed system, the distributed system including at least one type of metadata management node, the device includes: The receiving module is used to receive a request information for processing data to be processed, wherein the data to be processed is metadata; The determination module is used to determine the target node from multiple metadata management nodes according to the request information. The metadata management nodes of different node types process different types of data. The multiple metadata management nodes include management nodes for managing the metadata of the directory tree of the description file and management nodes for managing the metadata of the file attributes of the description file. The processing module is used to process the data to be processed based on the target proxy node among at least one proxy node when the target node is detected to be in an abnormal state, and to obtain the processing result.

13. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the storage medium to perform the data processing method according to any one of claims 1 to 11.

14. A data processing system, characterized in that, Applied to a distributed system, the distributed system including at least one type of metadata management node, including: Processor; and A memory, connected to the processor, for providing the processor with instructions to perform the following processing steps: A request to process data to be processed is received, wherein the data to be processed is metadata; The target node is determined from multiple metadata management nodes based on the request information. The metadata management nodes of different node types process different types of data. The multiple metadata management nodes include management nodes for managing the metadata of the directory tree of the description file and management nodes for managing the metadata of the file attributes of the description file. When the target node is detected to be in an abnormal state, the data to be processed is processed based on the target proxy node in at least one proxy node to obtain the processing result.

Images