Method for processing data using intermediate device, computer system, and intermediate device

Abstract

The embodiment of the invention provides a method for performing data processing by using intermediate equipment, a computer system and the intermediate equipment. The computing system comprises a computing node, a storage node and an intermediate device. Businesses run on the computing nodes, and data of the businesses are stored on the storage nodes. The business is provided with a storage space, the business operates data of the business in a storage node by accessing the storage space, and the storage space is a persistent log space (PLOG). According to the embodiment of the invention, the metadata of the storage space is stored in the intermediate equipment, the intermediate equipment manages the metadata of the storage space, and the data operation between the computing node and the storage node is realized according to the metadata of the storage space. The intermediate device is, for example, a switch. By using the scheme provided by the invention, the load of the computing node can be reduced, and the access delay of the data can also be reduced.

Description

[0001] This application is a divisional application. The application number of the original application is 202011269034.2, and the original application date is November 13, 2020. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a method for processing data using an intermediate device, a computer system, and an intermediate device. Background Art

[0003] In a multi-node computer system, it usually includes multiple computing nodes and multiple storage nodes. In order to achieve concurrent access to the storage space of an application by multiple computing nodes, the storage space of the application is managed by the storage node in the related art. When the storage space of the application is managed by the storage node, when the computing node accesses the data of the application stored in the storage node, the load of the computing node will be increased, and the delay of data processing will be affected. For example, when the computing node needs to write the data of the application into the storage space corresponding to the application, the computing node first requests the storage node to allocate a write address for the data in the storage space corresponding to the application, and then sends a write request to the storage node to write the data to the write address after obtaining the write address. In this way, when writing data, the computing node first needs to apply to the storage node for the allocation of the write address, and after the write address is allocated, it sends a write request to the storage node to write the data to the write address, that is, the computing node needs to send at least two requests to the storage node, which not only increases the load of the computing node, but also increases the delay of data writing. Summary of the invention

[0004] The embodiments of the present application aim to provide a data processing method, a computer system, and an intermediate device, which manages the storage space of an application by connecting an intermediate device between a computing node and a storage node, thereby effectively reducing the load on the computing node and reducing the delay in data writing.

[0005] To achieve the above-mentioned purpose, the first aspect of the present application provides a computer system. The computer system includes a computing node, a storage node and an intermediate device. The computing node runs a business, and the storage node stores the data of the business. The business is allocated a storage space, and the business operates on the data of the business in the storage node by accessing the storage space. The storage space is, for example, a persistent log space (PLOG). In an embodiment of the present application, the metadata of the storage space is stored in the intermediate device, and the intermediate device manages the metadata of the storage space, and implements data operations between the computing node and the storage node based on the metadata of the storage space.

[0006] By storing the metadata of the storage space of the service on the intermediate device, so that the intermediate device manages the storage space of the service through the metadata, the load of the computing node can be reduced and the delay of data writing can be reduced. For example, when the operation is a write operation, the computing node only needs to send a write request to the intermediate device, and the intermediate device allocates an address in the storage space for the write request according to the metadata, and writes the data to the storage device according to the allocated address. Since the computing node only needs to send a request to the intermediate device, the load of the computing node is reduced and the delay of data writing is also reduced.

[0007] In an implementation of the first aspect, the metadata of the storage space includes metadata of the logical storage space, and the metadata of the logical storage space includes an identifier of the logical storage space, address information of the logical storage space, and address information of a physical storage space corresponding to the logical storage space, and the physical storage space belongs to the space in the storage node.

[0008] By recording the address information of the application's logical storage space and the address information of the physical storage space in the storage node in the metadata, the intermediate device can map operations on the logical storage space to operations on the physical storage space in the storage node, thereby implementing operations on business data stored in the physical storage space in the storage node.

[0009] In an implementation of the first aspect, multiple intermediate devices are included between the computing node and the storage node, and the computing node specifies a first intermediate device for the logical storage space among the multiple intermediate devices. The first intermediate device is used to store metadata of the logical storage space. For example, the computing node can select the first intermediate device from the multiple intermediate devices based on the identifier of the logical storage space.

[0010] In this way, multiple storage spaces of the application can be distributed to different intermediate devices, thereby achieving load balancing of the intermediate devices and improving the data processing efficiency of the application.

[0011] In an implementation manner of the first aspect, the logical storage space is a persistent log space.

[0012] In an implementation manner of the first aspect, the intermediate device is a network switching device.

[0013] Since the communication between the computing nodes and the storage nodes passes through the network switch, storing the metadata of the application's storage space in the network switch can more conveniently implement the management of the application's storage space.

[0014] In an implementation of the first aspect of the present invention, when the computing node writes business data to the storage node, the write request is first sent to the intermediate device, the intermediate device allocates a first address in the logical storage space for the data to be written, and then determines the second address of the storage node corresponding to the first address to write the data to be written, and then requests the requesting node to write the data to be written to the second address.

[0015] When writing data, since the computing node only needs to send a write request to the intermediate device, the load on the computing node is reduced and the latency of the write request is reduced.

[0016] In an implementation of the first aspect, after the computing node allocates logical storage space to the application, it will request the intermediate device to allocate physical storage space in the storage node for the logical storage space. After receiving the request, the intermediate device will request the storage node to allocate physical storage space for the logical storage space. In this way, a correspondence between the logical storage space of the application and the physical storage space in the storage node can be established, and the application can access the data in the physical storage space through the logical storage space.

[0017] In an implementation of the first aspect, the intermediate device allocates an address of a logical storage space to each write request based on the order in which the at least two write requests are received, or the intermediate device returns a completion message of each write request to the computing node based on the order in which the at least two write requests are received.

[0018] By returning the completion message of the write request or the allocation address in the order of the write request, the to-be-written data of the write request can be stored sequentially in the storage node, thus avoiding holes in the storage medium of the storage node.

[0019] In an implementation of the first aspect, after receiving the read-only setting request for the logical storage space sent by the computing node, the intermediate device sets the state of the logical storage space to a read-only state if there is no conflict in the logical storage space.

[0020] In an implementation of the first aspect, the intermediate device is also used to, after receiving a deletion request for the logical storage space sent by the computing node, set the state of the logical storage space to a deleting state if there is no conflict in the logical storage space, and instruct the storage node to delete the physical storage space corresponding to the logical storage space.

[0021] In the above two implementations, the intermediate device determines whether there are conflicting operations on the logical storage space, thereby achieving state mutual exclusion control, reducing the number of communications between computing nodes or storage nodes, and improving the processing efficiency of the computer system.

[0022] In an implementation of the first aspect, after receiving a first write request sent by the computing node, the intermediate device is also used to obtain multiple shards of the data to be written in the first write request, wherein determining to write the data to be written in the first write request to the first address of the storage node includes determining to write the multiple shards to the first addresses of multiple storage nodes respectively.

[0023] In this implementation, multiple shards of the data to be written are stored in multiple storage nodes through an intermediate device, which ensures data reliability. At the same time, the location advantage of the intermediate device as a centralized point and its high-performance message processing capabilities are utilized to reduce the amount of data that the computing nodes need to transmit while ensuring reliability.

[0024] In an implementation of the first aspect, the intermediate device allocates the second address to the first write request from unallocated space of the logical storage space in ascending order of addresses.

[0025] In an implementation of the first aspect, the storage node includes a first storage node and a second storage node, the second address corresponds to a first address in the first storage node where the data to be written is written, and a first address in the second storage node where the data to be written is written, and the requesting the storage node to write the data to be written to the first address includes: requesting the first storage node to write the data to be written to the first address in the first storage node where the data to be written is written; requesting the second storage node to write the data to be written to the first address in the second storage node where the data to be written is written.

[0026] In this implementation, the intermediate device writes data copies to multiple storage nodes to ensure data reliability. At the same time, by utilizing the network location advantages and high-performance message processing capabilities of the intermediate device, the amount of data that the computing node needs to transmit to the intermediate device is reduced while ensuring reliability.

[0027] In an implementation of the first aspect, a queue is recorded in the intermediate device, and the queue records the pending write requests for accessing the logical storage space received from the computing node in the order of reception time. When it is determined that the first write request is the earliest request received in the queue, it is determined that other write requests for the logical storage space received before the first write request have been completed.

[0028] In this implementation, by recording the order in which write requests are received in a queue, the order in which completion information of the write requests is returned can be more conveniently and accurately controlled.

[0029] In an implementation of the first aspect, the intermediate device is further used to establish a QP queue corresponding to the computing node and N QP queues corresponding to the N storage nodes respectively to maintain connection information, thereby realizing one-to-N multicast reliable communication.

[0030] In an implementation of the first aspect, when the intermediate device receives a read request to access the logical storage space, if the status of the logical storage space is in a deleting state, the intermediate device blocks the read request, or returns information that the status of the logical storage space is in a deleting state to the computing node.

[0031] The second aspect of the present invention provides a method applied to the computer system provided by the first aspect of the present invention, and the various steps included in the method are executed by the computing nodes, intermediate devices, and storage nodes of the computer system. The function corresponding to each step in the method is the same as the function executed by the computing nodes, intermediate devices, and storage nodes in the computer system, and the beneficial effects achieved are also the same, which will not be repeated here.

[0032] The third aspect of the present invention provides a data processing method performed by an intermediate device, wherein the intermediate device is connected to a computing node and a storage node, the computing node runs a business, and the storage node stores the data of the business. The intermediate device saves the metadata of the storage space corresponding to the data of the business. The intermediate device first receives a first write request for the data of the business sent by the computing node, and then determines to write the data to be written in the first write request to the first address of the storage node according to the metadata of the storage space, and after determining the first address, requests the storage node to write the data to be written to the first address.

[0033] By storing the metadata of the business's storage space on the intermediate device, the computing node only needs to send a write request to the intermediate device. The intermediate device allocates an address in the storage space for the write request based on the metadata, and writes the data to the storage device based on the allocated address. Since the computing node only needs to send a request to the intermediate device, the load on the computing node is reduced and the delay in data writing is also reduced.

[0034] The functions performed by the other steps of the method executed by the intermediate device are the same as the functions and beneficial effects performed by the intermediate device in the computer system of the first aspect, and will not be repeated here.

[0035] The fourth aspect of the present application provides a data processing device, the device is deployed in an intermediate device, the intermediate device is connected to a computing node and a storage node, the computing node runs a business, and the storage node stores the data of the business. The processing device includes several modules, each module is used to execute each step in the data processing method provided in the third aspect of the present application, and the division of modules is not limited here. For the specific functions performed by each module of the data processing device and the beneficial effects achieved, please refer to the functions of each step of the data processing method provided in the third aspect of the present application, which will not be repeated here.

[0036] According to a fifth aspect of the present application, another computer system is provided, which includes a computing node, a storage node and an intermediate device, wherein the computing node accesses the storage node through the intermediate device; an application is run on the computing node, and data of the application is stored on the storage node; the computing node is used to specify storage space for the data of the application and send a first allocation request, wherein the first allocation request is used to request the physical storage space of the storage node corresponding to the storage space; the intermediate device is used to receive the first allocation request sent by the computing node, send a second allocation request to the storage node according to the first allocation request, obtain the physical storage space allocated by the storage node, and establish and save the metadata of the application according to the information of the logical storage space and the information of the physical storage space.

[0037] By storing metadata on an intermediate device and allocating storage space in a storage node to the application's storage space, the load on the storage node and the load on the computing node can be reduced, thereby improving data transmission efficiency.

[0038] In one implementation of the fifth aspect of the present application, the first allocation request carries the identification and size of the logical storage space, and the metadata includes: the identification of the logical storage space, the size of the logical storage space, the available address of the logical storage space, and the address information of the physical storage space corresponding to the logical storage space.

[0039] In one implementation, the logical storage space is a persistent log space, and the computing node specifies an identifier and a size of the persistent log space.

[0040] In one implementation of the fifth aspect of the present application, the computing node is also used to send a first write request, which carries the first data of the application and an identifier of the logical storage space corresponding to the first data; the intermediate device is also used to receive the first write request, determine the first physical address of the first data in the storage node according to the metadata of the application, and send a second write request to the storage node according to the first physical address; the storage node is also used to receive the second write request and store the first data at the first physical address.

[0041] In one implementation of the fifth aspect of the present application, after receiving the first write request, the intermediate device allocates a first logical space address to the first data according to the identifier of the logical storage space and the size of the first data, and determines the first physical address corresponding to the first logical space address according to the correspondence between the logical storage space and the physical storage space recorded in the metadata.

[0042] In an implementation of the fifth aspect of the present application, the intermediate device is further used to receive a storage completion notification sent by the storage node, and confirm the completion of execution of the first write request according to the notification.

[0043] In one implementation of the fifth aspect of the present application, the intermediate device updates the metadata information of the application according to the first write request.

[0044] In one implementation of the fifth aspect of the present application, the computer system includes multiple intermediate devices, and the computing node is further used to select an intermediate device from the multiple intermediate devices to send the first allocation request or the first write request to the selected intermediate device.

[0045] A sixth aspect of the present application provides a computer system, the computing system comprising: a computing node, a storage node and an intermediate device, the computing node accessing the storage node through the intermediate device, the computing node being used to send a first write request to the intermediate device, the first write request accessing a logical storage space, the logical storage space corresponding to the storage space in the storage node; the intermediate device being used to receive the first write request, and upon determining that all write requests received before the first write request and accessing the logical storage space have been completed, notifying the computing node that the first write request has been completed.

[0046] By implementing in the intermediate device that write requests return write request completion messages in the order in which they are received, it is possible to prevent storage holes in storage nodes from occurring while implementing parallel processing of write requests from different computing nodes.

[0047] For other functions performed by the computing system, reference may be made to the functions provided by the computer system provided in the first aspect, which will not be described in detail here.

[0048] The seventh aspect of the present application provides an intermediate device, the intermediate device includes a processing unit and a storage unit, the storage unit stores an executable code, and when the processing unit executes the executable code, the method provided in the second aspect and the third aspect of the present application is implemented

[0049] The eighth aspect of the present application provides an intermediate device, which includes: a communication interface for data transmission with a storage node and a computing node; a processing unit for processing data received by the communication interface to execute the method provided in the second and third aspects of the present application.

[0050] The ninth aspect of the present application provides a storage medium, in which executable instructions are stored. The processor of the intermediate device executes the executable instructions in the storage medium to implement the methods provided in the second and third aspects of the present application.

[0051] The tenth aspect of the present application provides a program product, and the processor of the intermediate device controls the processor to execute the methods provided in the second and third aspects of the present application by running the program product. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The embodiments of the present application may be described more clearly by referring to the accompanying drawings:

[0053] Figure 1 This is an architecture diagram of a computer system used in the embodiments of the present application;

[0054] Figure 2 A flow chart of a method for writing data in a computer system provided in an embodiment of the present application;

[0055] Figure 3 is a flowchart of process A executed when the remaining space of PLOG i is insufficient;

[0056] Figure 4 Schematic diagram of the process of computing nodes C0 and C1 writing to PLOG j in parallel;

[0057] Figure 5 is a schematic diagram of the contents of the write request queue corresponding to PLOG j;

[0058] Figure 6 is a flowchart of process B executed when the remaining space of PLOG i is sufficient;

[0059] Figure 7 A flow chart of a method for writing data in a computer system provided by another embodiment of the present application;

[0060] Figure 8 A schematic diagram of the process of writing data in a computer system by means of multi-shard storage;

[0061] Fig. 9 A schematic diagram of the process of reading data in a computer system through multi-shard storage;

[0062] Fig.10 A flow chart of a method for reading data in a computer system provided in an embodiment of the present application;

[0063] Fig.11 A flow chart of a method for deleting data in a computer system provided in an embodiment of the present application;

[0064] Fig.12 A flow chart of a method for setting data read-only in a computer system provided in an embodiment of the present application;

[0065] Fig.13 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0066] Fig.14 A schematic diagram of the structure of an intermediate device provided in an embodiment of the present application;

[0067] Fig.15 A schematic diagram of the structure of an intermediate device provided in an embodiment of the present application;

[0068] Fig.16 Schematic diagram of a cloud service system according to an embodiment of the present application. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0070] Figure 1 FIG. 1 is an architecture diagram of a computer system 100 used in an embodiment of the present application. Figure 1 As shown in FIG. 1 , the computer system 100 includes a computing layer 12, a network layer 13 and a storage layer 14. The computing layer 12 includes a plurality of computing nodes. Figure 1 The computing nodes C0, C1 and C2 are schematically shown in FIG. 1 ; the network layer 13 includes a plurality of intermediate devices, Figure 1 The intermediate devices St0 and St1 are schematically shown in FIG. 1 ; the storage layer 14 includes a plurality of storage nodes, Figure 1The storage nodes S0, S1 and S2 are schematically shown in the figure. The computing nodes and storage nodes may be physical servers, or may be virtual entities such as virtual machines and containers that are abstracted based on general hardware resources. The intermediate device may be a programmable network connection device, such as a switch, and the switch will be described as an example of an intermediate device below. The multiple computing nodes may, for example, be used as application servers for multiple applications (APPs) to provide business processing services to users of user terminals. The multiple storage nodes may, for example, be used to store business data of APPs. Business data may be stored in storage nodes through storage media such as memory or hard disks. Since the non-volatile memory medium not only has the characteristics of large capacity and high access speed, but also has non-volatile storage characteristics, the storage medium in the embodiment of the present invention is a non-volatile memory medium, and the non-volatile memory may, for example, include storage class memory (SCM). The non-volatile memory medium is only an example, but the embodiment of the present invention is not limited to the non-volatile memory.

[0071] exist Figure 1 In the multi-node distributed computer system shown, each computing node is connected to at least one switch, and each switch is connected to multiple storage nodes, so that the computing node can access multiple storage nodes through the switch. In addition, the switches are directly or indirectly connected to each other, so that the computing node can access the storage node through any switch. Among them, the connection between the computing node and the switch, and the connection between the switch and the storage node can be various connection methods, such as remote direct memory access (RDMA) connection method, or transmission control protocol (TCP) connection method. Multiple computing nodes can write and read business data to the storage node through the switch.

[0072] In the related art, the computing node is directly connected to the storage node, and the computing node accesses the storage space in the storage node by accessing a logical storage space corresponding to the storage space in the storage node. The logical storage space is, for example, a persistence log space (PLOG). The PLOG is identified by the unique identifier PLOG ID of the PLOG, and the storage data on the PLOG is stored in the form of append write, that is, the already stored data is not overwritten and modified, but the modification is appended and stored in a new address. Usually, the PLOG corresponds to a continuous physical storage space in a medium such as the SCM, solid-state drive (SSD) in the storage node. The physical storage space is the storage space provided by the storage node to the external device, and the external device can use the storage space to access the data. For the medium such as the SCM, the physical storage space is the physical address where the data is actually stored, but for the storage medium such as the SSD, it is not the physical address where the data is actually stored.

[0073] In this related art, metadata of PLOG is stored in a storage node, and the metadata includes information such as the ID of PLOG, the address of the storage space in the storage node corresponding to the PLOG, and the address of the unallocated space of the PLOG. The storage node manages the PLOG through the metadata of PLOG. For example, when a computing node wants to write data to a storage node through PLOG, the computing node first requests the storage node to allocate an address for the data in PLOG. After the storage node allocates an address for the data in the unallocated storage space in the PLOG according to the PLOG metadata, the allocated address is returned to the computing node. After obtaining the allocated address, the computing node sends a write request to the storage node to write the data to the storage node. In this related art, in the process of writing data to the storage node, the computing node needs to communicate with the storage node multiple times, which increases the load of the computing node and increases the delay of data writing.

[0074] In the embodiments of the present application, Figure 1As shown in , the computing node is connected to the storage node through a switch. The switch, as an intermediate device, stores the metadata of the PLOG and manages the metadata of the PLOG, thereby realizing the management of the PLOG through the management of the metadata of the PLOG. The metadata of the PLOG stored in the switch includes the ID of the PLOG, the address information of the storage space in the storage node corresponding to the PLOG (for example, the starting address and length of the storage space), the address information of the unallocated space of the PLOG (for example, the starting address of the unallocated space of the PLOG), etc. When the computing node needs to write data to the PLOG in response to the user's operation on the application, the computing node sends a write request to the switch to write the data to the PLOG. After receiving the write request, the switch allocates a write address in the PLOG to the write request based on the metadata of the PLOG, determines the address of the storage node to which the data to be written in the write request is written based on the write address, and sends a request to the storage node to write the data to be written to the address in the determined storage node, so that the storage node writes the data to be written to the storage space of the storage node. In the technical implementation of the embodiment of the present application, the metadata of the PLOG is stored in the switch, so that the switch manages the PLOG. In this way, when the computing node writes data to the storage node, it only needs to send a data write request to the switch without sending an address allocation request. In this way, not only the load of the computing node is reduced, but also the latency of the write request is reduced.

[0075] In addition, after the switch manages the PLOG, it processes each write request according to the order in which the write requests are received, thereby sequentially allocating addresses in the PLOG to each write request, and after allocating the addresses, writes the data to be written in the write request to the storage node. After the storage node has written the data, the switch returns the write request according to the order in which the write requests are received, thereby preventing holes from appearing in the storage space of the storage node corresponding to the PLOG.

[0076] The following will describe in detail Figure 1 The data processing process performed by the computer system shown.

[0077] Figure 2 A flow chart of a method for writing data in a computer system provided in an embodiment of the present application.

[0078] like Figure 2 As shown, first in step S201, the computing node sends a write request 1 to PLOG i.

[0079] In this manual Figure 2 , Figure 3 , Figure 6 , Figure 7 , Fig.10 , Fig.11 and Fig.12 In the flowchart of Figure 1 Any computing node among the multiple computing nodes, the switch is the switch corresponding to the operated PLOG, and the storage node is the storage node corresponding to the PLOG. In an embodiment of the present application, a PLOG is managed by only one switch. After the computing node generates a new PLOG, the PLOG is determined to belong to a certain switch, and the determined switch is the switch corresponding to the PLOG. The metadata of the PLOG is recorded in the corresponding switch, and the PLOG is managed based on the metadata. The storage node corresponding to the PLOG is the storage node designated by the computing node for the PLOG after the computing node generates a new PLOG. The specific method for determining the switch to which the PLOG belongs can be, for example: after generating a new PLOG, the computing node can perform a hash calculation on the ID of the PLOG, thereby attributing the PLOG to a certain switch, and send the information of the PLOG to the switch so that the switch can manage the PLOG. Another method for determining the switch to which the PLOG belongs may be: after generating a new PLOG, the computing node may send the information of the PLOG to any switch to which it is connected, so that multiple switches negotiate to determine and record the mapping relationship between the PLOG and the switch. Afterwards, the switch that receives the PLOG information may forward the PLOG information to the corresponding switch based on the mapping relationship, and the corresponding switch may manage the PLOG.

[0080] The following description will be made by taking the computing node C0, the switch St0, and the storage nodes S0, S1, and S2 as examples.

[0081] Assumptions Figure 2 The computing nodes in Figure 1 The computing node C0 in the computer system. The computing node C0 is provided with a PLOG function layer, which provides a calling interface for the application, and the calling interface includes, for example, a PLOG application interface, a PLOG write interface, etc. When the application in the computing node C0 needs to write data to the computer system, first, the application obtains the ID of the currently writable PLOG that is pre-recorded from the PLOG function layer, and determines that the currently writable PLOG is PLOG i, so that the application calls the write interface in the PLOG function layer, so that the computing node C0 sends a write request 1 for PLOG i to the switch.

[0082] Specifically, the application in the computing node C0 calls the write interface, and inputs the identifier of PLOG i, the storage address of the data to be written, and the length of the data to be written. The PLOG function layer calls the data transmission interface of the network card according to the input parameters, obtains the data to be written according to the storage address of the data to be written and the length of the data to be written, and assembles a data packet of write request 1 based on the data to be written and sends it to the switch, wherein the write request 1 includes the identifier of PLOG i and the data to be written. In one implementation, the computing node C0 can determine the switch corresponding to PLOG i by performing a hash operation on "PLOG i", and send the data packet to the corresponding switch. In another implementation, the computing node C0 can send the data packet to any switch to which it is connected, and the switch forwards the data packet to the switch corresponding to PLOG i based on the mapping relationship between PLOG and switch stored locally. Assume that PLOG i corresponds to switch St0, therefore, the write request 1 for PLOG i is sent to switch St0.

[0083] In step S202, the switch determines whether the remaining space of PLOG i is sufficient.

[0084] As described above, the switch St0 corresponds to PLOG i, and therefore, the switch St0 records metadata of PLOG i to manage PLOG i through the metadata. The metadata of PLOG i includes, for example, the identifier of PLOG i, address information of PLOG i (for example, the starting address of the unallocated storage space of PLOG i and the size of PLOG i, etc.), address information of the storage space in the storage node corresponding to PLOG i, and the like. Since PLOG i is a logical storage space, the address thereof is an offset address starting from 0, and thus, the switch can obtain the remaining space size of PLOG i by subtracting the starting address of the unallocated storage space of PLOG i from the size of PLOG i.

[0085] After receiving write request 1, the switch compares the length of the data to be written with the size of the remaining space of PLOG i to determine whether the remaining space is sufficient to write the data to be written. If the remaining space is insufficient, the switch executes Figure 3 In step S301 of process A shown in FIG. 1 , if the remaining space is sufficient, the switch executes Figure 6 Step S601 in process B in FIG. Figure 3 Process A and Figure 6 The process B in is described separately.

[0086] Figure 3Flow chart of process A executed when the remaining space of PLOG i is insufficient. Process A includes the process of generating a new PLOG j and writing the data to be written into PLOG j.

[0087] In step S301, after determining that the remaining space of PLOG i is insufficient in step S202, the switch sends information that the space of PLOG i is insufficient to the computing node.

[0088] In step S302, the computing node generates PLOG j.

[0089] For example, in the computing node C0, after receiving the information that the PLOG i is insufficient in space, the application may call the PLOG application interface in the PLOG functional layer to send a request to the PLOG functional layer to apply for a new PLOG. In the application request, the storage node corresponding to the newly applied PLOG, the size of the PLOG, and the initial state of the PLOG are specified. The initial state information is, for example, a readable and writable state. In a computer system, in order to ensure the reliability of data storage, the same data may be stored in multiple storage nodes, that is, each storage node stores a copy of the data, so that when an abnormality occurs in a storage node, data can be obtained from other storage nodes. In this case, multiple storage nodes corresponding to the newly applied PLOG are specified in the above application request. For example, the above application request may specify that the newly applied PLOG corresponds to storage nodes S0, S1, and S2, that is, the data written to the PLOG is stored in storage nodes S0, S1, and S2 respectively.

[0090] After receiving the application for the new PLOG from the application, the PLOG function layer in the computing node C0 generates a new PLOGID and returns the ID to the application. The newly generated PLOGID can be represented as "PLOG j", and the ID is generated according to the numbering sequence of the PLOGs, for example, if PLOG i is PLOG 1, then "PLOG j" is determined to be PLOG 2. It can be understood that the embodiment of the present application is not limited to determining the PLOG ID according to the numbering sequence of the PLOGs, as long as the PLOG ID can uniquely identify the newly generated PLOG. For example, the ID of the previously deleted PLOG can be reused as the ID of the new PLOG.

[0091] In step S303, the computing node sends metadata of PLOG j to the switch.

[0092] After the PLOG function layer of the computing node C0 generates PLOG j as described above, in one implementation, the computing node C0 may determine the switch corresponding to PLOG j based on a predetermined rule, and send the metadata of PLOG j to the switch corresponding to PLOG j; in another implementation, the computing node C0 may send the metadata information of PLOG j to any switch connected to it, which enables multiple switches to negotiate and determine the switch corresponding to PLOG j, and send the metadata of PLOG j to the switch corresponding to PLOG j. Assume that the switch corresponding to PLOG j is switch St0, that is, the metadata of PLOG j is sent to switch St0. The sent metadata includes information such as the identifier of PLOG j, the storage nodes corresponding to PLOG j (i.e., S0, S1, and S2), the size of PLOG j, and the initial state of PLOG j.

[0093] In step S304, the switch sends a storage space allocation request to the storage node.

[0094] After receiving the metadata of PLOG j, switch St0 stores the metadata of PLOG j locally and adds the address information of the unallocated storage space of PLOG j, such as the starting address of the unallocated storage space, to the metadata. Since PLOG j has not been used yet, the starting address of the unallocated storage space of PLOG j is the default offset address 0.

[0095] The switch St0 may record the metadata of each PLOG in a table form, as shown in Table 1, which is the metadata of the PLOG managed in the switch St0.

[0096] PLOG 0 Status=Delete, Offset=200,… PLOG 1 Status=R, Offset=100,… … PLOG Status=RW, Offset=0, Volume=2M, S0: Addr= / S1: Addr= / S2: Addr=,… …

[0097] Table 1

[0098] As shown in Table 1, "Status" indicates the status of PLOG, and the status of PLOG may include readable and writable status (RW), read-only status (R), and deleting status (Delete), wherein the RW status indicates that the storage space in the storage node corresponding to PLOG is readable and writable, the R status indicates that the storage space in the storage node corresponding to PLOG is only readable and not writable, and the Delete status indicates that the storage node corresponding to PLOG is performing a deletion operation on the storage space corresponding to the PLOG. "Offset" in Table 1 indicates the starting address of the unallocated storage space of PLOG, and "Volume" indicates the size of PLOG. In addition, in the metadata of PLOG j, "S0:Addr= / S1:Addr= / S2:Addr=" indicates that PLOG j corresponds to storage nodes S0, S1, and S2, and the address (Addr) corresponding to PLOG j in storage nodes S0, S1, and S2 has not yet been determined, and is therefore empty.

[0099] Afterwards, the switch St0 sends a storage space allocation request to the corresponding storage node based on the storage node corresponding to PLOG j in the metadata of PLOG j, and the storage space allocation request includes the size of PLOG j. The size of PLOG j is, for example, 2 megabytes (M), so the storage space allocation request is used to request to allocate 2M of storage space in the storage node. In the case of multiple copies of data being written, for example, the information of PLOG j sent by the computing node C0 to the switch St0 specifies the storage nodes S0, S1, and S2, then the switch St0 sends storage space allocation requests to the storage nodes S0, S1, and S2 respectively. Figure 3 Only one storage node is shown for illustration. It is understood that in the case of multiple copies of writing, the corresponding multiple storage nodes will all perform the same Figure 3 The same operation is shown in the storage node.

[0100] In step S305, the storage node allocates storage space according to the storage space allocation request and returns storage space information.

[0101] As described above, in the case of multi-copy storage, after receiving the storage space allocation request, the storage nodes S0, S1 and S2 will respectively allocate local storage space for PLOG j and return the storage space information to the switch St0. Here, only the operation in the storage node S0 is described as an example. After receiving the storage space allocation request, the storage node S0 obtains an unused 2M continuous storage space in the local storage medium (such as SCM), marks the storage space as allocated, and returns the address information of the storage space (such as the starting address or address range) to the switch St0.

[0102] In step S306, the switch records the relationship between PLOG j and the storage space information.

[0103] Specifically, after receiving the information of the storage space, such as the starting address of the storage space, from each storage node corresponding to PLOG j, the switch St0 records it in the metadata of PLOG j. Specifically, the switch St0 can record the starting address of the storage space allocated to PLOG j in the storage nodes S0, S1 and S2 into the corresponding "Addr" fields in Table 1.

[0104] After completing the above steps, the creation process of PLOG j is completed. After the creation process is completed, the switch can notify each computing node of the newly generated PLOG j, so that each computing node can write data to the PLOG j in parallel. Figure 4 Figure 1 is a schematic diagram of the process of computing nodes C0 and C1 writing to PLOG j in parallel. In order to distinguish the writing of computing nodes C0 and C1, Figure 4 In FIG. 1 , operations related to computing node C0 are shown with solid lines, and operations related to computing node C1 are shown with dotted lines. Figure 4 As mentioned above, the metadata of PLOG j is stored in switch St0. Figure 4 In the figure, PLOG j is schematically shown as a data strip, and a storage space is allocated to PLOG j in SCM0, SCM1 and SCM2 of storage nodes S0, S1 and S2 corresponding to PLOG j, respectively. The storage space is also schematically shown as a data strip. Figure 4 describe Figure 3 Steps in .

[0105] In addition, when an application in a computing node is used for the first time, the computing node will allocate an initial PLOG to the application and allocate storage space for the initial PLOG. The process of allocating storage space to the initial PLOG is the same as the process of allocating storage space to PLOG j. Please refer to the description of steps S303-S306 for details, which will not be repeated here.

[0106] In step S307, the computing node sends a write request 1' for PLOG j to the switch.

[0107] As described above, after obtaining the newly generated PLOG j, the application in the computing node C0 can re-initiate a write request 1′ to the PLOG j to write the unsuccessfully written data to be written. The write request 1′ is used to distinguish from the write request 1 to the PLOG i. Similarly, the write request 1′ includes the identifier of the PLOG j and the data to be written.

[0108] In step S308, the switch allocates address space 1' in PLOG j for write request 1'.

[0109] When the switch allocates address space 1′ for write requests, concurrent control of write requests from multiple computing nodes can be performed. For details, please refer to Figure 4 , assuming that computing node C0 sends a write request 1′ to computing node St0 for PLOG j in operation ①, and computing node C1 sends a write request 1″ to switch St0 for PLOG j in operation ②. Switch St0 processes each write request in sequence according to the order of received write requests based on the flow line mechanism. Assuming that switch St0 receives write request 1′ first, switch St0 can first allocate address space 1′ for write request 1′ in PLOG j, where address space 1′ can also be expressed as address 1′. In general, address space is expressed by starting address plus length of address space. Therefore, when switch St0 allocates address space 1′ for write request 1′, it first obtains PLOG j from metadata of PLOG j. j, and uses the offset1 as the starting address of the write request 1′. In this way, the address space 1′ allocated to the write request 1′ is an address space with the offset1 as the starting address and the length of the data to be written. After allocating the address space 1′ to the write request 1′, the switch St0 updates the offset of PLOG j in the metadata to offset1 plus the length of the data to be written. For example, assuming that before allocating the address to the write request 1′, the value of Offset is 10, that is, 10 bytes of address space have been allocated to other write requests in PLOG j, so the starting address in the current unallocated address space of PLOG j is 10. Assuming that the length of the data to be written of the write request 1′ is 10 bytes, the write address space allocated by the switch St0 to the write request 1′ is 10 to 19. Then, the switch St0 updates the value of the variable Offset in the metadata of PLOG j to 10+10=20, that is, the current PLOG The starting address of the unallocated address space in j is 20. When St0 allocates a write address space for a write request 1″ after processing the write request 1′, the address space is allocated to the write request 1″ with the address space 20 as the starting address.

[0110] In an embodiment of the present application, the switch allocates a write address space to each write request according to the order in which the write requests are received, thereby achieving mutual exclusion of the write address space allocations for each write request. In this way, for concurrent write requests from different computing nodes, there is no need to set a mutual exclusion lock, and concurrent control of access to the same PLOG by different computing nodes can be achieved, thereby achieving higher processing efficiency.

[0111] In step S309, the switch determines an address space 2' corresponding to the address space 1' in the storage node.

[0112] In one implementation, the switch may calculate the starting address of the address space 2′ corresponding to the address space 1′ in the storage node based on the starting address offset1 of the address space 1′ and the starting address of the storage space in the storage node corresponding to PLOG j, and may determine that the address space 2′ is the length of the data to be written of the write request 1′. For the case of multiple copy writes, the switch St0 may determine the address space 2′ corresponding to the address space 1′ in the storage nodes S0, S1, and S2, respectively. For example, assuming that the starting address of the storage space corresponding to PLOG j in the storage node S0 is 100, and as described above, the starting address of the address space 1′ is 10, so that the starting address of the address space 2′ = 100 + 10 = 110. The switch St0 may similarly calculate the starting address of the address space 2′ corresponding to the address space 1′ in the storage nodes S1 and S2.

[0113] It can be understood that in the above implementation, the address space 2′ is determined by determining the starting address and address space length of the address space 2′, but the embodiments of the present application are not limited to this. Other methods of determining the address space 2′ are also within the protection scope provided by the embodiments of the present invention.

[0114] In step S310 , the switch sends a write request 2 ′ for the address space 2 ′ to the storage node.

[0115] After determining the starting address of the address space 2' corresponding to the address space 1' in the corresponding storage node, the switch generates a write request 2' for writing to the corresponding storage node. The write request 2' includes information of the address space 2' (such as the starting address) and the data to be written in the write request 1'. Specifically, refer to Figure 4 , the switch St0 generates write requests 2′ to the storage nodes S0, S1 and S2 respectively, and sends the write requests 2′ to the storage nodes S0, S1 and S2 in operations ③, ④ and ⑤ in the figure, so as to store the storage spaces ( Figure 4The data to be written in the write request 1′ is written into the address space 2′ of the data strip in the write request 1′. Among them, in the RDMA connection mode, the switch St0 maintains the connection information by establishing a QP queue corresponding to the computing node and three QP queues corresponding to the storage nodes S0, S1 and S2 respectively, so as to realize one-to-three multicast reliable communication. The switch St0 copies the data to be written in the write request 1′ into three copies of data, and generates three write requests 2′ based on the three copies of data, so that the computing node C0 only needs to send one write request to the switch St0, and achieves the effect of writing to the three storage nodes. Compared with the method in the related art in which a write request is generated by the computing node for each storage node respectively, the number of write requests issued from the computing node C0 is greatly reduced, thereby reducing the data transmission volume of the computing node C0.

[0116] It can be seen from steps S308 to S310 that since the metadata of PlOGj is stored in the switch, the computing node only needs to send a write request 1′ to the switch St0, and the switch St0 allocates an address space 1′ for the write request 1′ according to the metadata of PlOGj. After allocating the address space 1′ to PlOGj, the switch St0 does not need to return the allocated address to the computing node C0, thereby effectively reducing the load of the computing node C0 and reducing the data writing delay, thereby improving the processing efficiency of the computer system.

[0117] After sending the write request 2′ corresponding to the write request 1′, the switch St0 may record the write request 1′ in the local write request queue corresponding to PLOG j, so as to return the write request in the order in which the write request is received after the data is written to the storage node. The specific return method will be described in detail below.

[0118] After executing step S310, the switch St0 can start processing the write request 1″, that is, allocate address space 1″ in PLOG j to the write request 1″ in PLOG j, determine the starting address of address space 2″ corresponding to address space 1″ in the storage node, and send a write request 2″ for address space 2″ to the storage node, such as Figure 4 As shown in operations ⑥, ⑦ and ⑧ in the flowchart, the switch St0 can send write request 2″ to storage nodes S0, S1 and S2 respectively, so as to write the data to be written in write request 1″ to each storage node. Among them, for example, in storage node S0, after executing operation ⑥, the writing to address space 2′ and the writing to address space 2″ can be executed in parallel, without waiting for the writing to address space 2′ to be completed before executing the writing to address space 2″. Similarly, after sending write request 2″ corresponding to write request 1″, write request 1″ can be recorded in the write request queue.

[0119] Figure 5is a schematic diagram of the contents of the write request queue corresponding to PLOG j. Figure 5 As shown, the write request queue is used to record the ID (Request ID) of the write request that accesses the PLOG j but has not been completed. The switch St0 records each write request in the write request queue according to the time sequence of receiving each write request. When a new write request is received, the switch St0 records the ID of the write request to the end of the write request queue. When the write request is completed, for example, after receiving the write completion confirmation information of the write request from the storage node, in the case where it is determined that the ID of the write request is the queue head of the write request queue, the write request completion information is returned to the computing node that sent the write request, and the ID of the write request is removed from the write request queue. For example, in Figure 5 In the queue shown, the head of the queue corresponds to the earliest received write request for accessing PLOG j among the uncompleted write requests, and the tail of the queue corresponds to the latest received write request for accessing PLOG j among the uncompleted write requests. Request ID5 is, for example, the ID of write request 1′, and Request ID6 is, for example, the ID of write request 1″.

[0120] It can be understood that the form of the write request queue is not limited to Figure 5 As shown, for example, in addition to the write request ID, the queue may also include the number of copies of the write request. When the write request queue includes the number of copies of the write request, the switch can determine whether each storage node corresponding to the write request has returned the write request based on the number of copies, without the need to query the number of copies additionally. For example, the write request queue records that the number of copies of the write request 1' is 3. When the write request 1' is located at the head of the queue of the write request queue, and when the three storage nodes corresponding to the write request 1' have returned the write request, the switch St0 can return the write request 1' completion information to the computing node C0. In another embodiment, the write request queue may include the offset address of each write request. The offset address is the starting address of the address space allocated to each write request in PLOG j. As described above, the size of the offset address allocated to the write request is related to the time when the write request arrives at the switch. The earlier the write request arrives at the switch, the smaller the offset address allocated. Therefore, the offset addresses allocated to each write request can be arranged in the write request queue in ascending order, and each offset address corresponds to a write request. For example, after receiving a new write request, switch St0 records the offset address of the write request to the end of the write request queue. When switch St0 receives write request completion confirmation information from the storage node corresponding to the write request, if it is determined that the offset address of the write request is the head of the write request queue, it returns the write request completion information to the computing node that sent the write request, and removes the offset address of the write request from the write request queue.

[0121] In step S311, the storage node writes the data to be written into the address space 2′ in the storage node.

[0122] After receiving the write request 2', the storage node writes the above-mentioned to-be-written data into the address space 2' in the storage space corresponding to PLOG j in response to the write request 2'. Figure 4 , in the storage nodes S0, S1 and S2, the data to be written is written into the address space 2′ in the corresponding storage space in SCM0, SCM1 and SCM2 respectively.

[0123] In step S312, the storage node returns write request 2' completion information to the switch.

[0124] After writing the to-be-written data in the write request 1 ′ into the address space 2 ′ in their respective storage spaces, the storage nodes S0 , S1 and S2 return write request 2 ′ completion information to the switch St0 .

[0125] In step S313, the switch determines that all write requests to access PLOG j received before write request 1' have been completed.

[0126] After the switch St0 receives the completion information of the write request 2′ from the storage nodes S0, S1 and S2 respectively, it indicates that the data to be written in the write request 1′ has been written into the three storage nodes corresponding to PLOG j. By controlling the order-preserving writing of the write request accessing PLOG j in the address space of PLOG j, it is possible to prevent a hole from appearing in the storage space of the storage node corresponding to PLOG j. In the embodiment of the present invention, the order-preserving writing of the write request in the address space of PLOG j is achieved by returning the write request to the computing node according to the order in which the switch receives the write request accessing PLOG j. Since the switch St0 allocates the address space in PLOG j to each write request accessing PLOG j according to the order in which the write request is received, the address in the address space allocated to the write request received earlier is smaller. By returning the write request to the computing node according to the order in which the write request is received, that is, returning the write request according to the order of the write address of the write request from small to large, it is possible to write in the address space of PLOG j in an order-preserving manner, thereby preventing a hole from appearing in the storage space corresponding to PLOG j in the storage node.

[0127] In the embodiment of the present application, based on Figure 5The write request queue shown in the figure is used to determine that all the write requests for accessing PLOG j received before the write request 1′ have been completed. For example, the ID of the write request 1′ is Request ID5. When Request ID5 becomes the queue head of the write request queue, it indicates that the write request 1′ is the earliest received write request in the write request queue, so that it can be determined that all the write requests received before the write request 1′ have been completed. If, after receiving the completion information of the write request 2′ received by the storage nodes S0, S1 and S2, the switch St0 determines that the Request ID5 is not the queue head in the write request queue, it will wait until the Request ID5 becomes the queue head in the write request queue, and then it is determined that all the write requests for accessing PLOG j received before the write request 1′ have been completed. It can be understood that the embodiments of the present application are not limited to confirming that all the write requests for accessing PLOG j received before the write request 1′ have been completed by means of a write request queue. Other ways of determining that all the write requests for accessing PLOG j received before the write request 1′ have been completed are also within the protection scope provided by the embodiments of the present invention.

[0128] In step S14, the switch returns relevant information of the write request 1' to the computing node.

[0129] After determining that all write requests to access PLOG j received before write request 1′ have been completed, switch St0 returns relevant information of write request 1′ to computing node C0, and the relevant information includes information that write request 1′ is successfully completed and the write address in PLOG j to which write request 1′ is assigned. After returning relevant information of write request 1′ to the computing node, switch St0 deletes the ID of write request 1′ from the write request queue.

[0130] Specifically, in one implementation, after determining that all write requests before write request 1′ have been completed, switch St0 returns confirmation information to computing node C0, where the confirmation information is used to indicate that write request 1′ has been completed. After receiving the confirmation information, computing node C0 sends a read request to switch St0 to read relevant information of write request 1′. After receiving the read request, switch St0 returns successful completion information and a write address of write request 1′ to computing node C0. In another implementation, after determining that all write requests before write request 1′ have been completed, switch St0 may directly return successful completion information and a write address of write request 1′ to computing node C0.

[0131] After obtaining the successful completion information of the write request 1′, the application in the computing node C0 records the write address of the write request 1′ (i.e., PLOG j, address space 1′) in the business that generates the write request 1′, so as to facilitate subsequent businesses to read the data written to the write address. When the application in the computing node C0 needs to read the data written by the above write request 1′, it can send a read request for the address space 1′ of PLOG j to the switch St0. The switch St0 can read the data from the address space 2′ corresponding to the address space 1′ in any storage node among the storage nodes S0, S1 and S2 according to the metadata of PLOG j, and return the data to the computing node C0.

[0132] Figure 6 Flowchart of process B executed when the remaining space of PLOG i is sufficient, and process B includes a process of writing in PLOG i.

[0133] In step S601, when the switch is Figure 2 After determining that the remaining space of PLOG i is sufficient in step S202, it can be determined that the data to be written in write request 1 can be written into PLOG i. Therefore, the switch allocates address space 1 for write request 1 in PLOG i. In step S602, the switch determines address space 2 corresponding to address space 1 in the storage node based on the metadata of PLOGi. Similar to the above, the storage node is at least one storage node corresponding to PLOG i. In step S603, the switch sends write request 2 for storage space 2 to the storage node. In step S604, the storage node writes to storage space 2. In step S605, the storage node returns completion information of write request 2. In step S606, the switch determines that all write requests for accessing PLOG i received before write request 1 have been completed. In step S607, the switch returns relevant information of write request 1 to the computing node. The manner in which the data in write request 1 is written into PLOG i in the above steps S601 to S607 is similar to the above. Figure 3 The method of writing the data of the write request 1′ into PLOG j is the same as that in the above. For details, please refer to the description of steps S308 to S314 above, which will not be described in detail here.

[0134] Similar to the above, after obtaining the successful completion information of the write request 1, the application in the computing node records the write address of the write request 1 (i.e., PLOG i, address space 1) in the business that generates the write request 1 to facilitate subsequent businesses to read the data written to the write address.

[0135] The above mainly describes the data writing method provided by the embodiment of the present application by taking the multi-copy storage method as an example to ensure data reliability. The embodiment of the present application can also be applied to the scenario of ensuring data reliability by storing data in shards, such as the scenario of ensuring the reliability of written data by using the Redundant Arrays of Independent Disks (RAID) algorithm and erasure coding (EC). Figure 7 Flowchart and Figure 8 As shown in the schematic diagram.

[0136] like Figure 7 As shown, first, in step S701, the computing node sends a write request 1 to PLOG i to the switch, where the write request 1 is used to write data 1.

[0137] The write request 1 includes, for example, the identifier of PLOG i and data 1 to be written.

[0138] In step S702 , after receiving the write request 1 , the switch obtains multiple fragments of the to-be-written data of the write request 1 .

[0139] After receiving the write request 1, the switch can divide the data 1 to be written in the write request 1 into multiple data slices according to the preset EC algorithm or RAID algorithm, and calculate the check slices of the multiple data slices. Alternatively, the computing node can divide the data 1 into multiple data slices before sending the write request 1, calculate the check slices of the multiple data slices, and include the data slices and the check slices of the data 1 in the sent write request 1, so that the switch can directly obtain the multiple data slices and the check slices of the data 1 from the write request 1. Figure 8 , the switch can evenly divide data 1 into two data fragments D1 and D2 of equal size ( Figure 8 A check slice C1 ( Figure 8 ), thereby obtaining three fragments D1, D2 and C1 of data 1, and the data lengths of the three fragments are consistent. The three fragments of data 1 are in the form of 2 data fragments + 1 check fragment, so that after the three fragments are stored in three storage nodes respectively, one of the three storage nodes can be allowed to be abnormal. Specifically, data 1 can be directly spliced ​​out through D1+D2, and if the storage node storing D1 is abnormal, data 1 can be restored through D2+C1 based on the EC algorithm or RAID algorithm.

[0140] In step S703, the switch determines whether the remaining space of PLOG i is sufficient.

[0141] In this embodiment, each fragment of data 1 is stored in each storage node instead of storing data 1. Therefore, in this step, it is determined whether the remaining space of PLOG i is sufficient to store a fragment of data 1. If the space is insufficient, execute Figure 3 The process A shown can be executed if there is enough space. Figure 6 Process B shown.

[0142] When process A is executed in this method, Figure 2 The method shown is different in that the reference Figure 8 , when allocating addresses in PLOG j for write request 1′ in switch St0, addresses (address space 1′) are allocated based on the size of each shard. In addition, when storing data in each storage node, each shard is stored in address space 2′ corresponding to address space 1′ in each storage node, instead of storing data 1 in address space 2′. By storing data 1 in the storage system in this way, compared with the multi-copy storage method, the amount of data storage is reduced and storage medium resources are saved.

[0143] Similar to the above, after writing each slice of data 1 to the storage node, switch St0 returns the successful completion information and write address of write request 1' to computing node C0. Therefore, after obtaining the successful completion information of the write request 1', the application in computing node C0 records the write address of the write request 1' (i.e., PLOG j, address space 1') in the business that generates the write request 1', so as to facilitate subsequent businesses to read the data written to the write address.

[0144] Fig. 9 A schematic diagram of the process of reading data in a computer system using multi-shard storage. Fig. 9 When the application in the computing node C0 wishes to read the stored data 1, the computing node may send a read request for the address space 1′ of PLOG j to the switch St0. The switch St0 may read the data slices D1 and D2 of the data 1 from the address space 2′ corresponding to the address space 1′ of the storage nodes S0 and S1, for example, respectively, and obtain the data 1 by splicing the data slices D1 and D2, and return the data 1 to the computing node C0.

[0145] By storing the metadata of the storage space of the business on the intermediate device, the intermediate device manages the storage space of the business through the metadata. In this way, when writing data, the computing node only needs to send a write request to the intermediate device. The intermediate device allocates an address in the storage space for the write request according to the metadata, and writes the data to the storage device according to the allocated address. Since the computing node only needs to send a request to the intermediate device, the load on the computing node is reduced and the delay in data writing is also reduced.

[0146] In the embodiment of the present application, in addition to running the control logic of concurrent writing through the switch, other control processes for PLOG can also be run through the switch, which will be described below by reference Fig.10 and Fig.11 Describe the control flow to avoid concurrent conflicts between read requests and delete requests, by referring to Fig.12 Describe the control flow that avoids concurrent conflicts between write requests and read-only set requests.

[0147] Fig.10 A flow chart of a method for reading data in a computer system provided in an embodiment of the present application, wherein the method can avoid concurrent conflicts between read requests and delete requests.

[0148] like Fig.10 As shown, in step S1001, the computing node generates a read request 1, which includes PLOG i and address space 1 corresponding to the data to be read.

[0149] For example, refer to Figure 1 , the computing node C0 may receive a request for writing data (i.e., data 1) of the write request 1 from the user terminal, and then the computing node C0 may generate a read request 1 in response to the request of the user terminal. Specifically, the computing node C0 may generate the read request 1 based on the pre-recorded write address of the write request 1, and the read request 1 includes PLOD i and address space 1, and the address space 1 includes, for example, the starting address of the address space 1 and the length of the address space 1.

[0150] In step S1002 , the computing node sends a read request 1 to the switch.

[0151] As described above, for example, PLOG i corresponds to the switch St0 , and therefore, the computing node C0 sends the read request 1 to the switch St0 .

[0152] In step S1003, the switch determines that PLOG i is readable.

[0153] Referring to the description of Table 1 above, the metadata of each corresponding PLOG is recorded in the switch St0, and the metadata includes the status of the PLOG, which includes, for example, a readable and writable status, a read-only status, and a deleting status. That is, the switch St0 can determine whether PLOG i is readable by querying the status of PLOG i in the metadata of PLOG i recorded locally. If the status of PLOG i is a readable and writable status or a read-only status, PLOG i is readable and can continue to be processed. Fig.10 If the state of PLOG i is in the deleting state, PLOG i is unreadable, so the switch St0 will block the read request 1, or return the information that PLOG i is in the deleting state to the computing node C0.

[0154] In step S1004 , the switch updates the information of the read request being executed to access PLOG i according to the read request 1 .

[0155] In the switch, information about the read request being executed to access PLOG i is recorded, and the switch updates the information about the read request being executed to access PLOG i when receiving the read request and completing the read request. By recording the information about the read request being executed to access PLOG i, when the switch has not yet completed processing the read request to access PLOG i, when the switch receives a delete request for PLOG i sent by other computing nodes, the switch can block the delete request based on the information about the read request being executed to access PLOG i.

[0156] Specifically, the information of the read request being executed to access PLOG i is the number of read requests being executed by the switch to access PLOG i, which is represented by variable a. After the switch receives read request 1, it adds 1 to the value of variable a, and subtracts 1 from the value of variable a each time a read request is completed.

[0157] In step S1005, the switch determines the address space 2 in the storage space corresponding to the address space 1. This step can refer to the description of step S309 above, and will not be repeated here.

[0158] In step S1006, the switch sends a read request 2 for address space 2 to the storage node. The read request 2 includes information of address space 2, for example, the start address of address space 2 and the length of address space 2.

[0159] In step S1007 , after receiving the read request 2 , the storage node reads the address space 2 according to the read request 2 .

[0160] In step S1008, the storage node returns the read data (ie, the above-mentioned data 1) to the switch.

[0161] In step S1009, the switch updates the information of the read request being executed to access PLOG i.

[0162] After receiving the read request 2 returned from the storage node, switch St0 indicates that the read operation on address space 2 in the storage node has been completed, and updates the information of the read request being executed to access PLOG i, that is, switch St0 can reduce the value of variable a by 1.

[0163] In step S1010 , the switch returns the read data to the computing node.

[0164] The switch St0 may return the read data 1 to the computing node C0, so that the computing node C0 may return the data 1 to the user terminal.

[0165] Fig.11 A flow chart of a method for deleting data in a computer system provided in an embodiment of the present application, wherein the method can avoid concurrent conflicts between read requests and delete requests.

[0166] like Fig.11 As shown, in step S1101, the computing node sends a deletion request for PLOG i to the switch.

[0167] In one implementation, multiple upper layer applications are running in the computing node C0, and each application applies for a PLOG to store the data generated in the application. That is, in this case, the PLOG corresponds to only one application. Each PLOG of the application has a different life cycle according to the settings in the application and the type of data recorded therein. For example, PLOG i is used to record the browsing history of the user, and the application is set to keep the browsing history for one week. That is, the life cycle of PLOG i is one week, and PLOG i will be deleted after one week. Therefore, in the computing node C0, when the life cycle of PLOG i ends, a deletion request for PLOG i can be sent to the switch St0.

[0168] In addition, the computing node may also generate a deletion request for PLOG i according to the user's deletion operation.

[0169] In step S1102, the switch determines that there is no ongoing read request for PLOG i after receiving the delete request for PLOG i.

[0170] After receiving the delete request for PLOG i, the switch St0 needs to determine whether there is an ongoing read request for PLOG i. If there is an ongoing read request for PLOG i, the execution of the delete request is suspended. After suspending the execution of the delete request, for example, polling can be continued to determine whether there is an ongoing read request, that is, polling to read the value of the variable a to determine whether the value of the variable a becomes 0, and then continue to execute after determining that the value of the variable a is zero (that is, determining that there is no ongoing read request). Fig.11 , so as not to conflict with the executing read request.

[0171] In step S1103, the switch records PLOG i as being in the deleting state.

[0172] After determining that there is no ongoing read request for PLOG i, the switch St0 may record the state of PLOG i as being in a deleting (Delete) state in the metadata of PLOG i shown in Table 1, so as to block subsequent read requests for PLOG i as described above.

[0173] In step S1104, the switch determines the storage space corresponding to PLOG i.

[0174] The switch St0 can determine the storage node corresponding to PLOG i and the storage space corresponding to PLOG i in each storage node based on the metadata of PLOG i. Specifically, the storage space corresponding to PLOG i in each storage node can be determined by determining the starting address of the storage space corresponding to PLOG i in each storage node and the size of the storage space.

[0175] In step S1105, the switch sends a deletion request for the storage space to the storage node.

[0176] The switch St0 sends a deletion request for the storage space corresponding to PLOG i to the storage nodes S0, S1 and S2 corresponding to PLOG i, respectively. The deletion request includes, for example, the start address and size of each corresponding storage space in the storage nodes S0, S1 and S2.

[0177] In step S1106, the storage node deletes the storage space.

[0178] After receiving the deletion request, the storage nodes S0, S1 and S2 delete the storage space. For example, in the storage node S0, the storage space is determined based on the starting address and size of the storage space in the deletion request, the data stored in the storage space is deleted, and the record of the storage space being allocated is cancelled, so that the storage space can be reallocated to other PLOGs.

[0179] In step S1107, the storage node returns deletion success information.

[0180] After completing the deletion of the storage space respectively, the storage nodes S0, S1 and S2 return deletion success information to the switch St0 respectively.

[0181] In step S1108, the switch deletes the information of PLOG i.

[0182] Based on the information of the storage node corresponding to PLOG i recorded in the metadata of PLOG i, switch St0 can delete the record of the metadata of PLOG i in Table 1 after determining that storage nodes S0, S1 and S2 all return deletion success information, thereby completing the deletion of PLOG i.

[0183] In step S1109, the switch returns deletion success information to the computing node.

[0184] After deleting the metadata of PLOG i, the switch St0 returns deletion success information to the computing node C0, so that the computing node C0 can delete the stored relevant information of PLOG i.

[0185] In this way, after receiving the deletion request for PLOGi sent by the computing node, the intermediate device sets the status of PLOGi to the deleting state when there is no conflict with the PLOGi, for example, when there is no read request to access the PLOGi, and instructs the storage node to delete the physical storage space corresponding to the logical storage space.

[0186] Fig.12 A flow chart of a method for setting data to read-only in a computer system provided in an embodiment of the present application, wherein the method can avoid concurrent conflicts between read-only setting requests and write requests.

[0187] like Fig.12 As shown, first, in step S1201, the computing node sends a read-only setting request for PLOG i to the switch.

[0188] For example, in Figure 6 When performing a write operation on PLOG i, it is assumed that after the switch St0 executes step S603 to send a write request 2 to address space 2, due to an abnormality in the storage node, the switch St0 has not received the completion information of the write request 2 from the storage node, and therefore has not returned the completion information of the write request 1 to the computing node C0. In this case, after the computing node C0 does not receive the completion information of the write request 1 within the preset time, it can be determined that the write request 1 has failed to be executed, and a read-only setting request for PLOG i is sent to the switch St0.

[0189] In step S1202, the switch determines that there is no ongoing write request to PLOG i.

[0190] After receiving the read-only setting request for PLOG i, the switch St0 needs to first determine whether there is a write request for PLOG i being executed. If there is such a write request, the read-only setting request conflicts with the write request, and the switch St0 suspends the execution of the read-only setting request. Moreover, after determining that there is an executing write request for PLOG i, the switch St0 can continue to poll to determine whether there is an executing write request, and then perform the read-only setting request after determining that there is no executing write request.

[0191] Specifically, similar to the above, the switch St0 may execute a write request to PLOGi sent by a computing node at the beginning of each execution (eg Figure 6 After receiving the write request 1 in the switch, the information of the write request being executed to access PLOG i is updated according to the write request 1. For example, the information of the write request being executed to access PLOG i is the number of write requests being executed by the switch to access PLOG i, which is represented by variable b. After receiving the write request 1, the switch adds 1 to the value of variable b, and subtracts 1 from the value of variable b each time a write request is completed. Thus, the switch St0 can determine whether there is currently an ongoing write request to PLOG i based on the value of variable b. If the value of variable b is 0, it can be determined that there is currently no ongoing write request to PLOG i.

[0192] In step S1203, the switch changes the state of PLOG i to read-only state.

[0193] After determining that there is no write request currently being executed for PLOG i, switch St0 modifies the state of PLOG i to read-only state in the metadata of PLOG i. Therefore, when switch St0 subsequently receives a write request for PLOG i, the write request can be blocked based on the read-only state of PLOG i.

[0194] In step S1204, the switch returns read-only setting success information to the computing node.

[0195] Thus, when the intermediate device receives the read-only setting request for PLOGi sent by the computing node, if there is no conflict in PLOGi, for example, there is no write request to access PLOGi, the state of PLOGi is set to read-only state.

[0196] Refer to the above Figure 10-12As can be seen from the description of the method shown, the switch can provide consistent PLOG information to each computing node by maintaining the status information and IO information of the PLOG, and can perform concurrent logic control based on the consistent information.

[0197] In an embodiment of the present application, the control logic originally run by the computing node or storage node is unloaded to a programmable intermediate device that serves as a network aggregation point. During the process of writing data, the computing node only needs to communicate with the switch once to write data to the storage node, thereby effectively reducing the number of communications between the computing node and the switch, reducing the load on the computing node, and reducing the delay in data writing, thereby improving access efficiency.

[0198] Fig.13 The schematic diagram of the structure of a data processing device 1300 provided in an embodiment of the present application, wherein the device 1300 is deployed in an intermediate device, wherein the intermediate device is connected to a computing node and a storage node, wherein a service is run on the computing node, and data of the service is stored on the storage node; wherein the device is used to execute the above-mentioned reference Figure 2 to Figure 12 The data processing method, the device 1300 includes:

[0199] The receiving unit 131 is configured to receive a first write request for data of the service sent by the computing node;

[0200] A determining unit 132 is configured to determine to write the to-be-written data in the first write request into a first address of the storage node; and

[0201] The writing unit 133 is used to request the storage node to write the data to be written into the first address.

[0202] In one implementation, the first write request accesses the logical storage space corresponding to the data of the business, and the intermediate device stores metadata of the logical storage space. The metadata of the logical storage space includes an identifier of the logical storage space, address information of the logical storage space, and address information of a physical storage space corresponding to the logical storage space. The physical storage space belongs to the space in the storage node, and the first address is an address in the physical storage space.

[0203] In one implementation, the determination unit 132 is further used to: allocate a second address in the logical storage space for the first write request; and determine a first address for writing the data to be written in the physical storage space corresponding to the logical storage space based on the metadata of the logical storage space.

[0204] In one implementation, the receiving unit 131 is also used to receive a notification message sent by the storage node to notify that the data to be written has been written; the device 1300 also includes a notification unit 134, which is used to notify the computing node that the first write request has been completed when it is determined that other write requests for the logical storage space received before the first write request have been completed.

[0205] In one implementation, the apparatus 1300 further includes: an allocating unit 135, configured to request the storage node to allocate physical storage space for the logical storage space according to the information of the logical storage space received from the computing node.

[0206] In one implementation, the determining unit 132 is configured to allocate the second address to the first write request from an unallocated space of the logical storage space in an ascending order of addresses.

[0207] In one implementation, the storage node includes a first storage node and a second storage node, the second address corresponds to a first address in the first storage node where the data to be written is written, and a first address in the second storage node where the data to be written is written, and the write unit 133 includes: a first write sub-unit 1331, used to request the first storage node to write the data to be written to the first address in the first storage node where the data to be written is written; and a second write sub-unit 1332, used to request the second storage node to write the data to be written to the first address in the second storage node where the data to be written is written.

[0208] In one implementation, the device 1300 further includes a conflict handling unit 136 for, upon receiving a read-only setting request for the storage space from the computing node, setting the state of the storage space to a read-only state if it is determined that the logical storage space has no conflict.

[0209] In one implementation, the device 1300 also includes a conflict processing unit 136, which is used to, when receiving a deletion request for the storage space from the computing node, set the state of the logical storage space to a deleting state and request the storage node to delete the physical storage space corresponding to the logical storage space when it is determined that there is no conflict in the logical storage space.

[0210] In one implementation, the device 1300 also includes: an acquisition unit 137, which is used to obtain N slices of data to be written in the first write request after receiving the first write request sent by the computing node, and the writing unit 133 is used to respectively request N storage nodes to write the N slices to the first addresses of the N storage nodes respectively.

[0211] Fig.14 FIG. 1 is a schematic diagram of the structure of an intermediate device 1400 provided in an embodiment of the present application. Fig.14 As shown, the intermediate device 1400 includes a storage unit 141 and a processing unit 142. The storage unit 141 stores executable code. When the processing unit 142 executes the executable code, the above reference Figure 2 to Figure 12 The data processing method.

[0212] Fig.15 FIG. 1 is a schematic diagram of the structure of an intermediate device 1500 provided in an embodiment of the present application. Fig.15 As shown, the intermediate device 1500 includes: a communication interface 151, which is used to transmit data with the storage node and the computing node; a processing unit 152, which is used to process the data received by the communication interface to execute the above reference Figure 2 to Figure 12 The data processing method.

[0213] It can be understood that the computing nodes and storage nodes mentioned above can be physical servers or cloud servers (such as virtual servers).

[0214] Fig.16 FIG. 1 is a schematic diagram of a cloud service system 1600 according to an embodiment of the present application. Fig.16 , the system 1600 includes: a computing device 1601, a storage device 1602 and an intermediate device 1603. The computing device 1601 includes a hardware layer 16016 and a virtual machine monitor (VMM) 16011 running on the hardware layer 16016, and multiple virtual machines (VMs i ) 16012. Any of the virtual machines 16012 can be used as a virtual computing node of the cloud service system 1600. The storage device 1602, similar to the computing device 1601, includes a hardware layer and a virtual machine monitor (VMM) running on the hardware layer, as well as multiple virtual machines (VMs). j ), any of the virtual machines can serve as a virtual storage node of the cloud service system 1600. The composition of the computing device 1601 will be described in detail as an example.

[0215] Specifically, the virtual machine 16012 is a virtual computer (server) simulated on public hardware resources by virtual machine software, on which an operating system and application programs can be installed, and the virtual machine can also access network resources. For the application programs running in the virtual machine, the virtual machine is like working in a real computer.

[0216] The hardware layer 16016 is a hardware platform for the virtualized environment to run, and can be abstracted from the hardware resources of one or more physical hosts. The hardware layer may include a variety of hardware, for example, the hardware layer 16016 includes a processor 16014 (such as a CPU) and a memory 16015, and may also include a network card (i.e., NIC) 16013, high-speed / low-speed input / output (I / O, Input / Output) devices, and other devices with specific processing functions. Among them, the memory 16015 can be a volatile memory, such as a random-access memory (RAM) and a dynamic random-access memory (DRAM); the memory 16015 can also be a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), a solid-state drive (SSD), a storage class memory (SCM), etc.; the memory 16015 can also include a combination of the above-mentioned types of memories. The virtual machine 16012 runs an executable program based on the VMM 16011 and the hardware resources provided by the hardware layer 16016 to implement the method steps performed by the computing node in the above embodiment. For the sake of brevity, it will not be repeated here.

[0217] It should be understood that the descriptions of "first", "second", etc. in this article are only used to distinguish similar concepts for the sake of simplicity of description and do not have other limiting effects. A person of ordinary skill in the art should be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention. The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or replacements that can be easily thought of by a person of ordinary skill in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A computer system, characterized in that: It includes a computing node, a storage node and an intermediate device, wherein the computing node accesses the storage node through the intermediate device; The computing node runs the service, and the storage node stores the data of the service; The intermediate device is used to store metadata of the storage space corresponding to the data of the business, and implement data operations between the computing node and the storage node according to the metadata of the storage space, wherein the metadata of the storage space includes metadata of the logical storage space.

2. The system according to claim 1, characterized in that The metadata of the logical storage space includes an identifier of the logical storage space, address information of the logical storage space, and address information of a physical storage space corresponding to the logical storage space, where the physical storage space belongs to a space in the storage node.

3. The system according to claim 2, characterized in that There are multiple intermediate devices between the computing node and the storage node. The computing node is also used to specify a first intermediate device for the logical storage space among the multiple intermediate devices. The first intermediate device is used to store metadata of the logical storage space.

4. The system according to claim 3, characterized in that The computing node selects the first intermediate device from the multiple intermediate devices according to the identifier of the logical storage space.

5. The system according to any one of claims 2 to 4, characterized in that: The logical storage space is a persistent log space.

6. The system according to any one of claims 1 to 5, characterized in that: The intermediate device is a network switching device.

7. The system according to any one of claims 2 to 6, characterized in that: The computing node is further used to send a write request, where the write request carries the data to be written and an identifier of the logical storage space; The intermediate device is specifically used to allocate a first address in the logical storage space to the data to be written according to the identifier of the logical storage space, determine a second address in the physical storage space corresponding to the first address, and request the storage node to write the data to be written to the second address in the physical storage space; The storage node is further configured to write the data to be written into the second address of the physical storage space according to a request of the intermediate device.

8. The system according to any one of claims 2 to 7, characterized in that: The computing node is further used to send an allocation request, where the allocation request is used to request space configuration for the logical storage space, and the allocation request carries an identifier of the first logical storage space; The intermediate device is further used to request the physical storage space corresponding to the logical storage space from the storage node according to the received allocation request.

9. The system according to claim 7 or 8, characterized in that: The intermediate device receives at least two write requests from the computing node, and the intermediate device is further configured to return a message indicating that the at least two write requests have been executed based on the order in which the at least two write requests are received.

10. The system according to claim 9, characterized in that The intermediate device allocates an address of a logical storage space to each write request based on the order in which the at least two write requests are received, or the intermediate device returns a completion message of the write request sent by each computing node to each computing node that sends the write request based on the order in which the at least two write requests are received.

11. The system according to claim 7 or 8, characterized in that: The intermediate device is also used to set the state of the logical storage space to a read-only state after receiving a read-only setting request for the logical storage space sent by the computing node and if there is no conflict in the logical storage space.

12. The system according to claim 7 or 8, characterized in that: The intermediate device is also used to set the state of the logical storage space to a deleting state after receiving a deletion request for the logical storage space sent by the computing node, if there is no conflict in the logical storage space, and instruct the storage node to delete the physical storage space corresponding to the logical storage space.

13. A data processing method, characterized in that: The method is performed by an intermediate device, the intermediate device is connected to a computing node and a storage node, the computing node runs a service, the storage node stores data of the service, and the intermediate device saves metadata of a storage space corresponding to the data of the service; the method includes: Receiving a first write request for data of the service sent by the computing node; Determining, according to metadata of the storage space, to write the data to be written in the first write request to a first address of the storage node, wherein the metadata of the storage space includes metadata of a logical storage space; and Request the storage node to write the data to be written into the first address.

14. The method according to claim 13, characterized in that The metadata of the logical storage space includes an identifier of the logical storage space, address information of the logical storage space, and address information of a physical storage space corresponding to the logical storage space. The physical storage space belongs to the space in the storage node, and the first address is an address in the physical storage space.

15. The method according to claim 14, characterized in that The determining to write the to-be-written data in the first write request into the first address of the storage node comprises: Allocating a second address in the logical storage space for the first write request; A first address for writing the to-be-written data is determined in a physical storage space corresponding to the logical storage space according to the metadata of the logical storage space.

16. The method according to claim 14 or 15, characterized in that The method further comprises: Receiving a notification message sent by the storage node for notifying that the writing of the to-be-written data has been completed; Determine whether other write requests for the logical storage space received before the first write request have been completed, and if so, notify the computing node that the first write request has been completed.

17. The method according to any one of claims 14 to 16, characterized in that: Before receiving the first write request, the method further includes: Receiving information of the logical storage space from a computing node; Sending a storage space allocation request to a storage node, wherein the storage space allocation request is used to request allocation of physical storage space to the logical storage space; The information of the physical storage space allocated by the storage node to the logical storage space is received from the storage node.

18. The method according to claim 15, characterized in that The allocating a second address in the logical storage space for the first write request comprises: The second address is allocated to the first write request from the unallocated space of the logical storage space in the order of addresses from small to large.

19. The method according to any one of claims 13 to 18, characterized in that: The storage node includes a first storage node and a second storage node, the second address corresponds to a first address in the first storage node where the data to be written is written, and a first address in the second storage node where the data to be written is written, The requesting the storage node to write the to-be-written data into the first address comprises: Requesting the first storage node to write the data to be written into a first address where the data to be written is written in the first storage node; The second storage node is requested to write the data to be written into a first address where the data to be written is written in the second storage node.

20. The method according to any one of claims 14 to 19, characterized in that: Also includes: receiving a read-only setting request for the logical storage space from the computing node; When it is determined that the logical storage space has no conflict, the state of the logical storage space is set to a read-only state.

21. The method according to any one of claims 14 to 20, characterized in that: Also includes: receiving a deletion request for the logical storage space from the computing node; In the case where it is determined that the logical storage space has no conflict, setting the state of the logical storage space to a deleting state; Request the storage node to delete the physical storage space corresponding to the logical storage space.

22. The method according to claim 13, characterized in that After receiving the first write request sent by the computing node, the method further includes: obtaining N slices of the data to be written in the first write request, wherein requesting the storage node to write the data to be written to the first address includes requesting N storage nodes to respectively write the N slices to the first addresses of the N storage nodes.

23. A data processing device, characterized in that: The device is deployed in an intermediate device, the intermediate device is connected to a computing node and a storage node, the computing node runs a service, the storage node stores data of the service, and the intermediate device stores metadata of a storage space corresponding to the data of the service; The device comprises: A receiving unit, configured to receive a first write request for data of the service sent by the computing node; a determining unit, configured to determine, according to metadata of the storage space, to write the data to be written in the first write request to a first address of the storage node, wherein the metadata of the storage space includes metadata of a logical storage space; and A writing unit is used to request the storage node to write the data to be written into the first address.

24. The method according to claim 23, characterized in that The metadata of the logical storage space includes an identifier of the logical storage space, address information of the logical storage space, and address information of a physical storage space corresponding to the logical storage space. The physical storage space belongs to the space in the storage node, and the first address is an address in the physical storage space.

25. The device according to claim 24, characterized in that The determining unit is further configured to: Allocating a second address in the logical storage space for the first write request; A first address for writing the to-be-written data is determined in a physical storage space corresponding to the logical storage space according to the metadata of the logical storage space.

26. The device according to claim 24 or 25, characterized in that The receiving unit is further used to receive a notification message sent by the storage node to notify that the writing of the to-be-written data has been completed; The apparatus further includes a notification unit for notifying the computing node that the first write request has been completed when it is determined that other write requests for the logical storage space received before the first write request have been completed.

27. The device according to any one of claims 24 to 26, characterized in that: The device also includes: The allocation unit is used to request the storage node to allocate physical storage space for the logical storage space according to the information of the logical storage space received from the computing node.

28. The device according to claim 25, characterized in that The determining unit is used to allocate the second address to the first write request from the unallocated space of the logical storage space in the order of addresses from small to large.

29. The device according to any one of claims 23 to 28, characterized in that The storage node comprises a first storage node and a second storage node, the second address corresponds to a first address in the first storage node where the data to be written is written, and a first address in the second storage node where the data to be written is written; The writing unit comprises: A first writing sub-unit, used for requesting the first storage node to write the data to be written into a first address of the first storage node where the data to be written is written; The second writing sub-unit is used to request the second storage node to write the data to be written into a first address of the second storage node where the data to be written is written.

30. The device according to any one of claims 24 to 29, characterized in that: The system further comprises a conflict handling unit for, when receiving a read-only setting request for the storage space from the computing node, setting the state of the storage space to a read-only state if it is determined that the logical storage space has no conflict.

31. The device according to any one of claims 24 to 29, characterized in that: It also includes a conflict processing unit for, when receiving a deletion request for the storage space from the computing node, setting the state of the logical storage space to a deleting state and requesting the storage node to delete the physical storage space corresponding to the logical storage space if it is determined that there is no conflict in the logical storage space.

32. The device according to claim 23, characterized in that The device further includes: an acquisition unit, configured to acquire N shards of the to-be-written data in the first write request after receiving the first write request sent by the computing node; The writing unit is used to request N storage nodes to write the N slices into the first addresses of the N storage nodes respectively.

33. An intermediate device comprising: Communication interface, used for data transmission with storage nodes and computing nodes; A processing unit, used to process the data received by the communication interface to execute the method described in any one of claims 13-22.

Images