Distributed data processing systems, methods, apparatus, media, and program products
By configuring the server to manage data location relationships and utilizing cache consistency protocols and remote direct data access technology, the complexity of distributed storage systems during expansion and contraction is solved, achieving efficient system expansion and data transmission.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- LANGCHAO ELECTRONIC INFORMATION IND CO LTD
- Filing Date
- 2025-07-18
- Publication Date
- 2026-07-21
AI Technical Summary
Existing distributed storage systems require updating the entire ring structure when expanding or shrinking, which increases the complexity of system updates and makes it difficult to improve expansion and shrinkage capabilities.
By using a configuration server to manage the relationship between data and its storage location, and through cache consistency protocols and remote direct data access technology, the communication efficiency between the client server and the data server is improved, and system expansion and contraction only require changing the index mapping relationship in the configuration server.
It improves the scalability and scaling-up capabilities of distributed systems, enhances data transmission efficiency between client servers and configuration servers, and reduces CPU-intensive response time.
Smart Images
Figure CN120512433B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a distributed data processing system, method, device, medium, and program product. Background Technology
[0002] In distributed storage systems, consistent hashing can be used to achieve distributed data storage. The basic principle of consistent hashing is to construct the entire hash value space into a closed ring structure. Each node and data item is mapped to this ring through a hash function, forming a continuous ring space. When accessing data, the algorithm searches for the nearest node clockwise for processing, thus ensuring a uniform distribution of data. If the system expands or shrinks, the entire ring structure needs to be updated accordingly, increasing the complexity of system updates.
[0003] Therefore, how to improve the scaling up and down capabilities of distributed systems is a problem that needs to be solved by those skilled in the art. Summary of the Invention
[0004] In view of this, the purpose of this application is to provide a distributed data processing system, method, device, medium and program product to improve the scalability and scaling-up capabilities of distributed systems.
[0005] In a first aspect, this application provides a distributed data processing system, comprising: a client server, a configuration server, and multiple data servers; the client server and the configuration server are connected via a cache consistency protocol; the configuration server and the multiple data servers are connected via a cache consistency protocol; the client server and the multiple data servers are connected via remote direct data access technology; the configuration server is used to: record index mapping relationships of the data stored by the multiple data servers in its own network interface card memory; the index mapping relationship is the relationship between any data and the location where the data is stored; the multiple data servers are used to: store data in their own storage devices and record metadata of the data they store in their own network interface card memory; the client server is used to: access the index mapping relationships in the configuration server; and / or access the data stored by the multiple data servers.
[0006] Secondly, this application provides a distributed data processing method applied to a distributed data processing system, which includes: a client server, a configuration server, and multiple data servers; the client server and the configuration server are connected via a cache consistency protocol; the configuration server and the multiple data servers are connected via a cache consistency protocol; the client server and the multiple data servers are connected via remote direct data access technology; the distributed data processing method includes: the configuration server recording an index mapping relationship of the data stored by the multiple data servers in its own network interface card (NIC) memory; the index mapping relationship is the relationship between any data and the location where the data is stored; the multiple data servers storing data in their own storage devices and recording metadata of the data they store in their own NIC memory; the client server accessing the index mapping relationship in the configuration server; and / or accessing the data stored by the multiple data servers.
[0007] Thirdly, this application provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the aforementioned disclosed distributed data processing method.
[0008] Fourthly, this application provides a distributed data processing method applied to a client server, comprising: sending a first access request to a configuration server, so that the configuration server processes the index mapping relationship of multiple data servers stored in its own network card memory according to the first access request, and returns the corresponding processing result to the client server; the index mapping relationship is the relationship between any data and the location where the data is stored; sending a second access request to the data server corresponding to the processing result, so that the data server operates on the corresponding data stored in its own storage device and / or on the corresponding metadata recorded in its own network card memory according to the second access request, and returns the corresponding operation result to the client server; wherein, the client server and the configuration server are connected through a cache consistency protocol; the configuration server and multiple data servers are connected through a cache consistency protocol; and the client server and multiple data servers are connected through remote direct data access technology.
[0009] Fifthly, this application provides a non-volatile storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned disclosed distributed data processing method.
[0010] Sixthly, this application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed distributed data processing method.
[0011] As can be seen from the above scheme, this application provides a distributed data processing system, including: a client server, a configuration server, and multiple data servers; the client server and the configuration server are connected via a cache consistency protocol; the configuration server and multiple data servers are connected via a cache consistency protocol; the client server and multiple data servers are connected via remote direct data access technology; the configuration server is used to: record the index mapping relationship of the data stored by the multiple data servers in its own network card memory; the index mapping relationship is the relationship between any data and the location where the data is stored; the multiple data servers are used to: store data in their own storage devices and record the metadata of the data they store in their own network card memory; the client server is used to: access the index mapping relationship in the configuration server; and / or access the data stored by the multiple data servers.
[0012] Therefore, the beneficial effects of this application are as follows: By utilizing a configuration server to manage the relationship between data and its storage location, system expansion and contraction only require changing the index mapping relationship of the data involved in the configuration server, without changing the global index mapping relationship, thus improving the expansion and contraction capabilities of the distributed system; furthermore, the client server and configuration server are connected via a cache consistency protocol; the configuration server is connected to multiple data servers via a cache consistency protocol; and the client server is connected to multiple data servers via remote direct data access technology. This improves the communication efficiency between the client server, configuration server, and multiple data servers. The client server and configuration server can synchronize the index mapping relationship through the cache consistency protocol, which can improve the data transmission efficiency between the client server and configuration server. Because the network card is closer to the network, the response speed is faster, and no CPU involvement is required.
[0013] Correspondingly, the distributed data processing method, device, medium, and program product provided in this application also have the above-mentioned technical effects. Attached Figure Description
[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0015] Figure 1 This is a schematic diagram of a distributed data processing system disclosed in this application; Figure 2 This is a schematic diagram of the second type of distributed data processing system disclosed in this application; Figure 3This is a schematic diagram of the third distributed data processing system disclosed in this application; Figure 4 This is a schematic diagram of a data node server disclosed in this application; Figure 5 This is a data query flowchart disclosed in this application; Figure 6 This is a data modification flowchart disclosed in this application; Figure 7 A server architecture diagram provided for this application; Figure 8 A terminal structure diagram provided for this application. Detailed Implementation
[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.
[0017] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.
[0018] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0019] Currently, consistent hashing algorithms can be used to achieve distributed data storage in distributed storage systems. The basic principle of consistent hashing is to construct the entire hash value space into a closed ring structure. Each node and data item is mapped to this ring through a hash function, forming a continuous ring space. When accessing data, the algorithm searches for the nearest node clockwise for processing, thus ensuring a uniform distribution of data. If the system expands or shrinks, the entire ring structure needs to be updated accordingly, increasing the complexity of system updates. Therefore, this application provides a distributed data processing scheme that can improve the expansion and shrinkage capabilities of distributed systems.
[0020] See Figure 1As shown in the figure, this application discloses a distributed data processing system, including: a client server, a configuration server, and multiple data servers; the client server and the configuration server are connected through a cache consistency protocol; the configuration server and multiple data servers are connected through a cache consistency protocol; the client server and multiple data servers are connected through remote direct data access technology.
[0021] The configuration server is used to record the index mapping relationship of data stored by multiple data servers in its own network card memory. The index mapping relationship is the relationship between any data and the location where the data is stored. Specifically, the index mapping relationship is a key-value pair, where the key is calculated based on the corresponding data (i.e., the data stored by the data server), and the value is calculated based on the location where the data is stored. The location where the data is stored includes: data server identification information (which may be the data server IP address, device number, etc.) and data address information (which may be RDMA address information).
[0022] Multiple data servers are used to: store data on their own storage devices and record metadata of the stored data in their own network interface card (NIC) memory. Each data server can also periodically divide the data stored on its own storage device into cold data and hot data; record hot data to its own NIC memory and cold data to its own processor memory. The storage device can be a hard drive, etc.
[0023] The client server is used to: access the index mapping relationships in the configuration server; and / or access data stored on multiple data servers. The client server accesses the configuration server and data servers, enabling corresponding CRUD operations on the data.
[0024] In one implementation, the query operation includes: the client server receiving a query request, querying the index mapping relationship corresponding to the first data in the local machine or configuration server according to the first data to be queried in the query request; determining the storage location of the first data based on the queried index mapping relationship; and querying the first data in the data server to which the location belongs using remote direct data access technology.
[0025] In one implementation, the deletion operation includes: the client server receiving a deletion request, querying the index mapping relationship corresponding to the second data to be deleted in the local machine or configuration server according to the second data to be deleted in the deletion request; determining the storage location of the second data based on the queried index mapping relationship; and deleting the second data in the data server to which the location belongs using remote direct data access technology; correspondingly, the client server is used to: delete the index mapping relationship corresponding to the second data in the local machine and configuration server; and correspondingly, the configuration server is used to: delete the corresponding metadata in the data server to which the storage location of the second data belongs using a cache consistency protocol.
[0026] In one implementation, the modification operation includes: the client server receiving a modification request, querying the index mapping relationship corresponding to the third data in the local machine or configuration server according to the third data to be modified in the modification request; determining the storage location of the third data based on the queried index mapping relationship; and modifying the third data in the data server to which the location belongs using remote direct data access technology. Accordingly, the client server is used to: modify the index mapping relationship corresponding to the third data in the local machine and configuration server; and the configuration server is used to: modify the corresponding metadata in the data server to which the storage location of the third data belongs using a cache consistency protocol.
[0027] In one implementation, the process of adding an operation includes: the client server receiving a data addition request, adding an index mapping relationship corresponding to the fourth data according to the fourth data to be added in the data addition request, either locally or on a configuration server; determining the storage location of the fourth data based on the added index mapping relationship; adding the fourth data in the data server to which the location belongs using remote direct data access technology; and correspondingly, the configuration server is used to: add corresponding metadata in the data server to which the storage location of the fourth data belongs using a cache consistency protocol.
[0028] In one implementation, the client server is used to: synchronize the index mapping relationship in the configuration server to its own cache using a cache consistency protocol. Correspondingly, the client server is used to: query the index mapping relationship locally or in the configuration server using the cache consistency protocol. It is evident that the client server and configuration server can achieve synchronous caching of the index mapping relationship at both ends through the cache consistency protocol, and the configuration server's use of its own network interface card (NIC) memory to record the index mapping relationship further improves the data transmission efficiency between the client server and the configuration server. Correspondingly, the configuration server connects to each data server through the cache consistency protocol, achieving synchronous caching of metadata in the data servers at both ends, and the data servers' use of their own NIC memory to record metadata further improves the data transmission efficiency between the configuration server and each data server. The client server connects to multiple data servers through remote direct data access technology, enabling direct data read and write between the client server and each data server, also improving the data transmission efficiency between the client server and each data server.
[0029] When the system is expanded, the configuration server records the identification information of the new data server after detecting it. The client server then creates an index mapping relationship for the new data server on the configuration server and stores the new data in the new data server; the configuration server can also create corresponding metadata in the new data server accordingly.
[0030] To handle system downsizing, when the configuration server detects that any data server is offline, it migrates the data stored on the offline data server to other active data servers and modifies the index mapping relationships corresponding to the migrated data. This achieves the goal of improving the scalability of the distributed system by only changing the index mapping relationships of the data involved in the configuration server, without altering the global index mapping relationships.
[0031] This embodiment utilizes a configuration server to manage the relationship between data and its storage location. System expansion and contraction only require changing the index mapping relationship of the data involved in the configuration server, without altering the global index mapping relationship, thus improving the expansion and contraction capabilities of the distributed system. Furthermore, the client server connects to the configuration server via a cache consistency protocol; the configuration server connects to multiple data servers via a cache consistency protocol; and the client server connects to multiple data servers via remote direct data access technology. This improves communication efficiency between the client server, configuration server, and multiple data servers. The client server and configuration server can synchronize their index mapping relationships through the cache consistency protocol, enhancing data transmission efficiency between them. Because the network interface card (NIC) is closer to the network, its response speed is faster, eliminating the need for CPU intervention.
[0032] Please see Figure 2 The client server uses CXL technology to access the configuration node server (i.e., the configuration server), the configuration node server uses CXL technology to access the data node server (i.e., the data server), and the client server uses RDMA technology to access the data node server. Generally, one configuration node server is configured, while there may be multiple data node servers and client servers.
[0033] When a client server accesses a configuration node server, it can query the key node index mapping (i.e., the index mapping relationship) to determine which data node server to access the desired data. The configuration node server accesses the data node server to update the corresponding metadata when data is added or deleted. Both of these accesses involve small amounts of data and are used for data configuration queries or metadata updates. Client server accesses the data node server to transmit the specific business data corresponding to the key node index mapping, which can be large in volume. This business data is the data stored on the data node server, and the metadata manages the attributes and other related information of this data.
[0034] Please see Figure 3The client server CPU directly manipulates the key node index mapping on the network interface card (NIC) of the configuration node server via CXL. This means the client server CPU directly accesses the NIC memory of the configuration node server and has the ability to cache this memory. The CPU's direct access to the remote NIC memory is a CXL type 2 device feature, which improves the performance of accessing the key node index mapping because CXL has cache consistency, thus accelerating the key node index mapping cache. Specifically, the key node index mapping includes node information and the address information of the key corresponding to the value. The inclusion of the value's address information is necessary because RDMA operations require address information. Directly obtaining the address reduces CPU operations on the data node server, thereby improving performance. The client server directly accesses the key node index mapping data in the NIC memory of the configuration node server using CXL, and supports caching. In this approach, only the CPU of the client server is involved, while only the NIC of the configuration node server is involved. With caching in place, performance can be significantly improved.
[0035] The configuration node server CPU directly manipulates the metadata on the data node server's network card through CXL. In other words, the configuration node server CPU directly manipulates the network card memory of the data node server and has the ability to cache this memory.
[0036] It's important to note that the key node index mapping and metadata reside in the network interface card (NIC) memory. This is because the NIC is closer to the network, resulting in faster response times and eliminating the need for CPU intervention on the server. The NIC here doesn't necessarily refer specifically to an Ethernet card; it can also be a CXL network device that supports cross-node data transmission. Data transmission between the client server and the data node server's NICs is achieved via RDMA. RDMA allows for data transmission through pure hardware, thereby improving system performance.
[0037] Please see Figure 4 The data node server also implements a hot / cold data partitioning system, dividing business data into hot and cold data, which are stored in the network interface card (NIC) memory and the host memory, respectively. Hot data is stored in the NIC memory, and cold data is stored in the host memory. When the client server makes an RDMA data read / write request to the data node server, it directly uses the host memory address or the NIC memory address of the data node server for addressing. If the addressing is completed in the NIC memory, one DMA operation from the NIC to the host memory can be reduced, which can improve the system response speed.
[0038] Please see Figure 5The data query process includes: the client server queries the key node index mapping data through CXL, or it can directly read its own cached data for querying. After obtaining the corresponding node information and the address information of the key-value pair, the client server uses RDMA to retrieve / update the stored data of the data node server.
[0039] Please see Figure 6 The data modification process includes: the client server modifying the key node index mapping data via CXL. This modification operation can be performed jointly with the CPU of the configuration node on the key node index mapping data. Modifying the key node index mapping data causes the CPU of the configuration node server to modify the corresponding metadata on the data node server via CXL. After the modification is complete, the configuration node server updates its own key node index mapping data. Only after the mapping data is updated does the client server know which address to retrieve the data from.
[0040] The deletion process includes: the client server deletes the key node index mapping data through CXL, which causes the CPU of the configuration node server to delete the corresponding metadata on the data node server through CXL.
[0041] The following describes a distributed data processing method provided by an embodiment of this application. The distributed data processing method described below can be referred to in conjunction with other embodiments described herein.
[0042] This application discloses a distributed data processing method applied to a distributed data processing system, which includes: a client server, a configuration server, and multiple data servers; the client server and the configuration server are connected via a cache consistency protocol; the configuration server and the multiple data servers are connected via a cache consistency protocol; the client server and the multiple data servers are connected via remote direct data access technology; the distributed data processing method includes: the configuration server recording an index mapping relationship of the data stored by the multiple data servers in its own network interface card (NIC) memory; the index mapping relationship is the relationship between any data and the location where the data is stored; the multiple data servers storing data in their own storage devices and recording metadata of the data they store in their own NIC memory; the client server accessing the index mapping relationship in the configuration server; and / or accessing the data stored by the multiple data servers.
[0043] In one implementation, the client server is used to: synchronize the index mapping relationship in the configuration server to its own cache using a cache consistency protocol.
[0044] In one implementation, the client server is used to: query index mapping relationships locally or on a configuration server using a cache consistency protocol.
[0045] In one implementation, the client server is configured to: receive a query request; query the index mapping relationship corresponding to the first data to be queried according to the query request in a local or configuration server; determine the storage location of the first data based on the queried index mapping relationship; and query the first data in the data server to which the location belongs using remote direct data access technology.
[0046] In one implementation, the client server is configured to: receive a deletion request; query the index mapping relationship corresponding to the second data to be deleted in the local machine or configuration server according to the deletion request; determine the storage location of the second data based on the queried index mapping relationship; and delete the second data in the data server to which the location belongs using remote direct data access technology. Accordingly, the client server is configured to: delete the index mapping relationship corresponding to the second data in the local machine and configuration server. Accordingly, the configuration server is configured to: delete the corresponding metadata in the data server to which the storage location of the second data belongs using a cache consistency protocol.
[0047] In one implementation, the client server is configured to: receive a modification request; query the index mapping relationship corresponding to the third data to be modified in the modification request in the local machine or the configuration server; determine the storage location of the third data based on the queried index mapping relationship; and modify the third data in the data server to which the location belongs using remote direct data access technology. Accordingly, the client server is configured to: modify the index mapping relationship corresponding to the third data in the local machine and the configuration server. Accordingly, the configuration server is configured to: modify the corresponding metadata in the data server to which the storage location of the third data belongs using a cache consistency protocol.
[0048] In one implementation, the client server is configured to: receive a data addition request; add an index mapping relationship corresponding to the fourth data to be added according to the data addition request; determine the storage location of the fourth data based on the added index mapping relationship; and add the fourth data in the data server to which the location belongs using remote direct data access technology; correspondingly, the configuration server is configured to: add corresponding metadata in the data server to which the storage location of the fourth data belongs using a cache consistency protocol.
[0049] In one implementation, the index mapping relationship is a key-value pair, where the key is calculated based on the corresponding data and the value is calculated based on the location where the data is stored. The location where the data is stored includes: data server identification information and data address information.
[0050] In one implementation, multiple data servers are used to: periodically divide the data stored in their own storage devices into cold data and hot data; record the hot data to their own network card memory, and record the cold data to their own processor memory.
[0051] In one implementation, the configuration server is used to: record the identification information of the newly added data server after it is detected.
[0052] In one implementation, the configuration server is used to: migrate the data stored in the offline data server to other active data servers after detecting that any data server is offline, and modify the index mapping relationship corresponding to the migrated data.
[0053] For more detailed information on the working process of each module and unit in this embodiment, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.
[0054] As can be seen, this embodiment provides a distributed data processing method that improves the communication efficiency between the client server, configuration server, and multiple data servers. The client server and configuration server can synchronize index mapping relationships through a cache consistency protocol, thereby improving data transmission efficiency between them. Because the network interface card (NIC) is closer to the network, its response speed is faster, and no CPU involvement is required.
[0055] The following describes another distributed data processing method provided by an embodiment of this application. The distributed data processing method described below can be referred to in conjunction with other embodiments described herein.
[0056] This application discloses a distributed data processing method applied to a client server, including: S1. Send a first access request to the configuration server so that the configuration server processes the index mapping relationship of multiple data servers stored in its own network card memory according to the first access request, and returns the corresponding processing result to the client server; the index mapping relationship is the relationship between any data and the location where the data is stored.
[0057] S2. Send a second access request to the data server corresponding to the processing result, so that the data server can operate on the corresponding data stored in its own storage device and / or the corresponding metadata recorded in its own network card memory according to the second access request, and return the corresponding operation result to the client server.
[0058] Specifically, the client server and the configuration server are connected via a cache consistency protocol; the configuration server is connected to multiple data servers via a cache consistency protocol; and the client server is connected to multiple data servers via remote direct data access technology.
[0059] In this embodiment, the first access request can be a request to add, delete, modify, or query the index mapping relationship; correspondingly, the second access request is used to add, delete, modify, or query the corresponding data and / or corresponding metadata stored in the data server. For example, the first access request can be a query request for first data, a deletion request for second data, a modification request for third data, or a addition request for fourth data. For details, please refer to the relevant descriptions in other embodiments, which will not be repeated here.
[0060] The following describes an electronic device provided by an embodiment of this application. The electronic device described below can be referred to in conjunction with other embodiments described herein.
[0061] This application discloses an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the method disclosed in any of the above embodiments.
[0062] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: record the index mapping relationship of data stored by multiple data servers in its own network card memory; the index mapping relationship is the relationship between any data and the location where the data is stored.
[0063] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: storing data in its own storage device and recording the metadata of the stored data in its own network card memory.
[0064] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: accessing the index mapping relationship in the configuration server; and / or accessing the data stored by multiple data servers.
[0065] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: using a cache coherence protocol to synchronize the index mapping relationship in the configuration server to its own cache.
[0066] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: using a cache coherence protocol to query the index mapping relationship locally or on a configuration server.
[0067] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: receiving a query request, querying the index mapping relationship corresponding to the first data in the local machine or configuration server according to the first data to be queried in the query request; determining the storage location of the first data based on the queried index mapping relationship; and querying the first data in the data server to which the location belongs using remote direct data access technology.
[0068] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: receiving a deletion request, querying the index mapping relationship corresponding to the second data in the local machine or configuration server according to the second data to be deleted in the deletion request; determining the storage location of the second data based on the queried index mapping relationship; and deleting the second data in the data server to which the location belongs using remote direct data access technology.
[0069] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: delete the index mapping relationship corresponding to the second data in the local machine and the configuration server.
[0070] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: using a cache coherence protocol to delete the corresponding metadata in the data server to which the second data is stored.
[0071] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: receiving a modification request; querying the index mapping relationship corresponding to the third data in the local machine or configuration server according to the third data to be modified in the modification request; determining the storage location of the third data based on the queried index mapping relationship; and modifying the third data in the data server to which the location belongs using remote direct data access technology.
[0072] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: modifying the index mapping relationship corresponding to the third data in the local machine and the configuration server.
[0073] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: modify the corresponding metadata in the data server to which the third data is stored using a cache coherence protocol.
[0074] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: receiving a data addition request; adding an index mapping relationship corresponding to the fourth data according to the data addition request; determining the storage location of the fourth data based on the added index mapping relationship; and adding the fourth data in the data server to which the location belongs using remote direct data access technology.
[0075] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: using a cache coherence protocol to add corresponding metadata to the data server to which the fourth data is stored.
[0076] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: periodically divide the data stored in its own storage device into cold data and hot data; record the hot data to its own network card memory and record the cold data to its own processor memory.
[0077] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: after detecting a new data server, record the identification information of the new data server.
[0078] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: after detecting that any data server is offline, migrate the data stored in the offline data server to other active data servers, and modify the index mapping relationship corresponding to the migrated data.
[0079] Furthermore, embodiments of this application also provide an electronic device. The aforementioned electronic device can be, for example,... Figure 7 The server shown can also be as follows: Figure 8 The terminal shown. Figure 7 and Figure 8 These are all diagrams illustrating the structure of an electronic device according to an exemplary embodiment. The content in the diagrams should not be considered as any limitation on the scope of this application.
[0080] Figure 7 This is a schematic diagram of a server structure provided in an embodiment of this application. The server may specifically include: at least one processor, at least one memory, a power supply, a communication interface, an input / output interface, and a communication bus. The memory stores a computer program, which is loaded and executed by the processor to implement the relevant steps in the distributed data processing disclosed in any of the foregoing embodiments.
[0081] In this embodiment, the power supply is used to provide operating voltage for each hardware device on the server; the communication interface can create a data transmission channel between the server and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0082] In addition, the memory, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system, computer programs and data, etc., and the storage method can be temporary storage or permanent storage.
[0083] The operating system manages and controls the various hardware devices and computer programs on the server to enable the processor to perform operations and processes on the data in the memory. It can be Windows Server, Netware, Unix, Linux, etc. In addition to computer programs capable of performing the distributed data processing method disclosed in any of the foregoing embodiments, the computer programs may further include computer programs capable of performing other specific tasks. The data may include application update information and application developer information.
[0084] Figure 8 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. The terminal may include, but is not limited to, a smartphone, tablet computer, laptop computer, or desktop computer.
[0085] Typically, the terminal in this embodiment includes a processor and a memory.
[0086] The processor may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor can be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor may also include a main processor and coprocessors. The main processor, also known as the CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which handles computational operations related to machine learning.
[0087] The memory may include one or more computer non-volatile storage media, which may be non-transitory. The memory may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory is used to store at least the following computer programs, which, after being loaded and executed by the processor, are capable of implementing the relevant steps in the distributed data processing method executed on the terminal side as disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory may also include operating systems and data, and the storage method may be temporary or permanent storage. The operating system may include Windows, Unix, Linux, etc. The data may include, but is not limited to, application update information.
[0088] In some embodiments, the terminal may further include a display screen, an input / output interface, a communication interface, a sensor, a power supply, and a communication bus.
[0089] Those skilled in the art will understand that Figure 8 The structure shown does not constitute a limitation on the terminal and may include more or fewer components than illustrated.
[0090] The following describes a non-volatile storage medium provided in an embodiment of this application. The non-volatile storage medium described below can be referred to in conjunction with other embodiments described herein.
[0091] A non-volatile storage medium is provided for storing a computer program, wherein the computer program, when executed by a processor, implements the distributed data processing method disclosed in the foregoing embodiments. The non-volatile storage medium is a computer-readable non-volatile storage medium, which, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon include an operating system, computer programs, and data, and the storage method can be temporary storage or permanent storage.
[0092] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: record the index mapping relationship of data stored by multiple data servers in its own network card memory; the index mapping relationship is the relationship between any data and the location where the data is stored.
[0093] In this embodiment, when the processor executes a computer program stored in a non-volatile storage medium, it can specifically implement the following steps: storing data in its own storage device and recording the metadata of the stored data in its own network card memory.
[0094] In this embodiment, when the processor executes a computer program stored in a non-volatile storage medium, it can specifically implement the following steps: accessing the index mapping relationship in the configuration server; and / or accessing data stored by multiple data servers.
[0095] In this embodiment, when the processor executes a computer program stored in a non-volatile storage medium, it can specifically implement the following steps: using a cache coherence protocol to synchronize the index mapping relationship in the configuration server to its own cache.
[0096] In this embodiment, when the processor executes a computer program stored in a non-volatile storage medium, it can specifically implement the following steps: using a cache coherence protocol to query the index mapping relationship locally or on a configuration server.
[0097] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: receiving a query request, querying the index mapping relationship corresponding to the first data in the local machine or configuration server according to the first data to be queried in the query request; determining the storage location of the first data based on the queried index mapping relationship; and querying the first data in the data server to which the location belongs using remote direct data access technology.
[0098] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: receiving a deletion request, querying the index mapping relationship corresponding to the second data in the local machine or configuration server according to the second data to be deleted in the deletion request; determining the storage location of the second data based on the queried index mapping relationship; and deleting the second data in the data server to which the location belongs using remote direct data access technology.
[0099] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: delete the index mapping relationship corresponding to the second data in the local machine and the configuration server.
[0100] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: using a cache coherence protocol to delete the corresponding metadata in the data server to which the second data is stored.
[0101] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: receiving a modification request; querying the index mapping relationship corresponding to the third data in the local machine or configuration server according to the third data to be modified in the modification request; determining the storage location of the third data based on the queried index mapping relationship; and modifying the third data in the data server to which the location belongs using remote direct data access technology.
[0102] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: modifying the index mapping relationship corresponding to the third data in the local machine and the configuration server.
[0103] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: modify the corresponding metadata in the data server to which the third data is stored using a cache coherence protocol.
[0104] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: receiving a data addition request; adding an index mapping relationship corresponding to the fourth data according to the data addition request; determining the storage location of the fourth data based on the added index mapping relationship; and adding the fourth data in the data server to which the location belongs using remote direct data access technology.
[0105] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: using a cache coherence protocol to add corresponding metadata to the data server to which the fourth data is stored.
[0106] In this embodiment, when the processor executes a computer program stored in a non-volatile storage medium, it can specifically implement the following steps: periodically dividing the data stored in its own storage device into cold data and hot data; recording the hot data to its own network card memory and recording the cold data to its own processor memory.
[0107] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: after detecting a new data server, record the identification information of the new data server.
[0108] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: after detecting that any data server is offline, migrate the data stored in the offline data server to other non-offline data servers, and modify the index mapping relationship corresponding to the migrated data.
[0109] The following describes a computer program product provided by an embodiment of this application. The computer program product described below can be referred to in conjunction with other embodiments described herein.
[0110] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed distributed data processing method.
[0111] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the steps in any of the above embodiments.
[0112] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0113] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of non-volatile storage medium known in the art.
[0114] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A distributed data processing system, characterized in that, include: Client server, configuration server, and multiple data servers; The processor of the client server and the network card of the configuration server are connected through a cache consistency protocol to realize synchronous caching of the index mapping relationship at both ends; The client server's processor directly operates the network card memory of the configuration server through a cache coherence protocol and has the ability to cache that memory. The processor of the configuration server is connected to the network interface card of the multiple data servers through a cache consistency protocol to realize synchronous caching of metadata at both ends; The processor of the configuration server directly operates the network card memory of the data server and has the ability to cache that memory; The network interface cards of the client server and the multiple data servers are connected via remote direct data access technology, reducing the processor operation of the data servers; The network interface card (NIC) is an Ethernet NIC or a CXL network device that supports cross-node data transmission; The configuration server is used to: record the index mapping relationship of the data stored by the multiple data servers in its own network card memory; the index mapping relationship is the relationship between any data and the location where the data is stored; The multiple data servers are used to: store data in their own storage devices and record metadata of the data they store in their own network card memory; The client server is used to: access the index mapping relationship in the configuration server; and / or access the data stored by the plurality of data servers; The multiple data servers are used to: periodically divide the data stored in their own storage devices into cold data and hot data; record the hot data into their own network card memory, and record the cold data into their own processor memory; When the client server makes an RDMA data read / write request to the data node server, it directly addresses the data node server through the host memory address or network card memory address. The configuration server is used to: record the identification information of the newly added data server after detecting it; the client server creates an index mapping relationship for the newly added data server on the configuration server and stores the newly added data in the newly added data server; the configuration server creates corresponding metadata in the newly added data server; wherein, the client server and the configuration server synchronize the index mapping relationship through a cache consistency protocol. The configuration server is used to: migrate the data stored in the offline data server to other active data servers after detecting that any data server is offline, and modify the index mapping relationship corresponding to the migrated data, without changing the global index mapping relationship.
2. The distributed data processing system according to claim 1, characterized in that, The client server is used to: synchronize the index mapping relationship in the configuration server to its own cache using a cache consistency protocol.
3. The distributed data processing system according to claim 1, characterized in that, The client server is used to: query the index mapping relationship locally or in the configuration server using a cache consistency protocol.
4. The distributed data processing system according to claim 1, characterized in that, The client server is configured to: receive a query request, and query the index mapping relationship corresponding to the first data to be queried according to the query request, either locally or in the configuration server; The location where the first data is stored is determined based on the queried index mapping relationship; The first data is queried from the data server at that location using remote direct data access technology.
5. The distributed data processing system according to claim 1, characterized in that, The client server is configured to: receive a deletion request, and query the index mapping relationship corresponding to the second data in the local machine or the configuration server according to the second data to be deleted in the deletion request; The location where the second data is stored is determined based on the queried index mapping relationship; The second data was deleted from the data server at that location using remote direct data access technology. Accordingly, the client server is configured to: delete the index mapping relationship corresponding to the second data in both the local machine and the configuration server; Accordingly, the configuration server is configured to: delete the corresponding metadata in the data server to which the location where the second data is stored using a cache consistency protocol.
6. The distributed data processing system according to claim 1, characterized in that, The client server is configured to: receive a modification request, and query the index mapping relationship corresponding to the third data to be modified in the local machine or the configuration server according to the modification request; The location of the third data is determined based on the queried index mapping relationship; the third data is modified in the data server to which the location belongs using remote direct data access technology; Accordingly, the client server is configured to: modify the index mapping relationship corresponding to the third data locally and in the configuration server; Accordingly, the configuration server is used to modify the corresponding metadata in the data server to which the location where the third data is stored, using a cache consistency protocol.
7. The distributed data processing system according to claim 1, characterized in that, The client server is configured to: receive a data addition request; add an index mapping relationship corresponding to the fourth data to be added according to the data addition request; determine the storage location of the fourth data based on the added index mapping relationship; and add the fourth data in the data server to which the location belongs using remote direct data access technology. Accordingly, the configuration server is used to: add corresponding metadata to the data server to which the location where the fourth data is stored using a cache consistency protocol.
8. The distributed data processing system according to claim 1, characterized in that, The index mapping relationship is a key-value pair, where the key is calculated based on the corresponding data and the value is calculated based on the location where the data is stored. The location where the data is stored includes: data server identification information and data address information.
9. A distributed data processing method, characterized in that, This system is applied to a distributed data processing system, which includes a client server, a configuration server, and multiple data servers. The processor of the client server is connected to the network interface card (NIC) of the configuration server via a cache coherence protocol to achieve synchronous caching of index mapping relationships at both ends. The processor of the client server directly operates on the NIC memory of the configuration server via the cache coherence protocol and has the ability to cache this memory. The processor of the configuration server is connected to the NICs of the multiple data servers via a cache coherence protocol to achieve synchronous caching of metadata at both ends. The processor of the configuration server directly operates on the NIC memory of the data servers and has the ability to cache this memory. The NICs of the client server and the multiple data servers are connected via remote direct data access technology to reduce processor operations on the data servers. The NICs are Ethernet NICs or CXL network devices that support cross-node data transmission. This distributed data processing method includes: The configuration server records the index mapping relationship of the data stored by the multiple data servers in its own network card memory; the index mapping relationship is the relationship between any data and the location where the data is stored. The multiple data servers store data in their own storage devices and record metadata of the stored data in their own network card memory; The client server accesses the index mapping relationship in the configuration server; and / or accesses the data stored by the multiple data servers; The multiple data servers periodically divide the data stored in their own storage devices into cold data and hot data; record the hot data into their own network card memory and the cold data into their own processor memory; when the client server makes an RDMA data read / write request to the data node server, it directly addresses the data node server through the host memory address or network card memory address. After detecting a new data server, the configuration server records the identification information of the new data server; the client server creates an index mapping relationship for the new data server on the configuration server and stores the new data in the new data server; the configuration server creates corresponding metadata in the new data server; wherein, the client server and the configuration server synchronize the index mapping relationship through a cache consistency protocol. When the configuration server detects that any data server is offline, it migrates the data stored in the offline data server to other active data servers and modifies the index mapping relationship corresponding to the migrated data, without changing the global index mapping relationship.
10. A distributed data processing method, characterized in that, Applied to client servers, including: A first access request is sent to the configuration server, so that the configuration server processes the index mapping relationship of multiple data servers stored in its own network card memory according to the first access request, and returns the corresponding processing result to the client server; the index mapping relationship is the relationship between any data and the location where the data is stored; the network card is an Ethernet network card or a CXL network device that supports cross-node data transmission; A second access request is sent to the data server corresponding to the processing result, so that the data server can operate on the corresponding data stored in its own storage device and / or the corresponding metadata recorded in its own network card memory according to the second access request, and return the corresponding operation result to the client server; Specifically, the client server's processor is connected to the configuration server's network interface card (NIC) via a cache coherence protocol, enabling synchronous caching of index mapping relationships at both ends. The client server's processor directly operates on the configuration server's NIC memory via the cache coherence protocol and has the ability to cache that memory. The configuration server's processor is connected to the NICs of the multiple data servers via a cache coherence protocol, enabling synchronous caching of metadata at both ends. The configuration server's processor directly operates on the data server's NIC memory and has the ability to cache that memory. The client server and the multiple data servers' NICs are connected via remote direct data access technology, reducing processor operations on the data servers. The multiple data servers periodically divide the data stored in their own storage devices into cold data and hot data; record the hot data into their own network card memory and the cold data into their own processor memory; when the client server makes an RDMA data read / write request to the data node server, it directly addresses the data node server through the host memory address or network card memory address. After detecting a new data server, the configuration server records the identification information of the new data server; the client server creates an index mapping relationship for the new data server on the configuration server and stores the new data in the new data server; the configuration server creates corresponding metadata in the new data server; wherein, the client server and the configuration server synchronize the index mapping relationship through a cache consistency protocol. When the configuration server detects that any data server is offline, it migrates the data stored in the offline data server to other active data servers and modifies the index mapping relationship corresponding to the migrated data, without changing the global index mapping relationship.
11. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the method as described in claim 9 or 10.
12. A non-volatile storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the method as described in claim 9 or 10.
13. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method of claim 9 or 10.