Communication method, communication system and electronic equipment

Abstract

The invention discloses a communication method, a communication system and an electronic device, and relates to the technical field of information, a first communication participant and a second communication participant do not need to negotiate before communication, and only the first communication participant needs to write metadata corresponding to communication into an annular queue. And the communication server side can allocate the shared memory, namely the specified data block, required by data transmission of the communication design by calling the memory allocator. In the embodiment of the invention, the first communication participant writes the communication data in the specified data block, and the second communication participant reads the communication data from the specified data block, so that the data transmission can be completed under the conditions of zero negotiation and zero copy, the transmission overhead can be reduced, and the communication delay can be reduced.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a communication method, a communication system and electronic equipment. Background Art

[0002] Currently, inter-process communication methods based on shared memory using CXL (Compute Express Link, a high-speed serial protocol) can be extended to inter-node process communication. However, the shared memory message queue pointers and request and response storage areas that both communicating parties rely on must be negotiated before communication. This negotiation process incurs additional transmission overhead and is not practical.

[0003] Therefore, how to reduce the communication overhead of inter-node communication based on shared memory is a technical problem that those skilled in the art urgently need to solve. Summary of the Invention

[0004] The present invention provides a communication method, a communication system and an electronic device. Before communicating based on a shared memory, the two communicating parties do not need to conduct communication negotiation, which can reduce transmission overhead and communication delay.

[0005] The present invention provides a first communication method, which is applied to a communication service end, including: using a circular queue located in a metadata area in a shared memory device to provide a data writing service; reading metadata from the circular queue and determining a first communication participant who writes the metadata; using the metadata, allocating a designated data block located in a communication data area in the shared memory device to the first communication participant; and feeding back the allocation result to the first communication participant so that after the first communication participant writes the communication data into the designated data block, the second communication participant is informed to read the communication data from the designated data block.

[0006] The present invention provides a second communication method, which is applied to a communication client, including: determining request metadata corresponding to request data; writing the request metadata to a circular queue located in a metadata area in a shared memory device, so that a communication server calls a memory allocator to allocate a request data block located in a communication data area in the shared memory device to the requester based on the request metadata; receiving an allocation result fed back by the memory allocator; using the allocation result, writing the request data into the request data block; and sending a data read notification carrying the request data block address to a responder, so that the responder can read the request data from the request data block.

[0007] The present invention provides a third communication method, which is applied to a communication client, including: receiving a data read notification carrying a request data block address sent by a requester; reading the request data from the request data block located in a communication data area in a shared memory device; processing the request data to obtain response data, and using the response data to determine response metadata; writing the response metadata to a circular queue located in a metadata area in the shared memory device, so that a communication server calls a memory allocator to allocate a response data block located in the communication data area in the shared memory device to the responder based on the response metadata; receiving an allocation result fed back by the memory allocator, and using the allocation result, writing the response data into the response data block; sending a data read notification carrying the response data block address to the requester, so that the requester can read the response data from the response data block.

[0008] The present invention also provides a communication system, comprising: a communication server and a communication client; wherein the communication server executes the steps of the first communication method mentioned above; the communication client includes a requester and a responder; the steps of the second communication method mentioned above are executed in the requester; and the steps of the third communication method mentioned above are executed in the responder.

[0009] The present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any one of the above communication methods when executing the computer program.

[0010] The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above communication methods are implemented.

[0011] The present invention also provides a computer program product, comprising a computer program, which implements the steps of any one of the above communication methods when executed by a processor.

[0012] Through the present invention, since the communication server uses a circular queue located in the metadata area of ​​the shared memory device to provide data writing services, the communication participants can write metadata to the circular queue. The communication server calls the memory allocator to read the metadata from the circular queue and determines the first communication participant to write the metadata. Then, the metadata can be used to allocate a specified data block located in the communication data area of ​​the shared memory device to the first communication participant. After completing the allocation of the specified data block, the memory allocator feeds back the allocation result to the first communication participant. Based on the allocation result, the first communication participant can write the communication data to the specified data block and then inform the second communication participant to read the communication data from the specified data block. The second communication participant can then read the communication data from the specified data block. In this way, the communication data transmission between the first communication participant and the second communication participant based on the shared memory can be completed.

[0013] As can be seen, the first and second communication participants do not need to negotiate before communicating. The first communication participant only needs to write the metadata corresponding to the communication to the circular queue. The communication server then calls the memory allocator to allocate the shared memory required for the data transfer of this communication design, that is, the specified data block. The first communication participant writes the communication data to the specified data block, and the second communication participant reads the communication data from the specified data block. This completes the data transfer with zero negotiation and zero copy, which can reduce transmission overhead and communication latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0015] Figure 1 A schematic diagram of an implementation of the first communication method provided in an embodiment of the present invention; Figure 2 A schematic diagram of a communication system provided by an embodiment of the present invention; Figure 3 A schematic diagram of a request / response process provided by an embodiment of the present invention; Figure 4 A schematic diagram of memory allocation and recovery provided by an embodiment of the present invention; Figure 5 A schematic diagram of an implementation of the second communication method provided in an embodiment of the present invention; Figure 6 A schematic diagram of an implementation of a third communication method provided in an embodiment of the present invention; Figure 7A schematic structural diagram of an electronic device provided by an embodiment of the present invention; Figure 8 A schematic diagram of the specific structure of an electronic device provided by an embodiment of the present invention; Figure 9 A schematic diagram of a communication method implementation is provided. DETAILED DESCRIPTION

[0016] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0017] It should be noted that, in the description of the present invention, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. The terms "first," "second," etc., in the present invention are used to distinguish similar objects, and are not used to describe a particular order or precedence.

[0018] The solution provided by the embodiment of the present invention involves network communication. For ease of understanding, the relevant technologies are explained below.

[0019] Network communication can be used to transmit data across nodes. For example, the TCP / IP (Transmission Control Protocol / Internet Protocol) protocol suite, as the core Internet protocol, employs a layered architecture (application layer, transport layer, network layer, and data link layer). TCP provides reliable connections through a three-way handshake and timeout retransmission mechanisms, but the kernel's involvement in data processing leads to high latency. IP handles connectionless routing and relies on upper-layer protocols for reliability, while multiple data replications increase CPU overhead. HTTP (Hypertext Transfer Protocol) is built on top of TCP and uses a request-response model. HTTP / 1.1 reduces connection overhead through persistent connection multiplexing, while HTTP / 2 introduces binary framing to improve efficiency. However, the overlapping protocol stack layers still restricts real-time scenarios. RDMA (Remote Direct Memory Access) can achieve zero copy and kernel bypass through a dedicated network card, and data is transferred directly between application memories, reducing latency to microseconds; its mainstream branches include InfiniBand (high-performance dedicated network), RoCE (RDMA based on Ethernet, taking into account compatibility), etc., but it relies on expensive hardware and multi-node memory sharing requires software coordination. CXL (Compute Express Link) is a high-speed serial protocol based on PCIe. Using CXL switches, it can allow multiple hosts to share memory devices. In CXL3.0 protocol mode, multiple hosts can achieve cross-node memory sharing by mapping CXL memory areas with the same physical offset. This communication method allows multiple hosts connected to CXL devices to communicate based on shared memory. For details, please refer to Figure 9 The shared memory is divided into a message queue and several storage areas. Each item in the message queue records the offset of a storage area, and the request and response information are stored in different storage areas. Before two nodes communicate, they need to negotiate and determine two pointer items in the message queue to store the request and response during the communication process. During communication, the requester writes the request to the request storage area and waits for the response data to arrive in the response storage area. The responder reads the data in the request storage area and writes the response information to the response storage area.

[0020] That is, although these protocols can achieve network communication, they still have their own corresponding defects. The present invention improves the solution of achieving inter-node communication based on CXL shared memory, thereby achieving inter-node communication with zero negotiation, zero copy and zero core switching.

[0021] Specifically, the solution provided by the present invention is implemented based on the CS architecture in the CXL pooled storage scenario. Figure 3As shown, a CXL shared memory device is formatted into two areas: a metadata area for storing metadata and a communication data area for storing request and response data. The entire CXL memory area is managed by a global CXL memory allocator on the master (master device). During initialization, the master memory allocator maps the entire CXL physical memory space and allocates memory for subsequent master and client metadata, requests, and responses, while also reclaiming garbage collected memory.

[0022] Based on the above system, corresponding communication methods are implemented on the memory allocator and the client. The memory allocator is the server in the CS architecture, and the client is the communicating party that needs to communicate, including the requester and the responder. Because the present invention implements communication based on the CS architecture, the server is referred to as the communication server, and the client is referred to as the communication client.

[0023] Please refer to Figure 1 ,The specific implementation steps of the communication method executed in the ,communication server are as follows.

[0024] S101: Provide data writing service by utilizing a circular queue located in a metadata area of ​​a shared memory device.

[0025] In this embodiment, the shared memory in the shared memory device can be divided into a metadata area and a communication data area. The metadata area can store communication-related metadata, and the communication data area can store request data and response data required for communication transmission.

[0026] The ring queue is a producer-consumer model, where the producer is the request / response client (i.e., the communication participants, such as the requester and responder), and the consumer is the memory allocator on the master server. The stored messages are the request / response metadata themselves, including the request / response index, the source client, the target client, and the message length. Based on the ring queue, the master's memory allocator can asynchronously process the request / response metadata of multiple clients, allocating CXL memory based on the message size in the metadata to support the client request / response process.

[0027] Specifically, when the ring queue provides data writing service, the communication participants can write corresponding metadata into the ring queue according to communication requirements.

[0028] For example, if the communication participant corresponds to the requester, the request metadata may be written into the circular queue; if the communication participant corresponds to the responder, the response metadata may be written into the circular queue.

[0029] Request metadata includes the request index, source client, target client, and message length. Response metadata includes the response index, source client, target client, and message length. In practice, if there's no partitioning requirement for request and response data, this metadata won't include the request or response index.

[0030] In a specific embodiment of the present invention, a data write service is provided using a circular queue located in the metadata area of ​​a shared memory device, including: providing data write services using the circular queue and maintaining atomic operations during the metadata write process. Since the circular queue can provide data write services to clients, and the number of clients is multiple, concurrent writes from multiple clients may occur. Therefore, in this embodiment, when providing data write services, it is necessary to maintain atomic operations during the metadata write process. That is, metadata writes only correspond to two states: one is fully written, and the other is not written, and there is no half-written state. In this way, concurrency safety can be guaranteed.

[0031] Furthermore, for this circular queue, only communication participants are allowed to write metadata, but not modify or delete written metadata. Only the memory allocator is allowed to consume metadata.

[0032] S102: Read metadata from the ring queue and determine the first communication participant who writes the metadata.

[0033] The memory allocator can monitor the circular queue and, after a communication participant writes metadata to the circular queue, can read the metadata from the circular queue. This metadata can be either request metadata or response metadata. For ease of description, in this embodiment, the communication participant that writes the metadata is referred to as the first communication participant, which can be either the requester or the responder.

[0034] S103: Allocate a designated data block located in a communication data area in a shared memory device to the first communication participant using the metadata.

[0035] After the memory allocator reads the metadata from the ring queue, it can allocate a specified data block located in the communication data area of ​​the relational memory device to the first communication participant based on the metadata.

[0036] If the metadata is request metadata, the specified data block can be called a request data block; if the metadata is response metadata, the specified data block can be called a response data block.

[0037] In this embodiment, the metadata may specify the length of the request data or response data, the source and destination of the communication. Therefore, when allocating a specified data block in the shared memory based on the metadata, the data block may be explicitly specified based on the length of the request data or response data to be transferred, as well as the source and destination.

[0038] In a specific embodiment of the present invention, metadata is read from a circular queue and a first communication participant to whom the metadata is written is determined, including: reading metadata in a triplet format from the circular queue; the triplet includes a source, a target, and a length; determining a client corresponding to the source as the first communication participant; accordingly, using the metadata, allocating a specified data block located in a communication data area in a shared memory device to the first communication participant, including: determining the length of the specified data block using the length; determining the client corresponding to the target as the second communication participant; searching the communication data area for a data block that meets the length and to which the second communication participant has access rights; determining the data block as a specified data block, and allocating the specified data block to the first communication participant.

[0039] That is to say, the metadata can be specifically a triplet of the source, destination, and length of the data to be transmitted corresponding to the communication, which is written into the circular queue by the communication participants as metadata. After the shared memory device reads the metadata, a triplet is read. Based on the source in the triplet, the first communication participant can be determined, based on the destination in the triplet, the second communication participant can be determined, and based on the length, the length of the data to be transmitted can be determined. The length of the specified data block can be directly determined based on the length in the triplet. For example, if the triplet is 5M, a length of not less than 5M is used as the length of the specified data block. Then, the space to which the second communication participant has access rights is searched from the communication data area in the shared memory, and a data block with a length not less than the length in the triplet is found as the specified data block, and the specified data block is allocated to the first communication participant.

[0040] That is, in this embodiment, access rights can be divided for different clients, thereby implementing permission control for different clients, such as limiting the capacity of the shared memory used by the client for communication, or setting different communication volume limits for different clients.

[0041] S104: Feedback the allocation result to the first communication participant, so that after the first communication participant writes the communication data into the designated data block, the second communication participant is informed to read the communication data from the designated data block.

[0042] When allocating a designated data block to a first communication participant, the first communication participant needs to be informed of an allocation result, which may be an offset address and a length of the allocated designated data block.

[0043] After the allocation result is fed back to the first communication participant, the first communication participant can easily know the offset address and length of the allocated designated data block. At this time, without having to negotiate with the second communication participant, the first communication participant can determine the available shared memory, write the communication data to the designated data block, and then notify the second communication participant to read the communication data from the designated data block. After receiving the notification, the second communication participant can obtain the communication data by directly reading the designated data block. In other words, during this communication process, the communication data itself is not actually transferred. Instead, the first communication participant writes the communication data to the designated data block, and the second communication participant reads the communication data from the designated data block, thus achieving zero-copy data communication.

[0044] In a specific implementation of the present invention, feeding back the allocation result to the first communication participant includes: searching the index table to determine the control queue of the first communication participant; and writing the allocation result into the control queue.

[0045] In this embodiment, the metadata area of ​​the CXL shared memory can be formatted into several data structures, including an index table, a request queue, a response queue, a control queue, and a circular queue. The index table maintains registered online clients and records the offsets and lengths of each client's request, response, and control queues. Each online client maintains a request queue, a response queue, and a control queue. The request queue records the offset and address of data requested for the target client. The response queue records the offset and address of the corresponding data sent to the target client. The control queue is used to request / respond to the initiator client's request for CXL memory from the master, completing the request / response process. Therefore, when feeding back the allocation result to the first communication participant, the control queue of the first communication participant can be determined by searching the index table and the allocation result is written to the control queue. Specifically, the offset address and length of the specified data block are written to the control queue. This allows the first communication participant to read the offset address and length of the specified data block from its own control queue and write the communication data to the specified data block.

[0046] In a specific embodiment of the present invention, after the allocation result is fed back to the first communication participant, it also includes: monitoring the control queue of the first communication participant; the control queue records the metadata and metadata status written by the first communication participant into the ring queue; when it is determined that the metadata status corresponding to the specified data block is completed, the specified data block is marked as a data block to be recycled; when the garbage collection condition is triggered, garbage collection is performed on the data block to be recycled.

[0047] Please refer to Figure 4That is, when request / response metadata enters the ring queue, the ring queue listener thread calls the memory allocator to complete memory allocation. When request metadata in the control queue is marked as completed, the control queue listener thread calls the memory allocator to complete memory reclaim. In other words, the Master provides a global CXL memory allocator to allocate and reclaim request / response data blocks on demand, reducing memory fragmentation and improving CXL shared memory utilization.

[0048] One specific embodiment of the present invention further includes: receiving a registration request from a newly added node; and recording the newly added node's control queue, request queue, and response queue in an index table, so that the newly added node can be added to the communication domain based on the index table. As can be seen above, during communication, no pre-communication negotiation is required; instead, the memory allocator allocates the specified data blocks. Therefore, when a new node is needed, it simply registers with the master to join the communication domain, eliminating the need for inter-node negotiation. Dynamic channels can effectively improve scalability.

[0049] Because the communication server provides data writing services using a circular queue located in the metadata area of ​​the shared memory device, communication participants can write metadata to the circular queue. The communication server calls a memory allocator to read the metadata from the circular queue and identifies the first communication participant to write the metadata. The server then uses the metadata to allocate a specified data block located in the communication data area of ​​the shared memory device to the first communication participant. After allocating the specified data block, the memory allocator provides feedback to the first communication participant. Based on the allocation result, the first communication participant can write the communication data to the specified data block and then notify the second communication participant to read the communication data from the specified data block. The second communication participant can then read the communication data from the specified data block. In this way, communication data transfer between the first and second communication participants is completed based on shared memory.

[0050] As can be seen, the first and second communication participants do not need to negotiate before communicating. The first communication participant only needs to write the metadata corresponding to the communication to the circular queue. The communication server then calls the memory allocator to allocate the shared memory required for the data transfer of this communication design, that is, the specified data block. The first communication participant writes the communication data to the specified data block, and the second communication participant reads the communication data from the specified data block. This completes the data transfer with zero negotiation and zero copy, which can reduce transmission overhead and communication latency.

[0051] In other words, the requester and responder clients in this invention do not need to negotiate shared memory offsets and storage areas before transmission. They only need to specify the source host, target host, and request and response data lengths to complete the communication process. The requester and responder only need to start the receive and send threads to complete the asynchronous transmission, reception, and processing of communication information without blocking the business flow. Based on the master node to manage each client node, the CXL shared memory space is allocated and recycled to achieve dynamic channels, reduce memory fragmentation, and improve memory utilization. The master request and response ring queue and CXL atomic lock ensure the concurrent security of multi-tenant writes.

[0052] Since the client in the present invention is the communication participant, and the communication participant is divided into two roles, one is the requester and the other is the responder, the communication methods executed in the requester and the responder are respectively explained in detail below.

[0053] Please refer to Figure 5 This method can be applied to a communication client. When the communication client acts as a requester, the specific implementation steps of the communication method executed are as follows.

[0054] S201: Determine request metadata corresponding to request data.

[0055] When a communication client is the requester and has a communication requirement, it can determine the request metadata corresponding to the request data. The request data is the data content that initiates a data request to the responder. For example, it requests access to a target file or data on the responder.

[0056] In a specific embodiment of the present invention, determining the request metadata corresponding to the communication includes: determining the length according to the data volume of the request data; determining the local client as the source and the responder as the target; encapsulating the length, source and target into a triplet, and determining the triplet as the request metadata.

[0057] Both request metadata and response metadata can be triples. Their source and destination differ depending on the direction of the request or response data. For both communicating parties, the source and destination are opposite. The length of the triple corresponds to the length of the request data for request metadata and the length of the response data for response metadata.

[0058] In a specific real-time method of the present invention, the requester can establish communication with multiple responders in parallel. Specifically, when determining the request metadata, multiple request metadata can be generated based on the same request data. That is, only the responding end is different in multiple different request metadata. For example, if the request metadata is 2M, the multiple request metadata can be request 1: source A, target B, length 2M, request 2: source A, target C, length 2M, request 3: source A, target C, length 2M, request 4: source A, target D, length 2M. Among them, source A is requester A, and target B, target C, and target D can be responders B, C, and D. These multiple request metadata are written into the circular queue respectively, so that the communication service end will allocate multiple request data blocks to the requester. In this way, communication with multiple responders can be triggered. Of course, the request data can also be of different lengths.

[0059] S202: Write request metadata to the circular queue in the metadata area of ​​the shared memory device, so that the communication server calls the memory allocator to allocate the request data block in the communication data area of ​​the shared memory device to the requester based on the request metadata.

[0060] Since the corresponding ring queue of the communication server provides metadata writing service based on the ring queue, the requester can write the requested metadata into the ring queue. Figure 1 The steps of the method shown are to allocate the requested data block to the requester.

[0061] S203: Receive the allocation result fed back by the memory allocator.

[0062] The requester can receive the allocation result fed back by the memory allocator, which is the offset address and length of the requested data block.

[0063] In a specific embodiment of the present invention, receiving the allocation result fed back by the memory allocator includes: providing an allocation result writing service to the memory allocator using a control queue corresponding to the local client; and reading the allocation result from the control queue.

[0064] In this embodiment, different clients can set corresponding control queues, which can provide allocation result writing services to the memory allocator. Therefore, the allocation result can be obtained by monitoring the control queue and reading data.

[0065] S204: Using the allocation result, write the requested data into the requested data block.

[0066] The allocation result corresponds to the offset address and length of the requested data block, so the requested data can be written into the requested data block based on the offset address and length.

[0067] S205: Send a data read notification carrying the address of the requested data block to the responder, so that the responder reads the requested data from the requested data block.

[0068] After the request data is written into the request data block, the responder may be notified to read the request data block so as to transmit the request data to the responder.

[0069] Specifically, sending a data read notification carrying the requested data block address to the responder includes: writing the offset address and length of the requested data block into the request queue of the responder.

[0070] In other words, the offset address and length of the request data block are written to the responder's request queue, so that the responder can monitor the request queue and read the data in it to know that it can read the request data from the request data block. In this way, the requester completes the process of transmitting the request data to the responder.

[0071] Generally, after receiving the request data, the responder will process the request data and generate corresponding response data, which needs to be fed back to the requester. This process can be referred to another method embodiment below and will not be described in detail here.

[0072] In a specific embodiment of the present invention, after sending a data read notification carrying the address of the request data block to the responder, the method further includes: receiving the data read notification sent by the responder; the data read notification carries the offset address and length of the response data block; and reading the response data from the response data block located in the communication data area of ​​the shared memory device according to the offset address and length. In other words, after the responder processes based on the request data and clearly determines the response data that needs to be fed back, it can send a data read notification to the requester according to the method steps provided in another method embodiment described below. After the requester receives the data read notification, it can read the response data from the response data block located in the communication data area of ​​the shared memory device based on the offset address and length of the response data block corresponding to the notification. In this way, the communication process of the requester requesting data from the responder and obtaining the response data is completed.

[0073] Please refer to Figure 6 This method can be applied to a communication client. When the communication client acts as a responder, the specific implementation steps of the communication method executed are as follows.

[0074] S301: Receive a data read notification sent by a requester that carries a requested data block address.

[0075] As mentioned above, in a shared memory device, there are control queues, request queues, and response queues corresponding to different clients. The client only needs to monitor these queues to obtain the corresponding content.

[0076] In a specific embodiment of the present invention, receiving a data read notification carrying a request data block address from a first communication participant includes: monitoring a response queue and a request queue of a local client; the response queue providing a data read notification write service for response data, and the request queue providing a data read notification write service for request data; reading the offset address and length of the response data block from the response queue; and reading the offset address and length of the request data block from the request queue. In other words, the client can obtain the offset addresses and lengths of the request and response data blocks by monitoring the response and request queues and then reading the offset addresses and lengths from the corresponding queues.

[0077] In particular, since the client's control queue provides write services to the memory allocator in the server, by monitoring the control queue and reading the offset address and length therein, the specified data block allocated to its own use by the memory allocator is obtained; since the client's request queue provides write services to the requester, by monitoring the request queue and reading the offset address and length therein, the request data block with the request data already written therein is obtained; since the client's response queue provides write services to the responder, by monitoring the response queue and reading the offset address and length therein, the request data block with the response data already written therein is obtained.

[0078] S302: Read the requested data from the requested data block in the communication data area of ​​the shared memory device.

[0079] After the request data block is determined, the request data can be read from the request data block.

[0080] S303: Process the request data to obtain response data, and use the response data to determine response metadata.

[0081] The requester parses the request data to obtain the response data requested by the requester. Specifically, the response data can be data generated by computing certain data in the request data, or data obtained from local storage or local services based on the request data.

[0082] Once the response data is determined, the response metadata can be determined.

[0083] S304: Write response metadata to the circular queue in the metadata area in the shared memory device, so that the communication server calls the memory allocator to allocate a response data block in the communication data area in the shared memory device to the responder based on the response metadata.

[0084] In order to feed back the response data to the requester, in this embodiment, the responder can write the response metadata into the ring queue, so that the communication server executes the following Figure 1 The method steps shown are used to allocate the response data block to the responder.

[0085] S305: Receive the allocation result fed back by the memory allocator, and use the allocation result to write the response data into the response data block.

[0086] The responder can monitor its own control queue to obtain the allocation result fed back by the memory allocator, that is, to clearly know the offset address and length of the response data block, and based on the offset address and length, the response data can be written into the allocated response data block.

[0087] S306: Send a data read notification carrying the response data block address to the requesting party, so that the requesting party can read the response data from the response data block.

[0088] After the responder has written the response data into the response data block, it can send a data read notification to the requester. For example, the responder writes the offset address and length of the response data block into the requester's response queue. The requester can then read the response data from the response data block. This completes the communication process where the responder obtains the request data and sends the response data back to the requester.

[0089] Corresponding to the above method embodiment, an embodiment of the present invention further provides a communication system. The communication system described below and the communication method described above can refer to each other.

[0090] Please refer to Figure 2 The communication system includes: a communication server and a communication client; wherein the communication server executes the steps of the first communication method mentioned above; the communication client includes a requester and a responder; the steps of the second communication method mentioned above are executed in the requester; and the steps of the third communication method mentioned above are executed in the responder.

[0091] Among them, the first communication method includes: using a circular queue located in the metadata area of ​​a shared memory device to provide data writing services; reading metadata from the circular queue and determining the first communication participant who writes the metadata; using the metadata to allocate a designated data block located in the communication data area of ​​the shared memory device to the first communication participant; and feeding back the allocation result to the first communication participant so that after the first communication participant writes the communication data into the designated data block, the second communication participant is informed to read the communication data from the designated data block.

[0092] Among them, metadata is read from the circular queue, and the first communication participant who writes the metadata is determined, including: reading metadata in a triple format from the circular queue; the triple includes a source, a target and a length; the client corresponding to the source is determined as the first communication participant; accordingly, using the metadata, a specified data block located in a communication data area in a shared memory device is allocated to the first communication participant, including: using the length to determine the length of the specified data block; determining the client corresponding to the target as the second communication participant; searching the communication data area for a data block that meets the length and to which the second communication participant has access rights; determining the data block as a specified data block, and allocating the specified data block to the first communication participant.

[0093] Feedback of the allocation result to the first communication participant includes: searching the index table to determine the control queue of the first communication participant; and writing the allocation result into the control queue.

[0094] Among them, after the allocation result is fed back to the first communication participant, it also includes: monitoring the control queue of the first communication participant; the control queue records the metadata and metadata status written by the first communication participant into the ring queue; when it is determined that the metadata status corresponding to the specified data block is completed, the specified data block is marked as a data block to be recycled; when the garbage collection condition is triggered, the data block to be recycled is subjected to garbage collection processing.

[0095] The method provides data writing service by utilizing a circular queue in a metadata area of ​​a shared memory device, including: providing data writing service by utilizing the circular queue and maintaining atomic operation during metadata writing.

[0096] The method further includes: receiving a registration request of a newly added node; and recording a control queue, a request queue, and a response queue of the newly added node in an index table, so as to add the newly added node to the communication domain based on the index table.

[0097] Among them, the first communication method includes: determining the request metadata corresponding to the request data; writing the request metadata to the circular queue located in the metadata area of ​​the shared memory device, so that the communication server calls the memory allocator to allocate the request data block located in the communication data area of ​​the shared memory device to the requester based on the request metadata; receiving the allocation result feedback from the memory allocator; using the allocation result, writing the request data into the request data block; sending a data read notification carrying the request data block address to the responder, so that the responder can read the request data from the request data block.

[0098] Determining the request metadata corresponding to the communication includes: determining the length according to the amount of the request data; determining the local client as the source and the responder as the target; encapsulating the length, source and target into a triplet, and determining the triplet as the request metadata.

[0099] Receiving the allocation result fed back by the memory allocator includes: providing an allocation result writing service to the memory allocator using a control queue corresponding to the local client; and reading the allocation result from the control queue.

[0100] The sending of a data read notification carrying the address of the requested data block to the responder includes: writing the offset address and length of the requested data block into the request queue of the responder.

[0101] Among them, after sending a data read notification carrying the address of the request data block to the responder, it also includes: receiving the data read notification sent by the responder; the data read notification carries the offset address and length of the response data block; according to the offset address and length, reading the response data from the response data block located in the communication data area in the shared memory device.

[0102] Among them, the first communication method includes: receiving a data read notification carrying the address of a request data block sent by a requester; reading the request data from the request data block located in the communication data area of ​​a shared memory device; processing the request data to obtain response data, and using the response data to determine response metadata; writing the response metadata to a circular queue located in the metadata area of ​​the shared memory device, so that the communication server calls a memory allocator to allocate a response data block located in the communication data area of ​​the shared memory device to the responder based on the response metadata; receiving the allocation result fed back by the memory allocator, and using the allocation result, writing the response data into the response data block; sending a data read notification carrying the address of the response data block to the requester, so that the requester can read the response data from the response data block.

[0103] Among them, receiving the data read notification carrying the request data block address sent by the requesting party includes: monitoring the response queue and request queue of the local client; the response queue provides a data read notification write service for the response data, and the request queue provides a data read notification write service for the request data; reading the offset address and length of the response data block from the response queue; reading the offset address and length of the request data block from the request queue.

[0104] In actual applications, the above communication method can be implemented simultaneously on the communication server and the communication client, and is described in detail below with reference to a specific application scenario as an example.

[0105] The scenario is for communication between multi-node distributed applications. Participants are divided into requesters and responders, who access the CXL switch device together with the service provider Master, and the CXL memory is shared between the participants. Figure 2 shown.

[0106] CXL shared memory devices are formatted into two areas: metadata and request / response data. The entire CXL memory area is managed by a global CXL memory allocator process on the master. During initialization, the master memory allocator maps all of the CXL physical memory space and allocates memory for subsequent master and client metadata, requests, and responses, while also reclaiming garbage memory.

[0107] Based on the above system, this section will explain in detail the communication method between client nodes.

[0108] like Figure 3 CXL shared memory metadata is formatted into several data structures: index table, request queue, response queue, control queue, and ring queue. The index table maintains the registered online clients and records the offset and length of each client's request, response, and control queue.

[0109] Each online client maintains a request queue, a response queue, and a control queue. The request queue records the offset and address of the data requested by the target client. The response queue records the offset and address of the corresponding data sent to the target client. The control queue is used to request / respond to the initiator client's request for CXL memory from the master, completing the request / response process.

[0110] A ring queue is a producer-consumer model, where the producer is the request / response client and the consumer is the master server. The stored messages are the request / response metadata, including the request / response index, the source client (the requester), the target client (the responder), and the message length. Based on the ring queue, the master can asynchronously process request / response metadata from multiple clients, allocating CXL memory based on the message size in the metadata to support the client request / response process.

[0111] Based on this, the client can locate the current host's request, response, and control queues using the index table. The requesting client creates two threads pointing to the response queue and the control queue, respectively, and keeps them listening. The former is used to receive response information, and the latter is used to control the request process. The responding client creates two threads pointing to the request queue and the control queue, respectively. The former is used to receive request information, and the latter is used to control the response process.

[0112] After the preparation is completed, Figure 3 As shown, taking the communication between client A (i.e., the requester) and client C (i.e., the responder) as an example, the request and response process instance steps are summarized as follows: 1. The requester (hereinafter referred to as client A) encapsulates the request metadata and writes it into the ring queue; 2. The Master processes the request metadata in the ring queue, uses the global memory allocator to allocate CXL shared memory for the request data, and pushes the allocation result (CXL memory offset address and length, corresponding to the request data block) into the control queue of Client A; 3. Client A controls the listening thread to write the request data into the request data block; 4. Client A controls the listening thread to write the CXL memory offset address and length of the requested data block into the request queue corresponding to the responder (hereinafter referred to as client C); 5. Client C requests the listening thread to read the request data pointed to by the request data block in the corresponding request queue; 6. Client C executes the business process and generates response information; 7. Client C encapsulates the response metadata and writes it into the ring queue; 8. As in step 2, the Master processes the response metadata in the ring queue, allocates shared memory (corresponding to the response data block) based on the response data size, and pushes it into the client C control queue; 9. Client C controls the listening thread to write the response data into the response data block; 10. Client C controls the listening thread to write the CXL memory offset address and length of the response data block into the response queue corresponding to client A; 11. Client A responds to the listening thread and reads the response data pointed to by the response data block in the corresponding response queue. This completes the request-response communication process.

[0113] It should be noted that the concurrency control in the above process, such as the index table being determined during initialization and read-only after being written once, belongs to a concurrency-safe structure; the request queue, response queue, and control queue require concurrency-safe flags (such as atomic operations and segment locks) to be set during operation due to the existence of concurrent read and write scenarios; the ring queue belongs to a multi-producer single-consumer model, and atomic operations are required to ensure concurrency safety during the message production phase, but no concurrency control is required during the consumption phase due to its single-threaded nature.

[0114] The Master server allocates CXL shared memory for request / response data based on the CXL memory allocator. The present invention also provides a complete CXL memory allocation and recovery method, and the details can be found in the above description.

[0115] When the Master server starts, it sets up two threads: one for listening to metadata messages in the ring queue, and the other for listening to each client control queue. When request / response metadata enters the ring queue, the ring queue listening thread calls the memory allocator to complete memory allocation; when the control queue's request metadata status is marked as completed, the control queue listening thread calls the memory allocator to complete memory reclaim.

[0116] In addition, the CXL memory allocator may be a third-party memory allocation tool, which will not be described in detail.

[0117] It can be seen that the present invention proposes a multi-node communication method based on CXL shared memory, including but not limited to the following technical effects.

[0118] First, low-latency communication with zero negotiation, zero copy, and zero kernel switching. This invention is based on request / response metadata (source, destination, and length triples), eliminating the need for pre-negotiation of CXL shared memory offsets or capacity. Shared memory-based transmission achieves zero copy. The entire process utilizes a two-stage submission process for requests and responses, all implemented in user mode, resulting in extremely low latency.

[0119] Second, dynamic channels and high scalability. The Master provides a global CXL memory allocator to allocate and recycle request / response data blocks on demand, reducing memory fragmentation and improving CXL shared memory utilization. New nodes only need to register with the Master to join the communication domain, eliminating the need for inter-node negotiation.

[0120] Third, it supports high concurrency and concurrency security in multi-tenant scenarios. This invention uses atomic operations, circular queues, and request and response queue separation modes to achieve lock-free or low-lock concurrency, ensuring concurrency security in multi-tenant scenarios.

[0121] Fourth, observability and operability: This invention uses a single Master server to maintain global information, which facilitates data observation and operation and maintenance display.

[0122] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0123] Corresponding to the above method embodiment, an embodiment of the present invention further provides an electronic device. The electronic device described below and the communication method described above can refer to each other.

[0124] See also Figure 7 As shown, the electronic device includes: a memory 332 for storing computer programs; a processor 322 for implementing the steps of the communication method of the above method embodiment when executing the computer program.

[0125] For details, please refer to Figure 8 The electronic device may vary significantly depending on its configuration or performance. It may include one or more central processing units (CPUs) (e.g., one or more processors) and memory 332. Memory 332 stores one or more computer programs 342 or data 344. Memory 332 may be either transient or persistent storage. The program stored in memory 332 may include one or more modules (not shown), each of which may include a series of instruction operations within the data processing device. Furthermore, processor 322 may be configured to communicate with memory 332 to execute the series of instruction operations stored in memory 332 on electronic device 301.

[0126] The electronic device 301 may further include one or more power supplies 326 , one or more wired or wireless network interfaces 350 , one or more input / output interfaces 358 , and / or one or more operating systems 341 .

[0127] The steps in the communication method described above may be implemented by the structure of an electronic device.

[0128] Corresponding to the above method embodiments, embodiments of the present invention further provide a readable storage medium. The readable storage medium described below and the communication method described above can be referenced in a corresponding manner. Embodiments of the present invention further provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps of any of the above communication method embodiments when executed.

[0129] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0130] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above communication method embodiments are implemented.

[0131] An embodiment of the present invention further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned communication method embodiments are implemented.

[0132] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0133] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above examples is only intended to help understand the method and core concept of the present invention. It should be noted that those skilled in the art may make various improvements and modifications to the present invention without departing from the principles of the present invention, and such improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A communication method, characterized in that: Applied to communication servers, including: Data writing service is provided by using a circular queue located in the metadata area of ​​the shared memory device; Reading metadata from the circular queue and determining a first communication participant that writes the metadata; Allocating, for the first communication participant, a designated data block located in a communication data area in the shared memory device using the metadata; The allocation result is fed back to the first communication participant so that after the first communication participant writes the communication data into the designated data block, the second communication participant is informed to read the communication data from the designated data block.

2. The method according to claim 1, characterized in that Reading metadata from the ring queue and determining a first communication participant that writes the metadata includes: Reading metadata in triple format from the circular queue; the triple includes source, destination and length; Determining the client corresponding to the source as the first communication participant; Accordingly, using the metadata to allocate a specified data block located in a communication data area in the shared memory device to the first communication participant includes: Determining the length of the specified data block using the length; Determining the client corresponding to the target as the second communication participant; Searching the communication data area for a data block that meets the length and to which the second communication participant has access rights; The data block is determined as the designated data block, and the designated data block is allocated to the first communication participant.

3. The method according to claim 1, characterized in that Feeding back the allocation result to the first communication participant includes: Searching the index table to determine the control queue of the first communication participant; The allocation result is written into the control queue.

4. The method according to claim 1, wherein Data write services are provided using a circular queue located in the metadata area of ​​the shared memory device, including: The circular queue is used to provide data writing service and maintain atomic operation during metadata writing.

5. The method according to claim 1, wherein Also includes: Receive registration requests from new nodes; The control queue, request queue, and response queue of the newly added node are recorded in the index table, so that the newly added node is added to the communication domain based on the index table.

6. The method according to any one of claims 1 to 5, characterized in that After feeding back the allocation result to the first communication participant, the method further includes: Monitoring a control queue of the first communication participant; the control queue records metadata and metadata status written by the first communication participant to the ring queue; When determining that the metadata status corresponding to the designated data block is completed, marking the designated data block as a data block to be recycled; When a garbage collection condition is triggered, garbage collection is performed on the data block to be recycled.

7. A communication method, characterized in that: Applicable to communication clients, including: Determine the request metadata corresponding to the request data; Writing the request metadata to a circular queue in a metadata area in a shared memory device so that the communication server calls a memory allocator to allocate a request data block in a communication data area in the shared memory device to the requester based on the request metadata; Receive the allocation result fed back by the memory allocator; Using the allocation result, writing the requested data into the requested data block; A data read notification carrying the address of the requested data block is sent to the responder, so that the responder reads the requested data from the requested data block.

8. The method according to claim 7, characterized in that Determine the request metadata corresponding to the communication, including: Determine the length according to the amount of the requested data; Determine the local client as a source and the responder as a target; The length, the source, and the target are encapsulated into a triple, and the triple is determined as the request metadata.

9. The method according to claim 7, characterized in that Receiving the allocation result fed back by the memory allocator, including: Utilize the control queue corresponding to the local client to provide the memory allocator with allocation result writing service; The allocation result is read from the control queue.

10. The method according to claim 7, characterized in that Send a data read notification carrying the requested data block address to the responder, including: The offset address and length of the requested data block are written into the request queue of the responder.

11. The method according to any one of claims 7 to 10, characterized in that After sending a data read notification carrying the requested data block address to the responder, the following steps are also included: receiving a data read notification sent by the responder; the data read notification carries the offset address and length of the response data block; Read response data from a response data block in a communication data area in a shared memory device according to the offset address and the length.

12. A communication method, characterized in that: Applicable to communication clients, including: Receive a data read notification from the requester carrying the requested data block address; Read the requested data from the requested data block located in the communication data area of ​​the shared memory device; Processing the request data to obtain response data, and using the response data to determine response metadata; Writing the response metadata to a circular queue in a metadata area in a shared memory device so that the communication server calls a memory allocator to allocate a response data block in a communication data area in the shared memory device to the responder based on the response metadata; receiving an allocation result fed back by the memory allocator, and using the allocation result to write the response data into the response data block; A data read notification carrying the address of a response data block is sent to the requesting party, so that the requesting party reads the response data from the response data block.

13. The method according to claim 12, characterized in that Receive a data read notification from the requester carrying the requested data block address, including: Monitor the response queue and request queue of the local client; the response queue provides external data read notification and write services for response data, and the request queue provides external data read notification and write services for request data; Reading the offset address and length of the response data block from the response queue; The offset address and length of the requested data block are read from the request queue.

14. A communication system, characterized in that: include: Communication server and communication client; Wherein, the communication server performs the steps of the communication method according to any one of claims 1 to 6; The communication client includes a requester and a responder; Performing the steps of the communication method according to any one of claims 7 to 11 in the requesting party; The steps of the communication method according to claim 12 or 13 are performed in the responder.

15. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the communication method according to any one of claims 1 to 13 when executing the computer program.

Images