Communication method, communication system, and electronic device

Abstract

The application discloses a communication method, a communication system and an electronic device, and relates to the technical field of information, wherein a first communication participant and a second communication participant do not need to negotiate before communication, and only need the first communication participant to write metadata corresponding to the communication into a ring queue, so that a communication server can allocate shared memory required by data transmission of the present communication design by calling a memory allocator, that is, a specified data block. The first communication participant writes communication data into the specified data block, and the second communication participant reads the communication data from the specified data block, so that data transmission can be completed under zero negotiation and zero copying, transmission overhead can be reduced, and communication delay can be reduced.

Description

Technical Field

[0001] This invention relates to the field of information technology, and more particularly to communication methods, communication systems and electronic devices. Background Technology

[0002] Currently, inter-process communication methods based on CXL (Compute Express Link, a high-speed serial protocol) shared memory can be extended to inter-node process communication. However, the shared memory message queue pointers and request and response storage areas that the communicating parties rely on need to be negotiated and determined before communication. This negotiation process incurs additional transmission overhead, making it impractical for real-world applications.

[0003] Therefore, how to reduce the communication overhead of inter-node communication based on shared memory is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] This invention provides a communication method, a communication system, and an electronic device. Before the two parties communicate based on shared memory, there is no need for communication negotiation, which can reduce transmission overhead and communication latency.

[0005] This invention provides a first communication method applied to a communication server, comprising: providing a data writing service using a circular queue located in a metadata area of ​​a shared memory device; reading metadata from the circular queue and determining a first communication participant writing the metadata; allocating a designated data block located in a communication data area of ​​the shared memory device to the first communication participant using the metadata; and feeding back the allocation result to the first communication participant so that after the first communication participant writes communication data into the designated data block, it informs a second communication participant to read the communication data from the designated data block.

[0006] This invention provides a second communication method applied to a communication client, comprising: determining request metadata corresponding to request data; writing the request metadata into a circular queue located in the metadata area of ​​a shared memory device, so that a communication server calls a memory allocator to allocate a request data block located in the communication data area of ​​the shared memory device to the requester based on the request metadata; receiving the allocation result fed back by the memory allocator; using the allocation result, writing the request data into the request data block; and sending a data read notification carrying the address of the request data block to the responder, so that the responder reads the request data from the request data block.

[0007] This invention provides a third communication method applied to a communication client, comprising: receiving a data read notification carrying the address of a requested data block sent by a requester; reading requested data from a requested data block located in a communication data area of ​​a shared memory device; processing the requested data to obtain response data, and using the response data to determine response metadata; writing the response metadata into a circular queue located in a metadata area of ​​the shared memory device, so that a communication server calls a memory allocator to allocate a response data block located in the communication data area of ​​the shared memory device to the responder based on the response metadata; receiving the allocation result fed back by the memory allocator, and using the allocation result to write the response data into the response data block; and sending a data read notification carrying the address of the response data block to the requester, so that the requester reads the response data from the response data block.

[0008] The present invention also provides a communication system, comprising: a communication server and a communication client; wherein the communication server performs the steps of the first communication method described above; the communication client includes a requester and a responder; the requester performs the steps of the second communication method described above; and the responder performs the steps of the third communication method described above.

[0009] The present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described communication methods.

[0010] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described communication methods.

[0011] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described communication methods.

[0012] Through this invention, since the communication server utilizes a circular queue located in the metadata area of ​​the shared memory device to provide data writing services, communication participants can write metadata to the circular queue. The communication server calls the memory allocator to read the metadata from the circular queue and determines the first communication participant to write the metadata. Then, using the metadata, a specified data block located in the communication data area of ​​the shared memory device can be allocated to the first communication participant. After the allocation of the specified data block is completed, the memory allocator feeds back the allocation result to the first communication participant. Based on the allocation result, the first communication participant can write communication data to the specified data block and then inform the second communication participant to read the communication data from the specified data block. The second communication participant can then read the communication data from the specified data block. In this way, communication data transfer between the first and second communication participants can be completed based on shared memory.

[0013] As can be seen, the first and second communication participants do not need to negotiate before communication. The first participant only needs to write the metadata corresponding to the communication to a circular queue. The communication server can then allocate the shared memory required for data transmission in this communication design, i.e., the specified data block, by calling the memory allocator. The first participant writes communication data to the specified data block, and the second participant reads communication data from the specified data block. Data transmission can be completed with zero negotiation and zero copying, which can reduce transmission overhead and communication latency. Attached Figure Description

[0014] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram illustrating an implementation of the first communication method provided in an embodiment of the present invention;

[0016] Figure 2 A schematic diagram of a communication system provided in an embodiment of the present invention;

[0017] Figure 3 A schematic diagram of a request / response process provided in an embodiment of the present invention;

[0018] Figure 4 A schematic diagram of memory allocation and reclamation provided in an embodiment of the present invention;

[0019] Figure 5 This is a schematic diagram illustrating an embodiment of the second communication method provided in this invention.

[0020] Figure 6 This is a schematic diagram illustrating the implementation of the third communication method provided in this embodiment of the invention;

[0021] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention;

[0022] Figure 8 This is a schematic diagram of the specific structure of an electronic device provided in an embodiment of the present invention;

[0023] Figure 9 This is a schematic diagram illustrating the implementation of a communication method. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0025] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0026] The solutions provided in the embodiments of the present invention relate to network communication. For ease of understanding, the relevant technologies are explained below.

[0027] Network communication can be used for cross-node data transmission. For example, the TCP / IP (Transmission Control Protocol / Internet Protocol) protocol suite, as the core protocol of the Internet, adopts a layered architecture (application layer / transport layer / network layer / data link layer). TCP provides reliable connections through a three-way handshake and timeout retransmission mechanism, but kernel involvement in data processing leads to high latency. The IP protocol is responsible for connectionless routing and relies on upper-layer protocols to ensure reliability, and multiple data copies increase CPU overhead. The HTTP (Hypertext Transfer Protocol) protocol is built on top of TCP and adopts a request-response model. HTTP / 1.1 reduces connection overhead through persistent connection reuse, and HTTP / 2 introduces binary framing to improve efficiency, but the layered protocol stack still limits real-time scenarios. RDMA (Remote Direct Memory Access) enables zero-copy and kernel bypass through a dedicated network interface card (NIC), allowing data to be transferred directly between application memory, reducing latency to the microsecond level. Its mainstream branches include InfiniBand (high-performance private network) and RoCE (Ethernet-based RDMA, prioritizing compatibility), but these rely on expensive hardware and require software coordination for multi-node memory sharing. CXL (Compute Express Link) is a high-speed serial protocol based on PCIe. Using CXL switches, it allows multiple hosts to share memory devices. In CXL 3.0 protocol mode, multiple hosts can achieve cross-node memory sharing by mapping CXL memory regions with the same physical offset. This communication method allows multiple hosts connected to CXL devices to communicate based on shared memory. For details, please refer to... Figure 9 The shared memory is divided into a message queue and several storage areas. Each entry in the message queue records the offset of a storage area, and request and response information are stored in different storage areas. Before two nodes communicate, they need to negotiate and determine two pointer entries in the message queue to store the requests and responses during communication. During communication, the requesting node writes the request to the request storage area and waits for the response data to arrive in the response storage area; the responding node reads the data from the request storage area and writes the response information to the response storage area.

[0028] In other words, while these protocols can achieve network communication, they each have their own shortcomings. This invention improves the scheme for inter-node communication based on CXL shared memory, thereby achieving inter-node communication with zero negotiation, zero copy, and zero kernel switching.

[0029] Specifically, the solution provided by this invention is implemented based on a client-server (CS) architecture in a CXL pooled storage scenario. For example... Figure 3As shown, the CXL shared memory device is formatted into two areas: a metadata area for storing metadata and a communication data area for storing request and response data. The entire CXL memory area is managed by a global CXL memory allocator on the Master. During initialization, the memory allocator on the Master maps the entire CXL physical memory space and allocates memory for subsequent metadata, request, and response data for both the Master and clients, while simultaneously reclaiming garbage memory.

[0030] Based on the above system, corresponding communication methods are implemented on the memory allocator and the client. The memory allocator is the server in the client-server architecture, and the client is the communicating party, including the requester and the responder. Since this invention implements communication based on the client-server architecture, the server is referred to as the communication server, and the client is referred to as the communication client.

[0031] Please refer to Figure 1 The specific implementation steps of the communication method executed in the communication server are as follows.

[0032] S101. Provide data writing services using a circular queue located in the metadata area of ​​a shared memory device.

[0033] In this embodiment, the shared memory in the shared memory device can be divided into a metadata area and a communication data area. The metadata area can store communication-related metadata, and the communication data area can store the request data and response data that need to be transmitted during communication.

[0034] The circular queue operates on a producer-consumer model. Producers are the request / response clients (i.e., the communication participants, such as the requester and responder), and consumers are the memory allocator on the Master server. The consumer stores the request / response metadata itself, including the request / response index identifier, the source client, the target client, and the message length. Based on the circular queue, the memory allocator in the Master can asynchronously process the request / response metadata from multiple clients, allocating CXL memory according to the message size in the metadata, thus supporting the client request / response flow.

[0035] Specifically, when a circular queue provides data writing services, communication participants can write corresponding metadata into the circular queue according to their communication needs.

[0036] For example, if a communication participant corresponds to a requester, request metadata can be written to a circular queue; if a communication participant corresponds to a responder, response metadata can be written to a circular queue.

[0037] The request metadata includes the request index identifier, source client, target client, and message length; the response metadata includes the response index identifier, source client, target client, and message length. However, in practical applications, if there is no requirement for partitioned storage of request and response data, this metadata will not include the request or response index identifier.

[0038] In one specific embodiment of this invention, a circular queue located in the metadata region of a shared memory device is used to provide a data write service. This includes: providing the data write service using the circular queue and maintaining atomic operations during the metadata write process. Since the circular queue can provide data write services to clients, and there can be multiple clients, concurrent writes from multiple clients may occur. Therefore, in this embodiment, when providing the data write service, atomic operations must be maintained during the metadata write process. That is, metadata writing corresponds to only two states: fully written or not written, and there is no state of half-written. This ensures concurrency safety.

[0039] Furthermore, for this circular queue, only communication participants are allowed to write metadata; they are not permitted to modify or delete already written metadata. Metadata consumption is limited to the memory allocator.

[0040] S102. Read metadata from the circular queue and determine the first communication participant to write the metadata.

[0041] The memory allocator can monitor the circular queue and read the metadata from it after a communication participant writes it to the queue. This metadata can be either request or response metadata. For ease of description, in this embodiment, the communication participant that writes the metadata is referred to as the first communication participant, which can be either the requester or the responder.

[0042] S103. Using metadata, allocate a specified data block located in the communication data area of ​​the shared memory device to the first communication participant.

[0043] After the memory allocator reads the metadata from the circular queue, it can allocate a specified data block located in the communication data region of the relational memory device to the first communication participant based on the metadata.

[0044] If the metadata is request metadata, then the specified data block can be called a request data block; if the metadata is response metadata, then the specified data block can be called a response data block.

[0045] In this embodiment, metadata can specify the length of the request or response data, as well as the source and destination of the communication. Therefore, when allocating a specified data block in shared memory based on metadata, the data block can be explicitly specified based on the length of the request or response data to be transmitted, as well as the source and destination.

[0046] In one specific embodiment of the present invention, reading metadata from a circular queue and determining a first communication participant to write the metadata includes: reading metadata in triple format from the circular queue; the triple includes a source, a destination, and a length; determining the client corresponding to the source as the first communication participant; accordingly, using the metadata, allocating a specified data block located in the communication data area of ​​a shared memory device to the first communication participant includes: determining the length of the specified data block using the length; determining the client corresponding to the destination as a second communication participant; searching the communication data area for a data block that meets the length requirement and that the second communication participant has access rights to; determining the data block as the specified data block and allocating the specified data block to the first communication participant.

[0047] In other words, metadata can be specifically defined as a triple containing the source, destination, and length of the data to be transmitted for the communication. This triple is written into a circular queue by the communication participants. After the shared memory device reads the metadata, it obtains a triple. Based on the source in the triple, the first communication participant can be identified; based on the destination, the second communication participant can be identified; and based on the length, the length of the data to be transmitted can be determined. The length of a specified data block can be directly determined based on the length in the triple. For example, if the triple is 5M, then a length not less than 5M is used as the length of the specified data block. Then, the space that the second communication participant has access to is searched in the communication data area of ​​the shared memory, and a data block with a length not less than the length in the triple is found as the specified data block and allocated to the first communication participant.

[0048] In other words, in this embodiment, access permissions can be divided for different clients, thereby achieving permission control for different clients. For example, the capacity of shared memory used by clients for communication can be limited, or different communication limits can be set for different clients.

[0049] S104. Feedback the allocation result to the first communication participant so that after the first communication participant writes the communication data into the specified data block, it can inform the second communication participant to read the communication data from the specified data block.

[0050] To allocate a specified data block to the first communication participant, the first participant needs to be informed of the allocation result. This allocation result can be the offset address and length of the allocated specified data block.

[0051] After the allocation result is fed back to the first communication participant, the first participant knows the offset address and length of the allocated specified data block. At this point, without consulting the second communication participant, it can determine the available shared memory, write the communication data to the specified data block, and then inform the second communication participant to read the communication data from the specified data block. Upon receiving the notification, the second communication participant can obtain the communication data by directly reading the specified data block. That is, in this communication process, the communication data itself is not actually transferred; instead, the first communication participant writes the communication data to the specified data block, and the second communication participant reads the communication data from the specified data block, thereby achieving zero-copy data communication.

[0052] In one specific embodiment of the present invention, feeding back the allocation result to the first communication participant includes: looking up an index table to determine the control queue of the first communication participant; and writing the allocation result into the control queue.

[0053] In this embodiment, the metadata area of ​​CXL shared memory can be formatted into several data structures, including an index table, a request queue, a response queue, a control queue, and a circular queue. The index table maintains the registered online clients and records the offsets and lengths of each client's request, response, and control queues. Each online client maintains its own request queue, response queue, and control queue. The request queue records the offset and address of data requested to the target client. The response queue records the offset and address of corresponding data sent to the target client. The control queue is used by the request / response initiator client to interact with the Master and request CXL memory, completing the request / response process. Therefore, when reporting the allocation result to the first communication participant, the control queue of that participant can be determined by looking up the index table, and the allocation result can be written to the control queue. That is, the offset address and length of the specified data block are written to the control queue. This allows the first communication participant to read the offset address and length of the specified data block from its own control queue and write the communication data into the specified data block.

[0054] In one specific embodiment of the present invention, after feeding back the allocation result to the first communication participant, the method further includes: monitoring the control queue of the first communication participant; the control queue records the metadata and metadata status written by the first communication participant to the circular queue; if the metadata status corresponding to the specified data block is determined to be completed, the specified data block is marked as a data block to be reclaimed; and if the garbage collection condition is triggered, the data block to be reclaimed is garbage collected.

[0055] Please refer to Figure 4In other words, when request / response metadata enters the circular queue, the circular queue listening thread will call the memory allocator to complete memory allocation; when request metadata in the control queue is marked as completed, the control queue listening thread will call the memory allocator to complete memory reclamation. That is, the Master provides a global CXL memory allocator to allocate and reclamation of request / response data blocks on demand, reducing memory fragmentation and improving the utilization of CXL shared memory.

[0056] In one specific embodiment of this invention, the method further includes: receiving a registration request from a new node; and recording the control queue, request queue, and response queue of the new node in an index table, so as to add the new node to the communication domain based on the index table. As can be seen from the above, during communication, there is no need for negotiation before communication; instead, the memory allocator allocates the specified data blocks. Therefore, when a new node needs to be added, it only needs to register with the Master to join the communication domain, without the need for inter-node negotiation. Dynamic channels can effectively improve scalability.

[0057] Because the communication server utilizes a circular queue located in the metadata area of ​​the shared memory device to provide data writing services, communication participants can write metadata to the circular queue. The communication server calls the memory allocator to read the metadata from the circular queue and determines the first communication participant to write the metadata. Then, using the metadata, a specified data block located in the communication data area of ​​the shared memory device can be allocated to the first communication participant. After the allocation of the specified data block is completed, the memory allocator feeds back the allocation result to the first communication participant. Based on the allocation result, the first communication participant can write communication data to the specified data block and then inform the second communication participant to read the communication data from the specified data block. The second communication participant can then read the communication data from the specified data block. In this way, communication data transfer between the first and second communication participants based on shared memory can be completed.

[0058] As can be seen, the first and second communication participants do not need to negotiate before communication. The first participant only needs to write the metadata corresponding to the communication to a circular queue. The communication server can then allocate the shared memory required for data transmission in this communication design, i.e., the specified data block, by calling the memory allocator. The first participant writes communication data to the specified data block, and the second participant reads communication data from the specified data block. Data transmission can be completed with zero negotiation and zero copying, which can reduce transmission overhead and communication latency.

[0059] In other words, the requesting and responding clients in this invention do not need to negotiate shared memory offsets and storage areas before transmission; they only need to specify the source host, target host, and the length of the request and response data to complete the communication process. The requesting and responding clients only need to start receiving and sending threads to asynchronously send, receive, and process communication information without blocking business flow. Based on the Master node managing each client node, dynamic channels are implemented by allocating and reclaiming CXL shared memory space, reducing memory fragmentation and improving memory utilization. The Master request-response circular queue and CXL atomic locks ensure concurrent safety for multi-tenant writes.

[0060] Since the client in this invention is the communication participant, and the communication participant has two roles, one is the requester and the other is the responder, the communication methods executed by the requester and the responder will be described in detail below.

[0061] Please refer to Figure 5 This method can be applied to a communication client. When the communication client acts as the requester, the specific implementation steps of the communication method are as follows.

[0062] S201. Determine the request metadata corresponding to the request data.

[0063] When the communication client is the requester, and the client has a communication need, it can determine the request metadata corresponding to the requested data. This request data is the data content that initiates a certain data request to the responder. For example, it may request access to a specific target file or data from the responder.

[0064] In one specific embodiment of the present invention, determining the request metadata corresponding to the communication includes: determining the length based on the data volume of the request data; determining the local client as the source and the responder as the target; encapsulating the length, source, and target into a triple, and determining the triple as the request metadata.

[0065] Both request and response metadata can be triple structures, with their source and destination differing depending on the direction of data transmission (request or response). For the communicating parties, the source and destination are reversed. The length in the triple structure corresponds to the length of the request data for request metadata and the length of the response data for response metadata.

[0066] In one specific real-time method of this invention, the requester can establish communication with multiple responders in parallel. Specifically, when determining the request metadata, multiple request metadata can be generated based on the same request data. That is, only the responders differ among the multiple different request metadata. For example, if the request metadata is 2M, then the multiple request metadata can be: Request 1: Source A, Target B, Length 2M; Request 2: Source A, Target C, Length 2M; Request 3: Source A, Target C, Length 2M; Request 4: Source A, Target D, Length 2M. Here, Source A is requester A, and Target B, Target C, and Target D can be responders B, C, and D. These multiple request metadata are written into a circular queue, so the communication server allocates multiple request data blocks to the requester. This triggers communication with multiple responders. Of course, the request data can also have different lengths.

[0067] S202. Write request metadata to a circular queue located in the metadata area of ​​the shared memory device, so that the communication server can call the memory allocator to allocate a request data block located in the communication data area of ​​the shared memory device to the requester based on the request metadata.

[0068] Because the circular queue corresponding to the communication server provides metadata writing services based on the circular queue, the requesting party can write request metadata to this circular queue. In this way, the communication server can perform actions such as... Figure 1 The steps of the method shown are to allocate the request data block to the requester.

[0069] S203, Receive the allocation result from the memory allocator.

[0070] The requesting party can receive the allocation result from the memory allocator. This allocation result specifies the offset address and length of the requested data block.

[0071] In one specific embodiment of the present invention, receiving the allocation result fed back by the memory allocator includes: providing the memory allocator with an allocation result writing service using the control queue corresponding to the local client; and reading the allocation result from the control queue.

[0072] In this embodiment, different clients can set up corresponding control queues, which can provide allocation result writing services to the memory allocator. Therefore, by monitoring the control queue and reading the data, the allocation result can be obtained.

[0073] S204. Using the allocation result, write the request data into the request data block.

[0074] The allocation result corresponds to the offset address and length of the requested data block, so the requested data can be written into the requested data block based on the offset address and length.

[0075] S205. Send a data read notification carrying the address of the requested data block to the responder so that the responder can read the requested data from the requested data block.

[0076] After the request data is written into the request data block, the responder can be notified to read the request data block, thereby passing the request data to the responder.

[0077] Specifically, a data read notification carrying the address of the requested data block is sent to the responder, including writing the offset address and length of the requested data block into the responder's request queue.

[0078] In other words, the offset address and length of the request data block are written to the responder's request queue. This allows the responder to monitor the request queue and read its data, thus determining that request data can be read from the request data block. This completes the process of the requester transmitting request data to the responder.

[0079] Generally, after receiving the request data, the responder will process the request data and generate corresponding response data, which needs to be sent back to the requester. This process can be referred to in another method embodiment below, which will not be elaborated here.

[0080] In one specific embodiment of the present invention, after sending a data read notification carrying the address of the requested data block to the responder, the method further includes: receiving the data read notification sent by the responder; the data read notification carrying the offset address and length of the response data block; and reading response data from the response data block located in the communication data area of ​​the shared memory device according to the offset address and length. That is, after the responder processes the requested data and determines the response data to be returned, it can send a data read notification to the requester according to the method steps provided in another method embodiment described below. When the requester receives the data read notification, it can read the response data from the response data block located in the communication data area of ​​the shared memory device based on the offset address and length of the response data block corresponding to the notification. Thus, the communication process of the requester requesting data from the responder and obtaining the response data is completed.

[0081] Please refer to Figure 6 This method can be applied to a communication client. When the communication client acts as the responder, the specific implementation steps of the communication method are as follows.

[0082] S301. Receive a data read notification sent by the requester, carrying the address of the requested data block.

[0083] As discussed above, in a shared memory device, there are control queues, request queues, and response queues corresponding to different clients. Clients only need to monitor their own queues to obtain the relevant content.

[0084] In one specific embodiment of this invention, receiving a data read notification carrying the address of a requested data block sent by a first communication participant includes: monitoring the response queue and request queue of the local client; the response queue providing a data read notification writing service for response data, and the request queue providing a data read notification writing service for request data; reading the offset address and length of the response data block from the response queue; and reading the offset address and length of the request data block from the request queue. In other words, the client can obtain the offset address and length of the request data block and the response data block by monitoring the response queue and the request queue, and then reading the offset address and length from the corresponding queue.

[0085] Specifically, since the client's control queue provides write services to the server's memory allocator, monitoring the control queue and reading its offset address and length reveals the specified data block allocated to the client by the memory allocator. Similarly, since the client's request queue provides write services to the requesting party, monitoring the request queue and reading its offset address and length reveals the request data block containing the request data. Likewise, since the client's response queue provides write services to the responding party, monitoring the response queue and reading its offset address and length reveals the request data block containing the response data.

[0086] S302: Read the requested data from the requested data block located in the communication data area of ​​the shared memory device.

[0087] Once the request data block is identified, the request data can be read from it.

[0088] S303. Process the request data to obtain response data, and use the response data to determine the response metadata.

[0089] The requesting party parses and processes the request data to obtain the response data requested by the requesting party. Specifically, the response data can be data generated after processing certain data in the request data, or data obtained from local storage or based on local business logic.

[0090] Once the response data is identified, the response metadata can be determined.

[0091] S304. Write response metadata to a circular queue located in the metadata area of ​​the shared memory device, so that the communication server can call the memory allocator to allocate a response data block located in the communication data area of ​​the shared memory device to the responder based on the response metadata.

[0092] In order to send the response data back to the requester, in this embodiment, the responder can write the response metadata into a circular queue, thereby enabling the communication server to perform actions such as... Figure 1 The method and steps shown enable the allocation of response data blocks to the responder.

[0093] S305: Receive the allocation result from the memory allocator and use the allocation result to write the response data into the response data block.

[0094] The responder can monitor its own control queue to obtain the allocation result from the memory allocator, that is, to know the offset address and length of the response data block. Based on the offset address and length, the response data can be written into the allocated response data block.

[0095] S306. Send a data read notification carrying the address of the response data block to the requester so that the requester can read the response data from the response data block.

[0096] After the responder writes response data into the response data block, it can send a data read notification to the requester. For example, the responder writes the offset address and length of the response data block into the requester's response queue. In this way, the requester can read the response data from the response data block. That is, the communication process of the responder obtaining the requested data and sending the response data back to the requester is complete.

[0097] Corresponding to the above method embodiments, this invention also provides a communication system. The communication system described below can be referred to in correspondence with the communication method described above.

[0098] Please refer to Figure 2 The communication system includes: a communication server and a communication client; wherein the communication server performs the steps of the first communication method described above; the communication client includes a requester and a responder; the requester performs the steps of the second communication method described above; and the responder performs the steps of the third communication method described above.

[0099] The first communication method includes: providing data writing services using a circular queue located in the metadata area of ​​a shared memory device; reading metadata from the circular queue and determining a first communication participant writing the metadata; allocating a specified data block located in the communication data area of ​​the shared memory device to the first communication participant using the metadata; and feeding back the allocation result to the first communication participant so that after the first communication participant writes the communication data into the specified data block, it can inform a second communication participant to read the communication data from the specified data block.

[0100] The process of reading metadata from a circular queue and determining the first communication participant to write the metadata includes: reading metadata in triple format from the circular queue; the triple includes source, destination, and length; determining the client corresponding to the source as the first communication participant; and accordingly, allocating a specified data block located in the communication data area of ​​the shared memory device to the first communication participant using the metadata, including: determining the length of the specified data block using the length; determining the client corresponding to the destination as the second communication participant; searching for a data block in the communication data area that meets the length requirement and that the second communication participant has access rights to; determining the data block as the specified data block and allocating the specified data block to the first communication participant.

[0101] The process of feeding back the allocation result to the first communication participant includes: looking up the index table to determine the control queue of the first communication participant; and writing the allocation result into the control queue.

[0102] The process of feeding back the allocation results to the first communication participant also includes: monitoring the control queue of the first communication participant; recording the metadata and metadata status of the first communication participant writing to the circular queue in the control queue; marking the specified data block as a data block to be reclaimed when it is determined that the metadata status of the specified data block is completed; and performing garbage collection on the data block to be reclaimed when the garbage collection conditions are triggered.

[0103] The provision of data writing services using a circular queue located in the metadata region of a shared memory device includes: providing data writing services using a circular queue while maintaining atomic operations during the metadata writing process.

[0104] This also includes: receiving registration requests from new nodes; and recording the control queue, request queue, and response queue of new nodes in the index table so that new nodes can be added to the communication domain based on the index table.

[0105] The first communication method includes: determining the request metadata corresponding to the request data; writing the request metadata to a circular queue located in the metadata area of ​​the shared memory device, so that the communication server calls the memory allocator to allocate a request data block located in the communication data area of ​​the shared memory device to the requester based on the request metadata; receiving the allocation result fed back by the memory allocator; using the allocation result, writing the request data into the request data block; and sending a data read notification carrying the address of the request data block to the responder, so that the responder can read the request data from the request data block.

[0106] The process of determining the request metadata corresponding to the communication includes: determining the length based on the amount of request data; identifying the local client as the source and the responder as the target; encapsulating the length, source, and target into a triple, and defining the triple as the request metadata.

[0107] The process of receiving allocation results from the memory allocator includes: providing allocation result writing services to the memory allocator using the control queue corresponding to the local client; and reading allocation results from the control queue.

[0108] Sending a data read notification carrying the address of the requested data block to the responder includes writing the offset address and length of the requested data block into the responder's request queue.

[0109] The process includes, after sending a data read notification carrying the address of the requested data block to the responder, receiving the data read notification sent by the responder; the data read notification carrying the offset address and length of the response data block; and reading the response data from the response data block located in the communication data area of ​​the shared memory device according to the offset address and length.

[0110] The first communication method includes: receiving a data read notification carrying the address of a requested data block sent by the requester; reading the requested data from the requested data block located in the communication data area of ​​the shared memory device; processing the requested data to obtain response data, and using the response data to determine response metadata; writing the response metadata to a circular queue located in the metadata area of ​​the shared memory device, so that the communication server calls the memory allocator to allocate a response data block located in the communication data area of ​​the shared memory device for the responder based on the response metadata; receiving the allocation result fed back by the memory allocator, and using the allocation result to write the response data into the response data block; and sending a data read notification carrying the address of the response data block to the requester, so that the requester can read the response data from the response data block.

[0111] The process of receiving a data read notification carrying the address of the requested data block sent by the requester includes: monitoring the response queue and request queue of the local client; providing a data read notification writing service for response data to the outside world through the response queue, and providing a data read notification writing service for request data to the outside world through the request queue; reading the offset address and length of the response data block from the response queue; and reading the offset address and length of the request data block from the request queue.

[0112] In practical applications, the above communication methods can be implemented simultaneously on both the communication server and the communication client. The following is a detailed explanation using specific application scenarios as examples.

[0113] The intended scenario is communication between multi-node distributed applications. Participants are divided into requesters and responders, all connected to the CXL switch device along with the service provider Master, and sharing CXL memory. A system diagram of this embodiment is attached. Figure 2 As shown.

[0114] The CXL shared memory device is formatted into two areas: metadata and request / response data. The entire CXL memory area is managed by the global CXL memory allocator process on the Master. During implementation initialization, the memory allocator in the Master maps the entire CXL physical memory space and allocates memory for subsequent metadata, request, and response data for the Master and clients, while simultaneously reclaiming garbage memory.

[0115] Based on the above system, this section will elaborate on the communication methods between client nodes.

[0116] like Figure 3 CXL shared memory metadata is formatted into several data structures: an index table, a request queue, a response queue, a control queue, and a circular queue. The index table maintains the registered online clients and records the offset and length of each client's request, response, and control queues.

[0117] Each online client maintains a request queue, a response queue, and a control queue. The request queue records the offset and address of data requested from the target client. The response queue records the offset and address of data sent to the target client in response to the request. The control queue is used by the request / response initiating client to interact with the Master and request CXL memory, thus completing the request / response process.

[0118] A circular queue is a producer-consumer model. Producers are request / response clients, and consumers are the Master server. The messages stored are the request / response metadata itself, including the request / response index identifier, the source client (requester), the target client (responder), and the message length. Based on the circular queue, the Master can asynchronously process request / response metadata from multiple clients, allocating CXL memory according to the message size in the metadata, thus supporting the client request / response process.

[0119] Based on this, and using the index table, the client can locate the request, response, and control queues of the current host. The requesting client creates two threads pointing to the response queue and the control queue respectively, and keeps listening. The former is used to receive response information, and the latter is used to control the request flow. The responding client creates two threads pointing to the request queue and the control queue respectively. The former is used to receive request information, and the latter is used to control the response flow.

[0120] After the preparations are completed, such as Figure 3 As shown, taking communication between client A (the requester) and client C (the responder) as an example, the steps of the request and response process are summarized below:

[0121] 1. The requesting party (hereinafter referred to as Client A) encapsulates the request metadata and writes it into a circular queue;

[0122] 2. The Master processes the request metadata in the circular queue, uses the global memory allocator to allocate CXL shared memory for the request data, and pushes the allocation result (CXL memory offset address and length, corresponding to the request data block) into the client A control queue;

[0123] 3. Client A controls the listening thread to write the request data into the request data block;

[0124] 4. Client A controls the listening thread to write the CXL memory offset address and length of the requested data block into the request queue corresponding to the responder (hereinafter referred to as Client C);

[0125] 5. Client C's request listening thread reads the request data pointed to by the request data block in the corresponding request queue;

[0126] 6. Client C executes the business process and generates response information;

[0127] 7. Client C encapsulates response metadata and writes it to a circular queue;

[0128] 8. Similar to step 2, the Master processes the response metadata in the circular queue, allocates shared memory (corresponding to the response data block) based on the response data size, and pushes it into the client C control queue;

[0129] 9. Client C controls the listening thread to write the response data into the response data block;

[0130] 10. Client C controls the listening thread to write the CXL memory offset address and length of the response data block into the response queue corresponding to client A;

[0131] 11. Client A's response listening thread reads the response data pointed to by the response data block in the corresponding response queue. This completes one request-response communication process.

[0132] It is important to note that the concurrency control in the above process, such as the index table being determined during initialization and read-only after being written once, belongs to a concurrency-safe structure; the request queue, response queue, and control queue have concurrent read and write scenarios, so concurrency-safe flags (such as atomic operations, segmented locks, etc.) need to be set during operation; the circular queue belongs to a multi-producer single-consumer model, and atomic operations are needed to ensure concurrency safety during the message production stage, while concurrency control is not required during the consumption stage due to its single-threaded nature.

[0133] The Master server allocates CXL shared memory for request / response data based on the CXL memory allocator. This embodiment of the invention also provides a complete CXL memory allocation and reclamation method, details of which can be found in the description above.

[0134] The Master server sets up two threads upon startup: one to listen for metadata messages in the circular queue, and the other to listen for messages in each client's control queue. When request / response metadata enters the circular queue, the circular queue listening thread calls the memory allocator to allocate memory; when request metadata in the control queue is marked as completed, the control queue listening thread calls the memory allocator to reclaim memory.

[0135] Furthermore, the CXL memory allocator can be a third-party memory allocation tool, which will not be elaborated further.

[0136] Therefore, this invention proposes a multi-node communication method based on CXL shared memory, which includes, but is not limited to, the following technical effects.

[0137] Firstly, it enables low-latency communication with zero negotiation, zero copy, and zero kernel switching. This invention is based on request / response metadata (source, destination, length triplet), eliminating the need for pre-negotiation of CXL shared memory offsets or capacities; it achieves zero-copy transmission based on shared memory; the entire process employs a "two-stage" submission for requests and responses, all implemented in user space, resulting in extremely low latency.

[0138] Secondly, dynamic channels and high scalability. The Master provides a global CXL memory allocator to allocate and reclaim request / response data blocks on demand, reducing memory fragmentation and improving the utilization of CXL shared memory; new nodes only need to register with the Master to join the communication domain, without the need for inter-node negotiation.

[0139] Thirdly, it supports high concurrency and concurrency safety in multi-tenant scenarios. This invention uses atomic operations, circular queues, and separate request and response queues to achieve lock-free or lock-less concurrency, ensuring concurrency safety in multi-tenant scenarios.

[0140] Fourth, observability and maintainability. This invention uses a single Master server to maintain global information, facilitating data observation and operational display.

[0141] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0142] Corresponding to the above method embodiments, this invention also provides an electronic device. The electronic device described below and the communication method described above can be referred to in correspondence.

[0143] See Figure 7 As shown, the electronic device includes: a memory 332 for storing a computer program; and a processor 322 for executing the computer program to implement the steps of the communication method described in the above method embodiment.

[0144] For details, please refer to Figure 8 The electronic device can vary considerably depending on its configuration or performance. It may include one or more central processing units (CPUs) (e.g., one or more processors) and memory 332, which stores one or more computer programs 342 or data 344. Memory 332 may be temporary or permanent storage. The program stored in memory 332 may include one or more modules (not shown in the figure), each module including a series of instruction operations on the data processing device. Furthermore, processor 322 may be configured to communicate with memory 332 and execute the series of instruction operations stored in memory 332 on electronic device 301.

[0145] Electronic device 301 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341.

[0146] The steps in the communication method described above can be implemented by the structure of an electronic device.

[0147] Corresponding to the above method embodiments, this invention also provides a readable storage medium, which can be referred to in conjunction with the communication method described above. This invention also provides a computer-readable storage medium storing a computer program configured to execute the steps of any of the communication method embodiments described above during runtime.

[0148] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0149] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described communication method embodiments.

[0150] Embodiments of the present invention also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described communication method embodiments.

[0151] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the invention.

[0152] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only intended to help understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A communication method, characterized in that, Applied to communication servers, including: A circular queue located in the metadata area of ​​the shared memory device is used to provide data writing services; the shared memory in the shared memory device is divided into a metadata area and a communication data area; the metadata area stores communication-related metadata, and the communication data area stores the request data and response data that need to be transmitted for communication. Metadata is read from the circular queue, and the first communication participant writing the metadata is determined; wherein, the metadata includes request metadata and response metadata, the request metadata includes a request index identifier, the source client, the target client, and the message length, and the response metadata includes a response index identifier, the source client, the target client, and the message length; Using the metadata, a designated data block located in the communication data region of the shared memory device is allocated to the first communication participant; The allocation result is fed back to the first communication participant so that after the first communication participant writes the communication data into the designated data block, it informs the second communication participant to read the communication data from the designated data block. After the allocation result is fed back to the first communication participant, the control queue of the first communication participant is monitored; the control queue records the metadata and metadata status written by the first communication participant to the circular queue. If the metadata status corresponding to the specified data block is determined to be completed, then the specified data block is marked as a data block to be reclaimed. When the conditions for garbage collection are triggered, the data block to be collected is subjected to garbage collection processing; Receive registration requests from newly added nodes; The control queue, request queue, and response queue of the newly added node are recorded in the index table so that the newly added node can be added to the communication domain based on the index table; The process of reading metadata from the circular queue and determining the first communication participant writing the metadata includes: Metadata in triple format is read from the circular queue; the triple includes source, destination, and length. The client corresponding to the source is identified as the first communication participant; Accordingly, using the metadata, a designated data block located in the communication data region of the shared memory device is allocated to the first communication participant, including: The length of the specified data block is determined using the length; The client corresponding to the target is identified as the second communication participant; Search the communication data area for a data block that meets the specified length and that the second communication participant has access to; The data block is identified as the designated data block, and the designated data block is assigned to the first communication participant; The process of feeding back the allocation result to the first communication participant includes: Search the index table to determine the control queue of the first communication participant; Write the allocation result into the control queue; Specifically, data writing services are provided using a circular queue located in the metadata region of a shared memory device, including: The circular queue is used to provide data writing services and maintains atomic operations during metadata writing.

2. A communication method, characterized in that, Applied to communication clients, including: Determine the request metadata corresponding to the requested data; The request metadata is written to a circular queue located in the metadata area of ​​the shared memory device, so that the communication server calls the memory allocator to allocate a request data block located in the communication data area of ​​the shared memory device to the requester based on the request metadata; wherein, the communication server uses the circular queue located in the metadata area of ​​the shared memory device to provide data writing service; the communication server is used to execute the communication method as described in claim 1; Receive the allocation result fed back by the memory allocator; Using the allocation result, the request data is written into the request data block; Send a data read notification carrying the address of the requested data block to the responder so that the responder can read the requested data from the requested data block.

3. The method according to claim 2, characterized in that, Determine the request metadata corresponding to the communication, including: The length is determined based on the amount of data in the requested data; The local client is identified as the source, and the responder is identified as the target. The length, the source, and the target are encapsulated into a triple, and the triple is determined as the request metadata.

4. The method according to claim 2, characterized in that, Receive the allocation result fed back by the memory allocator, including: The local client uses its corresponding control queue to provide allocation result writing services to the memory allocator. The allocation result is read from the control queue.

5. The method according to claim 2, characterized in that, Send a data read notification carrying the address of the requested data block to the responder, including: Write the offset address and length of the requested data block into the request queue of the responder.

6. The method according to any one of claims 2 to 5, characterized in that, After sending a data read notification carrying the address of the requested data block to the responder, it also includes: Receive a data read notification sent by the responder; the data read notification carries the offset address and length of the response data block; Response data is read from the response data block located in the communication data region of the shared memory device according to the offset address and the length.

7. A communication method, characterized in that, Applied to communication clients, including: Receive a data read notification sent by the requester, carrying the address of the requested data block; Read the requested data from the requested data block located in the communication data area of ​​the shared memory device; The request data is processed to obtain response data, and the response data is used to determine response metadata; The response metadata is written to a circular queue located in the metadata region of the shared memory device, so that the communication server calls the memory allocator to allocate a response data block located in the communication data region of the shared memory device to the responder based on the response metadata; wherein the communication server uses the circular queue located in the metadata region of the shared memory device to provide data writing service; the communication server is used to execute the communication method as described in claim 1; Receive the allocation result fed back by the memory allocator, and use the allocation result to write the response data into the response data block; A data read notification carrying the address of the response data block is sent to the requesting party so that the requesting party can read the response data from the response data block.

8. The method according to claim 7, characterized in that, Receive the data read notification sent by the requester, which carries the address of the requested data block, including: The response queue and request queue of the local client are monitored; the response queue provides data reading notification and writing services for response data, and the request queue provides data reading notification and writing services for request data. Read the offset address and length of the response data block from the response queue; Read the offset address and length of the requested data block from the request queue.

9. A communication system, characterized in that, include: Communication server and communication client; The communication server executes the steps of the communication method as described in claim 1. The communication client includes a requester and a responder; The steps of the communication method as described in any one of claims 2 to 6 are performed on the requesting party; The responder performs the steps of the communication method as described in claim 7 or 8.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the communication method as described in any one of claims 1 to 8.

Images