A distributed storage message interaction method, system, device and storage medium
By adopting a multi-channel and semi-asynchronous distributed storage message interaction method in HCI scenarios, the problem of resource waste in traditional methods is solved, network bandwidth and IO speed are improved, and resource consumption is reduced.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JINAN INSPUR DATA TECH CO LTD
- Filing Date
- 2023-02-03
- Publication Date
- 2026-06-30
AI Technical Summary
In HCI scenarios, traditional distributed storage message interaction methods lead to a waste of CPU and memory resources, excessive use of file descriptors and buffers, and network performance becomes a bottleneck.
Messages are sent through multiple channels, while data is received through a single channel. Messages and file descriptors are reused, and the system is transformed into a semi-asynchronous mode. Socket time-division multiplexing and polling mode are adopted to reduce memory copying and the use of file descriptors.
It reduces CPU, memory, and file descriptor resource consumption in HCI scenarios, improves network bandwidth utilization and IO speed, and reduces IO latency.
Smart Images
Figure CN116414309B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to a distributed storage message interaction method, system, device, and storage medium. Background Technology
[0002] With the rapid development of computer technology, hardware devices have evolved from traditional magnetic tapes to physical hard drives, and now to SSDs, NVMe disks, and even PMEM and DRAM. Hardware performance is getting better and better. In HCI (Hyper Converged Infrastructure) scenarios, network performance has become the performance bottleneck of distributed storage. At the same time, because HCI scenarios expand computing and network resources by expanding nodes, the development of network technology has limited the development of traditional FC, FCoE, etc. Therefore, all development goals are based on network-based development.
[0003] In traditional HCI scenarios, HCI interaction mainly involves establishing multiple sockets with multiple hosts, exchanging messages with the remote end using different HDR information, and using a blocking method of waiting for the remote end's file descriptor (fd) and RSP information to determine whether the message was sent successfully or the task was successful. This method wastes a lot of CPU and memory resources. Summary of the Invention
[0004] In view of this, the purpose of this invention is to propose a distributed storage message interaction method, system, device, and storage medium. It sends messages through multiple channels, receives data through a single channel, and performs network interaction through message reuse and message file descriptor reuse. This reduces memory usage and file descriptor information in message transmission in HCI scenarios, and reduces the CPU, memory, and file descriptor resources used by HCI, thereby solving the problems of excessive file descriptors and excessive buffer usage in traditional message interaction processes of distributed storage.
[0005] To achieve the above objectives, in one respect, the present invention provides a distributed storage message interaction method, wherein the method includes the following steps:
[0006] By identifying message retransmission and discarding through IO context recognition, the entire IO context is wrapped so that the same context information, the same memory information, and the same message header are used throughout the IO process.
[0007] When sending messages, file descriptors in the socket cache are used to send data.
[0008] Each file descriptor performs data reception operations independently and waits for a reply message from the receiving file descriptor;
[0009] The message content is processed once by distinguishing retransmission messages through polling mode and by using IO context and unique information carried by the context.
[0010] As a further aspect of the present invention, after the entire IO context is packaged, no memory copying is required in the entire IO process. All content is collectively encapsulated to achieve consistency between upper and lower layers and complete memory reuse.
[0011] As a further aspect of the present invention, when sending data using file descriptors in the socket cache, if concurrency occurs, a new file descriptor is created, and after use, it is added to the socket cache. After the message is sent, the file descriptor is released.
[0012] As a further aspect of the present invention, when waiting for a reply message from a file descriptor, IO network message processing is performed using the IO context carried in the received message, and the data segment content of the IO context is reused.
[0013] As a further aspect of the present invention, the distributed storage message interaction method further includes: modifying the traditional HCI interaction mode from synchronous mode to semi-asynchronous mode; centralizing message header information and message reception; releasing the used file descriptor after sending is completed using time-division multiplexing of sockets; and receiving return messages through a unified listening port; and reusing unified message header information for message sending.
[0014] As a further aspect of the present invention, when receiving return messages through unified data interaction, the traditional synchronous waiting method is changed to a unified separate waiting mechanism, and the traditional epoll method is replaced by polling mode to directly process messages.
[0015] As a further aspect of the present invention, during IO context integration, the overall resources of local processing and socket processing in the entire IO process are integrated. Through memory reuse technology, local processing and socket processing resources are integrated, and the data headers in the socket interaction during the entire IO process are reused. The headers are filled in once during the entire IO process and retry process without modification, and are directly reused.
[0016] Another aspect of the present invention provides a distributed storage message interaction system, comprising:
[0017] The IO context wrapping module is used to identify message retransmission and discard based on the IO context. It wraps the entire IO context so that the same context information, the same memory information, and the same message header are used throughout the IO process.
[0018] The socket time-division multiplexing module is used to send data using file descriptors in the socket cache when sending messages.
[0019] The file descriptor receiving module is used to perform data receiving operations for each file descriptor individually and wait for the reply message from the file descriptor.
[0020] The message sending module is used to distinguish retransmitted messages by polling mode and by using IO context and unique information carried in the context, and to process the message content once.
[0021] In another aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, which, when executed by the processor, performs any of the above-described distributed storage message interaction methods according to the present invention.
[0022] In another aspect, the present invention provides a computer-readable storage medium storing computer program instructions that, when executed, implement any of the above-described distributed storage message interaction methods according to the present invention.
[0023] This invention has at least the following beneficial technical effects: It proposes a distributed storage message interaction method, system, device, and storage medium. Through IO context integration, it reduces memory consumption in the IO stack and improves memory utilization efficiency; context integration converts message transmission into semi-asynchronous processing, reducing CPU overhead; time-division multiplexing of sockets reduces file descriptor usage during message interaction; multi-channel socket technology improves overall network bandwidth utilization; IO context identification reduces the impact of data retransmission on the system; and polling mode accelerates IO and message processing speeds, thereby reducing CPU, memory, and file descriptor usage in the entire HCI environment, while simultaneously improving IO speed and reducing IO latency in the overall HCI scenario. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.
[0025] In the diagram:
[0026] Figure 1A flowchart of the distributed storage message interaction method according to the present invention;
[0027] Figure 2 This is a schematic block diagram of an embodiment of the distributed storage message interaction system according to the present invention;
[0028] Figure 3 A schematic diagram of the hardware structure of a computer device for implementing a distributed storage message interaction method according to an embodiment of the present invention;
[0029] Figure 4 This is a schematic diagram of an embodiment of a computer-readable storage medium for implementing a distributed storage message interaction method according to the present invention. Detailed Implementation
[0030] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.
[0031] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two different entities or different parameters with the same name. Therefore, "first" and "second" are merely for convenience of expression and should not be construed as limiting the embodiments of the present invention. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as other steps or units inherent in a process, method, system, product, or device that includes a series of steps or units.
[0032] In simple terms, in traditional HCI scenarios, HCI interaction mainly involves establishing multiple sockets with multiple hosts, exchanging messages with the remote end through different HDR information, and using a blocking wait for the remote end's file descriptor and RSP information to determine whether the message has been sent successfully or the task has succeeded. This process wastes a lot of CPU and memory resources.
[0033] Therefore, this invention mainly addresses the problems of excessive file descriptors (fds) and excessive buffer usage in traditional message interaction processes of distributed storage. By using the CTX method to include the context information of the entire IO stack and performing data interaction through single socket and single buffer methods, the number of sockets and HDR information in the IO process and data interaction process are reduced, thereby improving the overall IO performance, reducing the number of fds in the entire interaction process, and increasing the number of supported nodes in HCI (Hyper Converged Infrastructure) scenarios.
[0034] This invention uses a multi-channel approach for message sending, a single-channel approach for data reception, and message reuse and message file descriptor reuse for network interaction. This reduces memory usage and file descriptor information in message transmission in HCI scenarios, and also reduces the CPU, memory, and file descriptor resources used by HCI.
[0035] To this end, in a first aspect, the present invention provides a distributed storage message interaction method, which includes using polling mod to cancel past message interruption notification methods to reduce message processing latency; reducing the use of file descriptors during message interaction through socket reuse; reducing memory usage of message interaction return information through separate recv queues and other methods; and reducing CPU, memory and file descriptor resources in HCI scenarios through semi-asynchronous message transmission, thereby improving IO performance in HCI scenarios.
[0036] A distributed storage message interaction method according to an embodiment of the present invention includes the following steps:
[0037] S10. Message retransmission identification and discarding are performed by identifying message transmission through IO context. The entire IO context is wrapped so that the same context information, the same memory information, and the same message header are used throughout the IO process.
[0038] S20. When sending messages, use the file descriptors in the socket cache to send data;
[0039] S30. Each file descriptor performs a data receiving operation independently and waits for a reply message from the receiving file descriptor.
[0040] S40: Distinguish between retransmitted messages using polling mode and by using the IO context and the unique information carried in the context, and process the message content once.
[0041] In embodiments of the present invention, after the entire IO context is packaged, no memory copying is required in the entire IO process. All content is collectively encapsulated to achieve consistency between upper and lower layers and complete memory reuse.
[0042] In an embodiment of the present invention, when sending data using file descriptors in the socket cache, if concurrency occurs, a new file descriptor is created, and after use, it is added to the socket cache. After the message is sent, the file descriptor is released.
[0043] In an embodiment of the present invention, while waiting to receive a reply message from a file descriptor, IO network message processing is performed using the IO context carried in the received message, and the data segment content of the IO context is reused.
[0044] Therefore, when modifying the traditional HCI interaction mode, the traditional interaction mode generally checks for unused file descriptors (socket cache method). If an unused file descriptor exists, it transmits messages through that file descriptor and then listens to that file descriptor to wait for the other party's processing result. Because HCI scenarios require multi-machine sending and failure handling, a single message transmission typically consumes several replicas of file descriptors. Also, because message retries are required, the sent message header and received return message are generally not reused in memory, and this is because multi-machine processing is required.
[0045] Therefore, the memory consumed is generally (message header + return information) * number of replicas, which greatly occupies memory space and file descriptors. To address this, the traditional HCI interaction mode is modified by using time-division multiplexing of sockets. After sending is complete, the used file descriptors are released, and return messages are received through a unified listening port. The unified message header information is reused, reducing memory usage in HCI scenarios. When a message returns, the memory of the unified listening port is used, and there is no need to refill the sent data segment information when retransmitting messages.
[0046] Secondly, in the embodiments of the present invention, the traditional synchronous waiting method is changed to a unified separate waiting mechanism by receiving return messages through unified data interaction. This eliminates the need to wait for each interaction, reduces the number of waiting threads, lowers CPU usage, and improves CPU resource utilization in HCI scenarios.
[0047] The distributed storage message interaction method of the present invention also replaces the traditional epoll method with polling mode, reducing signal interruptions in the epoll method. The CPU does not need to switch and can directly perform message processing, thereby improving the sensitivity of message processing, reducing the time consumption of message processing, and improving the speed of the entire IO process.
[0048] In some embodiments of the present invention, the distributed storage message interaction method further includes: modifying the traditional HCI interaction mode from synchronous mode to semi-asynchronous mode, centralizing message header information and message reception, releasing the used file descriptor after sending is completed through time-division multiplexing of sockets, and receiving return messages through a unified listening port; and reusing unified message header information for message sending.
[0049] The specific implementation of this invention includes modifying the traditional HCI interaction mode from a traditional synchronous mode to a semi-asynchronous mode. By centralizing message header information and message reception, memory usage during the interaction process is reduced. Socket reuse reduces file descriptor usage during the interaction process. Message retransmission identification and discarding are performed through IO context recognition. Polling mode is used to improve IO message processing speed, thereby improving the efficiency of the entire interaction process, increasing IO speed, and reducing resource utilization.
[0050] In this embodiment, when receiving return messages through unified data interaction, the traditional synchronous waiting method is changed to a unified separate waiting mechanism. The polling mode replaces the traditional epoll method and directly processes messages.
[0051] In this embodiment, during IO context integration, the overall resources of local processing and socket processing in the entire IO process are integrated. Through memory reuse technology, local processing and socket processing resources are integrated, and the data headers in the socket interaction in the entire IO process are reused. The headers are filled in one go in the entire IO process and retry process without modification, and are directly reused.
[0052] The distributed storage message interaction method in this embodiment of the invention operates as follows:
[0053] (1) The entire IO context is packaged, that is, the same context information is used throughout the IO process, the same memory information is reused, and the same message header is used; no memory copying is required in the entire IO process, all content is collectively encapsulated to achieve consistency between the upper and lower layers and complete memory reuse.
[0054] (2) When sending messages, use the file descriptors in the socket cache to send data. If concurrency occurs, create new file descriptors. After use, add them to the socket cache. Release the file descriptors after the message is sent. That is, expand the bandwidth of the entire IO through single-machine multi-channel method and improve the utilization of network bandwidth.
[0055] (3) Each file descriptor performs data reception operation independently and waits for the reply message from the receiving file descriptor. By reusing the returned message content, the memory allocation for the returned message is reduced. At the same time, the sent message content is not reused, so the message content does not need to be refilled when retransmitting. The IO network message is processed by the IO context carried in the received message. By reusing the data segment content of the IO context, there is no need to make separate memory allocation, which reduces the memory usage in the entire IO process and improves the IO performance in the entire IO process.
[0056] (4) By polling mode, interrupt handling and CPU switching are reduced, message response speed is improved, CPU switching loss is reduced, thereby improving the performance of the entire IO process.
[0057] (5) Distinguish retransmission messages by using the IO context and the unique information carried by the context, process the message content only once, and avoid retransmission affecting the entire IO context.
[0058] Therefore, this invention reduces memory consumption in the I / O stack and improves memory utilization efficiency through I / O context integration; it converts message transmission to semi-asynchronous mode, reducing CPU consumption; it reduces file descriptor usage during message interaction through time-division multiplexing of sockets; it improves overall network bandwidth utilization through socket multi-channel technology; it reduces the impact of data retransmission on the system through I / O context identification; and it accelerates I / O and message processing speeds through polling mode, thereby reducing CPU, memory, and file descriptor usage in the entire HCI environment, while improving I / O speed and reducing I / O latency in the entire HCI scenario.
[0059] It should be noted that, in this embodiment of the invention, during IO context integration, the overall resources of local processing and socket processing in the entire IO process are integrated. Memory reuse technology is used to reduce memory consumption in each IO process and improve memory utilization throughout the IO process. Message retransmission and multiple transmissions are handled using a unique identifier in the context. When integrating local processing and socket processing resources, the memory resources for multi-machine network transmission are integrated, and the data headers in the socket interactions throughout the IO process are reused. The headers are filled once during the entire IO process and retry process without modification, and are directly reused. Message retransmission and multiple transmissions are handled using a unique identifier in the context. The key feature is that, through context integration, it is possible to distinguish whether a returned message is a retransmission or multiple transmission. Invalid data is directly discarded to avoid the impact of retransmissions and multiple transmissions on the entire IO process.
[0060] When transforming messages into semi-asynchronous messages, a socket cache is used during sending. After sending is complete, the occupied file descriptor can be released without waiting for the result, allowing other IO contexts to reuse the file descriptor. The receiving of messages is processed asynchronously, thereby reducing the need for each IO context to wait for message returns in different threads, reducing the consumption of CPU and memory resources during the IO process. At the same time, there is no need to wait for events, reducing the impact of CPU interrupts.
[0061] When using network time-division multiplexing and multi-channel technology, with the development of network technology, the traditional single-channel method cannot fully utilize the network bandwidth. Therefore, multi-channel technology is needed to utilize the network bandwidth by sending messages through multiple file descriptors, thereby increasing network bandwidth. By using network time-division multiplexing, the utilization rate of file descriptors is increased, the use of file descriptors in the I / O process is reduced, and the consumption of system resources is reduced.
[0062] When transforming message reception to be asynchronous, each file descriptor is processed separately without blocking. That is, each file descriptor waits to receive the reply information sent back by the other party, finds the IO context corresponding to the reply message through the reply information, performs IO context processing, and completes the data transmission and reception of multiple copies.
[0063] When receiving file descriptors independently without blocking, the message sender avoids out-of-order messages by using time-division multiplexing of file descriptors. After receiving the message header, the target end can find the IO context based on the IO context carried in the header for network message processing. At the same time, it can handle message retransmission and multiple transmission based on the message processing status of the IO context to avoid the impact of retransmission and multiple transmission on the IO process. Meanwhile, during message reception, the header information of the received message is reused, that is, the header information of the received message reuses the context of the entire file descriptor, eliminating the need to allocate memory information separately for each IO, thus reducing memory allocation during the IO process.
[0064] Among them, polling mode reduces context switching and kernel switching for message notification during message reception and processing, thereby improving the response speed of the entire I / O process, reducing message loss during I / O, and improving the overall performance of the I / O process.
[0065] In a second aspect, the present invention also provides a distributed storage message interaction system. Figure 2 This is a schematic block diagram of an embodiment of the distributed storage message interaction system according to the present invention. Figure 2 As shown, the system includes:
[0066] The IO context wrapping module 210 is used to identify message retransmission and discard through IO context recognition message transmission, and to wrap the entire IO context so that the same context information, the same memory information, and the same message header are used throughout the IO process.
[0067] The socket time-division multiplexing module 220 is used to send data using file descriptors in the socket cache when sending messages.
[0068] The file descriptor receiving module 230 is used to perform data receiving operations for each file descriptor individually and wait for the reply message from the file descriptor.
[0069] The message sending module 240 is used to distinguish retransmitted messages by polling mode and by using IO context and unique information carried in the context, and to process the message content once.
[0070] In this embodiment, after the IO context packaging module 210 packages the entire IO context, no memory copying is required in the entire IO process. All content is collectively encapsulated to achieve consistency between the upper and lower layers and complete memory reuse.
[0071] When the socket time-division multiplexing module 220 sends messages and uses file descriptors in the socket cache to send data, if concurrency occurs, a new file descriptor is created. After use, the file descriptors are added to the socket cache. After the message is sent, the file descriptors are released.
[0072] When waiting for a reply message from a file descriptor, the descriptor receiving module 230 performs IO network message processing using the IO context carried in the received message, and reuses the data segment content of the IO context.
[0073] The distributed storage message interaction system of this invention modifies the traditional HCI interaction mode from synchronous to semi-asynchronous. It centralizes message header information and message reception, utilizes time-division multiplexing of sockets, releases the used file descriptors after sending, and receives return messages through a unified listening port; it reuses unified message header information for sending. When receiving return messages through unified data interaction, the traditional synchronous waiting method is replaced with a unified, separate waiting mechanism, using polling mode instead of the traditional epoll method for direct message processing.
[0074] A third aspect of the present invention also provides a computer device 400, including a memory 420 and a processor 410. The memory stores a computer program, which, when executed by the processor, implements the method of any of the above embodiments. The method includes the following steps:
[0075] By identifying message retransmission and discarding through IO context recognition, the entire IO context is wrapped so that the same context information, the same memory information, and the same message header are used throughout the IO process.
[0076] When sending messages, file descriptors in the socket cache are used to send data.
[0077] Each file descriptor performs data reception operations independently and waits for a reply message from the receiving file descriptor;
[0078] The message content is processed once by distinguishing retransmission messages through polling mode and by using IO context and unique information carried by the context.
[0079] This method modifies the traditional HCI interaction mode from synchronous to semi-asynchronous mode, centralizes message header information and message reception, uses time-division multiplexing of sockets, releases the used file descriptors after sending is completed, and receives return messages through a unified listening port; it also reuses the unified message header information for message sending.
[0080] like Figure 3 The diagram shown is a hardware structure schematic of an embodiment of a computer device for executing a distributed storage message interaction method provided by the present invention. Figure 3 Taking the computer device 400 shown as an example, this computer device includes a processor 410 and a memory 420, and may also include an input device 430 and an output device 440. The processor 410, memory 420, input device 430, and output device 440 can be connected via a bus or other means. Figure 3 Taking a bus connection as an example, input device 430 can receive input digital or character information and generate signal inputs related to distributed storage message interaction. Output device 440 may include display devices such as a display screen.
[0081] Memory 420, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the resource monitoring method in this embodiment. Memory 420 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created by the use of the resource monitoring method, etc. In addition, memory 420 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 420 may optionally include memory remotely located relative to processor 410, and these remote memories can be connected to the local module via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0082] The processor 410 executes various server functions and data processing by running non-volatile software programs, instructions, and modules stored in the memory 420, thereby implementing the resource monitoring method of the above method embodiment.
[0083] A fourth aspect of the present invention also provides a computer-readable storage medium. Figure 4 This is a schematic diagram of a computer-readable storage medium for a distributed storage message interaction method provided according to an embodiment of the present invention. Figure 4 As shown, the computer-readable storage medium 300 stores computer program instructions 310, which can be executed by a processor. When executed, the computer program instructions 310 implement the method of any of the above embodiments, the method comprising the following steps:
[0084] By identifying message retransmission and discarding through IO context recognition, the entire IO context is wrapped so that the same context information, the same memory information, and the same message header are used throughout the IO process.
[0085] When sending messages, file descriptors in the socket cache are used to send data.
[0086] Each file descriptor performs data reception operations independently and waits for a reply message from the receiving file descriptor;
[0087] The message content is processed once by distinguishing retransmission messages through polling mode and by using IO context and unique information carried by the context.
[0088] This method modifies the traditional HCI interaction mode from synchronous to semi-asynchronous mode, centralizes message header information and message reception, uses time-division multiplexing of sockets, releases the used file descriptors after sending is completed, and receives return messages through a unified listening port; it also reuses the unified message header information for message sending.
[0089] It should be understood that, where there is no conflict, all the embodiments, features and advantages described above for the distributed storage message interaction method according to the present invention are equally applicable to the distributed storage message interaction system and storage medium according to the present invention.
[0090] Finally, it should be noted that the computer-readable storage medium (e.g., memory) described herein can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. By way of example, and not limitation, non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which can act as external cache memory. By way of example, and not limitation, RAM can be obtained in various forms, such as synchronous RAM (DRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The storage devices disclosed herein are intended to include, but are not limited to, these and other suitable types of memory.
[0091] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.
[0092] It should be understood that, as used herein, the singular form "a" is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations of one or more of the associatedly listed items. The embodiment numbers disclosed above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0093] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.
Claims
1. A distributed storage message interaction method, characterized in that, Includes the following steps: Message retransmission identification and discarding are performed by recognizing message transmission through the IO context, and the entire IO context is wrapped; When sending messages, file descriptors in the socket cache are used to send data. Each file descriptor performs data reception operations independently and waits for a reply message from the receiving file descriptor; The message content is processed once by distinguishing retransmitted messages through polling mode and by using IO context and unique information carried by the context. When sending data using file descriptors in the socket cache, if concurrency occurs, a new file descriptor is created. After use, the file descriptors are added to the socket cache. The file descriptors are released after the message is sent. The traditional HCI interaction mode is modified from synchronous mode to semi-asynchronous mode. The message header information and message reception are centrally processed. The time-division multiplexing of sockets is used to release the file descriptor after the sending is completed. The return message is received through a unified listening port. The message header information for message sending is also reused.
2. The distributed storage message interaction method according to claim 1, characterized in that, After wrapping the entire IO context, all content is collectively encapsulated to ensure consistent memory reuse across the upper and lower layers.
3. The distributed storage message interaction method according to claim 1, characterized in that, While waiting for a reply message from the file descriptor, perform IO network message processing using the IO context carried in the received message.
4. The distributed storage message interaction method according to claim 1, characterized in that, When receiving return messages through unified data interaction, the traditional synchronous waiting method is changed to a unified, separate waiting mechanism. The polling mode replaces the traditional epoll method and directly processes messages.
5. The distributed storage message interaction method according to claim 4, characterized in that, During IO context integration, the overall resources of local processing and socket processing in the entire IO process are integrated. Through memory reuse, local processing and socket processing resources are integrated, and the data headers in the socket interaction in the entire IO process are reused. The headers are filled in once in the entire IO process and reused directly without modification in the retry process.
6. A distributed storage message interaction system, characterized in that, include: The IO context wrapping module is used to identify message retransmission and discard based on the IO context. It wraps the entire IO context so that the same context information, the same memory information, and the same message header are used throughout the IO process. The socket time-division multiplexing module is used to send data using file descriptors in the socket cache when sending messages. The file descriptor receiving module is used to perform data receiving operations for each file descriptor individually and wait for the reply message from the file descriptor. The message sending module is used to distinguish retransmitted messages by polling mode and by using IO context and unique information carried in the context, and to process the message content once. When sending data using file descriptors in the socket cache, if concurrency occurs, a new file descriptor is created. After use, the file descriptors are added to the socket cache. The file descriptors are released after the message is sent. And the modules configured for the following functions: The traditional HCI interaction mode is modified from synchronous mode to semi-asynchronous mode. The message header information and message reception are centrally processed. The time-division multiplexing of sockets is used to release the file descriptor after the sending is completed. The return message is received through a unified listening port. The message header information for message sending is also reused.
7. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program, which, when executed by the processor, performs the distributed storage message interaction method as described in any one of claims 1-5.
8. A computer-readable storage medium, characterized in that, The system stores computer program instructions, which, when executed, implement the distributed storage message interaction method as described in any one of claims 1-5.