A data transmission method, system and product

Abstract

The embodiments of the present application provide a data transmission method, system and product, belonging to the technical field of data transmission, the method comprising: executing a message sending process, during which the sending end sends multiple messages in the corresponding sending buffer queue to multiple receiving ends at one time, and the receiving end adds any received message to its own corresponding receiving buffer queue; when it is detected in real time that there is queue congestion in the receiving buffer queue corresponding to any of the multiple receiving ends, a sleep strategy is executed after the message sending process is completed; when the sleep strategy is executed, the sending end stops sending messages to the multiple receiving ends within the sleep time. The embodiments of the present application are intended to avoid the problem of the length of the buffer queue of the receiving end constantly growing, resulting in memory fullness and crash.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of data transmission, and specifically, to a data transmission method, system and product. Background Art

[0002] With the development of Internet technology in recent years, network communication can provide users with a variety of services. For example, the mobile terminal can communicate with the server through the network, thereby obtaining applications and resources stored on high-performance servers, so that the mobile terminal can provide users with a variety of applications.

[0003] In the current major network management systems, such as switch systems, the Client / Server model of Socket communication is used. The Client / Server model includes two types of data transmission. One is that the client subscribes to the message and the server sends the received message to the client. The other is that the server sends the message to the client only after receiving the client's request message.

[0004] During the message sending process, the message sender can first store the messages to be sent in a buffer queue, and then send the messages in the buffer queue to the message receiver one by one through Socket communication; the receiver will also set up a buffer queue to cache the messages sent by the sender to the receiver, and the receiver will take out the messages one by one from its own buffer queue for processing.

[0005] However, since the messages received by the message sender may come from different hardware platforms, the difference in message transmission speeds of hardware platforms may cause the size of each message to be non-fixed. When the message processing speed of any receiving end is fixed, if the speed at which the messages sent by the sender are written into the receiving end's buffer queue is greater than the receiving end's processing speed, the length of the receiving end's buffer queue will continue to grow, and the entire system may even crash because the receiving end's memory is full. Summary of the invention

[0006] The embodiments of the present application provide a data transmission method, system and product, which are intended to avoid the problem of memory being full and crashing due to the continuous growth of the length of the buffer queue at the receiving end.

[0007] In a first aspect, an embodiment of the present application provides a data transmission method, which is applied to a sending end, and the method includes:

[0008] Execute a message sending process, wherein in the message sending process, the sending end sends multiple messages in the corresponding sending buffer queue to multiple receiving ends at one time, and the receiving end adds any received message to its own corresponding receiving buffer queue;

[0009] When it is detected in real time that a receiving buffer queue corresponding to any of the multiple receiving ends is congested, a sleep strategy is executed after the message sending process is completed;

[0010] Wherein, when the sleep strategy is executed, the sending end stops sending messages to the multiple receiving ends during the sleep time.

[0011] Optionally, the method further comprises:

[0012] Real-time detection of memory usage of each of the multiple receiving ends;

[0013] When the memory occupancy corresponding to any receiving end is greater than a preset congestion condition, it is determined that queue congestion exists in the receiving buffer queue corresponding to the receiving end.

[0014] Optionally, real-time detection of memory usage of each of the multiple receiving ends includes:

[0015] Real-time monitoring of the current memory usage of each of the multiple receiving ends;

[0016] The proportion of the current memory occupancy size corresponding to any receiving end to the total memory size corresponding to the receiving end is determined, and the occupancy proportion is used as the memory occupancy status corresponding to any receiving end.

[0017] Optionally, when the memory usage corresponding to any receiving end is greater than a preset congestion condition, determining that a receiving buffer queue corresponding to the receiving end has queue congestion includes:

[0018] When the occupancy ratio corresponding to any receiving end is greater than a preset first ratio, it is determined that the receiving buffer queue corresponding to the receiving end is congested.

[0019] Optionally, executing the sleep strategy includes:

[0020] Determine the maximum sleep time according to the total amount of messages in the sending buffer queue during the current message sending process;

[0021] After each message sending process is completed, the message sending to any receiving end is stopped within the maximum sleep time, and the next message sending process is performed after the maximum sleep time is reached.

[0022] Optionally, determining the maximum sleep time according to the total amount of messages in the sending buffer queue during the current message sending process includes:

[0023] Determine the minimum number of processing units included in the sending buffer queue according to the total number of messages in the sending buffer queue and the size of the minimum processing unit during the current message sending process;

[0024] Determining a maximum sleep time according to the minimum number of processing units and the minimum unit processing time;

[0025] The minimum processing unit is the minimum processing character quantity among the multiple receiving ends, and the minimum unit processing time is the time used by the receiving end to process the minimum processing character quantity.

[0026] Optionally, according to the minimum number of processing units and the minimum unit processing time, a formula for determining the maximum sleep time is:

[0027] MAX_SLEEP_TIME=len(QUEUE_RECV)*MUPT;

[0028] Among them, MAX_SLEEP_TIME is the maximum sleep time; QUEUE_RECV is the sending buffer queue; len(QUEUE_RECV) is the minimum number of processing units included in the sending buffer queue; MUPT is the minimum unit processing time.

[0029] Optionally, after each message sending process is completed, stopping sending messages to any receiving end within the maximum sleep time until the maximum sleep time is reached, and then executing the next message sending process, including:

[0030] After each message sending process is completed, start the first timer and stop sending messages to any receiving end;

[0031] When the current time of the first timer is the maximum sleep time, the next message sending process is executed.

[0032] Optionally, executing the sleep strategy includes:

[0033] Determine the waiting time according to the message growth amount in the sending buffer queue during the current message sending process and the last message sending process and the change in the queue balance of the sending buffer queue during the last sending process;

[0034] After the current message sending process is completed, the message sending to any receiving end is stopped within the waiting time, and the next message sending process is performed after the waiting time is reached.

[0035] Optionally, determining the waiting time according to the message growth amount in the sending buffer queue between the current message sending process and the last message sending process and the change in the queue balance of the sending buffer queue during the last sending process includes:

[0036] The waiting time is calculated based on the message growth in the sending buffer queue between the current message sending process and the last message sending process, the queue balance of the sending buffer queue during the last sending, and the total length of the sending buffer queue; wherein the total length of the sending buffer queue is fixed.

[0037] Optionally, the waiting time is calculated according to the message growth amount in the sending buffer queue between the current message sending process and the last message sending process, the queue balance of the sending buffer queue at the last sending, and the total length of the sending buffer queue, including:

[0038] The optimized maximum sleep time is calculated according to the message growth amount in the sending buffer queue during the current message sending process and the last message sending process, the queue balance of the sending buffer queue during the last sending process, and the total length of the sending buffer queue;

[0039] The waiting time is calculated according to the message growth amount, the queue balance of the sending buffer queue at the last sending, and the optimized maximum sleep time.

[0040] Optionally, the formula for calculating the waiting time is:

[0041] The optimized maximum sleep time = (LEN_TOTAL-LEN_LEFT+GROWTH)*MUPT;

[0042] Waiting time = sqrt(GROWTH / LEN_LEFT)*optimized maximum sleep time;

[0043] Wherein, LEN_TOTAL is the total length of the sending buffer queue; LEN_LEFT is the queue balance of the sending buffer queue at the last sending; GROWTH is the message growth amount in the sending buffer queue during the current message sending process and the last message sending process; MUPT is the minimum unit processing time; sqrt is the square root;

[0044] Among them, the total length of the sending buffer queue, the queue balance of the sending buffer queue during the last sending, and the message growth amount are all multiples of the minimum processing unit, the minimum processing unit is the minimum processing character amount among the multiple receiving ends, and the minimum unit processing time is the time used by the receiving end to process the minimum processing character amount.

[0045] Optionally, after the current message sending process is completed, stopping sending messages to any receiving end within the waiting time, and executing the next message sending process after the waiting time is reached, including:

[0046] After the message sending process is completed, the second timer is started and the message sending to any receiving end is stopped;

[0047] When the current time of the second timer is the waiting time, the next message sending process is executed.

[0048] Optionally, the method further comprises:

[0049] Real-time detection of memory usage of each of the multiple receiving ends;

[0050] When the memory occupancy corresponding to any receiving end is less than the preset congestion condition, and the memory occupancy corresponding to the receiving end with queue congestion is less than the release condition, the sleep strategy is stopped.

[0051] Optionally, real-time detection of memory usage of each of the multiple receiving ends includes:

[0052] Monitor the current memory usage of each of the multiple receiving ends in real time, and determine the proportion of the current memory usage corresponding to any receiving end to the total memory size corresponding to the receiving end;

[0053] When the occupancy ratio corresponding to any receiving end is less than a preset first ratio, and the memory occupancy corresponding to the receiving end with queue congestion is less than a second ratio, stop executing the sleep strategy; wherein the first ratio is greater than the second ratio.

[0054] Optionally, executing the sleep strategy includes:

[0055] When the sending end stops sending messages to the multiple receiving ends, the sending end adds the received messages to the sending buffer queue in sequence;

[0056] When the sending buffer queue is full, the sending end adds the received message to the temporary storage queue;

[0057] When the sending end completes sending the messages in the full sending buffer queue in the next message sending process, the messages in the temporary storage queue are added to the head of the sending buffer queue.

[0058] In a second aspect, an embodiment of the present application provides a data transmission system, the system comprising a transmitting end and multiple receiving ends, the transmitting end being used to execute the data transmission method described in the first aspect of the embodiment.

[0059] In a third aspect, an embodiment of the present application provides a computer device, comprising: at least one processor, and a memory, wherein the memory stores a computer program that can be run on the processor, wherein when the processor executes the computer program, the data transmission method described in the first aspect of the embodiment is performed.

[0060] In a fourth aspect, an embodiment of the present application provides a non-volatile readable storage medium, wherein the non-volatile readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the data transmission method described in the first aspect of the embodiment is performed.

[0061] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the data transmission method described in the first aspect of the embodiment.

[0062] Beneficial effects:

[0063] During the message sending process, the sending end sends multiple messages in the corresponding sending buffer queue to multiple receiving ends at one time, and each of the multiple receiving ends adds any received message to its own corresponding receiving buffer queue; the sending end monitors in real time whether there is queue congestion in the multiple receiving ends, and when it is detected in real time that there is queue congestion in the receiving buffer queue corresponding to any of the multiple receiving ends, a sleep strategy is executed after each message sending process is completed. When the sleep strategy is executed, the sending end stops sending messages to the multiple receiving ends during the sleep time, so that the receiving end has time to process the messages in its own receiving buffer queue, thereby avoiding the problem that the receiving end processes messages slowly and the number of messages in the receiving buffer queue increases, resulting in the length of the buffer queue of the receiving end constantly increasing, causing the memory to be full and causing the entire system to crash. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art.

[0065] Figure 1 A flowchart showing the steps of the data transmission method provided by an embodiment of the present application is shown;

[0066] Figure 2 The application architecture diagram provided by the embodiment of the present application is shown;

[0067] Figure 3 A schematic diagram of a message sending cache queue provided in an embodiment of the present application is shown;

[0068] Figure 4 shows a function curve graph provided by an embodiment of the present application;

[0069] Figure 5 A schematic diagram of a data transmission system provided by an embodiment of the present application is shown;

[0070] Figure 6A data processing flow chart of a transmitting end provided in an embodiment of the present application is shown;

[0071] Figure 7 A data processing flow chart of a receiving end provided in an embodiment of the present application is shown;

[0072] Figure 8 A schematic diagram of a computer device provided in an embodiment of the present application is shown;

[0073] Fig. 9 A schematic diagram of a non-volatile readable storage medium provided in an embodiment of the present application is shown;

[0074] Fig.10 A schematic diagram of a computer program product provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0076] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in each embodiment of the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not constitute any limitation, and the various embodiments can be combined with each other and referenced to each other without contradiction.

[0077] With the development of Internet technology in recent years, mobile terminals communicate with servers through the network, so as to obtain applications and resources stored on high-performance servers, so that mobile terminals can provide users with a variety of applications. Each subject, such as the communication between mobile terminals and servers, and the communication between servers, will use a basic component Socket for inter-process communication. Socket is designed for client and service mode, that is, Client / Server mode, that is, one of the two communicating parties plays the role of client and the other plays the role of server; this mode is also divided into two types of data transmission, one is that the client subscribes to the message, and the server sends the received message to the client, and the other is that the server sends the message to the client after receiving the client's request message; the Client / Server mode of Socket communication is also used in major network management systems. With the development of technology, in order to standardize the data format in the management system, a database layer is added on top of Socket, so that the process in the management system only needs to interact with the database layer, which is convenient for development and maintenance.

[0078] For example, in the management system of multiple switches, the process responsible for receiving hardware messages is called MSG_SERVER. At the same time, MSG_SERVER is also responsible for serializing the messages received from the hardware and actively sending them to the receiving end that subscribes to the message, namely, the CLIENT. In the management system of the switch, there may be multiple independent CLIENTs receiving the messages from MSG_SERVER.

[0079] Considering that the processing speed of some receiving ends may be slow, or there may be a large number of messages that need to be sent, but the physical memory of the sending end is limited, in order to prevent the accumulation of too many messages to be sent and cause memory shortage, the sending end of the message, i.e., MSG_SERVER, can first store the messages to be sent to the receiving end, i.e., CLIENT, in a fixed-length buffer queue QUEUE_RECV, and then send the messages in the buffer queue QUEUE_RECV to the receiving end. The size of the buffer queue QUEUE_RECV can be set according to the memory size of the sending end. The larger the memory of the sending end, the larger the buffer queue QUEUE_RECV can be set. In addition, the buffer queue QUEUE_RECV allows messages to be discarded, that is, when a huge number of messages are received from the hardware in a short period of time, and the length of the buffer queue QUEUE_RECV is not enough to store them, some messages can be discarded. The discarded messages will be added to the queue again and sent when the length of the buffer queue is sufficient.

[0080] The receiving end side of the CLIENT is also equipped with a buffer queue QUEUE_BUFFER, which is used to cache messages sent from the sender to the receiving end. The receiving end will take out messages one by one from its own buffer queue QUEUE_BUFFER for processing; messages cannot be lost in the buffer queue QUEUE_BUFFER, because multiple receiving ends may receive the same message. If the same message is received by some receiving ends but not by others, there will be differences in the messages received by multiple receiving ends, which will cause a fatal error of asynchrony in the system.

[0081] However, the message transmission speeds of different hardware platforms may vary. For example, some platforms will optimize the message transmission speed and include a lot of information in one message, such as merging multiple messages into one message for sending. As a result, the size of each message is not fixed. Because the processing speed of the receiving end as the CLIENT is fixed, the speed at which the sending end writes messages to the receiving end's buffer queue QUEUE_BUFFER is different. If the speed at which the sending end writes messages to the receiving end's buffer queue QUEUE_BUFFER is greater than the receiving end's processing speed, the length of the receiving end's buffer queue QUEUE_BUFFER will grow without limit, which poses a great risk in the limited memory environment of the receiving end and may even cause the entire system to crash due to memory fullness.

[0082] Therefore, in order to avoid the problem of the length of the buffer queue at the receiving end constantly growing, causing the memory to be full and causing crash, an embodiment of the present application provides a data transmission method, which can avoid the problem of the memory of the receiving end being full and causing the system crash of the receiving end when the memory of the receiving end is limited.

[0083] Reference Figure 1 , shows a flowchart of a data transmission method provided by an embodiment of the present application, the method is applied to a sending end, and the method may include the following steps:

[0084] S101: Execute a message sending process, wherein in the message sending process, the sending end sends multiple messages in the corresponding sending buffer queue to multiple receiving ends at one time, and the receiving end adds any received message to its own corresponding receiving buffer queue.

[0085] Specifically, the sending end and the receiving end in this embodiment can be any switch in the switch system. For example, the sending end is a lower-layer switch and the receiving end is an upper-layer switch. The data transmission between the sending end and the receiving end is based on Socket communication, that is, the Client / Server mode is adopted. After the sending end receives the message sent by the hardware, it adds it to the sending buffer queue. The sending end sends multiple messages in the sending buffer queue to multiple receiving ends at one time. Multiple receiving ends can adopt the mode of subscribing to the message, or multiple receiving ends can send requests corresponding to the message to the sending end respectively, and then the sending end sends the message to multiple receiving ends.

[0086] Reference Figure 2 , showing an application architecture diagram provided by an embodiment of the present application, in a switch system, multiple devices that can send messages form a sending end cluster, any sending end in the sending end cluster adds the message sent by the hardware to its corresponding sending buffer queue, and then the sending end sends multiple messages in the sending buffer queue to the corresponding receiving end cluster through Socket communication at one time, and any of the multiple receiving ends in the receiving end cluster adds the message sent by the sending end to its corresponding receiving buffer queue after receiving the message.

[0087] In this embodiment, the message is characterized in a fine-grained manner. Specifically, the minimum processing unit corresponding to the receiving end with the smallest processing unit among multiple receiving ends is determined, that is, the MPU (minimum process unit). The minimum processing unit is the minimum processing character amount among the multiple receiving ends, and the minimum unit processing time is the time taken by the receiving end to process the minimum processing character amount, which is recorded as MUPT (minimum unit process time).

[0088] For example, the sending end sends 100 characters, the receiving end 1 needs 30 characters to process the message, and the receiving end 2 needs 20 characters to start processing the message. The minimum processing character amount 20 of the receiving end 2 is taken as the minimum processing unit.

[0089] Reference Figure 3 , showing a message schematic diagram of the sending cache queue provided in an embodiment of the present application. When the hardware platforms on which the sending end receives the message are different, the message transmission speeds of different hardware platforms may be different. Some platforms may optimize the message transmission speed and may include a lot of information in one message, such as merging multiple messages into one message for sending, resulting in the size of each message being not fixed; for example, message 1 corresponds to 1 minimum processing unit, message 2 corresponds to 5 minimum processing units, and message n corresponds to x minimum processing units.

[0090] Usually, the receiving end has a fixed message processing speed. If the speed of writing messages to the receiving buffer queue is greater than the receiving end's processing speed, the length of the receiving buffer queue will increase, and the entire system will crash because the memory is full.

[0091] Assuming that a receiver processes 1 MPU per second and a sender sends 10 MPUs per second, the exact ratio of the receiver's processing rate to the sender's sending rate is 10:1; that is, if the sender keeps sending messages, the receiver may not be able to process all the messages, and the receive buffer queue will continue to grow.

[0092] S102: When it is detected in real time that there is queue congestion in the receiving buffer queue corresponding to any of the multiple receiving ends, a sleep strategy is executed after the message sending process is completed; wherein, when the sleep strategy is executed, the sending end stops sending messages to the multiple receiving ends during the sleep time.

[0093] In order to limit the problem that the receiving buffer queue of the receiving end does not occupy more memory due to an increase in messages, the sending end, when sending a message, detects in real time whether there is queue congestion in the receiving buffer queue corresponding to any of the multiple receiving ends. If there is queue congestion in the receiving buffer queue corresponding to any of the receiving ends, a sleep strategy is executed to stop the sending end from sending messages to the multiple receiving ends during the sleep time, thereby providing the receiving end with sufficient time to process the messages in the receiving buffer queue, thereby avoiding the problem that the length of the buffer queue of the receiving end continues to grow due to an increase in the number of messages in the receiving buffer queue, causing the memory to be full and leading to the crash of the entire system.

[0094] In a feasible implementation manner, the process of detecting in real time whether there is queue congestion in a receiving buffer queue corresponding to any receiving end among multiple receiving ends may be:

[0095] The memory occupancy of each of the multiple receiving ends is detected in real time; when the memory occupancy corresponding to any receiving end is greater than a preset congestion condition, it is determined that the receiving buffer queue corresponding to the receiving end has queue congestion.

[0096] Specifically, by detecting the memory occupancy of each of the multiple receiving ends in real time, the current memory occupancy size of each of the multiple receiving ends can be monitored in real time; the proportion of the current memory occupancy size corresponding to any receiving end to the total memory size corresponding to the receiving end is determined, and the occupancy ratio is used as the memory occupancy size corresponding to any receiving end.

[0097] During the actual implementation process, the sending end can send current memory occupancy size requests to the corresponding multiple receiving ends at intervals. After any receiving end responds to the current memory occupancy size request, it sends its own current memory occupancy size to the sending end. The sending end stores the total memory size of each receiving end, and the sending end calculates the proportion of the current memory occupancy size corresponding to any receiving end to the total memory size corresponding to the receiving end.

[0098] It is also possible that the sending end sends occupancy ratio requests to the corresponding multiple receiving ends at intervals. After responding to the occupancy ratio request, any receiving end determines the ratio of its own current memory occupancy size to its own total memory size, and then returns the occupancy ratio to the sending end; it is also possible that the receiving end actively reports the ratio of its own current memory occupancy size to its own total memory size at intervals; the interval mentioned in this embodiment can be 1s, and the specific size of the interval can be customized according to the needs of the actual application, and this embodiment does not impose any restrictions.

[0099] In a feasible implementation, after the sending end learns the occupancy ratio corresponding to any receiving end, it can compare the occupancy ratio with a preset first ratio. When the occupancy ratio corresponding to any receiving end is greater than the preset first ratio, it is determined that there is queue congestion in the receiving buffer queue corresponding to the receiving end.

[0100] For example, the first ratio can be set to 60%, that is, when the current memory occupancy size of any receiving end accounts for more than 60% of its total memory size, it indicates that there are more unprocessed messages stored in the receiving buffer queue of the receiving end, which further indicates that there is queue congestion in the receiving buffer queue of the receiving end.

[0101] In order to synchronize messages between all upper layers, the messages sent by the sender to multiple receiving ends each time need to be consistent. However, when there is a queue congestion problem at any receiving end, if the sender continues to send messages to multiple receiving ends, the queue length of the receiving buffer queue of the receiving end will become longer, causing the memory to become full and data in the receiving buffer queue to be lost. Therefore, when it is detected that there is queue congestion in the receiving buffer queue corresponding to any of the multiple receiving ends, a sleep strategy is executed after each message sending process is completed.

[0102] In a feasible implementation, executing the sleep strategy includes the following steps:

[0103] A1: Determine the maximum sleep time according to the total amount of messages in the sending buffer queue during the current message sending process.

[0104] Specifically, when determining the maximum sleep time based on the total number of messages in the sending buffer queue during the current message sending process, the minimum number of processing units included in the sending buffer queue can be determined based on the total number of messages in the sending buffer queue during the current message sending process and the size of the minimum processing unit; then the maximum sleep time is determined based on the minimum number of processing units and the minimum unit processing time.

[0105] The minimum processing unit is the minimum processing character quantity among the multiple receiving ends, and the minimum unit processing time is the time used by the receiving end to process the minimum processing character quantity.

[0106] For example, during the message sending process, the total number of messages in the sending buffer queue is A characters, and the size of the minimum processing unit is 20 characters. Then the minimum number of processing units included in the sending buffer queue is X=A / 20; then the maximum sleep time is determined based on the minimum number of processing units X and the minimum unit processing time MUPT.

[0107] In a feasible implementation manner, according to the minimum number of processing units and the minimum unit processing time, the formula for determining the maximum sleep time is:

[0108] MAX_SLEEP_TIME=len(QUEUE_RECV)*MUPT;

[0109] Among them, MAX_SLEEP_TIME is the maximum sleep time; QUEUE_RECV is the sending buffer queue; len(QUEUE_RECV) is the minimum number of processing units included in the sending buffer queue; MUPT is the minimum unit processing time.

[0110] A2: After each message sending process is completed, stop sending messages to any receiving end within the maximum sleep time, and execute the next message sending process after the maximum sleep time is reached.

[0111] In the actual implementation process, after each message sending process is completed, the first timer can be started and the sending of messages to any receiving end can be stopped; when the current time of the first timer is the maximum sleep time, the next message sending process is executed.

[0112] Because the receiving end will have one more message processing time than the sending end, when the receiving buffer queue of any receiving end is congested, a maximum sleep time can be calculated by sending the total number of messages sent in the sending buffer queue during this message sending process and the minimum processing unit, so that the receiving end that processes the slowest messages can also have sufficient time to process the messages sent in this message sending process. Under the premise of ignoring the time consumption of sending and receiving, the sending rate of the sending end and the processing rate of the receiving end are balanced, avoiding the receiving message queue of the receiving end with the slowest message processing speed occupying too much memory, resulting in message loss or system crash.

[0113] In a feasible implementation, considering that adopting the maximum sleep time will cause waste of resources and affect the overall performance, in the actual implementation process, it is not necessary to wait for the maximum sleep time every time before starting the next message sending, so the maximum sleep time can be optimized; specifically, executing the sleep strategy can also include the following steps:

[0114] B1: Determine the waiting time according to the message growth amount in the sending buffer queue between the current message sending process and the last message sending process and the change in the queue balance of the sending buffer queue during the last sending process.

[0115] Specifically, when determining the waiting time based on the changes in the message growth amount in the sending buffer queue between the current message sending process and the previous message sending process and the queue balance of the sending buffer queue at the last sending, the waiting time can be calculated based on the message growth amount in the sending buffer queue between the current message sending process and the previous message sending process, the queue balance of the sending buffer queue at the last sending and the total length of the sending buffer queue, and the total length of the sending buffer queue is fixed.

[0116] Specifically, the optimized maximum sleep time is calculated based on the message growth amount in the sending buffer queue between the current message sending process and the last message sending process, the queue balance of the sending buffer queue at the last sending, and the total length of the sending buffer queue; then the waiting time is calculated based on the message growth amount, the queue balance of the sending buffer queue at the last sending, and the optimized maximum sleep time.

[0117] During the calculation, the total length of the sending buffer queue, the queue balance of the sending buffer queue at the last sending, and the message growth amount are all represented by the minimum processing unit MPU, that is, the total length of the sending buffer queue, the queue balance of the sending buffer queue at the last sending, and the message growth amount are all converted into multiples of the minimum processing unit.

[0118] For example, the formula for calculating the waiting time is:

[0119] The optimized maximum sleep time = (LEN_TOTAL-LEN_LEFT+GROWTH)*MUPT;

[0120] Waiting time = sqrt(GROWTH / LEN_LEFT)*optimized maximum sleep time;

[0121] Among them, LEN_TOTAL is the total length of the sending buffer queue; LEN_LEFT is the queue balance of the sending buffer queue at the last sending; GROWTH is the message growth amount in the sending buffer queue between this message sending process and the last message sending process; MUPT is the minimum unit processing time; sqrt is the square root.

[0122] The value range of message growth amount GROWTH is (0-LEN_LEFT); when GROUTH is the minimum value 0, that is, the message length in the sending buffer queue has not changed between this and the previous message sending process, so there is no need to wait; when the value of message growth amount GROWTH is small and LEN_LEFT is large, the waiting time is relatively short; but when the message growth amount GROWTH and LEN_LEFT are both small, but the ratio is relatively large, the waiting time is closer to the maximum sleep time.

[0123] Reference Figure 4 , shows a function curve graph provided by an embodiment of the present application, wherein the x-axis represents the message growth amount GROWTH, and the y-axis represents the waiting time; when the message growth amount GROWTH is between 0 and 4, the waiting time on the y-axis grows very fast, and when GROWTH is between 4 and 10, the waiting time is close to the longest waiting time; ultimately, the faster the message growth amount grows and the less the queue margin is, the longer the waiting time is, but the waiting time will not exceed the maximum sleep time, which can ensure the processing speed of the messages in the receiving buffer queue and maximize the use of the central processing unit (CPU) of the receiving end without causing excessive memory usage.

[0124] For example, assuming that the total length of the sending buffer queue is 10 MPUs in total, there were 2 MPUs of messages in the sending buffer queue during the last message sending process, and there are 2 MPUs in the sending buffer queue during this message sending process, indicating that the message growth in the sending buffer queue is 0, so the waiting time is also 0.

[0125] Assuming that there are 3 MPUs in the sending buffer queue during this message sending process, the waiting time is *The optimized maximum sleep time; if there are 10 MPUs in the sending buffer queue during this message sending process, then WATING_TIME is the optimized maximum sleep time.

[0126] B2: After the current message sending process is completed, stop sending messages to any receiving end within the waiting time, and execute the next message sending process after the waiting time is reached.

[0127] In the actual implementation process, after the current message sending process is completed, the second timer can be started and the message sending to any receiving end can be stopped; when the current time of the second timer is the waiting time, the next message sending process is executed.

[0128] In a feasible implementation, the method also includes real-time detection of the memory occupancy of each of the multiple receiving ends. When the memory occupancy corresponding to any receiving end is less than a preset congestion condition, and the memory occupancy corresponding to the receiving end with queue congestion is less than a release condition, the sleep strategy is stopped and the normal message sending process is executed.

[0129] By way of example, the current memory occupancy size of each of the multiple receiving ends can be monitored in real time to determine the proportion of the current memory occupancy size corresponding to any receiving end to the total memory size corresponding to the receiving end; when the occupancy ratio corresponding to any receiving end is less than a preset first ratio, and the memory occupancy corresponding to the receiving end with queue congestion is less than a second ratio, stop executing the sleep strategy.

[0130] The first ratio is greater than the second ratio. In this embodiment, the first ratio can be set to 60% and the second ratio can be set to 30%. In the actual implementation process, the settings can be customized according to the needs of the actual application, and this embodiment does not impose any restrictions.

[0131] In a feasible implementation manner, when the sleep strategy is executed, when the sending end stops sending messages to the multiple receiving ends, the sending end adds the messages received from the hardware to the sending buffer queue in sequence.

[0132] Since the length of the sending buffer queue is fixed and is related to the memory size of the sending end, when the sending buffer queue is full, the messages received by the sending end can no longer be added to the sending buffer queue, and the messages received by the sending end can be added to the temporary storage queue; when the sending end completes sending the messages in the full sending buffer queue in the next message sending process, the messages in the temporary storage queue are added to the head of the sending buffer queue.

[0133] For example, during the sleep time, the sender receives message 1, message 2, and message 3. Assuming that messages 1 and 2 have filled the sending buffer queue, message 3 will be placed in the temporary storage queue. After the sleep time ends, during the next message sending process, messages 1 and 2 in the sending buffer queue will be sent to multiple receiving ends. At this time, the sending buffer queue is empty, and message 3 in the temporary storage queue can be migrated to the head of the sending buffer queue for sending, thereby avoiding a sudden increase in the number of messages during the sleep process, which will cause the sending waiting time of some messages to increase.

[0134] In the data transmission method provided by the present embodiment, the sending end monitors in real time whether there is queue congestion at multiple receiving ends. When it is detected in real time that there is queue congestion in the receiving buffer queue corresponding to any of the multiple receiving ends, a sleep strategy is executed after each message sending process is completed. When the sleep strategy is executed, the sending end stops sending messages to the multiple receiving ends during the sleep time, so that the receiving end has time to process the messages in its own receiving buffer queue, thereby avoiding the problem that the receiving end processes messages slowly and the number of messages in the receiving buffer queue increases, resulting in the length of the buffer queue of the receiving end constantly increasing, causing the limited memory to be filled up and causing the entire system to crash.

[0135] You can calculate a maximum sleep time based on the total amount of messages sent in the sending buffer queue during this message sending process and the minimum processing unit, so that the receiving end that processes messages the slowest can have sufficient time to process the messages sent in this message sending process. You can also optimize the maximum sleep time and determine a more accurate sleep time based on the message growth amount between this message sending process and the last message sending process and the change in the queue balance of the sending buffer queue during the last sending, so as to maximize the use of the CPU processing power of the receiving end and avoid system crashes caused by excessive memory usage at the receiving end.

[0136] Reference Figure 5 , showing a schematic diagram of a data transmission system provided in an embodiment of the present application, the system includes a transmitting end and multiple receiving ends, and the transmitting end is used to execute the data transmission method described in the embodiment.

[0137] Reference Figure 6 , showing a data processing flow chart of the sending end provided by an embodiment of the present application. After the sending end process starts, it is determined whether there is a message in the sending buffer queue of the sending end. If there is a message in the sending buffer queue, all the messages in the sending buffer queue are sent to any corresponding receiving end. The receiving end writes the message into its own receiving buffer queue, and then continues to determine whether there is a message in the sending buffer queue; if there is no message in the sending buffer queue, it continues to wait for the message.

[0138] Reference Figure 7, showing a data processing flow chart of the receiving end provided by an embodiment of the present application. After the receiving end process starts, it is determined whether there is a message in the receiving buffer queue of the receiving end. If there is a message in the receiving buffer queue, the message is read from the receiving buffer queue, and the message is processed, and then the detection continues to be whether there is a message in the receiving buffer queue; if there is no message in the receiving buffer queue, the message continues to be waited for.

[0139] Reference Figure 8 , showing a schematic diagram of a computer device provided by an embodiment of the present application, the computer device comprising: at least one processor 801, and a memory 802, the memory 802 storing a computer program that can be run on the processor 801, wherein the processor 801 executes the data transmission method described in the embodiment when executing the computer program, specifically: executing a message sending process, during which the sending end sends multiple messages in the corresponding sending buffer queue to multiple receiving ends at one time, and each of the multiple receiving ends adds any received message to its own corresponding receiving buffer queue; when it is detected in real time that the receiving buffer queue corresponding to any of the multiple receiving ends is congested, a sleep strategy is executed after each message sending process is completed; when the sleep strategy is executed, the sending end stops sending messages to the multiple receiving ends during the sleep time.

[0140] Reference Fig. 9 , showing a schematic diagram of a non-volatile readable storage medium provided by an embodiment of the present application, wherein the non-volatile readable storage medium 900 stores a computer program 901, wherein the computer program 901, when executed by a processor, executes the data transmission method described in the embodiment, specifically: executing a message sending process, during which the sending end sends multiple messages in the corresponding sending buffer queue to multiple receiving ends at one time, and each of the multiple receiving ends adds any received message to its own corresponding receiving buffer queue; when it is detected in real time that the receiving buffer queue corresponding to any of the multiple receiving ends is congested, a sleep strategy is executed after each message sending process is completed; when the sleep strategy is executed, the sending end stops sending messages to the multiple receiving ends during the sleep time.

[0141] Reference Fig.10, showing a schematic diagram of a computer program product provided by an embodiment of the present application, the computer program product 1001 includes a computer program / instruction 1002, and the computer program / instruction 1002, when executed by a processor, implements the data transmission method described in the embodiment, specifically: executing a message sending process, during which the sending end sends multiple messages in the corresponding sending buffer queue to multiple receiving ends at one time, and each of the multiple receiving ends adds any received message to its own corresponding receiving buffer queue; when it is detected in real time that the receiving buffer queue corresponding to any of the multiple receiving ends is congested, a sleep strategy is executed after each message sending process is completed; when the sleep strategy is executed, the sending end stops sending messages to the multiple receiving ends during the sleep time.

[0142] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0143] It should be understood by those skilled in the art that the embodiments of the present application can be provided as methods, devices or computer program products. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0144] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0145] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0146] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0147] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0148] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0149] Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data transmission method, characterized in that: Applied to the sending end, the method includes: Execute a message sending process, wherein in the message sending process, the sending end sends multiple messages in the corresponding sending buffer queue to multiple receiving ends at one time, and the receiving end adds any received message to its own corresponding receiving buffer queue; When it is detected in real time that a receiving buffer queue corresponding to any of the multiple receiving ends is congested, a sleep strategy is executed after the message sending process is completed; Among them, when executing the sleep strategy, the sending end stops sending messages to the multiple receiving ends within the sleep time. Executing the sleep strategy includes determining the minimum number of processing units included in the sending buffer queue according to the total number of messages in the sending buffer queue and the size of the minimum processing unit during this message sending process, and determining the maximum sleep time according to the minimum number of processing units and the minimum unit processing time. After each message sending process is completed, stop sending messages to any receiving end within the maximum sleep time until the maximum sleep time is reached, and then execute the next message sending process. The minimum processing unit is the minimum processing character amount among the multiple receiving ends, and the minimum unit processing time is the time used by the receiving end to process the minimum processing character amount.

2. The method according to claim 1, characterized in that The method further comprises: Real-time detection of memory usage of each of the multiple receiving ends; When the memory occupancy corresponding to any receiving end is greater than a preset congestion condition, it is determined that queue congestion exists in the receiving buffer queue corresponding to the receiving end.

3. The method according to claim 2, characterized in that Real-time detection of the memory usage of each of the multiple receiving ends includes: Real-time monitoring of the current memory usage of each of the multiple receiving ends; The proportion of the current memory occupancy size corresponding to any receiving end to the total memory size corresponding to the receiving end is determined, and the occupancy proportion is used as the memory occupancy status corresponding to any receiving end.

4. The method according to claim 3, characterized in that When the memory usage corresponding to any receiving end is greater than a preset congestion condition, determining that the receiving buffer queue corresponding to the receiving end has queue congestion includes: When the occupancy ratio corresponding to any receiving end is greater than a preset first ratio, it is determined that the receiving buffer queue corresponding to the receiving end is congested.

5. The method according to claim 1, characterized in that According to the minimum number of processing units and the minimum unit processing time, the formula for determining the maximum sleep time is: MAX_SLEEP_TIME=len(QUEUE_RECV)*MUPT; Among them, MAX_SLEEP_TIME is the maximum sleep time; QUEUE_RECV is the sending buffer queue; len(QUEUE_RECV) is the minimum number of processing units included in the sending buffer queue; MUPT is the minimum unit processing time.

6. The method according to claim 1, characterized in that After each message sending process is completed, stop sending messages to any receiving end within the maximum sleep time until the maximum sleep time is reached, and then execute the next message sending process, including: After each message sending process is completed, start the first timer and stop sending messages to any receiving end; When the current time of the first timer is the maximum sleep time, the next message sending process is executed.

7. The method according to claim 1, characterized in that Execution of the sleep strategy includes: Determine the waiting time according to the message growth amount in the sending buffer queue during the current message sending process and the last message sending process and the change in the queue balance of the sending buffer queue during the last sending process; After the current message sending process is completed, the message sending to any receiving end is stopped within the waiting time, and the next message sending process is performed after the waiting time is reached.

8. The method according to claim 7, characterized in that Determining the waiting time according to the message growth amount in the sending buffer queue during the current message sending process and the last message sending process and the change of the queue balance of the sending buffer queue during the last sending process, including: The waiting time is calculated based on the message growth in the sending buffer queue between the current message sending process and the last message sending process, the queue balance of the sending buffer queue during the last sending, and the total length of the sending buffer queue; wherein the total length of the sending buffer queue is fixed.

9. The method according to claim 8, characterized in that The waiting time is calculated according to the message growth amount in the sending buffer queue during the current message sending process and the last message sending process, the queue balance of the sending buffer queue during the last sending, and the total length of the sending buffer queue, including: The optimized maximum sleep time is calculated according to the message growth amount in the sending buffer queue during the current message sending process and the last message sending process, the queue balance of the sending buffer queue during the last sending process, and the total length of the sending buffer queue; The waiting time is calculated according to the message growth amount, the queue balance of the sending buffer queue at the last sending, and the optimized maximum sleep time.

10. The method according to claim 9, characterized in that The formula for calculating the waiting time is: The optimized maximum sleep time = (LEN_TOTAL-LEN_LEFT+GROWTH)*MUPT; Waiting time = sqrt(GROWTH / LEN_LEFT)*optimized maximum sleep time; Wherein, LEN_TOTAL is the total length of the sending buffer queue; LEN_LEFT is the queue balance of the sending buffer queue at the last sending; GROWTH is the message growth amount in the sending buffer queue during the current message sending process and the last message sending process; MUPT is the minimum unit processing time; sqrt is the square root; Among them, the total length of the sending buffer queue, the queue balance of the sending buffer queue during the last sending, and the message growth amount are all multiples of the minimum processing unit, the minimum processing unit is the minimum processing character amount among the multiple receiving ends, and the minimum unit processing time is the time used by the receiving end to process the minimum processing character amount.

11. The method according to claim 7, characterized in that After the current message sending process is completed, stop sending messages to any receiving end during the waiting time, and execute the next message sending process after the waiting time is reached, including: After the message sending process is completed, the second timer is started and the message sending to any receiving end is stopped; When the current time of the second timer is the waiting time, the next message sending process is executed.

12. The method according to claim 1, characterized in that The method further comprises: Real-time detection of memory usage of each of the multiple receiving ends; When the memory occupancy corresponding to any receiving end is less than the preset congestion condition, and the memory occupancy corresponding to the receiving end with queue congestion is less than the release condition, the sleep strategy is stopped.

13. The method according to claim 12, characterized in that Real-time detection of the memory usage of each of the multiple receiving ends includes: Monitor the current memory usage of each of the multiple receiving ends in real time, and determine the proportion of the current memory usage corresponding to any receiving end to the total memory size corresponding to the receiving end; When the occupancy ratio corresponding to any receiving end is less than a preset first ratio, and the memory occupancy corresponding to the receiving end with queue congestion is less than a second ratio, stop executing the sleep strategy; wherein the first ratio is greater than the second ratio.

14. The method according to claim 1, characterized in that Executing the sleep strategy includes: When the sending end stops sending messages to the multiple receiving ends, the sending end adds the received messages to the sending buffer queue in sequence; When the sending buffer queue is full, the sending end adds the received message to the temporary storage queue; When the sending end completes sending the messages in the full sending buffer queue in the next message sending process, the messages in the temporary storage queue are added to the head of the sending buffer queue.

15. A data transmission system, characterized in that: The system comprises a transmitting end and a plurality of receiving ends, wherein the transmitting end is used to execute the data transmission method according to any one of claims 1 to 14.

16. A computer device, characterized in that: include: At least one processor, and a memory, wherein the memory stores a computer program that can be run on the processor, wherein the processor executes the data transmission method according to any one of claims 1 to 14 when executing the computer program.

17. A non-volatile readable storage medium, characterized in that: The non-volatile readable storage medium stores a computer program, wherein the computer program, when executed by a processor, performs the data transmission method according to any one of claims 1 to 14.

18. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the data transmission method according to any one of claims 1 to 14 is implemented.

Images