Data transmission method and apparatus, computer storage medium, and electronic device

Abstract

The application discloses a data transmission method and device, a computer storage medium and an electronic device. It relates to the field of data communication. The method comprises the following steps: receiving data from a data sender, identifying the service type corresponding to the data; determining the index for flow control parameter calculation and the weight corresponding to the index; determining the target data flow according to the service type; dynamically adjusting the initial flow control window parameter corresponding to the target data flow based on the index, the weight corresponding to the index, and the historical data sending record of the data sender, to obtain the adjusted flow control window parameter; optimizing the adjusted flow control window parameter based on the current data load condition and the data sending characteristics, to obtain the target flow control window parameter; and transmitting data based on the target flow control window parameter. Through the application, the problem that, in the related art, too many request pressures cause the accumulation of too many request data in the cache area of the service provider during the service peak, eventually leading to memory overflow and affecting the business handling, is solved.

Description

Technical Field

[0001] This application relates to the field of data communication, and more specifically, to a data transmission method, apparatus, computer storage medium, and electronic device. Background Technology

[0002] In a distributed microservice architecture, consumers communicate with service providers via RPC (Remote Procedure Call). When a consumer sends a request to a service provider, the request traffic first goes to the system's TCP buffer. The size of each data packet sent is controlled by the flow control window size. However, the consumer does not consider the service provider's load when sending requests. Therefore, during peak business periods, excessive request pressure can cause the service provider's buffer to accumulate too much request data, leading to memory overflow and other problems, ultimately impacting business operations.

[0003] There is currently no effective solution to the problem that during peak business periods, excessive request pressure can cause service providers to accumulate too much request data in their caches, eventually leading to memory overflow and affecting business operations. Summary of the Invention

[0004] This application provides a data transmission method, apparatus, computer storage medium, and electronic device to solve the problem in related technologies where excessive request pressure during peak business periods causes service providers' caches to accumulate too much request data, ultimately leading to memory overflow and affecting business processing.

[0005] According to one aspect of this application, a data transmission method is provided. The method includes: receiving data from a data sender and identifying the service type corresponding to the data; determining an indicator for calculating flow control parameters and a weight corresponding to the indicator based on the service type; acquiring historical data transmission records and data transmission characteristics of the data sender; determining a target data stream from multiple data streams based on the service type; dynamically adjusting an initial flow control window parameter corresponding to the target data stream based on the indicator, the weight corresponding to the indicator, and the historical data transmission records to obtain adjusted flow control window parameters; optimizing the adjusted flow control window parameters based on the current data load and the data transmission characteristics to obtain target flow control window parameters; and transmitting the data from the data sender based on the target flow control window parameters.

[0006] Optionally, before receiving data from the data sender and identifying the service type corresponding to the data, the method further includes: establishing a multiplexed connection with the data sender to obtain the multiple data streams.

[0007] Optionally, the method further includes: initializing the flow control window size of the connection; initializing the flow control window size corresponding to each of the multiple data streams.

[0008] Optionally, the method further includes: obtaining the priorities corresponding to the multiple data streams respectively; determining the flow control window parameters corresponding to the multiple data streams respectively based on the priorities; and receiving data from the multiple data streams respectively according to the flow control window parameters corresponding to the multiple data streams respectively.

[0009] Optionally, the step of dynamically adjusting the initial flow control window parameters corresponding to the target data stream based on the indicator, the weight corresponding to the indicator, and the historical data transmission records to obtain the adjusted flow control window parameters includes: determining the change trend of the indicator of the data sender over a predetermined time period based on the historical data transmission records; calculating the score of the flow control parameters based on the weight corresponding to the indicator and the change trend; and dynamically adjusting the initial flow control window parameters corresponding to the target data stream based on the score of the flow control parameters to obtain the adjusted flow control window parameters.

[0010] Optionally, transmitting the data of the data sender based on the target flow control window parameters includes: using the window size in the target flow control window parameters as the size of a single data packet for transmitting the data of the data sender; and updating the window size according to the window update frequency in the target flow control window parameters.

[0011] Optionally, after transmitting the data from the data sender based on the target flow control window parameters, the method further includes: performing a service operation based on the received data; and after the service operation is completed, sending a flow control window update data frame to the data sender, wherein the flow control window update data frame is used to allow the data sender to send data again.

[0012] According to one aspect of this application, a data transmission apparatus is provided, comprising: an identification module for receiving data from a data sender and identifying the service type corresponding to the data; a first determination module for determining an indicator for calculating flow control parameters and a weight corresponding to the indicator based on the service type; an acquisition module for acquiring historical data transmission records and data transmission characteristics of the data sender; a second determination module for determining a target data stream from multiple data streams based on the service type; an adjustment module for dynamically adjusting an initial flow control window parameter corresponding to the target data stream based on the indicator, the weight corresponding to the indicator, and the historical data transmission records to obtain adjusted flow control window parameters; an optimization module for optimizing the adjusted flow control window parameters based on the current data load and the data transmission characteristics to obtain target flow control window parameters; and a receiving module for transmitting data from the data sender based on the target flow control window parameters.

[0013] According to another aspect of the present invention, a computer storage medium is also provided for storing a program, wherein the program, when running, controls the device where the computer storage medium is located to execute a data transmission method.

[0014] According to another aspect of the present invention, an electronic device is also provided, comprising one or more processors and a memory; the memory stores computer-readable instructions, and the processor is configured to execute the computer-readable instructions, wherein the computer-readable instructions execute a data transmission method when executed.

[0015] This application addresses the problem in related technologies where excessive request pressure during peak business periods leads to an accumulation of excessive request data in the service provider's cache, ultimately causing memory overflow and impacting business processing. It achieves the ability to adaptively adjust the service provider's cache size, update frequency, and update strategy based on system operation, protecting the service provider from being overwhelmed by high traffic. This allows for the adaptive adjustment of the service provider's cache size, update frequency, and update strategy according to system operation, protecting it from system overload. Attached Figure Description

[0016] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 This is a flowchart of a data transmission method provided according to an embodiment of this application;

[0018] Figure 2 This is a schematic diagram of the overall adaptive flow control architecture provided according to an optional implementation of this application;

[0019] Figure 3 This is an adaptive flow control processing flowchart provided according to an optional embodiment of this application;

[0020] Figure 4 This is a schematic diagram of a data transmission device provided according to an embodiment of this application. Detailed Implementation

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent information from the aforementioned user or organization.

[0025] It should be noted that the data transmission methods, apparatus, computer storage media and electronic devices defined in this disclosure can be used in the field of data communication, or in any field other than data communication. The application fields of the data transmission methods, apparatus, computer storage media and electronic devices defined in this disclosure are not limited.

[0026] In the current microservice architecture, when a consumer sends a request to a service provider, the request traffic first goes to the system's TCP buffer, where it is split and reassembled into data packets for network transmission. The size of each data packet is controlled by a flow control window (windowsize). After the consumer sends a data packet, the flow control window decreases. When the flow control window is 0, the data packet is not actually sent over the network but is temporarily stored in the buffer. After receiving the data packet, the provider sends a flow control window update data packet to the consumer, informing the consumer that the window can be increased again to send data. During this process, the provider updates its window upon receiving data and puts the data into the buffer, awaiting processing and consumption by the business process. When there is a surge in business traffic, if the provider cannot handle the load, it can cause buffer overflows and other anomalies, impacting business operations.

[0027] Therefore, this application aims to provide a solution that can solve the above-mentioned technical problems, the details of which will be described in subsequent embodiments.

[0028] According to an embodiment of this application, a data transmission method is provided.

[0029] Figure 1 This is a flowchart of a data transmission method provided according to an embodiment of this application. For example... Figure 1 As shown, the method includes the following steps:

[0030] Step S102: Receive data from the data sender and identify the service type corresponding to the data;

[0031] Step S104: Based on the business type, determine the indicators for calculating flow control parameters and the corresponding weights of the indicators;

[0032] Step S106: Obtain the historical data transmission records and data transmission characteristics of the data sender;

[0033] Step S108: Determine the target data stream from multiple data streams based on the business type;

[0034] Step S110: Based on the indicators, the weights corresponding to the indicators, and historical data transmission records, dynamically adjust the initial flow control window parameters corresponding to the target data stream to obtain the adjusted flow control window parameters.

[0035] Step S112: Based on the current data load and data transmission characteristics, optimize the adjusted flow control window parameters to obtain the target flow control window parameters;

[0036] Step S114: Based on the target flow control window parameters, transmit the data from the data sender.

[0037] This application embodiment receives data from the data sender and identifies the corresponding service type; based on the service type, it determines the indicators for calculating flow control parameters and the corresponding weights; it obtains the historical data sending records and data sending characteristics of the data sender; based on the service type, it determines the target data stream from multiple data streams; based on the indicators, the corresponding weights of the indicators, and the historical data sending records, it dynamically adjusts the initial flow control window parameters corresponding to the target data stream to obtain the adjusted flow control window parameters; based on the current data load and data sending characteristics, it optimizes the adjusted flow control window parameters to obtain the target flow control window parameters; based on the target flow control window parameters, it receives data from the data sender. This solves the problem in related technologies where, during peak business periods, excessive request pressure causes the service provider's cache to accumulate too much request data, ultimately leading to memory overflow and affecting business processing. Furthermore, it achieves the effect of adaptively adjusting the flow control size, update frequency, and update strategy of the service provider's cache according to system operation, thus protecting the service provider from being overwhelmed by high traffic.

[0038] The flow control window is a mechanism used to control the data transmission rate, primarily balancing the speed of data transmission between the sender and receiver. It's a flow control method used in computer networks. During data transmission, the sender sends a certain amount of data to the receiver and then waits for confirmation of receipt. The flow control window controls the amount of data sent. The sender adjusts the amount of data sent based on feedback from the receiver. The size of the flow control window depends on the receiver's processing capacity and network conditions. When the receiver can process more data promptly, the flow control window increases, allowing the sender to send more data. Conversely, when the receiver's processing capacity is insufficient or the network is congested, the flow control window decreases, limiting the amount of data sent. The purpose of the flow control window is to prevent the sender from sending too much data, leading to situations where the receiver cannot process it in time or network congestion occurs. By dynamically adjusting the size of the flow control window, a balance can be maintained between the sender and receiver, thus achieving efficient data transmission.

[0039] In this embodiment of the application, by identifying the data when the data receiver receives data from the data sender, the service type corresponding to the data sent by the data sender at this time can be determined, and the target data stream corresponding to the service type can be determined according to different service types.

[0040] It can also monitor different important indicators corresponding to different business scenarios and determine the weight of each indicator. This weight can be used to dynamically adjust the size and / or update frequency of the flow control window by combining the historical data sending records of the data sender, so as to obtain the parameters of the adjusted flow control window.

[0041] It should be noted that since the indicators and corresponding weights of the indicators are determined by the service type to adjust the window parameters, the selection of which data stream to use is based on the service type. This prevents the window on that data stream from becoming too large when transmitting data of one service type and too small when transmitting data of another service type, thus avoiding frequent changes in the flow control window. Moreover, if the flow control window needs to change from very large to very small, data still needs to be transmitted during this process. In this case, the flow control window may not have had time to change before the data is transmitted, which may cause problems during transmission. Therefore, the solution described in this application can avoid the above problems and make the data transmission process smoother.

[0042] Subsequently, this embodiment of the application further optimizes the parameters of the adjusted flow control window based on the data load on the data receiver side and the data transmission characteristics of the data sender to obtain the target flow control window parameters. Data transmission between the data sender and the data receiver based on the target flow control window parameters can effectively alleviate the data processing pressure on the buffer of the data receiver when the demand for business data transmission is large.

[0043] It should be noted that the above-mentioned business types include at least one of the following: video streaming or large file transmission scenarios, real-time interactive business scenarios, and large-scale concurrent connection scenarios. The embodiments of this application can identify the business type in the traffic through deep learning algorithms, which can be specifically calculated based on the traffic pattern, user behavior, data packet content, etc. during the data transmission process.

[0044] Different business types may correspond to different key metrics. For example, key metrics for video streaming or large file transfer scenarios are throughput and latency, key metrics for real-time interactive business scenarios are packet loss rate and latency, and key metrics for large-scale concurrent connection scenarios are connection establishment success rate and connection rejection rate, etc. By focusing on monitoring different metrics for different business scenarios, the weight of each metric can be determined more accurately. This weight is the dynamic adjustment weight of flow control calculation.

[0045] The aforementioned data transmission characteristics are determined based on the data sender's behavioral data, including the data sender's activity level in data transmission over different time periods, and / or the data sender's access frequency, etc.

[0046] It should be noted that, in the process of optimizing the adjusted flow control window parameters based on the current data load and data transmission characteristics to obtain the target flow control window parameters, this embodiment of the application collects the server load (i.e., data load) and data transmission characteristics of each node in the data transmission process, and performs comprehensive analysis and processing to collaboratively optimize multiple targets in order to obtain the optimized target flow control window parameters.

[0047] Optionally, in the data transmission method provided in this application embodiment, before receiving data from the data sender and identifying the service type corresponding to the data, the method further includes: establishing a multiplexed connection with the data sender to obtain multiple data streams.

[0048] Before data transmission can begin, a connection between the data sender and the data receiver must be established. In this embodiment, the connection is established by multiplexing the same connection, meaning that multiple data streams can be established between the data sender and the data receiver.

[0049] Optionally, in the data transmission method provided in the embodiments of this application, the method further includes: initializing the flow control window size of the connection; initializing the flow control window size corresponding to each of the multiple data streams.

[0050] After the connection between the data sender and the data receiver is established, the size of the flow control window on the connection needs to be initialized. Since the same connection is multiplexed and there are multiple data streams in this embodiment, the window size of the flow control window corresponding to the entire connection and the flow control windows corresponding to multiple data streams in the connection are initialized separately.

[0051] The flow control window size corresponding to the entire connection limits the amount of data that can be sent over the entire connection. When the flow control window size at the connection level is 0, the entire connection is blocked, meaning that data transmission between the data sender and the data receiver can no longer occur over this connection. The flow control window size corresponding to a data stream limits the amount of data that can be sent over the current data stream. When the flow control window size at the data stream level is 0, the data sender cannot transmit data over this data stream until the data sender receives a flow control window update data frame from the data receiver. Only then can the flow control window size at the data stream level be increased and used to continue sending data from the data sender to the data receiver.

[0052] Optionally, in the data transmission method provided in the embodiments of this application, the method further includes: obtaining the priorities corresponding to the multiple data streams respectively; determining the flow control window parameters corresponding to the multiple data streams respectively based on the priorities; and receiving data from the multiple data streams respectively according to the flow control window parameters corresponding to the multiple data streams respectively.

[0053] When multiple data streams exist between the data receiver and the data sender, each data stream can be used to transmit data for the same service to improve data transmission efficiency, or it can be used to transmit data for different services to achieve the effect of simultaneously requesting and processing multiple services. However, different services may have different levels of importance or urgency. Therefore, in this embodiment, priorities are set for multiple data streams, and the flow control window parameters corresponding to each data stream are adjusted based on the priorities of the multiple data streams to ensure that high-priority data streams can obtain sufficient bandwidth and window size to ensure the transmission efficiency and reliability of important or urgent data.

[0054] In determining the priority of each data stream among multiple data streams, the system can be preset for custom settings, or machine learning and other methods can be used to intelligently identify the business priority of each data stream to quickly and accurately determine the priority of each data stream, ensuring that a larger window and a faster window size update frequency are provided for high-priority businesses.

[0055] Optionally, in the data transmission method provided in this application embodiment, the initial flow control window parameters corresponding to the target data stream are dynamically adjusted based on the indicators, the weights corresponding to the indicators, and historical data transmission records to obtain the adjusted flow control window parameters. This includes: determining the changing trend of the data sender's indicators over a predetermined time period based on historical data transmission records; calculating the score of the flow control parameters based on the weights corresponding to the indicators and the changing trend; and dynamically adjusting the initial flow control window parameters corresponding to the target data stream based on the scores of the flow control parameters to obtain the adjusted flow control window parameters.

[0056] When adjusting the initial flow control parameters, the changing trend of the data sender's indicators within a predetermined time period can be determined based on the data sender's historical data transmission records. This changing trend can also be used to predict the amount of data the data sender may send to the data receiver within the predetermined time period. Thus, based on the weight and changing trend of the indicators, the score of the flow control parameters can be calculated. Based on this score, the initial flow control window parameters can be dynamically adjusted. For example, the size of the flow control window can be dynamically adjusted according to the amount of data the data sender may send to the data receiver within the predetermined time period, thereby limiting the amount of data transmitted from the data sender to the data receiver, thus avoiding the buffer overflow caused by the data receiver receiving too much data.

[0057] Optionally, in the data transmission method provided in this application embodiment, transmitting the data sender's data based on the target flow control window parameters includes: using the window size in the target flow control window parameters as the size of a single data packet for transmitting the data sender's data; and updating the window size according to the window update frequency in the target flow control window parameters.

[0058] In this embodiment, the flow control window parameters used to realize data transmission between the data sender and the data receiver include window size and / or window update frequency. The window size is used to limit the flow control window size, that is, to limit the size of the data packets that the data sender can transmit in a single data transmission, which is also the size of the data packets that the data receiver can transmit in a single data transmission. The window update frequency is used to control the frequency of updating the flow control window size. Since the corresponding flow control window may decrease after each data transmission, the size of the flow control window can be updated according to the window update frequency so that the data sender can continue to send data to the data receiver and then transmit data.

[0059] Optionally, in the data transmission method provided in the embodiments of this application, after transmitting the data of the data sender based on the target flow control window parameters, the method further includes: performing a service operation based on the received data; and after the service operation is completed, sending a flow control window update data frame to the data sender, wherein the flow control window update data frame is used to allow the data sender to send data again.

[0060] In this embodiment, to avoid excessive data in the buffer on the data receiver side, the amount of data in the buffer on the data receiver side can be controlled by changing the sending strategy of the data receiver sending the flow control window update data frame to the data sender. For example, when the traffic volume is small, the original sending method can be used. That is, after the data receiver receives the data sent by the data sender and stores the data in the buffer, the data receiver sends the flow control window update data frame to the data sender to allow the data sender to continue sending data. However, when the traffic volume is large, the sending method can be changed so that after the data receiver receives the data sent by the data sender and completes the business operation based on the data, the data receiver sends the flow control window update data frame to the data sender. That is, after the data enters the buffer, no new data will be stored in the buffer before the corresponding business operation is completed. This can effectively avoid data overload and memory overflow problems in the buffer on the data receiver side.

[0061] The data transmission method provided in this application embodiment receives data from a data sender and identifies the corresponding service type; based on the service type, it determines the indicators for calculating flow control parameters and the corresponding weights of the indicators; it obtains the historical data transmission records and data transmission characteristics of the data sender; based on the service type, it determines the target data stream from multiple data streams; based on the indicators, the corresponding weights of the indicators, and the historical data transmission records, it dynamically adjusts the initial flow control window parameters corresponding to the target data stream to obtain the adjusted flow control window parameters; based on the current data load and data transmission characteristics, it optimizes the adjusted flow control window parameters to obtain the target flow control window parameters; based on the target flow control window parameters, it receives data from the data sender. This solves the problem in related technologies where excessive request pressure during peak business periods causes excessive request data to accumulate in the service provider's cache, ultimately leading to memory overflow and affecting business processing. Furthermore, it achieves the effect of adaptively adjusting the flow control size, update frequency, and update strategy of the service provider's cache according to the system's operating conditions, thereby protecting the service provider from being overwhelmed by high traffic.

[0062] Based on the above embodiments and optional embodiments, this application proposes an optional implementation method, which will be described below.

[0063] An optional implementation of this application proposes an adaptive flow control backpressure method. Figure 2 This is a schematic diagram of the overall adaptive flow control architecture provided according to an optional embodiment of this application. Figure 3 This is an adaptive flow control processing flowchart provided according to an optional implementation of this application, such as... Figure 2 and Figure 3 As shown, the method includes the following steps:

[0064] Step 1): The service consumer and the service provider establish a connection. Multiplexing is used on the same connection, meaning that multiple data streams can be established.

[0065] Step 2): Initialize the flow control window size for each data stream on the connection. This represents the amount of data the sender can transmit. When the window size is 0, the sender must wait for the receiver to send a windowupdate frame to increase the window size in order to continue sending data.

[0066] Step 3): In addition to the window size for each data stream, the connection-level window size also needs to be initialized. This represents the total amount of data that the entire connection can send. When the connection-level window size is 0, the entire connection is blocked.

[0067] Step 4): While monitoring traffic in real time, automatically identify different service types and adjust the weights of various indicators in the flow control calculation according to each service type, including but not limited to:

[0068] 1. In video streaming or large file transfer scenarios, focus on monitoring throughput and latency;

[0069] 2. In real-time interactive business scenarios, focus on monitoring packet loss rate and latency;

[0070] 3. In scenarios with large-scale concurrent connections, focus on monitoring the connection establishment success rate and connection rejection rate;

[0071] 4. Identify the business type in the traffic through deep learning algorithms, based on information such as traffic patterns, user behavior, and data packet content.

[0072] Step 5): Based on the key business indicators in Step 4, dynamically adjust the flow control calculation weight according to the important indicators of different business scenarios, and calculate the changing trend of the provider instance indicators in real time based on the historical data of the same period, analyze the trend changes in a specific time period, and calculate the data flow score for the connection of the provider instance.

[0073] Step 6): Multi-source input collaborative optimization. Collect data other than network data, such as node server load and user behavior data (based on activity level and user access frequency over time). Perform comprehensive analysis and processing to collaboratively optimize multiple objectives, such as simultaneously optimizing network efficiency, response time, and resource utilization.

[0074] Step 7): Based on the previous input, dynamically adjust the flow control backpressure window size and window size update frequency using an optimization algorithm to achieve the system's optimal solution.

[0075] Step 8): Dynamically adjust the flow control backpressure window update frequency. The previous strategy of sending window update frames immediately after receiving data packets and placing them in the buffer is changed to sending them only after receiving data and completing the business operation within the business process. Simultaneously, the update frequency is dynamically adjusted. Sending a window update frame is also a network data packet transmission. The data packet is first sent to the memory buffer, and then periodically refreshed and sent to the system kernel via the I / O thread. For window update data packets, the frequency of refreshing and sending to the system kernel via the I / O thread is dynamically adjusted based on the data flow score.

[0076] Step 9): Since a single connection multiplexes multiple data streams, considering the priority of each stream, when adjusting the flow control window size and the flow control update policy window update frequency, priority must also be taken into account to ensure that high-priority streams receive sufficient bandwidth and window size to guarantee transmission efficiency and reliability. Through system presets, integrated critical service monitoring, machine learning, and other methods, the service priority of each data stream is intelligently identified. Higher-priority services will be provided with larger windows and faster update frequencies.

[0077] Each data stream (video call, chat, file sharing, and online quiz) is assigned a unique identifier. This identifier is used to distinguish different data streams at the receiving end. Before sending data, each data stream is encapsulated into a frame, and the data stream identifier is added to the frame header. At the receiving end, using the same multiplexing protocol as the sending end, different data streams are separated based on the data stream identifier in the frame header and transmitted to the corresponding processing modules (such as video players, chat windows, etc.). In other words, different data streams are determined for transmission based on the different service types of the data.

[0078] In addition, error handling mechanisms can be designed to handle potential data loss or corruption. Flow control strategies can also be implemented to prevent one data stream from consuming excessive bandwidth, causing performance degradation in other data streams.

[0079] In summary, the optional implementation methods of this application can bring the following beneficial effects:

[0080] 1. Adjust the service provider's flow control window update strategy to put pressure on the service consumer and prevent memory overflow.

[0081] 2. Adaptively adjust flow control strategies to dynamically achieve optimal performance.

[0082] 3. Intelligently identify business types and adaptively optimize flow control and backpressure strategies.

[0083] 4. Intelligently identify service priorities and optimize flow control backpressure strategies when multiple connections are reused.

[0084] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0085] This application also provides a data transmission apparatus. It should be noted that the data transmission apparatus of this application can be used to execute the data transmission method provided in this application. The data transmission apparatus provided in this application will be described below.

[0086] Figure 4 This is a schematic diagram of a data transmission apparatus provided according to an embodiment of this application. Figure 4 As shown, the device includes: an identification module 41, a first determination module 42, an acquisition module 43, a second determination module 44, an adjustment module 45, an optimization module 46, and a receiving module 47. The device will be described below.

[0087] The system comprises: an identification module 41 for receiving data from the data sender and identifying the corresponding service type; a first determination module 42 connected to the identification module 41 for determining the indicators for calculating flow control parameters and the corresponding weights based on the service type; an acquisition module 43 connected to the first determination module 42 for acquiring historical data transmission records and data transmission characteristics of the data sender; a second determination module 44 connected to the acquisition module 43 for determining the target data stream from multiple data streams based on the service type; an adjustment module 45 connected to the second determination module 44 for dynamically adjusting the initial flow control window parameters corresponding to the target data stream based on the indicators, the corresponding weights, and historical data transmission records to obtain the adjusted flow control window parameters; an optimization module 46 connected to the adjustment module 45 for optimizing the adjusted flow control window parameters based on the current data load and data transmission characteristics to obtain the target flow control window parameters; and a receiving module 47 connected to the optimization module 46 for transmitting data from the data sender based on the target flow control window parameters.

[0088] Optionally, the data transmission apparatus provided in this application embodiment further includes: an establishment unit, used to establish a multiplexed connection with the data sender to obtain multiple data streams.

[0089] Optionally, the data transmission device provided in this application embodiment further includes: a first initialization unit for initializing the flow control window size of the connection; and a second initialization unit for initializing the flow control window size corresponding to each of the multiple data streams.

[0090] Optionally, the data transmission apparatus provided in this application embodiment further includes: a first acquisition unit, configured to acquire the priorities corresponding to the multiple data streams respectively; a first determination unit, configured to determine the flow control window parameters corresponding to the multiple data streams respectively based on the priorities; and a first receiving unit, configured to receive data from the multiple data streams respectively according to the flow control window parameters corresponding to the multiple data streams respectively.

[0091] Optionally, in the data transmission apparatus provided in this application embodiment, the adjustment module 45 includes: a second determining unit, used to determine the changing trend of the data sender's index over a predetermined time period based on historical data transmission records; a calculation unit, used to calculate the score of the flow control parameter based on the weight and changing trend corresponding to the index; and an adjustment unit, used to dynamically adjust the initial flow control window parameter corresponding to the target data stream based on the score of the flow control parameter, to obtain the adjusted flow control window parameter.

[0092] Optionally, in the data transmission apparatus provided in this application embodiment, the receiving module 47 includes: a second receiving unit, used to transmit the data of the data sender by using the window size in the target flow control window parameters as the size of a single data packet; and an updating unit, used to update the window size according to the window update frequency in the target flow control window parameters.

[0093] Optionally, the data transmission apparatus provided in this application embodiment further includes: an operation unit for performing service operations based on the received data; and a sending unit for sending a flow control window update data frame to the data sender after the service operation is completed, wherein the flow control window update data frame is used to allow the data sender to send data again.

[0094] The data transmission apparatus provided in this application embodiment receives data from a data sender and identifies the corresponding service type; based on the service type, it determines the indicators for calculating flow control parameters and the corresponding weights of the indicators; it obtains the historical data transmission records and data transmission characteristics of the data sender; based on the service type, it determines the target data stream from multiple data streams; based on the indicators, the corresponding weights of the indicators, and the historical data transmission records, it dynamically adjusts the initial flow control window parameters corresponding to the target data stream to obtain the adjusted flow control window parameters; based on the current data load and data transmission characteristics, it optimizes the adjusted flow control window parameters to obtain the target flow control window parameters; based on the target flow control window parameters, it receives data from the data sender. This solves the problem in related technologies where, during peak business periods, excessive request pressure causes the service provider's cache to accumulate too much request data, ultimately leading to memory overflow and affecting business processing. Furthermore, it achieves the effect of adaptively adjusting the flow control size, update frequency, and update strategy of the service provider's cache according to the system's operating conditions, thereby protecting the service provider from being overwhelmed by high traffic.

[0095] The aforementioned data transmission device includes a processor and a memory. The identification module 41, the first determination module 42, the acquisition module 43, the second determination module 44, the adjustment module 45, the optimization module 46, and the receiving module 47 are all stored in the memory as program units. The processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0096] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured. By adjusting kernel parameters, the problem in related technologies where excessive request pressure during peak business periods causes the service provider's cache to accumulate too much request data, ultimately leading to memory overflow and affecting business processing can be addressed. This allows the processor to adaptively adjust the service provider's cache size, update frequency, and update strategy based on system operating conditions, thus protecting the service provider from being overwhelmed by high traffic.

[0097] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0098] This invention provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the data transmission method.

[0099] This invention provides a processor for running a program, wherein the program executes the data transmission method during runtime.

[0100] This invention provides an electronic device, including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: receiving data from a data sender and identifying the service type corresponding to the data; determining the metrics for calculating flow control parameters and the corresponding weights based on the service type; acquiring historical data transmission records and data transmission characteristics from the data sender; determining the target data stream from multiple data streams based on the service type; dynamically adjusting the initial flow control window parameters corresponding to the target data stream based on the metrics, the corresponding weights, and the historical data transmission records to obtain adjusted flow control window parameters; optimizing the adjusted flow control window parameters based on the current data load and data transmission characteristics to obtain target flow control window parameters; and transmitting the data from the data sender based on the target flow control window parameters. The device described herein can be a server, PC, PAD, mobile phone, etc.

[0101] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: receiving data from a data sender and identifying the service type corresponding to the data; determining the indicators for calculating flow control parameters and the corresponding weights based on the service type; obtaining historical data transmission records and data transmission characteristics of the data sender; determining the target data stream from multiple data streams according to the service type; dynamically adjusting the initial flow control window parameters corresponding to the target data stream based on the indicators, the weights corresponding to the indicators, and the historical data transmission records to obtain the adjusted flow control window parameters; optimizing the adjusted flow control window parameters based on the current data load and data transmission characteristics to obtain the target flow control window parameters; and transmitting the data from the data sender based on the target flow control window parameters.

[0102] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0104] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0105] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0106] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0107] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0108] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0109] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0110] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A data transmission method, characterized by, include: Receive data from the data sender and identify the service type corresponding to the data; Based on the business type, determine the indicators for calculating the flow control parameters, and the corresponding weights of the indicators; Obtain the historical data sending records and data sending characteristics of the data sender, wherein the data sending characteristics are determined based on the data sender's behavioral data, including the data sender's data sending activity in different time periods and / or the data sender's access frequency; Based on the business type, the target data stream is determined from multiple data streams; Based on the metric, the weight corresponding to the metric, and the historical data transmission record, the initial flow control window parameters corresponding to the target data stream are dynamically adjusted to obtain the adjusted flow control window parameters. Based on the current data load and the data transmission characteristics, the adjusted flow control window parameters are optimized to obtain the target flow control window parameters; Based on the target flow control window parameters, the data sent by the data sender is transmitted; The step of dynamically adjusting the initial flow control window parameters corresponding to the target data stream based on the indicator, the weight corresponding to the indicator, and the historical data transmission records to obtain the adjusted flow control window parameters includes: determining the change trend of the indicator of the data sender over a predetermined time period based on the historical data transmission records; calculating the score of the flow control parameters based on the weight corresponding to the indicator and the change trend; and dynamically adjusting the initial flow control window parameters corresponding to the target data stream based on the score of the flow control parameters to obtain the adjusted flow control window parameters.

2. The method of claim 1, wherein, Before receiving data from the data sender and identifying the service type corresponding to the data, the method further includes: A multiplexed connection is established with the data sender to obtain the multiple data streams.

3. The method of claim 2, wherein, The method further includes: Initialize the flow control window size of the connection; Initialize the flow control window size corresponding to each of the multiple data streams.

4. The method of claim 2, wherein, The method further includes: Obtain the priority of each of the multiple data streams; Based on the priority, the flow control window parameters corresponding to the multiple data streams are determined respectively; Data from the multiple data streams is received according to the flow control window parameters corresponding to each of the multiple data streams.

5. The method of claim 1, wherein, The transmission of data from the data sender based on the target flow control window parameters includes: The window size in the target flow control window parameters is used as the size of a single data packet for transmitting data from the data sender. The window size is updated according to the window update frequency in the target flow control window parameters.

6. The method according to any one of claims 1 to 5, characterized in that, After transmitting the data from the data sender based on the target flow control window parameters, the method further includes: Perform business operations based on the received data; After the business operation is completed, a flow control window update data frame is sent to the data sender, wherein the flow control window update data frame is used to allow the data sender to send data again.

7. A data transmission device, characterized in that, include: The identification module is used to receive data from the data sender and identify the service type corresponding to the data; The first determining module is used to determine the indicators for calculating the flow control parameters and the weights corresponding to the indicators based on the business type. The acquisition module is used to acquire the historical data transmission records and data transmission characteristics of the data sender, wherein the data transmission characteristics are determined based on the behavioral data of the data sender, including the data sender's data transmission activity in different time periods and / or the data sender's access frequency. The second determining module is used to determine the target data stream from multiple data streams based on the business type. The adjustment module is used to dynamically adjust the initial flow control window parameters corresponding to the target data stream based on the indicator, the weight corresponding to the indicator and the historical data transmission record, so as to obtain the adjusted flow control window parameters. The optimization module is used to optimize the adjusted flow control window parameters based on the current data load and the data transmission characteristics to obtain the target flow control window parameters. The receiving module is used to transmit the data from the data sender based on the target flow control window parameters; The adjustment module is further configured to determine the trend of the indicator of the data sender over a predetermined time period based on the historical data sending records; calculate the score of the flow control parameter based on the weight corresponding to the indicator and the trend of change; and dynamically adjust the initial flow control window parameter corresponding to the target data stream based on the score of the flow control parameter to obtain the adjusted flow control window parameter.

8. A computer storage medium, characterized in that, The computer storage medium is used to store a program, wherein the program, when running, controls the device where the computer storage medium is located to execute the data transmission method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the data transmission method according to any one of claims 1 to 6.

Images