Data transmission control method, device and computer readable storage medium

Abstract

The present application discloses a data transmission control method, device and computer-readable storage medium. The method comprises the following steps: obtaining historical network performance data of multiple data transmission objects, and calculating group network performance data of each object group; performing weighted calculation on the group network performance data to obtain target group network performance data; performing reinforcement learning on the target group network performance data and the group network performance data of each object group based on a reinforcement learning model to obtain a data transmission index of each object group; calculating data transmission parameters of each object group based on the data transmission index and the group network performance data; when receiving a data transmission connection request sent by a target data transmission object, determining a target data transmission parameter based on the object group to which the target data transmission object belongs, and controlling the data transmission process based on the target data transmission parameter. The method can improve data transmission performance.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a data transmission control method, device and computer-readable storage medium. Background Art

[0002] In recent years, with the continuous rise of various short video services, audio and video traffic has shown explosive growth. The contradiction between the ever-increasing user experience requirements of clients and the lagging network transmission performance has become the main contradiction that needs to be solved urgently. In this regard, technicians in this field have proposed a series of congestion control technologies, such as the Cubic (full name Cubic Binary Increase Congestion) algorithm based on packet loss information, the BBR algorithm based on bandwidth detection (Bottleneck Bandwidth and RTT, an algorithm for network congestion control provided by Google), etc. The above congestion control algorithm can improve the quality of network transmission to a certain extent.

[0003] However, current congestion control algorithms are all based on fixed rules set based on expert experience, which are difficult to adapt to changing network conditions and network services, making it impossible to achieve good results in improving network transmission quality in some new scenarios. Summary of the invention

[0004] The embodiments of the present application provide a data transmission control method, device and computer-readable storage medium, which can improve the transmission performance of network traffic.

[0005] A first aspect of the present application provides a data transmission control method, the method comprising:

[0006] Acquire historical network performance data of a plurality of data transmission objects, and calculate group network performance data of each object group based on the historical network performance data, wherein the object group is a group obtained by dividing the plurality of data transmission objects;

[0007] Performing weighted calculation on the group network performance data to obtain target group network performance data;

[0008] Performing reinforcement learning on the target group network performance data and the group network performance data of each object group based on a reinforcement learning model to obtain a data transmission index of each object group;

[0009] Calculate a data transmission parameter for each object group according to the data transmission index and the group network performance data;

[0010] When a data transmission connection request sent by a target data transmission object is received, a target data transmission parameter is determined according to the object group to which the target data transmission object belongs, and the data transmission process is controlled based on the target data transmission parameter.

[0011] A second aspect of the present application provides a data transmission control device, the device comprising:

[0012] an acquisition unit, configured to acquire historical network performance data of a plurality of data transmission objects, and calculate group network performance data of each object group based on the historical network performance data, wherein the object group is a group obtained by dividing the plurality of data transmission objects;

[0013] A first calculation unit, configured to perform weighted calculation on the group network performance data to obtain target group network performance data;

[0014] A learning unit, configured to perform reinforcement learning on the target group network performance data and the group network performance data of each object group based on a reinforcement learning model to obtain a data transmission index of each object group;

[0015] A second calculation unit, configured to calculate a data transmission parameter of each object group according to the data transmission index and the group network performance data;

[0016] The control unit is used to determine the target data transmission parameters according to the object group to which the target data transmission object belongs when receiving a data transmission connection request sent by the target data transmission object, and control the data transmission process based on the target data transmission parameters.

[0017] In some embodiments, the learning unit includes:

[0018] A first calculation subunit, configured to calculate a reinforcement learning parameter of each object group according to the group network performance data of each object group;

[0019] The learning subunit is used to input the target group network performance data as the state value of the reinforcement learning model and the reinforcement learning parameter as the reward value of the reinforcement learning model into the reinforcement learning model to obtain the data transmission index of each object group as output.

[0020] In some embodiments, the first computing subunit includes:

[0021] An extraction module, used for extracting effective throughput data and smoothed round-trip transmission time data from the group network performance data of each object group;

[0022] The first calculation module is used to calculate the ratio between the effective throughput data corresponding to each object group and the corresponding smoothed round-trip transmission time data to obtain the reinforcement learning parameters of each object group.

[0023] In some embodiments, the acquisition unit includes:

[0024] A first acquisition subunit, configured to acquire Internet Protocol address data of a plurality of data transmission objects and to acquire historical network performance data of the plurality of data transmission objects;

[0025] a division subunit, configured to divide the plurality of data transmission objects into a plurality of object groups based on the Internet Protocol address data;

[0026] The second calculation subunit is used to calculate the group network performance data of each object group according to the historical network performance data of the data transmission objects in each object group.

[0027] In some embodiments, the second computing subunit includes:

[0028] A determination module, configured to determine a set of historical network performance data corresponding to each object group according to the historical network performance data of each data transmission object in each object group;

[0029] The second calculation module is used to calculate the average value of the historical network performance data in each historical network performance data set to obtain the group network performance data corresponding to each object group.

[0030] In some embodiments, the first computing unit includes:

[0031] A second acquisition subunit, used to acquire a weight coefficient of each object group;

[0032] The third calculation subunit is used to perform weighted calculation on the group network performance data of each object group based on the weight coefficient to obtain the target group network performance data.

[0033] In some embodiments, the second acquisition subunit includes:

[0034] An acquisition module, used for acquiring the number of data transmission objects included in each object group;

[0035] The third calculation module is used to calculate the ratio between the number of data transmission objects in each group and the number of the plurality of data transmission objects to obtain a weight coefficient of each object group.

[0036] In some embodiments, the second computing unit includes:

[0037] an extraction subunit, configured to extract effective throughput data and minimum round-trip transmission time data of each object group from the group network performance data;

[0038] The fourth calculation subunit is used to calculate the initial window data of each object group based on the effective throughput data, the minimum round-trip transmission time data and the data transmission index of each object group, and determine the data transmission parameters of each object group based on the initial window data of each object group.

[0039] In some embodiments, the control unit comprises:

[0040] A third acquisition subunit is used to acquire a target Internet Protocol address of the target data transmission object when receiving a data transmission connection request sent by the target data transmission object;

[0041] A determination subunit, configured to determine a target object group to which the target data transmission object belongs according to the target Internet Protocol address;

[0042] The transmission subunit is used to obtain target data transmission parameters of the target object group, and perform data transmission on the target data transmission objects based on the target data transmission parameters.

[0043] The third aspect of the present application also provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the steps in the data transmission control method provided in the first aspect of the present application.

[0044] The fourth aspect of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the data transmission control method provided in the first aspect of the present application when executing the computer program.

[0045] A fifth aspect of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps in the data transmission control method provided in the first aspect.

[0046] The data transmission control method provided in the embodiment of the present application obtains historical network performance data of multiple data transmission objects, and calculates group network performance data of each object group based on the historical network performance data, where the object group is a group obtained by dividing multiple data transmission objects; performs weighted calculation on the group network performance data to obtain target group network performance data; performs reinforcement learning on the target group network performance data and the group network performance data of each object group based on a reinforcement learning model to obtain a data transmission index for each object group; calculates data transmission parameters for each object group based on the data transmission index and the group network performance data; when a data transmission connection request sent by the target data transmission object is received, determines the target data transmission parameters according to the object group to which the target data transmission object belongs, and controls the data transmission process based on the target data transmission parameters.

[0047] Thus, the data transmission control method provided by the present application divides the data transmission objects into groups, and then uses the historical network performance data of each group in the recent period to perform reinforcement learning to obtain the most suitable network data transmission parameters for each group at the current time. This method can perform fine-grained control of network data transmission parameters, thereby matching the most suitable network data transmission parameters for each data transmission object at the current time, which can greatly improve the data transmission performance of the data transmission object. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 It is a schematic diagram of a scenario of data transmission control in this application;

[0050] Figure 2 It is a flowchart of the data transmission control method provided by the present application;

[0051] Figure 3 is another flow chart of the data transmission control method provided by the present application;

[0052] Figure 4a and Figure 4b This is a schematic diagram of the division of IP addresses;

[0053] Figure 5 It is another flow chart of the data transmission control method provided by the present application;

[0054] Figure 6It is a structural schematic diagram of the data transmission control device provided by the present application;

[0055] Figure 7 It is a structural schematic diagram of the computer device provided by this application. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0057] The embodiment of the present invention provides a data transmission control method, device, computer-readable storage medium and computer equipment. Among them, the data transmission control method can be used in a data transmission control device. The data transmission control device can be integrated in a computer device, which can be a terminal or a server. Among them, the terminal can be a mobile phone, a tablet computer, a laptop computer, a smart TV, a wearable smart device, a personal computer (PC, Personal Computer) and a vehicle-mounted terminal and other devices. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and big data and artificial intelligence platforms. Among them, the server can be a node in a blockchain.

[0058] See also Figure 1 , which is a schematic diagram of a scenario of the data transmission control method provided by the present application. As shown in the figure, the gateway server A obtains the historical network performance data of multiple data transmission objects, and calculates the group network performance data of each object group based on the historical network performance data, where the object group is a group obtained by dividing multiple data transmission objects; weighted calculation is performed on the group network performance data to obtain the target group network performance data; reinforcement learning is performed on the target group network performance data and the group network performance data of each object group based on the reinforcement learning model to obtain the data transmission index of each object group; the data transmission parameters of each object group are calculated based on the data transmission index and the group network performance data; when the gateway server A receives the data transmission request sent by the terminal B, it can determine the corresponding target data transmission parameters according to the object group to which the terminal B belongs, and then control the transmission process of data transmission to the terminal B based on the target data transmission parameters.

[0059] It should be noted that Figure 1 The data transmission control scenario diagram shown is only an example. The data transmission control scenario described in the embodiment of the present application is to more clearly illustrate the technical solution of the present application and does not constitute a limitation on the technical solution provided by the present application. It is known to those skilled in the art that with the evolution of data transmission control scenarios and the emergence of new business scenarios, the technical solution provided by the present application is also applicable to similar technical problems.

[0060] Based on the above implementation scenarios, detailed descriptions are given below.

[0061] In the network data transmission scenario, when the amount of audio and video data to be transmitted is large, data transmission congestion will occur. In order to solve the problem of data transmission congestion and improve data transmission efficiency and data transmission quality, relevant technical personnel in this field have proposed a variety of congestion control algorithms to improve network transmission quality. In this regard, with the increasing maturity of artificial intelligence technology in recent years, adaptive congestion control based on machine learning can automatically adapt to changes in network status and obtain benefits in network transmission performance through parameter setting and other methods. These methods can set the relevant parameters of congestion control through reinforcement learning methods to improve the quality of network data transmission. However, these methods are difficult to be feasible because the parameter sets are set by human experience and these methods are completely separated from the existing congestion control mechanism. In addition, these methods have the problem of lack of stability, which makes them unable to be deployed and used in the existing network. In this regard, the present application provides a data transmission control method, in order to alleviate the congestion problem in data transmission to a certain extent and improve data transmission performance.

[0062] The embodiments of the present application will be described from the perspective of a data transmission control device, which can be integrated into a computer device. The computer device can be a terminal or a server. The terminal can be a mobile phone, a tablet computer, a laptop computer, a smart TV, a wearable smart device, a personal computer (PC, Personal Computer), a vehicle-mounted terminal and other devices. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), as well as basic cloud computing services such as big data and artificial intelligence platforms. For example Figure 2 FIG. 1 is a flow chart of a data transmission control method provided by the present application, and the method includes:

[0063] Step 101 : Acquire historical network performance data of a plurality of data transmission objects, and calculate group network performance data of each object group based on the historical network performance data.

[0064] Among them, multiple data transmission objects can be all registered users of a certain application. For example, an instant messaging application or a short video application has an application server that provides data services, and multiple application clients that register the application. Since each application client can be the object of data transmission by the application server of the application, then all application clients registered with the application can be determined here as multiple data transmission objects. Or, in some cases, multiple data transmission objects of some clients registered with the application can also be set. Similarly, for example, in the aforementioned short video application, only some of the application clients registered with the application are member user clients, so data transmission control can be performed only for these member user clients to optimize the data transmission performance during data transmission, and provide them with good data transmission services and smooth viewing experience. Or, in some cases, some users have less demand for data transmission and another part of users have higher demand for data transmission, then targeted optimization can be performed for these users with higher data transmission demand, and these users with higher data transmission demand can be determined as multiple data transmission objects. That is, in the embodiment of the present application, multiple data transmission objects can be personalized, which is not limited here.

[0065] The historical network performance data of the data transmission object may be the network performance data of the data transmission object counted over a period of time. Specifically, the network performance data of the data transmission object may be periodically counted, for example, the network performance data between the last statistical time and the current statistical time may be counted every T time. When it is necessary to control the data transmission, the network performance data of the data transmission object counted the most recently may be used as its historical network performance data. The network performance data of the data transmission object may specifically be its network service quality information (Quality of Service, QoS), and the QoS information may specifically include: minimum round trip time (round trip time, RTT) (min_rtt), maximum round trip time (max_rtt), smooth round trip time (srtt), maximum in-flight (max_inflight), effective throughput (goodput), retransmission rate (retran_ratio), initial window (init_cwnd) and maximum window (max_cwnd) and other information.

[0066] In the embodiment of the present application, the multiple data transmission objects can be divided into multiple groups, which can be referred to as object groups. In this way, the historical network performance data corresponding to each group can be calculated based on the historical network performance data of the data transmission objects in each group, that is, the group network performance data of each object group can be calculated.

[0067] In some embodiments, historical network performance data of multiple data transmission objects are obtained, and group network performance data of each object group is calculated based on the historical network performance data, where the object group is a group obtained by dividing the multiple data transmission objects, including:

[0068] 1. Obtaining Internet Protocol address data of multiple data transmission objects and obtaining historical network performance data of multiple data transmission objects;

[0069] 2. Divide multiple data transmission objects into multiple object groups based on Internet Protocol address data;

[0070] 3. Calculate the group network performance data of each object group based on the historical network performance data of the data transmission objects in each object group.

[0071] In an embodiment of the present application, a plurality of data transmission objects are grouped, specifically, they can be grouped according to the Internet Protocol Address (IP address) of each data transmission object. As mentioned above, the data transmission object can be a client of an application, and the IP address of the data transmission object can be the IP address of the terminal on which the client is loaded. The terminal can be any terminal that can connect to the Internet and transmit data, such as a smart phone, a tablet, a smart watch, or a personal computer.

[0072] Specifically, the IP address data of multiple data transmission objects and the historical network performance data of each data transmission object may be obtained first. Then, the multiple data transmission objects may be divided into multiple object groups based on the IP address data of each data transmission object. The group network performance data of each object group is then calculated based on the historical network performance data of the data transmission objects contained in each object group. Among them, the group division is performed based on the IP address data of the data transmission object, and the division may be performed based on the region, operator number, autonomous system (AS) number, access mode and other rules of the IP address. When the group division is performed based on the IP address data of the data transmission object, the division may be performed based on all rules or based on some rules. For example, when the division is performed based on all rules, the division may be performed based on the region+operator+AS number+access mode; when the division is performed based on some rules, the division may be performed based on the region+operator, region+access mode or region and other rules. It can be understood that the data transmission process of the data transmission objects in the same object group has similar network forwarding paths and also has similar network status information. That is, the data transmission objects in the same object group may be controlled by the same data transmission parameters when performing data transmission control.

[0073] In some embodiments, calculating the group network performance data of each object group according to the historical network performance data of the data transmission objects in each object group includes:

[0074] 3.1. Determine a set of historical network performance data corresponding to each object group according to the historical network performance data of each data transmission object in each object group;

[0075] 3.2. Calculate the average value of the historical network performance data in each historical network performance data set to obtain the group network performance data corresponding to each object group.

[0076] Among them, in the embodiment of the present application, the calculation of the group network performance data corresponding to each object group can be obtained by averaging the historical network performance data of the data transmission objects contained in each object group. Specifically, the data transmission objects contained in each object group can be determined first, and then the historical network performance data of each data transmission object in the group can be determined to obtain a set of historical network performance data corresponding to each object group. Furthermore, the average value of the historical network performance data in each set of historical network performance data can be calculated to obtain the group network performance data corresponding to each object group.

[0077] In some embodiments, before calculating the average of the historical network performance data of multiple data transmission objects in each object group to calculate the group network performance data of the object group, anomaly detection can be performed on the historical network performance data of each data transmission object to determine that there is no abnormal data in the historical network performance data of the data transmission objects in the group, thereby ensuring the accuracy of the calculation of the group network performance data. In some embodiments, the anomaly detection of the historical network performance data of each data transmission object can be performed based on the historical network performance data in a longer time dimension. For example, if the historical network performance data of the data transmission object is the QoS data within 1 hour before the current time, the QoS data of the data transmission object within 1 month before the current time can be used to detect the QoS data of the data transmission object within 1 hour before the current time to determine that there is no obvious abnormality in the QoS data. Specifically, the QoS data of the data transmission object within 1 hour before the current time can be predicted based on the QoS data of the data transmission object within 1 month before the current time to obtain the predicted QoS data; then the difference between the predicted QoS data and the actual statistical QoS is calculated. If the difference rate is greater than the preset ratio, it is determined that the historical network performance data of the data transmission object is abnormal.

[0078] If it is determined that the QoS data of the data transmission object has obvious abnormalities, the historical network performance data of the data transmission object can be removed when calculating the group network performance data.

[0079] Step 102: Perform weighted calculation on the group network performance data to obtain target group network performance data.

[0080] After calculating the group network performance data of each object group, the group network performance data of all object groups may be further weighted to obtain the overall network performance data of all data transmission objects, which may be referred to as target group network performance data.

[0081] In some embodiments, weighted calculation is performed on the group network performance data to obtain target group network performance data, including:

[0082] 1. Obtain the weight coefficient of each object group;

[0083] 2. Perform weighted calculation on the group network performance data of each target group based on the weight coefficient to obtain the target group network performance data.

[0084] In an embodiment of the present application, the target group network performance data is calculated based on the group network performance data of each object group. Specifically, the weight coefficient corresponding to each object group can be obtained first, and then the group network performance data of each object group can be weighted calculated based on the weight coefficient of each object group to obtain the target group network performance data.

[0085] In some embodiments, obtaining the weight coefficient of each object group includes:

[0086] 1.1. Obtain the number of data transfer objects contained in each object group;

[0087] 1.2. Calculate the ratio between the number of data transmission objects in each group and the number of multiple data transmission objects to obtain a weight coefficient of each object group.

[0088] In the embodiment of the present application, the weight coefficient of each object group can be calculated based on the number of data transmission objects included in each object group. Specifically, the number of data transmission objects included in each object group can be obtained first, and then the ratio between the number of data transmission objects in each group and the number of the aforementioned multiple data transmission objects can be calculated, that is, the ratio of the number of members in each group to the total number of members is calculated, and then the weight coefficient of each object group can be obtained.

[0089] Step 103 , performing reinforcement learning on the target group network performance data and the group network performance data of each object group based on the reinforcement learning model to obtain a data transmission index of each object group.

[0090] Among them, the reinforcement learning model (RL) is one of the fields of machine learning. Inspired by behavioral psychology, it focuses on how agents take different actions in the environment to maximize cumulative rewards. Reinforcement learning is mainly composed of agents, environments, states, actions, and rewards. After the agent performs an action, the environment will switch to a new state, and the environment will give a reward signal (positive reward or negative reward) for the new state. Subsequently, the agent performs new actions according to a certain strategy based on the new state and the reward fed back by the environment. The above process is the way in which the agent and the environment interact through state, action, and reward. Through reinforcement learning, the agent can know what state it is in and what actions it should take to get the maximum reward. Since the way the agent interacts with the environment is similar to the way humans interact with the environment, reinforcement learning can be considered a general learning framework that can be used to solve the problem of general artificial intelligence. Therefore, reinforcement learning is also called the machine learning method of general artificial intelligence.

[0091] In the embodiment of the present application, the network performance data setting in the data transmission process can be controlled based on the above characteristics of the reinforcement learning model. Specifically, the corresponding rewards and environmental states can be set based on the target group network performance data and the group network performance data of each object group, so that the preset reinforcement learning model learns the data transmission index corresponding to each object group, and the data transmission index can be used to control the data transmission parameters of the corresponding object group to obtain a better reward value.

[0092] Specifically, in some embodiments, reinforcement learning is performed on the target group network performance data and the group network performance data of each object group based on the reinforcement learning model to obtain the data transmission index of each object group, including:

[0093] 1. Calculate the reinforcement learning parameters of each object group based on the group network performance data of each object group;

[0094] 2. The target group network performance data is used as the state value of the reinforcement learning model, and the reinforcement learning parameter is used as the reward value of the reinforcement learning model. The reinforcement learning model is input to obtain the data transmission index of each target group as output.

[0095] In an embodiment of the present application, the reinforcement learning parameters of each object group can be calculated based on the group network performance data of each object group as the reward value of the reinforcement learning model, and then the target group network performance data calculated above is used as the state value of the reinforcement learning model as the input of the reinforcement learning model, and then the reinforcement learning model is subjected to reinforcement learning processing based on the above reward value and state value to obtain the output parameters learned by the reinforcement learning model. It can be understood that since the reinforcement learning parameters calculated for different object groups are different, that is, the reward values ​​of the reinforcement learning model are different, the learned parameters are also different. That is, for different object groups, the reinforcement learning model can learn different output parameters, and the output parameters can control the data transmission parameters of each object group so that the reinforcement learning parameters of the object group are continuously improved, so the output parameters can also be called the data transmission index.

[0096] In some embodiments, calculating the reinforcement learning parameters of each group of objects according to the group network performance data of each group of objects includes:

[0097] 1.1. Extract effective throughput data and smoothed round-trip transmission time data from the group network performance data of each object group;

[0098] 1.2. Calculate the ratio between the effective throughput data corresponding to each object group and the corresponding smoothed round-trip transmission time data to obtain the reinforcement learning parameters of each object group.

[0099] In an embodiment of the present application, the reinforcement learning parameters of each object group are calculated, and specifically, the effective throughput data and the smooth round-trip transmission time data in the group network performance data of each object group can be used for calculation. Specifically, the effective throughput data and the smooth round-trip transmission time data can be first extracted from the group network performance data of each object group, and then the ratio between the effective throughput data and the smooth round-trip transmission time data is calculated as the reinforcement learning parameter of each object group. The ratio between the effective throughput and the smooth round-trip transmission time data is set as the reinforcement learning parameter of each object group, and specifically the ratio is used as the reward value of the reinforcement learning model. It is mainly considered that the learning goal of the reinforcement learning model is to obtain a higher reward value, that is, to obtain a larger effective throughput and less smooth round-trip transmission time, that is, to obtain a better QoS.

[0100] In some embodiments, a reinforcement learning model is used to learn the data transmission index of each object group. Multiple reinforcement learning can be performed based on the target group network performance data and the group network performance data of each object group using the reinforcement learning model to obtain multiple data transmission indexes for each object group. Then, the mean of these multiple data transmission indexes is further calculated to finally obtain a more accurate data transmission index for each object group. In an embodiment of the present application, the mean of the output parameters obtained by multiple learning is used as the final output, which can avoid the problem of inaccurate learned data transmission index due to learning errors, thereby improving the accuracy of the learned data transmission index.

[0101] Step 104 , calculating the data transmission parameters of each object group according to the data transmission index and the group network performance data.

[0102] After the data transmission parameters of each object group are learned by the reinforcement learning model, the current group network performance data of each object group can be further adjusted according to the data transmission parameters of each object group to obtain the data transmission parameters of each object group. By controlling the data transmission process of the corresponding object group through the data transmission parameters of each object group, a better data transmission effect can be obtained, that is, the data transmission performance can be improved.

[0103] In some embodiments, the data transmission parameters of each object group are calculated based on the data transmission index and the group network performance data, including:

[0104] 1. Extract the effective throughput data and minimum round-trip transmission time data of each object group from the group network performance data;

[0105] 2. Calculate the initial window data of each object group based on the effective throughput data, minimum round-trip transmission time data and data transmission index of each object group, and determine the data transmission parameters of each object group based on the initial window data of each object group.

[0106] Among them, in the embodiment of the present application, the group network performance data of the corresponding object group is adjusted based on the learned data transmission index of each object group. Specifically, effective throughput data and minimum round-trip transmission time data can be extracted from the group network performance data of each object group, and then the initial window data of each object group can be calculated based on the effective throughput data, the minimum round-trip transmission time data and the learned data transmission index, and then the initial window data of each object group can be determined as the data transmission parameter of each object group. Furthermore, the data transmission parameter of each object group can be associated with the corresponding object group and stored for use.

[0107] Step 105: when receiving a data transmission connection request sent by a target data transmission object, determining a target data transmission parameter according to the object group to which the target data transmission object belongs, and controlling the data transmission process based on the target data transmission parameter.

[0108] After the data transmission parameters corresponding to each object group are determined, the data transmission process of each data transmission object can be controlled based on the data transmission parameters corresponding to each object group before the historical network performance data statistics of the next cycle are completed.

[0109] Specifically, when a data transmission connection request sent by a target data transmission object is received, the object group corresponding to the target data transmission object can be determined first, and then the target data transmission parameters of the object group can be obtained, and the data transmission process of the target data transmission object can be controlled based on the target data transmission parameters. For example, when a data connection request is received from a client of a certain application to an application server, the target object group to which the client belongs can be determined, and the initial window data of the target object group can be determined, and then the process of data transmission of the client can be controlled using the initial window data object.

[0110] According to the above description, the data transmission control method provided in the embodiment of the present application obtains historical network performance data of multiple data transmission objects, and calculates group network performance data of each object group based on the historical network performance data, where the object group is a group obtained by dividing multiple data transmission objects; performs weighted calculation on the group network performance data to obtain target group network performance data; performs reinforcement learning on the target group network performance data and the group network performance data of each object group based on a reinforcement learning model to obtain a data transmission index for each object group; calculates data transmission parameters for each object group based on the data transmission index and the group network performance data; when a data transmission connection request sent by the target data transmission object is received, determines the target data transmission parameters according to the object group to which the target data transmission object belongs, and controls the data transmission process based on the target data transmission parameters.

[0111] Thus, the data transmission control method provided by the present application divides the data transmission objects into groups, and then uses the historical network performance data of each group in the recent period of time to perform reinforcement learning to obtain the most suitable network data transmission parameters for each group at the current time. This method can perform fine-grained control of network data transmission parameters, thereby matching the most suitable network data transmission parameters for each data transmission object at the current time, which can greatly improve the data transmission performance of the data transmission object. Moreover, the data transmission control method provided by the present application can make data transmission control more fine-grained and more accurate, and the setting of session parameters when the connection is reached is also more targeted. This method of setting data transmission control parameter values ​​for each user group that conforms to its network characteristics is conducive to improving the overall transmission quality of the network system.

[0112] The present application also provides a data transmission control method, which can be used in a computer device, and the computer device can be a server. Figure 3 FIG. 1 is another flow chart of the data transmission control method provided by the present application, and the method specifically includes:

[0113] Step 201: The server divides users into groups to obtain multiple user groups.

[0114] Specifically, the server can divide the IP addresses of the clients corresponding to the users according to the rules such as region, operator, AS number, access method, etc., and divide the client IP addresses into different user groups, where the connection (or session) of each user group has a similar network forwarding path and also has similar network status information.

[0115] The division of user groups is not limited to the division based on the IP addresses of all clients, but can also be divided based on certain important IP address prefixes or certain IP address prefixes with a relatively high traffic proportion.

[0116] In addition, the user group division rules can be divided according to all rules (i.e. region, operator, AS number, access method, etc.); or divided according to some rules, such as region + operator, region + access method, region. Figure 4a as well as Figure 4b As shown in FIG. 1 , it is a schematic diagram of a scheme for dividing the IP addresses of user clients. Figure 4a To divide according to all the rules, Figure 4b It is divided according to some rules (regions).

[0117] IP Prefix User group ID 192.168.0.0 / 24 UG_1 192.168.1.0 / 24 UG_1 192.168.2.0 / 24 UG_1 192.168.3.0 / 24 UG_2 192.168.4.0 / 24 UG_2

[0118] Table 1 User group division information table

[0119] As shown in Table 1, it is a user group division information table. As shown in the table, user groups can also be divided according to IP address prefixes. IP addresses belonging to the same user group have the same user group identifier.

[0120] Step 202: The server collects statistics on the QoS data within the statistical period, and calculates the QoS data of each user group based on the collected QoS data.

[0121] Among them, in an embodiment of the present application, the server can record the network QoS information and client IP address of each data stream. Specifically, when each stream ends, the server will write the relevant QoS information and client IP address of this stream into the server's log file. QoS information includes: minimum RTT (min_rtt), maximum RTT (max_rtt), smooth RTT (srtt), maximum in-flight (max_inflight), effective throughput (goodput), retransmission rate (retran_ratio), initial window (init_cwnd), maximum window (max_cwnd) and other information. That is, the above information indicates the service quality of the network. After each data stream transmission is completed, the server will record the IP address of the data stream and the minimum RTT (min_rtt), maximum RTT (max_rtt), smooth RTT (srtt), maximum in-flight (max_inflight), effective throughput (goodput), retransmission rate (retran_ratio), initial window (init_cwnd), maximum window (max_cwnd) and other information of the data stream, so as to obtain the QoS information of the data stream.

[0122] The server can count the QoS mean and related quantile values ​​(such as the 50th or 90th percentile QoS value) of each user group within a period of time (recorded as the statistical period T). Specifically, the server reads the QoS information and client IP address of each flow from the log file one by one, and obtains the user group identification of each flow by querying the correspondence between the IP address and the user group identification. And take the average value or related quantile (such as the 50th or 90th percentile) of the QoS information of the flows with the same user group identification.

[0123] For example, in the time period T=15 minutes, during this period, there are 100 sessions with IP address 192.168.0.0 / 24, 200 sessions with IP addresses 192.168.1.0 / 24 and 192.168.2.0 / 24, and 50 sessions with IP addresses 192.168.3.0 / 24 and 192.168.4.0 / 24. Therefore, the number of sessions of user group 1 (UG_1) is 500, and the number of sessions of user group 2 (UG_2) is 100. In this embodiment, it is assumed that the QoS averages (QoS_avg_1) of 500 sessions in UG_1 are respectively min_rtt_avg_1, max_rtt_avg_1, srtt_avg_1, goodput_avg_1, retran_ratio_avg_1, init_cwnd_avg_1 and max_cwnd_avg_1, and the QoS averages (QoS_avg_2) of 100 sessions in UG_2 are respectively min_rtt_avg_2, max_rtt_avg_2, srtt_avg_2, goodput_avg_2, retran_ratio_avg_2, init_cwnd_avg_2 and max_cwnd_avg_2. Therefore, the reward value of UG_1 is goodput_avg_1 / srtt_avg_1, and the reward value of UG_2 is goodput_avg_2 / srtt_avg_2.

[0124] In step 203, the server performs weighted calculation on the QoS data of each user group, and uses the data obtained by the weighted calculation as the state value of the reinforcement learning model.

[0125] Furthermore, the server can obtain a weight coefficient for each user group, which can be manually set by the user or calculated according to the number of users in the user group. After determining the weight coefficient for each user group, the QoS data of each user group can be weighted calculated based on the weight coefficient of each user group, and the result of the weighted calculation is used as the state value of the reinforcement learning model.

[0126] Specifically, the server weights the QoS information according to the user group, where the QoS of user group i is recorded as QoS_i, and the weight of QoS_i is recorded as w_i. At this time, the weighted QoS is shown in formula (1):

[0127] QoS=w_1*QoS_1+w_2*QoS_2+……+w_n*QoS_n Formula (1)

[0128] In step 204, the server extracts effective throughput data and smoothed round-trip transmission time data from the QoS data of each user group, and calculates a ratio of the effective throughput data to the smoothed round-trip transmission time data as a reward value of the reinforcement learning model.

[0129] Furthermore, the server extracts effective throughput data and smoothed round-trip transmission time data from the QoS data of each user group, and calculates the ratio of the effective throughput data to the smoothed round-trip time data as a reward value of the reinforcement learning model.

[0130] The specific calculation formula is as follows:

[0131] Reward=goodput / srtt formula (2)

[0132] In some embodiments, the numerator in formula (2) may also be throughput or the download speed (download_speed) of data from the application layer to the network layer. The numerator in formula (2) may also be average rtt, min_rtt or max_rtt.

[0133] Step 205: The server performs reinforcement learning on the state value and the reward value corresponding to each user group based on the reinforcement learning model to obtain a learning parameter corresponding to each user group output by the reinforcement learning model.

[0134] After determining the state value and reward value of the reinforcement learning model, the state value and reward value can be input into the reinforcement learning model for reinforcement learning to obtain the parameter α output by the reinforcement learning model. The reinforcement learning model here can be one reinforcement learning model or multiple reinforcement learning models. Different user groups provide different reward values ​​for the reinforcement learning model, so the learning parameters learned by the reinforcement learning model for different user groups are also different. The output of the reinforcement learning model is one of the parameter set {-1, -0.5, 0, 0.5, 1} (denoted as α, the default value is 0).

[0135] Step 206 : The server calculates the data transmission parameters corresponding to each user group based on the learning parameters corresponding to each user group and the QoS information of each user group.

[0136] After learning the learning parameters corresponding to each user group, the data transmission parameters corresponding to each user group can be further calculated according to the learning parameters and QoS information of each user group. The data transmission parameters here can also be called initialization congestion control parameters. Specifically, the data transmission parameters of each user group can be calculated according to the following formula:

[0137] Init_rtt=min_rtt formula (3)

[0138] Init_cwnd=2α*(goodput*init_rtt) Formula (4)

[0139] For example, for the above user group 1, namely UG1, the calculation formula for initializing the congestion parameter is as follows:

[0140] init_rtt_1=min_rtt_avg_1

[0141] init_cwnd_1=2α*(goodput_avg_1*init_rtt_1)

[0142] Furthermore, for each user group, the above method can be used to calculate the data transmission parameters of each user group, and then the data transmission parameters of each user group can be saved for future use.

[0143] As shown in Table 2, this is a schematic diagram of the saved initialization parameter table.

[0144] User Group ID Init_rtt Init_cwnd UG_1 Init_rtt_1 Init_cwnd_1 UG_2 Init_rtt_2 Init_cwnd_2

[0145] Table 2 Initialization parameter table

[0146] Step 207: When the server receives the data transmission connection request sent by the user client, the server obtains the IP address of the user client.

[0147] When the server receives a data transmission connection request sent by a user client, it can obtain the IP address included in the connection request.

[0148] Step 208: The server determines the corresponding group information according to the IP address of the user client, and determines the data transmission parameters corresponding to the group information.

[0149] After obtaining the IP address of the user client, the group information corresponding to the user client can be further determined, and the initialization parameters corresponding to the group can be found in the initialization parameter table based on the group information, that is, the corresponding data transmission parameters can be found.

[0150] Step 209: The server responds to the connection request of the user client according to the corresponding data transmission parameters and sends data to the user client.

[0151] After determining the data transmission parameters of the user group corresponding to the user client, the data connection request of the user client can be responded to according to the parameters, and the request data can be sent to the user client. Specifically, when the group described by the user is determined to be UG_2 according to the user ID, the initial rtt in its initial data transmission parameters can be determined to be Init_rtt_2, and the initial cwnd can be determined to be Init_cwnd_2. In this way, the rtt in the initial network parameters when sending data to the user client can be set to Init_rtt_2, and the cwnd in the initial network parameters can be set to Init_cwnd_2, and then the corresponding data transmission can be performed.

[0152] Among them, the existing congestion control algorithms can be divided into two categories. One is the "rule-based" congestion control algorithm based on expert experience. This type of algorithm determines whether congestion occurs in the current network based on packet loss information, delay information or bandwidth detection information, and then performs congestion avoidance and other actions. However, these methods often rely on the experience of network personnel to set the congestion avoidance method, and lack good adaptability to network status and related services (such as audio and video services), resulting in less benefits from network transmission performance optimization. The other type of congestion control algorithm is mainly based on artificial intelligence. This type of method either completely overturns the existing congestion control algorithm or has many problems in stability, fine control, etc. and cannot be truly deployed. The adaptive congestion control method (data transmission control method) for user groups proposed in the embodiment of the present application can solve the above problems well. By using network measurement technology, the advantages of traditional congestion control are retained. The new congestion control parameters are set on the basis of the traditional congestion control algorithm, avoiding the challenges faced by the existing congestion control algorithm based on machine learning in terms of convergence, and can ensure the stability of the proposed adaptive congestion control method to a certain extent; at the same time, the present invention divides the client IP into different user groups according to rules such as region, operator, access mode, etc., and sets a special initialization congestion control parameter for each user group through a single reinforcement learning module based on reinforcement learning, thereby realizing the refinement and lightweight control of congestion avoidance during network traffic transmission. The adaptive congestion control method proposed in the present invention can achieve greater benefits in the initialization parameter setting of the existing congestion control algorithm, which is conducive to improving the network transmission performance of audio and video traffic and enhancing the competitiveness of audio and video services.

[0153] like Figure 5As shown, it is another flow chart of the data transmission control method provided by the present application. As shown in the figure, the process can be specifically executed by four modules, which can specifically include a network measurement module 10, an intelligent learning module 20, a parameter calculation module 30 and a transmission control module 40. Among them, the network measurement module 10 can obtain the historical network performance data of multiple users and can divide the users into multiple user groups according to the IP address, obtain the user group information of each user, and determine the group network performance data corresponding to each user group based on the historical network performance data of each user. Further, the network measurement module 10 can also perform weighted calculation on the group network performance data of multiple user groups to obtain weighted network performance data. The intelligent learning module 20 can use the weighted network performance data as the state value, and the ratio of the effective throughput and the smoothed rtt in the group network performance data as the reward value to perform reinforcement learning through the reinforcement learning model to obtain the learned data transmission index. The parameter calculation module 30 can use the data transmission index learned by the reinforcement learning model to calculate the data transmission parameters corresponding to each user group to form a data transmission parameter list. When receiving a data transmission connection request uploaded by a user, the transmission control module 40 can determine the user group to which the user belongs based on the user's IP address, and then query the data transmission parameters corresponding to the user group in the data transmission parameter list, and then use the corresponding data transmission parameters to control the data transmission process, thereby improving the transmission performance of the data transmission.

[0154] According to the above description, the data transmission control method provided by the present application obtains historical network performance data of multiple data transmission objects, and calculates group network performance data of each object group based on the historical network performance data, where the object group is a group obtained by dividing multiple data transmission objects; performs weighted calculation on the group network performance data to obtain target group network performance data; performs reinforcement learning on the target group network performance data and the group network performance data of each object group based on a reinforcement learning model to obtain a data transmission index for each object group; calculates data transmission parameters for each object group based on the data transmission index and the group network performance data; when a data transmission connection request sent by the target data transmission object is received, determines the target data transmission parameters according to the object group to which the target data transmission object belongs, and controls the data transmission process based on the target data transmission parameters.

[0155] Thus, the data transmission control method provided by the present application divides the data transmission objects into groups, and then uses the historical network performance data of each group in the recent period to perform reinforcement learning to obtain the most suitable network data transmission parameters for each group at the current time. This method can perform fine-grained control of network data transmission parameters, thereby matching the most suitable network data transmission parameters for each data transmission object at the current time, which can greatly improve the data transmission performance of the data transmission object.

[0156] In order to better implement the above data transmission control method, an embodiment of the present application further provides a data transmission control device, which can be integrated in a terminal or a server.

[0157] For example, Figure 6 As shown, it is a structural schematic diagram of a data transmission control device provided in an embodiment of the present application, and the data transmission control device may include an acquisition unit 301, a first calculation unit 302, a learning unit 303, a second calculation unit 304 and a control unit 305, as follows:

[0158] An acquisition unit 301 is used to acquire historical network performance data of multiple data transmission objects, and calculate group network performance data of each object group based on the historical network performance data, where the object group is a group obtained by dividing the multiple data transmission objects;

[0159] A first calculation unit 302, configured to perform weighted calculation on the group network performance data to obtain target group network performance data;

[0160] A learning unit 303, configured to perform reinforcement learning on the target group network performance data and the group network performance data of each object group based on a reinforcement learning model to obtain a data transmission index of each object group;

[0161] A second calculation unit 304, configured to calculate a data transmission parameter for each object group according to the data transmission index and the group network performance data;

[0162] The control unit 305 is used to determine the target data transmission parameters according to the object group to which the target data transmission object belongs when receiving the data transmission connection request sent by the target data transmission object, and control the data transmission process based on the target data transmission parameters.

[0163] In some embodiments, the learning unit includes:

[0164] A first calculation subunit, configured to calculate a reinforcement learning parameter of each object group according to the group network performance data of each object group;

[0165] The learning subunit is used to input the target group network performance data as the state value of the reinforcement learning model and the reinforcement learning parameter as the reward value of the reinforcement learning model into the reinforcement learning model to obtain the data transmission index of each object group as output.

[0166] In some embodiments, the first computing subunit includes:

[0167] An extraction module, used for extracting effective throughput data and smoothed round-trip transmission time data from the group network performance data of each object group;

[0168] The first calculation module is used to calculate the ratio between the effective throughput data corresponding to each object group and the corresponding smoothed round-trip transmission time data to obtain the reinforcement learning parameters of each object group.

[0169] In some embodiments, the acquisition unit includes:

[0170] A first acquisition subunit is used to acquire Internet Protocol address data of multiple data transmission objects and to acquire historical network performance data of multiple data transmission objects;

[0171] a partitioning subunit, for partitioning a plurality of data transmission objects into a plurality of object groups based on Internet Protocol address data;

[0172] The second calculation subunit is used to calculate the group network performance data of each object group according to the historical network performance data of the data transmission objects in each object group.

[0173] In some embodiments, the second computing subunit includes:

[0174] A determination module, configured to determine a set of historical network performance data corresponding to each object group according to the historical network performance data of each data transmission object in each object group;

[0175] The second calculation module is used to calculate the average value of the historical network performance data in each historical network performance data set to obtain the group network performance data corresponding to each object group.

[0176] In some embodiments, the first computing unit includes:

[0177] A second acquisition subunit, used to acquire a weight coefficient of each object group;

[0178] The third calculation subunit is used to perform weighted calculation on the group network performance data of each object group based on the weight coefficient to obtain the target group network performance data.

[0179] In some embodiments, the second acquisition subunit includes:

[0180] An acquisition module, used for acquiring the number of data transmission objects included in each object group;

[0181] The third calculation module is used to calculate the ratio between the number of data transmission objects in each group and the number of multiple data transmission objects to obtain a weight coefficient of each object group.

[0182] In some embodiments, the second computing unit includes:

[0183] An extraction subunit, used to extract effective throughput data and minimum round-trip transmission time data of each object group from the group network performance data;

[0184] The fourth calculation subunit is used to calculate the initial window data of each object group based on the effective throughput data, the minimum round-trip transmission time data and the data transmission index of each object group, and determine the data transmission parameters of each object group based on the initial window data of each object group.

[0185] In some embodiments, the control unit comprises:

[0186] A third acquisition subunit is used to acquire a target Internet Protocol address of a target data transmission object when receiving a data transmission connection request sent by the target data transmission object;

[0187] a determination subunit, configured to determine a target object group to which a target data transmission object belongs according to a target Internet Protocol address;

[0188] The transmission subunit is used to obtain target data transmission parameters of the target object group, and perform data transmission on the target data transmission objects based on the target data transmission parameters.

[0189] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous method embodiments, which will not be repeated here.

[0190] According to the above description, the data transmission control device provided in the embodiment of the present application obtains the historical network performance data of multiple data transmission objects through the acquisition unit 301, and calculates the group network performance data of each object group based on the historical network performance data, where the object group is a group obtained by dividing multiple data transmission objects; the first calculation unit 302 performs weighted calculation on the group network performance data to obtain the target group network performance data; the learning unit 303 performs reinforcement learning on the target group network performance data and the group network performance data of each object group based on the reinforcement learning model to obtain the data transmission index of each object group; the second calculation unit 304 calculates the data transmission parameters of each object group based on the data transmission index and the group network performance data; when the control unit 305 receives the data transmission connection request sent by the target data transmission object, it determines the target data transmission parameters according to the object group to which the target data transmission object belongs, and controls the data transmission process based on the target data transmission parameters.

[0191] Thus, the data transmission control method provided by the present application divides the data transmission objects into groups, and then uses the historical network performance data of each group in the recent period to perform reinforcement learning to obtain the most suitable network data transmission parameters for each group at the current time. This method can perform fine-grained control of network data transmission parameters, thereby matching the most suitable network data transmission parameters for each data transmission object at the current time, which can greatly improve the data transmission performance of the data transmission object.

[0192] The present application also provides a computer device, which may be a terminal or a server. Figure 7 The figure is a schematic diagram of the structure of the computer device provided by this application. Specifically:

[0193] The computer device may include components such as a processing unit 401 of one or more processing cores, a storage unit 402 of one or more storage media, a power module 403, and an input module 404. Those skilled in the art will appreciate that Figure 7 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0194] The processing unit 401 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. It executes various functions of the computer device and processes data by running or executing software programs and / or modules stored in the storage unit 402, and calling data stored in the storage unit 402. Optionally, the processing unit 401 may include one or more processing cores; preferably, the processing unit 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, object interface and application program, etc., and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processing unit 401.

[0195] The storage unit 402 can be used to store software programs and modules. The processing unit 401 executes various functional applications and data processing by running the software programs and modules stored in the storage unit 402. The storage unit 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, and web page access, etc.); the data storage area may store data created according to the use of the computer device, etc. In addition, the storage unit 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the storage unit 402 may also include a memory controller to provide the processing unit 401 with access to the storage unit 402.

[0196] The computer device also includes a power module 403 for supplying power to each component. Preferably, the power module 403 can be logically connected to the processing unit 401 through a power management system, so as to achieve functions such as management of charging, discharging, and power consumption management through the power management system. The power module 403 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0197] The computer device may further include an input module 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to object settings and function controls.

[0198] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically, in this embodiment, the processing unit 401 in the computer device will load the executable files corresponding to the processes of one or more application programs into the storage unit 402 according to the following instructions, and the processing unit 401 will run the application programs stored in the storage unit 402, thereby realizing various functions, as follows:

[0199] Acquire historical network performance data of multiple data transmission objects, and calculate group network performance data of each object group based on the historical network performance data, where the object group is a group obtained by dividing multiple data transmission objects; perform weighted calculation on the group network performance data to obtain target group network performance data; perform reinforcement learning on the target group network performance data and the group network performance data of each object group based on a reinforcement learning model to obtain a data transmission index for each object group; calculate the data transmission parameters of each object group based on the data transmission index and the group network performance data; when a data transmission connection request sent by the target data transmission object is received, determine the target data transmission parameters according to the object group to which the target data transmission object belongs, and control the data transmission process based on the target data transmission parameters.

[0200] It should be noted that the computer device provided in the embodiment of the present application and the method in the above embodiment belong to the same concept, and the specific implementation of the above operations can be referred to the previous embodiment, which will not be repeated here.

[0201] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0202] To this end, an embodiment of the present invention provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any method provided in the embodiment of the present invention. For example, the instructions can execute the following steps:

[0203] Acquire historical network performance data of multiple data transmission objects, and calculate group network performance data of each object group based on the historical network performance data, where the object group is a group obtained by dividing multiple data transmission objects; perform weighted calculation on the group network performance data to obtain target group network performance data; perform reinforcement learning on the target group network performance data and the group network performance data of each object group based on a reinforcement learning model to obtain a data transmission index for each object group; calculate the data transmission parameters of each object group based on the data transmission index and the group network performance data; when a data transmission connection request sent by the target data transmission object is received, determine the target data transmission parameters according to the object group to which the target data transmission object belongs, and control the data transmission process based on the target data transmission parameters.

[0204] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0205] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0206] Since the instructions stored in the computer-readable storage medium can execute the steps in any method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0207] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a storage medium. A processor of a computer device reads the computer instructions from the storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional implementations of the above-mentioned data transmission control method.

[0208] The data transmission control method, device and computer-readable storage medium provided in the embodiments of the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A data transmission control method, characterized in that: The method comprises: Acquire historical network performance data of a plurality of data transmission objects, and calculate group network performance data of each object group based on the historical network performance data, wherein the object group is a group obtained by dividing the plurality of data transmission objects; Performing weighted calculation on the group network performance data to obtain target group network performance data; Performing reinforcement learning on the target group network performance data and the group network performance data of each object group based on a reinforcement learning model to obtain a data transmission index of each object group; Calculate a data transmission parameter for each object group according to the data transmission index and the group network performance data; When a data transmission connection request sent by a target data transmission object is received, a target data transmission parameter is determined according to the object group to which the target data transmission object belongs, and the data transmission process is controlled based on the target data transmission parameter.

2. The method according to claim 1, characterized in that The reinforcement learning is performed on the target group network performance data and the group network performance data of each object group based on the reinforcement learning model to obtain the data transmission index of each object group, including: Calculating reinforcement learning parameters for each group of objects based on group network performance data for each group of objects; The target group network performance data is used as the state value of the reinforcement learning model, and the reinforcement learning parameter is used as the reward value of the reinforcement learning model, which is input into the reinforcement learning model to obtain the data transmission index of each object group as an output.

3. The method according to claim 2, characterized in that The step of calculating the reinforcement learning parameters of each group of objects according to the group network performance data of each group of objects includes: Extracting effective throughput data and smoothed round-trip transmission time data from the group network performance data of each object group; The ratio between the effective throughput data corresponding to each object group and the corresponding smoothed round-trip transmission time data is calculated to obtain the reinforcement learning parameters of each object group.

4. The method according to claim 1, characterized in that: The acquiring of historical network performance data of a plurality of data transmission objects, and calculating group network performance data of each object group based on the historical network performance data, wherein the object group is a group obtained by dividing the plurality of data transmission objects, includes: Obtaining Internet Protocol address data for a plurality of data transmission objects and obtaining historical network performance data for the plurality of data transmission objects; dividing the plurality of data transmission objects into a plurality of object groups based on the Internet Protocol address data; The group network performance data of each object group is calculated according to the historical network performance data of the data transmission objects in each object group.

5. The method according to claim 4, characterized in that The calculating the group network performance data of each object group according to the historical network performance data of the data transmission objects in each object group includes: Determine a set of historical network performance data corresponding to each object group according to the historical network performance data of each data transmission object in each object group; The average value of the historical network performance data in each historical network performance data set is calculated to obtain the group network performance data corresponding to each object group.

6. The method according to any one of claims 1 to 5, characterized in that The performing weighted calculation on the group network performance data to obtain target group network performance data includes: Obtaining the weight coefficient of each object group; The group network performance data of each object group is weightedly calculated based on the weight coefficient to obtain the target group network performance data.

7. The method according to claim 6, characterized in that The obtaining of the weight coefficient of each object group includes: Get the number of data transfer objects contained in each object group; The ratio between the number of data transmission objects in each group and the number of the plurality of data transmission objects is calculated to obtain a weight coefficient of each object group.

8. The method according to any one of claims 1 to 5, characterized in that The calculating the data transmission parameter of each object group according to the data transmission index and the group network performance data includes: Extracting effective throughput data and minimum round-trip transmission time data of each object group from the group network performance data; Initial window data of each object group is calculated based on the effective throughput data, the minimum round-trip transmission time data and the data transmission index of each object group, and a data transmission parameter of each object group is determined based on the initial window data of each object group.

9. The method according to claim 4, characterized in that The method of, when receiving a data transmission connection request sent by a target data transmission object, determining a target data transmission parameter according to the object group to which the target data transmission object belongs, and controlling the data transmission process based on the target data transmission parameter, includes: When receiving a data transmission connection request sent by a target data transmission object, obtaining a target Internet Protocol address of the target data transmission object; Determine, according to the target Internet Protocol address, a target object group to which the target data transmission object belongs; The target data transmission parameters of the target object group are acquired, and data transmission is performed on the target data transmission objects based on the target data transmission parameters.

10. A data transmission control device, characterized in that: The device comprises: an acquisition unit, configured to acquire historical network performance data of a plurality of data transmission objects, and calculate group network performance data of each object group based on the historical network performance data, wherein the object group is a group obtained by dividing the plurality of data transmission objects; A first calculation unit, configured to perform weighted calculation on the group network performance data to obtain target group network performance data; A learning unit, configured to perform reinforcement learning on the target group network performance data and the group network performance data of each object group based on a reinforcement learning model to obtain a data transmission index of each object group; A second calculation unit, configured to calculate a data transmission parameter of each object group according to the data transmission index and the group network performance data; The control unit is used to determine the target data transmission parameters according to the object group to which the target data transmission object belongs when receiving a data transmission connection request sent by the target data transmission object, and control the data transmission process based on the target data transmission parameters.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the data transmission control method according to any one of claims 1 to 9.

12. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the data transmission control method according to any one of claims 1 to 9 when executing the computer program.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps in the data transmission control method according to any one of claims 1 to 9 are implemented.

Images